In my case, setting the metadata type to attribute solved the issue.
vich_uploader: metadata: type: attribute