This issue was caused by how the field was set.
I was setting the data through pydantic model_dump_json.
raw_data = DataRaw(
data=customer.model_dump_json(),
)
This however saved the JSON just as a string. Too bad SQLAlchemy gave no errors/warnings for that.
The problem was solved by replacing model_dump_json with model_dump.