Does anyone know which environment variable or configuration file might have been responsible for ensuring charset compatibility in the previous setup?

Any insights would be greatly appreciated!

You have to redo the migration. And before redoing the migration you have to migrate the serialized data first and fix the column type of the affected fields.

(OR: If your setup currently works with the workaround you may be able to do this without redoing the migration. After applying the fix (see below), it must work without the workaround and the workaround should be removed.)

As outlined in XXX (Q&A), serialized data must not be stored in text fields with a character encoding other than BINARY.

To remove the double negation: PHP serialized data should be stored in BINARY columns. If you do otherwise problems like the ones you describe might arise.

• Identify the affected columns
• Test unserialize() against migrated columns with a connection encoding that provokes the errors you are seeing.
• Alter the columns by transposing their encoding to BINARY.
• Test unserialize() against migrated columns, it should now be immune to any connection encoding.