TLDR; use this

import os

os.environ['TF_USE_LEGACY_KERAS'] = '1'

Explanation: Transformers package uses Keras 2 objects, current version is Keras 3, packed in Tensorflow since version 2.16. Fastest fix without downgrading tensorflow is to set legacy keras usage flag as above. More info can be found here.