My initial guess would be that your generator runs out of data and you need ot manually "refill" it.

While running a custom example, which was generated from Gemini, on colab, I got the following warning:

UserWarning: Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least steps_per_epoch * epochs batches. You may need to use the `.repeat()` function when building your dataset.

I suppose this could be the root cause of your problems.