Here is what my professor explained to me when I asked the same question. He explained by taking example of 4 fold cross validation.

Split each user’s ratings into 4 folds (e.g., User1’s 4 ratings - 4 test/train splits).

For each fold, hold out 1 rating/user as test data (if a user has <4 ratings, cycle/reuse splits).

Exclude single-rating users from testing (keep their data in training).Train on the remaining ratings (all users’ non-test data).

Repeat for all 4 folds, ensuring users always have training data. That's it ! Hope it makes sense