I'm getting the similar results with stratifiedKFold. For any k folds, only the last fold is good , while the first k-1 fold always perform worse. So, I did k iterations instead with better results as you did. No idea why.