Dear Ayesha Kiran,

The Hyperparameter tuning need to be put outside the loop, so that each training iteration can be fairly evaluated.

you can see the hyperparameter setting in the training beginning stage on the ref link.

ref: https://scikit-learn.org/stable/modules/cross_validation.html#