Set the model to evaluation mode when you are doing inference or testing.
Just add model.eval() after the model initialization
model.eval()
And, https://github.com/ultralytics/ultralytics/issues/4471 here you can find some explanation