Please share more details on your issue, like how you implemented torch.cuda.memory_reserved().
And what are the command line outputs of that you see that As the training progresses, the training slows down?
As the training progresses, the training slows down
And how do you monitor your memory?