Try removing the if in compute_loss. It could be said that the other branch is "unused".
compute_loss
https://github.com/Lightning-AI/pytorch-lightning/issues/17212#issuecomment-1632222390