This seems to work fine: torch.autograd.functional.jacobian(model.decoder, latent_l, strategy="forward-mode", vectorize=True), where only forward passes are needed instead of computing the whole jacobian.