For me this is because I am trying to do torch.zeros((all_actions_mask.shape[0], 1)).bool().to(device_id) ,do every operations to CPU solves this error.
torch.zeros((all_actions_mask.shape[0], 1)).bool().to(device_id)