When using huggingface transformer library, the output returned by the model includes the model loss. Before starting the training, simply perform a forward pass on the dataset and obtain the model loss.

with torch.no_grad():
    outputs = model(**inputs, labels=labels)

loss = outputs.loss
print(f"Loss: {loss.item()}")