I have been searching for an answer for a long time. And now, after posting the question I found it.

When returning the error messages as I am doing:

return new ValidationResult(errorMessage);

The error message isn't bound to the field. To bind the error messages to the field the return should instead be:

return new ValidationResult(errorMessage, [validationContext.DisplayName]);