I have gone through the problem statement you have mentioned above, below might be the solutions

1. Handle validation exceptions manually,you can catch its validation exception and manually throw a ResponseStatusException(HttpStatus.UNAUTHORIZED).

public TokenResponse login(LoginUserRequest request){

try {

    validationService.validate(request);

} catch (ConstraintViolationException | SomeOtherValidationException ex) {

    throw new ResponseStatusException(HttpStatus.UNAUTHORIZED, "Username or password wrong");

}

}

2: Use @Valid in the controller

I hope it really helps