I found the issue, earlier in the code I have added a comment, stating that the first 4 bytes of the message are the packet length and the packet number, so those must not participate in the message parsing.

const MYSQL_CLIENT_HANDSHAKE_USERNAME_START_POS = 36 // packet len + packet number + 32 bytes from the message(client caps and other fields)

Once I took this into account:

clientCaps := binary.LittleEndian.Uint32(input[4:8])

things started working properly.