QueueClient
expects a message that is in UTF-8 format. This is from the documentation:
A message must be in a format that can be included in an XML request with UTF-8 encoding.
The way to serialize your object was causing problems. Using something like this will work:
var queueMessage = System.Text.Json.JsonSerializer.Serialize(item);
because this implementation internally uses Utf8JsonWriter.
Most likely the old library that you ended up using does support Base64 encoding.