Found a way to make Azure SQL connections using private IP work:

Set the jdbc url to "jdbc:sqlserver://<private_ip>:1433;databaseName=<yourdatabase>;trustServerCertificate=true".

Set the SQL DB username to "<db_user>@<your_azure_sql_server>.database.windows.net".

It needs the SQL server name appended to the user name with an @ to let Azure SQL accept the connection.