I got the API to database connection to finally work by using the following tutorial
See step 2. Create a passwordless connection. As the tutorial mentions, I used the Azure Portal to automatically create the connection for a system-assigned managed identity.