I got the authentication working. Thanks to Shiraz Bhaiji's answer I started looking into the token sent by the client. I started implementing my own AuthenticationHandler on the server side to see what was going on. I think one of the issues was that the authentication issuer wasn't matching: the server was expecting login.microsoftonline.com, but the client was sending sts.windows.net.

After some googling I found a fix by changing requestedAccessTokenVersion from "null" to "2" in the App Registration manifest on Azure.