In case someone is in need in the future the problem was with with the azure Web App.

I had App Service authentication enabled. I found it by change on this github bug where a dev mentioned about "Easy auth": https://github.com/AzureAD/microsoft-identity-web/issues/2880

Didn't find this documented anywhere so leaving here for reference.