Finally found a way to do this using the apiPrefix property on the app service: https://techcommunity.microsoft.com/blog/azurenetworkingblog/understanding-azure-app-service-authentication-challenges-with-path-based-routin/4372730