The issue was in the nextjs app itself and not the deployment configuration. I disabled a redirect in the middleware.ts and now it is working.

The nextjs app worked without errors + responded with 200 on /api/healthcheck, so I am still not sure why that caused an issue but the face stays the problem was resolved.