So I found out (thanks @pratik-jadhav) that using the npx teamsapp auth command is not suitable for usage in a CI/CD pipeline due to the client credentials flow using a scope that cannot be changed (as far as I know).

Retrieving the AZURE_ACCESS_TOKEN by CURL and setting it as a environment variable seems to be working as a workaround.

    - >
      export AZURE_ACCESS_TOKEN=$(curl -s -X POST https://login.microsoftonline.com/$AAD_APP_TENANT_ID/oauth2/v2.0/token
      -H "Content-Type: application/x-www-form-urlencoded"
      -d "client_id=$AAD_APP_CLIENT_ID"
      -d "scope=https://management.azure.com/.default"
      -d "client_secret=$AAD_APP_CLIENT_SECRET"
      -d "grant_type=client_credentials" | jq -r .access_token)

I also tried using the az login command in combination with npx teamsapp deploy which authenticates me as well.

    - az login --service-principal -u "$AAD_APP_CLIENT_ID" -p "$AAD_APP_CLIENT_SECRET" --tenant "$AAD_APP_TENANT_ID"
    - az account set --subscription "$AAD_SUBSCRPTION_ID"

However, when the npx teamsapp deploy command starts, I get the following error now:

I tried fixing this by manually setting the SWA_CLI_DEPLOYMENT_TOKEN (that is used by Azure's az deploy command) as a CI/CD environment variable, but the error remains the same.