We were struggling with a similar issue (Invalid Signature on a JWT issued by Azure Entra ID). The comment from @Apps on the other answer here was the correct one for us - we just needed to add {clientId}/.default into the requested scopes when authenticating.