If you use the enhanced flow (GetCredentialsForIdentity) a scope down policy is applied to all guest identities which doesn't include Bedrock.

In order to allow guest identities to access Bedrock, you need to use the classic flow (GetOpenIdToken and STS AssumeRoleWithWebIdentity) that doesn't apply the scope down policy.

With that said, I would not recommend giving guest users access to Bedrock, bad actors can create any number of guests and run up your Bedrock bill.