This appears to be a weird/unintended behavior of OpenSearch. I was able to resolve it by setting the access policy to the root principal of my AWS account:
Principal: ['arn:aws:iam::$ACCOUNT_ID:root']