The issue was that the AWS IAM policy didn't have the "kms:Decrypt" permission for the KMS Encryption key associated with the S3 bucket