How can I enable cross-account KMS access so that Athena in Account B can read from S3 in Account A, where the KMS key is managed?

You need to add a statement to your key policy in account A to allow your IAM principal in account B to decrypt using the key.

    {
      "Sid": "role-xxxx decrypt",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<account-b>:role/role-xxxx"
      },
      "Action": "kms:Decrypt",
      "Resource": "*"
    }

Then you also need to add the decrypt permission to the identity policy of the principal accessing the bucket:

    {
      "Sid": "decrypt",
      "Effect": "Allow",
      "Action": "kms:Decrypt",
      "Resource": "arn:aws:kms:<region>:<account-a>:key/<key-id>"
    }

You can confirm the key used for bucket level encryption with aws s3api get-bucket-encryption --bucket <bucket-name> or for a specific object with aws s3api head-object --bucket <bucket-name> --key <key>.

Would updating the KMS key policy in Sandbox to allow decryption from the IAM role in QA resolve this? Any other settings I should check?

You also need to add to the identity policy but yeah, for a principal to read an S3 object encrypted with a KMS key, they need read access to that object and decrypt permission on the key. So if you add these permissions to the correct principal, for the correct key, then all should work. The only other thing to check that I can think of is if the key is in another region, then you'll need a multi-region key with a replica in your region.