Could you use kubectl cluster-info to verify if you are accessing the same EKS cluster from cloudshell and your terminal? The output shows the EKS apiserver endpoint address.

Also, I noticed that you use --profile when doing update-kubeconfig, are you using the same profile when executing all commands?

Another thing you can try is kubectl get node -v=9 and compare output from cloudshell and terminal.

To answer all your questions:

1. How can I ensure my admin user has the appropriate permissions to access the EKS cluster?

If your admin user is whom you used to execute eksctl create command, then you will be fine. The cluster creator will be granted access permission automatically. You don't need to manually configure it.

2. What steps should I take to modify the aws-auth ConfigMap to grant access?

If you want to grant access to other IAM users, EKS provides 2 ways to do that. The aws-auth configmap and EKS IAM Access Entries (or called EKS API).

I would suggest you to use EKS IAM Access Entries, since it is more simple way to configure it.