Just discovered that pgAdmin can't use the more secure ED25519 keys. The private key format needs to be RSA. With ED25519 key I was getting this error. Switched to RSA and the error was gone.