Someone already mentioned AWS Vault - this can be a good option, but it depends on longed-lived IAM Users and access keys which AWS now recommend avoiding.

I've built something that is macOS specific called awseal that uses keys generated in the Secure Enclave to encrypt your credentials, so every time they're accessed you're asked for Touch ID. A bit like what Secretive does for SSH keys. It uses AWS Identity Center to bootstrap credentials via OIDC, rather than IAM Users. If you're on a relatively modern Mac I think it's a good option.

If you're not on macOS and you have a private CA - or don't mind setting one up - you might want to look at https://github.com/aws/rolesanywhere-credential-helper. Has support for PKCS#11 and TPMv2.