As of now (2024) I find that devcontainers just work out of the box with the host ssh-agent. I just installed the ssh-agent plugin with zsh and it works with my host ssh credentials when I make a commit. I have not tried the gnupg keys yet.

How exactly it works, I have no clue.