We do something similar at my work where the code signing keys are generated in the HSM and we leverage a signing platform called GaraSign to do the actual signing. We don't have to RDP to the various servers to do the signing, although you could implement it that way. In our environment each developer can sign from their own workstation using the centralized key, and SSO from our AD Domain controls authentication and authorization. We don't allow many developers to sign anymore as we try to control that all from our CI/CD pipeline, although exceptions have been made for certain legacy use cases. Since we are a large company we have a few different HSMs that we use, Azure Key Vault being one of them but also Luna HSM.