Unfortunately I do not have an answer for you because I am currently going through the same process.
But I was wondering what you landed on here.
We have .NET Core (fortunately we're not on Framework) applications (batch and web) that we are moving to Azure VMs.
My initial thought was assign the VM access to KeyVault, then store client secrets for service principals in KeyVault and then grant the service principal rights to the databases and other resources as needed. This still sounds sub-optimal to me though for multiple reasons.
Access to the VM gives you all the keys you need, which seems like a hefty risk.
We're still ultimately dealing with client secrets (which is just a PW) and all the poor practice that comes along with passwords.
Somehow this seems absolutely no better than just storing our secrets in a config file on the VM, it's a lot of faffing about to wind up with the same exact issues we have had for decades.