I recently submitted a feature request to the Vagrant GitHub repo to allow disabling the communicator entirely, making Vagrant usable purely as a VM manager.
Given the community interest in this topic (including this thread), if you think this feature would be useful, feel free to give it a thumbs up or leave a comment to help give it some visibility! 🚀