I ended up using rbenv, creating a separate account to install it where most of the apps will run from. For cron jobs I added the shim dir to the path and explicitly invoked ruby <script> so that was relatively straight forward. Some programs were run from systemd so again made sure that the environment was set correctly script ruby invoked explicitly.

Then I went back to @pjs suggestion which I did not understand when I first saw it. Did a bit of research into the env command, though DOH! and tried it out and that seems to work fine. Obviously the program has to be invoked in the correct environment. That is a much simpler approach. There is one program that remains problematic - it is invoke from rsyslog's omprog (output to program) where I have no control over the environment. Luckily that one is not dependent on any of the broken gems.