You could do that, but it's probably better to use something more robust like AWS secret manager or any of the other SaaS products. I know SO doesn't like software recommendations (or at least asking for them) but really you need a secret manager.

What's right for you will be more specific to your stack and architecture but most cloud providers have some version of this. A quick google search for "secret manager" will set you straight :)