You are right, action masking is an established concept, but only for discrete action spaces.

There are also ways to enforce hard constraints for continuous actions. For 1d actions spaces (which I think your problem is?), this is pretty straight forward, since you can just clip the actions in the way you described.

However, for more complex action constraints in higher dimensional spaces, the problem becomes more difficult. To address this, we have published a paper on continuous action masking with convex sets in the 2024 NeurIPS conference.

You can find our paper here https://openreview.net/forum?id=yVzWlFhpRW&noteId=LISF9Uv6j6
Also feel free to reach out if you have any questions about it.