Look a this example: https://godbolt.org/z/1TYco6xM1

The compiler will store the variable, if volatile, after each modify and read it back again.

If your ISRs are non-concurrent, then you can get away with not making it volatile, since code would not get preempted. The access will be basically atomic.

That said, I would say, if you work not alone on this project, make it volatile. The speed impact will be small, as well as the memory footprint. And most important, if the variable will, at some point, be used at other places as well, you will not have have less issues with concurrent access.