So to resolve this, it seems like my issue was actually with SimpleCov and Rails 8 beta and simplecov was giving incorrect coverage reports, and it has fixed itself with the final release of rails 8.