I've gotten around this for now by implementing the guidance on this reddit thread. Basically, I refactored all the logic from the task into a service object, and then the Rake task becomes a single line call that I leave untested. It's probably a better architectural solution, but my OCD doesn't like leaving the task untested, even if it's just a single line, because it's production critical. Oh well.

It still doesn't really answer when/why/how the rakefile runs when running the test suite, but not a single file of tests, why the rake tasks loaded in the rakefile don't persist and need to be re-loaded (but apparently constant definitions do persist), how to load a task in tests without incurring this warning, and why this popped up in Rails 8.0 but was silent in Rails 7.

Anyways, I'll leave this answer as not accepted for a good long time in case someone else has insight on the core questions.