This looks like a system bug rather than an issue with your code.
You can reproduce it on iOS 26.1 Beta — even when the condition is false, SwiftUI still reserves space for the tabViewBottomAccessory.

At the moment, there doesn’t seem to be a workaround; we just have to wait until Apple fixes it.