Xcode writes out IDE workspace data constantly on a timer, on the main thread (WHY ON THE MAIN THREAD APPLE!?!), which is what causes this stuttering. Seeing it even on a M4 Pro (as revealed by using Instruments to sample Xcode).