Turns out the delay was caused by Xcode itself.

If I turn off Debug Executable in the Run scheme, the model loads instantly. According to an Apple engineer, this is a known issue in Xcode 26.0.1 (17A400). To disable it open Edit Scheme → Run → Info, then uncheck Debug Executable. After that, RealityKit scenes launch immediately on device.