3 years later and giving full disk access to Xcode is still the answer. Although for me it was Mac -> System Settings -> Privacy & Security -> Full Disk Access -> Toggle for Xcode