Just found out about SpacialEventGesture which seems to solve this now in iOS 18, SwiftUI docs: SpacialEventGesture I plan on making a usable example soon.