The problem is still there, but I was able to work around it by linking directly to the View with NavigationLink without using navigationDestination.
NavigationLink
navigationDestination