Smartphone GPS always drifts. Even good phones move a few meters while standing still. This is normal and not a bug in React Native or Expo. Raw GPS is simply too noisy for small areas like a sculpture park.

First, lower the accuracy mode. Balanced often gives steadier readings than BestForNavigation. The latter exposes more raw noise. Next, apply strong filtering. Your current smoothing lets too much noise pass. Use a low-pass filter with alpha around 0.1–0.2. Add a deadzone of 1–2 meters so tiny jumps get ignored.

A Kalman filter helps even more. It combines new data with past data and reduces jitter. Many GPS hardware projects use this exact approach. For example, the GPS tracker shown here uses filtering to stabilize noisy readings:
https://www.pcbway.com/project/shareproject/GPS_tracker_recorder_for_cars_and_other_vehicles.html

You can do the same. Run your GPS reading through a Kalman filter, then through your low-pass filter. The result will feel smooth and believable.

Also, remember that you only need rough GPS. Your routing works on your own map, not on real geography. You can snap the user to the nearest logical area or path and ignore tiny moves. This is how many indoor and park apps work.

So your recipe is simple: lower accuracy, filter hard, apply a deadzone, and use a Kalman filter. After that, your dot will stay calm, and your offline routes will make sense.