Got a response from an Apple engineer on how to correctly handle this:

.onPreferenceChange(HeightKey.self) { [$height] height in
  $height.wrappedValue = height
}

I feel a little sheepish for not realizing this earlier.

The approach that avoids preferences by @Benzy Neez is also a good one.