If you are able to support iOS 18, onScrollPhaseChange` modifier is the solution.
Here its documentation.