Something that has worked for me is to use a stateful widget to disable the scroll view once the user starts interacting with the gesture detector.

Also it is important to use a GlobalKey, to make sure that always the same instance of the widget is the one that is displayed and therefore the state is kept