I found a way to do what I needed.

1. I first created a cell with a dropdown list of the section headers in cell A2. This was not strightforward, because Google Sheets kept transforming the whole column into a dropdown column, which I didn't want. So I created the cell with the dropdown list in a temporary sheet and then I copied the cell, went to the cell where I needed the dropdown list and pasted only the data validation.

2. Next I used this formula in cell A3:

   =HYPERLINK(
      "#gid=12345678range=" &SUBSTITUTE(
      SUBSTITUTE(
         cell("address",xlookup(A2,B:B,B:B)), "data!", ""
      ),"$",""
   ),"GO TO SECTION")
   

So, I have to first select the header I wish in cell A2. Then click "GO TO SECTION" in A3. A popup with the link shows up. Clicking this link takes me to the header I need.

It is still two extra clicks than ideal, but it does the trick.