It's not that you have bad clicking precision, it's because the gutter is poorly designed. And, as to your question, I too have searched for an answer but have not found none.

It is frustrating because I will fold a block far more often then set a breakpoint.

There is an open issue.

Upvote/star it and maybe fixing the issue can gain some traction.