Christian has a good hack for an answer here, but perhaps a more updated CSS hack would be to use user-select: none:

.CodeMirror-gutter-wrapper {
  user-select: none;
}