There is now a checkbox to turn off references. Just go to settings search "code lens" and you should see it.