In my case it is caused by IIS. Simply exiting the IIS development server in the tray icon allows me to debug again. Still annoying, but faster than a VS restart.