Older question, but for us the only fix was to completely remove the Spark pool and recreate it. After that, all our notebooks ran succesfully again.