We ran into this issue too. In our case, it was caused by a duplicate folder name in the .git/refs/remotes/origin directory. When a new branch was created, the folder path was given with an uppercase starting letter, which resulted in a duplicate folder on a case-insensitive file system (like Windows). Once we removed the conflicting folder, the issue was resolved.
Delete the folder through website and did "git remote prune origin" as suggested in this thread. Then fetch and pull from VS works fine.