I guess there is a bug in VS. Somehow in my situation, I will require to restart my PC each time I want to do a publish.