The problem in my case was that I had a lot of virtual servers (~15). In the Server configuration there's an option for Virtual Servers and if not specified all are deployed to. Everything worked normally after I removed all but the default virtual server from my config.