crawler
Some urls can be climbed normally
Are these urls from the same site? It is possible your crawler is detected as a bot, or you just faced a rate limit.