J Eti's "accepted" answer directly answers my question, but it misses the intuition which I wanted to get when I asking this as well, which I understand a lot better now that I'm older. I wanted to leave this here to anyone discovering this thread again.

The issue with web scraping is that it's a very "I/O bound" task (eg. waiting for an internet packet to physically traverse across cables across the world have a server respond, etc. Another good one is waiting for a filesystem to respond). When your code runs to send this web request, you don't want function invocation to be "dead-waited". If you solve this via multi-threading (in a naive approach where you don't customize your own sockets/syscalls which 99.99% of developers are), you are effectively leaving your operating system's process scheduler to properly sleep your thread and free up that thread so other threads could do work. Asynchronous threads on the other hand effectively have this "process scheduler" built into a single thread, where dead-wait is explicitly defined by the programmer. And so, the dead-wait optimization is more explicit by the nature of asynchronous programming and thus easier to optimize around to the library writer, leading to faster code without us having to do that task scheduling manually :-)