I ended up storing the time each host was last contacted. I am storing in a global variable. If the time passed since the last contact, is less than the required crawl-delay, the thread is sent to sleep until crawl-delay requirement is met.

To be thread-safe, this logic is happening in a locked space in each thread, as described here: https://superfastpython.com/threadpoolexecutor-thread-safe/#How_to_Make_Critical_Sections_Thread-Safe