I think this is related to sharding the data into chunks and delegate it to process with CPU cores or Memory available. As you mentioned it would related with Multiprocessing and Batching strategy topic.