1/ Compare like with like
2/ use unit_scale to avoiding having to count every value on screen
3/ use chunksize to reduce map vs imap differences (credit @Richard Sheridan)
print("Running normaly...")
t0 = time.time()
with Pool(processes=cpu_count()) as pool:
results = list(pool.imap(partial(dummy_task, size=size), steps, chunksize=size))
print("Running with tqdm...")
t2 = time.time()
with Pool(processes=cpu_count()) as pool:
results = list(tqdm(pool.imap(partial(dummy_task, size=size), steps, chunksize=size), total=len(steps), unit_scale=True))
Running normaly...
Time taken: 2.151 seconds
Running with tqdm...
100%|███████████████████████████████████████████████████████████| 500k/500k [00:02<00:00, 237kit/s]
Time taken: 2.192 seconds
Pool Process with TQDM is 1.019 times slower.