In my case, the problem was solved by manually calling the garbage collector using gc.collect() after the with block.

I believe this does not agree with the multiprocessing documentation. There, it is suggested that a context manager is sufficient to properly handle the resources and that we should not rely on the garbage collector to do it.

Warning

multiprocessing.pool objects have internal resources that need to be properly 
managed (like any other resource) by using the pool as a context manager or by
 calling close() and terminate() manually. Failure to do this can lead to the
 process hanging on finalization.

Note that it is not correct to rely on the garbage collector to destroy the
 pool as CPython does not assure that the finalizer of the pool will be called
 (see object.__del__() for more information).