Skip to content

pathos.pools.ProcessPool deadlock/hang on exceptions #266

@oliver-s-lee

Description

@oliver-s-lee

Hi there, thanks for the nice code!

I'm encountering a problem where my scripts will hang indefinitely on close if an exception is raised after a ProcessPool has been created (even if the exception itself is handled in the body of the program, and even if the pool is not being used at the time of the exception). The script can only be stopped by using ctrl+c, which gives this monster stacktrace:

Stack Trace
...                                              <-- script does work here
^C                                               <-- deliberately interrupted here
Process ForkPoolWorker-10:
script exiting                                   <-- exit message from the script
Process ForkPoolWorker-7:
Process ForkPoolWorker-3:
Process ForkPoolWorker-13:
Process ForkPoolWorker-4:
Process ForkPoolWorker-8:
Process ForkPoolWorker-12:
Process ForkPoolWorker-5:
Process ForkPoolWorker-9:
Process ForkPoolWorker-1:
Process ForkPoolWorker-2:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Process ForkPoolWorker-6:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
Process ForkPoolWorker-11:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 368, in get
    res = self._reader.recv_bytes()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 224, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 422, in _recv_bytes
    buf = self._recv(4)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 387, in _recv
    chunk = read(handle, remaining)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 368, in get
    res = self._reader.recv_bytes()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 224, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 422, in _recv_bytes
    buf = self._recv(4)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 387, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
                                                     <-- Hang indefinitely
^C                                                   <-- Get tired of waiting and raise crtl-c
Process ForkPoolWorker-14:
Exception ignored in atexit callback: <function _exit_function at 0x7f889017ef80>
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 334, in _exit_function
    _run_finalizers(0)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 300, in _run_finalizers
    finalizer()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 224, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 695, in _terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 675, in _help_stuff_finish
    inqueue._rlock.acquire()
KeyboardInterrupt: 
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 368, in get
    res = self._reader.recv_bytes()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 224, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 422, in _recv_bytes
    buf = self._recv(4)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 387, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt

The problem is challenging to recreate exactly, but it can be at least partially recreated by doing the following at the console:

Stack Trace
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathos.pools
>>> pool = pathos.pools.ProcessPool(20)
>>> Process ForkPoolWorker-17:         <-- Raise ctrl-c here
Process ForkPoolWorker-13:
Process ForkPoolWorker-20:
Process ForkPoolWorker-18:
Process ForkPoolWorker-16:
Process ForkPoolWorker-19:
Process ForkPoolWorker-9:
Process ForkPoolWorker-7:
Process ForkPoolWorker-11:
Process ForkPoolWorker-5:
Process ForkPoolWorker-15:

KeyboardInterrupt
>>> Process ForkPoolWorker-10:
Traceback (most recent call last):
Traceback (most recent call last):
Process ForkPoolWorker-6:
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
Process ForkPoolWorker-14:
Process ForkPoolWorker-4:
Process ForkPoolWorker-3:
Traceback (most recent call last):
Process ForkPoolWorker-8:
Process ForkPoolWorker-2:
Process ForkPoolWorker-1:
Process ForkPoolWorker-12:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
KeyboardInterrupt
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 368, in get
    res = self._reader.recv_bytes()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 224, in recv_bytes
    buf = self._recv_bytes(maxlength)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 422, in _recv_bytes
    buf = self._recv(4)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/connection.py", line 387, in _recv
    chunk = read(handle, remaining)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
KeyboardInterrupt

>>>                                    <-- Hangs here
                                       <-- Ctrl-d (EOF) to to exit, hangs again
                                       <-- Ctrl-c again to exit
^CProcess ForkPoolWorker-36:
Process ForkPoolWorker-35:
Process ForkPoolWorker-32:
Process ForkPoolWorker-33:
Process ForkPoolWorker-31:
Process ForkPoolWorker-34:
Process ForkPoolWorker-39:
Process ForkPoolWorker-37:
Process ForkPoolWorker-29:
Process ForkPoolWorker-27:
Process ForkPoolWorker-38:
Process ForkPoolWorker-26:
Process ForkPoolWorker-30:
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
Exception ignored in atexit callback: <function _exit_function at 0x7f10820e55a0>
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 334, in _exit_function
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
Process ForkPoolWorker-23:
Process ForkPoolWorker-25:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Process ForkPoolWorker-24:
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
Process ForkPoolWorker-40:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Process ForkPoolWorker-28:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
Process ForkPoolWorker-21:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
Traceback (most recent call last):
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
KeyboardInterrupt
Traceback (most recent call last):
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
    _run_finalizers(0)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 300, in _run_finalizers
Process ForkPoolWorker-22:
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
KeyboardInterrupt
    finalizer()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/util.py", line 224, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 695, in _terminate_pool
Traceback (most recent call last):
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 675, in _help_stuff_finish
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
    inqueue._rlock.acquire()
KeyboardInterrupt: 
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt
Traceback (most recent call last):
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 314, in _bootstrap
    self.run()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/queues.py", line 367, in get
    with self._rlock:
  File "/home/oliver/.local/lib/python3.10/site-packages/multiprocess/synchronize.py", line 101, in __enter__
    return self._semlock.__enter__()
KeyboardInterrupt

In this latter example you'll notice that the pool has not even been given any work to do yet. My machine does not have 20 CPUs available, but the problem appears to be at least partially race-dependent, so the more processes that are started, the more likely for the bug to arise.

The problem can at least partially be avoided by calling terminate() on the pool before close:

Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathos.pools
>>> pool = pathos.pools.ProcessPool(20)
>>> pool.terminate()
>>> 
KeyboardInterrupt
>>> 

But using a context manager has no effect:

Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathos.pools
>>> with pathos.pools.ProcessPool(20) as pool:
...     pass
... 
>>> Process ForkPoolWorker-13:
(stack trace continues as above)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions