fork_map

fork_map is like python's builtin map function, but uses os.fork to execute the mapped function in child processes. Unlike the builtin multiprocessing map functions, fork_map requires only its outputs (not its inputs) to be pickleable.

from fork_map import fork_map

result = fork_map(lambda x: x*2, range(10))

The example above isn't possible with multiprocessing's Pool.map, because lambdas aren't pickleable.

Caveats:

Because fork_map uses os.fork, it has the same limitations as os.fork, namely:

• It only works on operating systems where it's available (e.g., not Windows).
• It is not safe to use in multithreaded code, because deadlocks may occur. See the warnings on os.fork or this discussion for more information.