-
Notifications
You must be signed in to change notification settings - Fork 36
Open
Description
The current worker implementation model uses multiprocessing for parallel execution but each process blocks on task execution (though it might be waiting on I/O). Is there a possibility to use asyncio/threaded approach to alleviate this problem and think of a model where processes could be used for parallelization (using multiple cores) and each process has an event loop/ThreadPool to poll and execute tasks to maximize performance on each core ?
I'd be happy to discuss more on this and submit a PR if required as well. Thanks for all the awesome work so far 👍
h-kawase, JCHacking and ClemDoum
Metadata
Metadata
Assignees
Labels
No labels