Skip to content

Support for asyncio based workers #167

@aditya-n-invcr

Description

@aditya-n-invcr

The current worker implementation model uses multiprocessing for parallel execution but each process blocks on task execution (though it might be waiting on I/O). Is there a possibility to use asyncio/threaded approach to alleviate this problem and think of a model where processes could be used for parallelization (using multiple cores) and each process has an event loop/ThreadPool to poll and execute tasks to maximize performance on each core ?

I'd be happy to discuss more on this and submit a PR if required as well. Thanks for all the awesome work so far 👍

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions