How many concurrent workflow tasks can Dify run at the same time?

[1210744256]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Content

How many concurrent workflow tasks can Dify run at the same time? My system has 4 cores and 8GB of RAM. How can I adjust the configuration?

[dosubot[bot]]

Dify’s concurrency for workflow tasks is managed by Celery workers. On your system with 4 CPU cores and 8GB RAM, you can control how many tasks run in parallel by setting environment variables:

• If you set CELERY_AUTO_SCALE=true, Dify will autoscale Celery workers up to the number of CPU cores (so, 4 workers for your system), unless you override with CELERY_MAX_WORKERS or CELERY_MIN_WORKERS source.
• If you don’t use autoscale, set CELERY_WORKER_AMOUNT=4 to match your core count. Each worker can handle a configurable number of concurrent tasks (default is 5 per worker) source.

Other parameters that impact concurrency include SERVER_WORKER_AMOUNT (recommended: CPU cores × 2 + 1, so 9 for you), SERVER_WORKER_CONNECTIONS (default 10), APP_MAX_ACTIVE_REQUESTS (default 0 for unlimited), and thread pool sizes (default 10 for workflow execution) source.

There’s no hardcoded maximum; actual concurrency depends on your hardware and these settings. After changing environment variables, restart Dify for changes to take effect. For best results, monitor system load and gradually increase concurrency to find the optimal setup for your workflows source.

If you need more parallel branches in workflows, you can increase WORKFLOW_PARALLEL_DEPTH_LIMIT (default 3) source.

Let me know if you want details on tuning for specific workflow types or batch processing!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}