File tree Expand file tree Collapse file tree 1 file changed +36
-0
lines changed Expand file tree Collapse file tree 1 file changed +36
-0
lines changed Original file line number Diff line number Diff line change 1+ ## Adjusting Scheduler Resources
2+
3+ In Airflow, the [ scheduler] ( https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/scheduler.html )
4+ monitors all tasks and DAGs, then triggers the task instances once their dependencies are complete.
5+
6+ You may find that you need to increase the number of schedulers that you are running as well as increase the resources
7+ available to each scheduler.
8+
9+ ### edit values.yaml
10+
11+ In order to update scheduler resources, you'll then want to edit ` airflow/helm/airflow/values.yaml ` in your installation
12+ repo. If you wanted 2 schedulers each with at least 0.5 cpu & 1.75Gi of resources, you'd make the following change:
13+
14+ ``` yaml
15+ airflow :
16+ airflow :
17+ airflow :
18+ scheduler :
19+ replicas : 2
20+ resources :
21+ limits :
22+ cpu : 1.5
23+ memory : 2.5Gi
24+ requests :
25+ cpu : 0.5
26+ memory : 1.75Gi
27+ ` ` `
28+
29+ > It is highly recommended to set limits on your scheduler(s) to ensure fair resource sharing across your cluster.
30+
31+ Alternatively, you should be able to do this in the configuration section for airflow in your plural console as well.
32+
33+ ### redeploy
34+
35+ From there, you should be able to run ` plural build --only airflow && plural deploy --commit "update scheduler
36+ resources"` to adjust your Airflow scheduler configuration
You can’t perform that action at this time.
0 commit comments