How to create custom roles when deploying airflow with helm chart

[RickyLeeeee]

Hello,

I want to create multiple custom roles while deploying Airflow using a Helm Chart. However, after searching extensively on Google, I couldn't find a relevant solution. I have tried adding the following Airflow CLI commands under webserver.extraInitContainers, but it did not work:

extraInitContainers:
  - name: setup-roles-and-permissions
    image: apache/airflow:2.10.0
    command:
      - /bin/bash
      - -c
    args:
      - |
        airflow db migrate
        airflow roles create Test &&
        airflow roles add-perms Test --resource "DAG Runs" --action "can_edit" &&
        airflow roles add-perms Test --resource "DAG Runs" --action "can_create" &&
        airflow roles add-perms Test --resource "DAG Runs" --action "can_edit"

Logs for the extraContainer:

/home/airflow/.local/lib/python3.12/site-packages/airflow/plugins_manager.py:30 DeprecationWarning: 'cgitb' is deprecated and slated for removal in Python 3.13
DB: sqlite:////opt/airflow/airflow.db
Performing upgrade to the metadata database sqlite:////opt/airflow/airflow.db
[2025-01-10T09:54:24.474+0000] {migration.py:215} INFO - Context impl SQLiteImpl.
[2025-01-10T09:54:24.479+0000] {migration.py:218} INFO - Will assume non-transactional DDL.
[2025-01-10T09:54:24.481+0000] {migration.py:215} INFO - Context impl SQLiteImpl.
[2025-01-10T09:54:24.482+0000] {migration.py:218} INFO - Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running stamp_revision  -> 22ed7efa9da2
Database migrating done!
/home/airflow/.local/lib/python3.12/site-packages/airflow/plugins_manager.py:30 DeprecationWarning: 'cgitb' is deprecated and slated for removal in Python 3.13
Added 1 role(s)
/home/airflow/.local/lib/python3.12/site-packages/airflow/plugins_manager.py:30 DeprecationWarning: 'cgitb' is deprecated and slated for removal in Python 3.13
Resource named 'DAG Runs' does not exist

Based on the logs, it shows that the metadata database being used is SQLite instead of PostgreSQL. It seems that it hasn’t connected to the correct database.

May I ask how to create custom roles during deployment?

The Helm Chart version I am using is 1.15.0, and the Airflow version is 2.10.0.

[potiuk]

Generally speaking helm is for installing apps not intializing the database with custom roles variables, connections etc., you should have additional synchronization and configuraiton scripts that are not part of "helm install" - that woudl be best way and you are free to do it as you want - but outside of Helm. We have airflow roles export/import and airflow users export/import tools for that - but generally it's something that is independent from "application" installation that helm provides.

If you wish to run it initialization script and init container, you need to do it in a way that will handle all ambiguities - i.e. what happens when you create or update roles in the future, how to delete those that were deleted/added, how not to override the roles that were created manually - and also handle the case that the init container is executed every time the main container starts (which means tha this init container will run many times even if you do not re-deploy helm - basically any time your containers start.

in your case yes - you miss all the environment variables that oother containers had - so your "airflow" command is not using the right credentials - you can look at the definition of those containers you have via helm install --dry-run for example.

[RickyLeeeee]

Got it, thank you for your suggestion. I will try using a script to create these custom roles instead of handling it through the Helm Chart.