make bundle_name not nullable #47592

gyli · 2025-03-11T02:46:52Z

Change bundle_name in dag not nullable and prepopulate the value in the migration: #46734

Closes: #46734

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py

remove migration query Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py

…ue46734

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py

gyli · 2025-05-05T04:05:41Z

Hi @ephraimbuddy @jedcunningham Do you think this PR looks good already, or you think more testing needs to be done?

ephraimbuddy · 2025-05-05T07:30:48Z

Hi @ephraimbuddy @jedcunningham Do you think this PR looks good already, or you think more testing needs to be done?

Since Airflow 3 has been released, we should update the PR with new migration file instead of editing the add-dagbundlemodel migration file. Sorry that it didn't make it to 3.0.0

gyli · 2025-05-06T22:00:48Z

PR is updated to use new migration file, as 3.0 is rolled out.

airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_bundle_name_not_nullable.py

…undle_name_not_nullable.py Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_bundle_name_not_nullable.py

…undle_name_not_nullable.py Update downgrade Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

gyli · 2025-07-15T00:38:27Z

@jedcunningham @eladkal Conflict is resolved, could you please review again? Thanks!

providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_job.py

airflow-core/tests/unit/api_fastapi/common/db/test_dags.py

airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_backfills.py

airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_dag_warning.py

gyli · 2025-07-19T20:54:41Z

Hi @jedcunningham, PR updated. All test functions now add dag bundle with fixture. For adding dag bundle in helper functions, DagBundleModel is still used to avoid creating dag bundle unnecessarily. Does it look good to you?

airflow-core/src/airflow/migrations/versions/0079_3_1_0_make_bundle_name_not_nullable.py

uranusjr · 2025-07-28T09:07:00Z

airflow-core/tests/unit/api_fastapi/common/db/test_dags.py

        dag_id = "dag_with_queued_run"

        # Create DagModel
        dag_model = DagModel(
            dag_id=dag_id,
+            bundle_name="testing",


Would testing_dag_bundle.name be better?

That fixture doesnt return the model, but it could. My 2c, we have so much of this pattern already, keeping consistent and refactoring separately, if we wanted to, would be the way to go.

uranusjr · 2025-07-28T09:15:25Z

devel-common/src/tests_common/pytest_plugin.py

+    if AIRFLOW_V_3_0_PLUS:
+        clear_db_dag_bundles()


This should use AIRFLOW_V_3_1_PLUS instead (since 3.0.x would not have this function)

By checking the function definition and the migration file of DagBundle, I think the function and the model exist with 3.0.x.
Actually, AIRFLOW_V_3_0_PLUG is used as the condition for clear_db_dag_bundles in other places as well, like here.

jedcunningham · 2025-07-28T18:59:00Z

airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_import_error.py

Why are these fixtures no longer class scoped?

This is because function scope fixture testing_dag_bundle is used here. If they are class scoped, it will raise error ScopeMismatch: You tried to access the function scoped fixture testing_dag_bundle with a class scoped request object.
I think switching to function scoped is easier, compared to creating bundle with DagBundleModel.

providers/common/sql/tests/unit/common/sql/operators/test_sql.py

make bundle_name not nullable

c21dee7

gyli requested review from ephraimbuddy, XD-DENG and ashb as code owners March 11, 2025 02:46

boring-cyborg bot added the area:db-migrations label Mar 11, 2025

ephraimbuddy reviewed Mar 11, 2025

View reviewed changes

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py Outdated Show resolved Hide resolved

Apply suggestions from code review

24ba85b

remove migration query Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

jedcunningham added the AIP-66: DAG Bundle/Manifest label Mar 12, 2025

jedcunningham added this to AIP-66: DAG Bundles & Parsing Mar 12, 2025

jedcunningham reviewed Mar 12, 2025

View reviewed changes

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py Outdated Show resolved Hide resolved

update erd

8431e2a

gyli requested a review from potiuk as a code owner March 15, 2025 22:24

GL added 2 commits March 15, 2025 18:25

update erd

b7013d3

Merge branch 'issue46734' of https://github.com/gyli/airflow into iss…

e2ce847

…ue46734

ephraimbuddy reviewed Mar 16, 2025

View reviewed changes

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py Outdated Show resolved Hide resolved

ephraimbuddy reviewed Mar 16, 2025

View reviewed changes

airflow/migrations/versions/0050_3_0_0_add_dagbundlemodel.py Outdated Show resolved Hide resolved

add existing_type

2da1068

jedcunningham moved this to In Progress in AIP-66: DAG Bundles & Parsing Mar 24, 2025

gyli requested review from ephraimbuddy and jedcunningham April 14, 2025 04:24

Guangyang Li added 3 commits May 5, 2025 23:39

merge main

5bf349d

add new revision

3b8bfa3

add new revision

ccc3205

ephraimbuddy reviewed May 8, 2025

View reviewed changes

airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_bundle_name_not_nullable.py Outdated Show resolved Hide resolved

Update airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_b…

2851fe6

…undle_name_not_nullable.py Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

ephraimbuddy reviewed May 9, 2025

View reviewed changes

airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_bundle_name_not_nullable.py Outdated Show resolved Hide resolved

Update airflow-core/src/airflow/migrations/versions/0069_3_1_0_make_b…

c65885f

…undle_name_not_nullable.py Update downgrade Co-authored-by: Ephraim Anierobi <splendidzigy24@gmail.com>

Guangyang Li added 4 commits July 14, 2025 14:45

fix compat test for providers

f5b69f8

fix db test mark

11a02dc

renme clean_dags_and_dagruns to clean_dags_dagruns_and_dag_bundles

732fb3f

rename clean_dags_and_dagruns to clean_dags_dagruns_and_dag_bundles

e529a92

ephraimbuddy reviewed Jul 15, 2025

View reviewed changes

providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_job.py Outdated Show resolved Hide resolved

jedcunningham reviewed Jul 18, 2025

View reviewed changes

Guangyang Li added 5 commits July 18, 2025 00:40

merge main

9cf1e01

use fixture as much as posssible in tests

57f3e1d

Merge branch 'main' of https://github.com/apache/airflow into issue46734

a26649d

fix test_mappedoperator.py

c6c8557

fix test_dag_run

b4ea819

Guangyang Li added 6 commits July 24, 2025 00:09

merge main

9d96795

update migration number

5b55c41

merge main

ce85d40

remove unnecessary nondb test class

e2d4367

Merge branch 'main' of https://github.com/apache/airflow into issue46734

a0ff1fc

fix newly added test in test_dag_command.py

c93c032

uranusjr reviewed Jul 28, 2025

View reviewed changes

airflow-core/src/airflow/migrations/versions/0079_3_1_0_make_bundle_name_not_nullable.py Outdated Show resolved Hide resolved

uranusjr reviewed Jul 28, 2025

View reviewed changes

jedcunningham reviewed Jul 28, 2025

View reviewed changes

Guangyang Li added 6 commits July 28, 2025 21:04

merge main

194c352

re-generate migration file

048c983

re-generate migration file

52027bc

merge main

a650d74

re-generate migration file

a2cc1c5

fix unit test

1d24321

ephraimbuddy approved these changes Aug 1, 2025

View reviewed changes

make bundle_name not nullable #47592

Are you sure you want to change the base?

make bundle_name not nullable #47592

Conversation

gyli commented Mar 11, 2025 • edited by eladkal Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gyli commented May 5, 2025

Uh oh!

ephraimbuddy commented May 5, 2025

Uh oh!

gyli commented May 6, 2025

Uh oh!

Uh oh!

Uh oh!

gyli commented Jul 15, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gyli commented Jul 19, 2025

Uh oh!

Uh oh!

uranusjr Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

jedcunningham Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

uranusjr Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

gyli Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jedcunningham Jul 28, 2025

Choose a reason for hiding this comment

Uh oh!

gyli Jul 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gyli commented Mar 11, 2025 •

edited by eladkal

Loading

gyli Jul 29, 2025 •

edited

Loading