Skip to content

Integrate alembic check in ci #49002

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

guan404ming
Copy link
Contributor

@guan404ming guan404ming commented Apr 9, 2025

Related Issue

closes #48998
cc: @ashb

Why

Since Alembic 1.9, a built-in command alembic check is available to detect database model changes that don't have corresponding migration files. This feature is crucial for maintaining database integrity by ensuring developers create appropriate migration files whenever they modify database models.

Currently, our CI pipeline runs various migration tests, but it doesn't verify if model changes have corresponding migrations. Integrating this check into our CI ensures that no model changes go unnoticed, preventing potential issues that could arise when applying migrations in production.

How

  • add a new step to the ci

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@guan404ming guan404ming changed the title CI: integrate alembic check ci: integrate alembic check Apr 9, 2025
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 5513306 to 9a67d8b Compare April 9, 2025 10:40
@ashb
Copy link
Member

ashb commented Apr 9, 2025

Could you add a change to a model in a second commit on this PR (that we will shortly revert/undo) to test that this fails as expected please?

@guan404ming
Copy link
Contributor Author

Sure! I would do it later after this CI finished (make sure it could pass now)

Comment on lines 1409 to 1415
ActionCommand(
name="check-models",
help="Check if there are model changes without a corresponding migration",
description="Check if the current models require new migrations to be generated",
func=lazy_load_command("airflow.cli.commands.db_command.check_models"),
args=(ARG_VERBOSE,),
),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I'm not sure we want this in here though actually.

The airflow db is designed for end users/Airflow administrators to run, but the check command is only useful for Airflow devs

Copy link
Contributor Author

@guan404ming guan404ming Apr 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reviewing, I would remove it from airflow db ~

@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 9a67d8b to f389818 Compare April 9, 2025 14:33
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from f389818 to b13a09c Compare April 9, 2025 14:50
@guan404ming guan404ming requested a review from XD-DENG as a code owner April 9, 2025 15:38
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 2 times, most recently from b3e4358 to 31fa445 Compare April 9, 2025 17:15
@guan404ming guan404ming marked this pull request as draft April 10, 2025 01:42
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 14 times, most recently from cc8b840 to f09bb95 Compare April 12, 2025 08:58
@guan404ming guan404ming marked this pull request as ready for review April 12, 2025 09:10
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 4 times, most recently from 9f22ec4 to b68002a Compare April 12, 2025 13:24
@guan404ming guan404ming marked this pull request as ready for review April 13, 2025 17:05
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from b68002a to e4ec1a5 Compare April 13, 2025 17:06
@guan404ming guan404ming requested a review from ashb April 13, 2025 17:07
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from e4ec1a5 to 4935d6e Compare April 13, 2025 19:42
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 4935d6e to 2604527 Compare April 14, 2025 01:29
@guan404ming guan404ming marked this pull request as draft April 15, 2025 19:20
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 2604527 to 3dee702 Compare April 15, 2025 19:20
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 5 times, most recently from 6def718 to 8fe2c39 Compare April 19, 2025 16:00
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 2 times, most recently from 66cd694 to 4c3b055 Compare April 29, 2025 07:22
@guan404ming guan404ming marked this pull request as ready for review April 29, 2025 07:22
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 4c3b055 to 9645890 Compare April 29, 2025 16:55
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch 3 times, most recently from 2cc7087 to ae67e5a Compare April 30, 2025 17:26
@guan404ming guan404ming changed the title ci: integrate alembic check Integrate alembic check in ci Apr 30, 2025
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from ae67e5a to 5e68e28 Compare May 1, 2025 00:29
@guan404ming guan404ming force-pushed the integrate-alembic-check-in-ci branch from 5e68e28 to 9354811 Compare May 1, 2025 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Integrate alembic check into CI to check for model changes without a matching migration
2 participants