Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Raise an error when a custom dbt_valid_to_current is configured for a pre-existing snapshot #10923

Open
3 tasks done
dbeatty10 opened this issue Oct 25, 2024 · 1 comment
Labels
enhancement New feature or request pre-release Bug not yet in a stable release snapshots Issues related to dbt's snapshot functionality triage user docs [docs.getdbt.com] Needs better documentation

Comments

@dbeatty10
Copy link
Contributor

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

The dbt_valid_to_current config is intended for net-new snapshots rather than being applied to snapshots that already exist (i.e., the relevant table exists in the database). The reason is that dbt is not supplying automated migration paths (#10325), and its up to users to migrate in the best way they see fit.

So similar to #10185 for the snapshot_meta_column_names config, dbt should throw an error if the user has an existing snapshot and they try to add/change the dbt_valid_to_current config.

  • this should error on insert if you change the meta field names, but [one idea] we could try to throw an error before running the snapshot DML by running adapters.get_columns_in_relation() on the existing snapshot and comparing the meta fields that exist in the current config to what exists in the warehouse

Describe alternatives you've considered

No response

Who will this benefit?

No response

Are you interested in contributing this feature?

No response

Anything else?

No response

@dbeatty10 dbeatty10 added enhancement New feature or request triage snapshots Issues related to dbt's snapshot functionality pre-release Bug not yet in a stable release user docs [docs.getdbt.com] Needs better documentation labels Oct 25, 2024
@dbeatty10 dbeatty10 changed the title [Feature] Raise and error when a custom dbt_valid_to_current is configured for a pre-existing snapshot [Feature] Raise an error when a custom dbt_valid_to_current is configured for a pre-existing snapshot Oct 28, 2024
@graciegoheen
Copy link
Contributor

this should error on insert if you change the meta field names

This config doesn't change the name of the meta fields. It changes the value that means "current" for the dbt_valid_to field. How could we throw an error in that case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request pre-release Bug not yet in a stable release snapshots Issues related to dbt's snapshot functionality triage user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

No branches or pull requests

2 participants