[Backport 1.9.latest] Backport #11698 to 1.9.latest #11703
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…precedence (#11698) * Handle explicit setting of null for source freshness config * Abstract out the creation of the target config This is useful because it makes that portion of code more re-usable/portable and makes the work we are about to do easier. * Fix bug in `merge_source_freshness` where empty freshness was preferenced over `None` The issue was that during merging of freshnesses, an "empty freshness", one where all values are `None`, was being preferenced over `None`. This was problematic because an "empty freshness" indicates that a freshness was not specified at that level. While `None` means that the freshness was _explicitly_ set to `None`. As such we should preference the thing that was specifically set. * Properly get dbt_project defined freshness and don't merge with schema defined freshness Previously we were only getting the "top level" freshness from the dbt_project.yaml. This was ignoring freshness settings for the direct, source, and table set in the dbt_project.yaml. Additionally, we were merging the dbt_project.yaml freshness into the schema freshness. Long term this merging would be desireably, however before we do that we need to ensure freshness at diffrent levels within the dbt_project.yml get properly merged (currently the different levels clobber each other). Fixing that is a larger issue though. So for the time being, the schema defintion of freshness will clobber any dbt_project.yml definition of freshness. * Add changie doc * Fix whitespace to make code quality happy * Set the parsed source freshness to an empty FreshnessThreshold if None This maintains backwards compatibility
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## 1.9.latest #11703 +/- ##
==============================================
- Coverage 89.09% 89.06% -0.04%
==============================================
Files 184 184
Lines 23935 23941 +6
==============================================
- Hits 21326 21322 -4
- Misses 2609 2619 +10
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
emmyoop
approved these changes
May 30, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we backported #11628 to 1.9.latest via #11683 we backported a bug along with it. We fixed the bug in #11698, and thus we need to backport that fix (which this does)