-
-
Notifications
You must be signed in to change notification settings - Fork 128
Explain strange nulls #4442
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Explain strange nulls #4442
Conversation
…1_agg column nulls.
4a9c2b5
to
26c5750
Compare
description: > | ||
This table has a lot of row_conditions because it has a few columns that | ||
are found in the EIA-860M generators table (and so appear in the most), | ||
recent monthly data) but most of the columns are specific to boilers, | ||
which are not reported in the EIA-860M. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that this table is included and has a lot of row conditions is a bit of an artifact of the fact that the row_conditions
were generated before I created the ignore_eia860m_nulls
argument. I chose to leave them in (even though I think the table would not have any failures in the fast ETL now) because I think we'll want to deploy this test more widely and it encodes real expectations about which column-years should be null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sweet, we'll be happy about these docs in a few months when we have to figure out what's going on :)
Thanks for putting so many signposts for the more trivial changes too - helped streamline the review process.
A single typo fix + a few non-blocking questions remain.
Co-authored-by: Dazhong Xia <[email protected]>
|
Overview
no_null_cols
validation out of pytest #4105 / Add no-null-cols dbt test to all tables #4382schema.yml
not_all_null.sql
dbt test definition.Closes #4407
However, note that there are a couple of issues in #4407 that I think warrant some discussion.
Testing
not_all_null.sql
wasn't being used somewhere.To-do list