-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
Just some definitions to assist in describing the ask.
(A) Implicit opt out - user does not configure anything in dbt and dbt sets the behaviour_change_flag
to False
automatically / default - i.e. dbt is choosing to set it to False
on behalf of the user.
# dbt_project.yml
...
(B) Explicit opt out - user sets in their own dbt_project.yml
and sets the behaviour_change_flag
to False
.
# dbt_project.yml
...
flags:
behaviour_change_flag: False
Currently, our docs on behaviour change warnings (https://docs.getdbt.com/reference/global-configs/behavior-changes#behavior-change-flags) say:
You will continue to see warnings for legacy behaviors you've opted out of, until you either:
- Resolve the issue (by switching the flag to True)
- Silence the warnings using the warn_error_options.silence flag
This means that even if a user explicitly sets the flag to False (i.e. opts out of a behaviour) - we continue to emit the warning until they take an extra action (silence the warning).
(A) Implicit opt out.
# dbt_project.yml
name: my_dbt_project
profile: all
version: "1.0.0"
models:
my_dbt_project:
+materialized: table
on-run-start: "{% do log('hello world', true) %}"
$ dbt source freshness --no-partial-parse
05:06:40 Running with dbt=1.10.0-b2
05:06:41 Registered adapter: snowflake=1.9.4
05:06:41 hello world
05:06:41 Found 1 model, 1 operation, 1 source, 476 macros
05:06:41
05:06:41 Concurrency: 1 threads (target='sf')
05:06:41
05:06:44 [WARNING]: In a future version of dbt, the `source freshness` command
will start running `on-run-start` and `on-run-end` hooks by default. For more
information: https://docs.getdbt.com/reference/global-configs/legacy-behaviors
05:06:44 Pulling freshness from warehouse metadata tables for 0 sources
05:06:44 1 of 1 START freshness of raw.customers ........................................ [RUN]
05:06:45 1 of 1 WARN freshness of raw.customers ......................................... [WARN in 0.96s]
05:06:46
05:06:46 Finished running 1 source in 0 hours 0 minutes and 4.48 seconds (4.48s).
05:06:46 Done.
Here, the warning is reasonable as the user has not explicitly instructed dbt of the behaviour that they want.
(B) Explicit opt out.
# dbt_project.yml
name: my_dbt_project
profile: all
version: "1.0.0"
models:
my_dbt_project:
+materialized: table
on-run-start: "{% do log('hello world', true) %}"
flags:
source_freshness_run_project_hooks: False # Tell dbt that I want to opt out.
$ dbt source freshness --no-partial-parse
05:07:45 Running with dbt=1.10.0-b2
05:07:45 Registered adapter: snowflake=1.9.4
05:07:45 hello world
05:07:45 Found 1 model, 1 operation, 1 source, 476 macros
05:07:45
05:07:45 Concurrency: 1 threads (target='sf')
05:07:45
05:07:48 [WARNING]: In a future version of dbt, the `source freshness` command
will start running `on-run-start` and `on-run-end` hooks by default. For more
information: https://docs.getdbt.com/reference/global-configs/legacy-behaviors
05:07:48 Pulling freshness from warehouse metadata tables for 0 sources
05:07:48 1 of 1 START freshness of raw.customers ........................................ [RUN]
05:07:49 1 of 1 WARN freshness of raw.customers ......................................... [WARN in 0.72s]
05:07:49
05:07:49 Finished running 1 source in 0 hours 0 minutes and 3.92 seconds (3.92s).
05:07:49 Done.
We can see here that the behaviour is pretty much as it is in (A) - which tallies with the documentation. However, in this case, the user has explicitly told dbt of the behaviour that they want exactly. So perhaps, what should happen here is to not emit that warning anymore.
Describe alternatives you've considered
Use the additional warn_error_options
config to silent the warning...
# dbt_project.yml
name: my_dbt_project
profile: all
version: "1.0.0"
models:
my_dbt_project:
+materialized: table
on-run-start: "{% do log('hello world', true) %}"
flags:
source_freshness_run_project_hooks: False # Tell dbt that I want to opt out.
warn_error_options:
silence:
- SourceFreshnessProjectHooksNotRun
$ dbt source freshness --no-partial-parse
05:10:32 Running with dbt=1.10.0-b2
05:10:32 Registered adapter: snowflake=1.9.4
05:10:33 hello world
05:10:33 Found 1 model, 1 operation, 1 source, 476 macros
05:10:33
05:10:33 Concurrency: 1 threads (target='sf')
05:10:33
05:10:36 Pulling freshness from warehouse metadata tables for 0 sources
05:10:36 1 of 1 START freshness of raw.customers ........................................ [RUN]
05:10:37 1 of 1 WARN freshness of raw.customers ......................................... [WARN in 0.98s]
05:10:37
05:10:37 Finished running 1 source in 0 hours 0 minutes and 4.39 seconds (4.39s).
05:10:38 Done.
Who will this benefit?
Users who are confused as to why "explicitly" setting a behaviour change flag results in continued warnings.
Are you interested in contributing this feature?
No response
Anything else?
No response