Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Support logging for python models #1117

Open
3 tasks done
mikealfare opened this issue Jul 16, 2024 · 0 comments
Open
3 tasks done

[Feature] Support logging for python models #1117

mikealfare opened this issue Jul 16, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@mikealfare
Copy link
Contributor

mikealfare commented Jul 16, 2024

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Python models should log to the account's event table like any other stored procedure. Snowflake supports altering the logging level (LOG_LEVEL) at three points. However, there are difficulties with all three of these approaches.

Account level

Setting LOG_LEVEL at the account level requires permissions that are typically not available for many users. It also has the effect of impacting everyone on the account, which is generally not desirable. This is currently the only available workaround.

Object level

Setting LOG_LEVEL on the stored procedure would minimize the impact of the setting change. However, the stored procedure only exists during the dbt run; it's created and then dropped as part of running the model (after pre-hooks and before post-hooks). Since the user has no entry point between the model being created and the model running, they cannot apply these permissions. Additionally, since the default behavior of dbt-snowflake is to use a temporary stored procedure, this approach would only apply to the non-default path.

Session level

Setting LOG_LEVEL for the session feels like the correct approach. This approach supports both the default and the non-default methods for implementing python models. Since the change is bound to the session, it also doesn't affect the state of the warehouse after the dbt run completes. This can be implemented as a pre-hook or it can be added as a one-liner before calling the stored procedure. The pre-hook is something that a user can implement on their own, but it's required overhead.

Who will this benefit?

This benefits dbt-snowflake users who want logging for python models.

Additional information

This feature is a result of #846.

@mikealfare mikealfare added enhancement New feature or request triage labels Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants