-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix for dbt compile overwriting source files #10887
base: main
Are you sure you want to change the base?
Conversation
Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA. In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR. CLA has not been signed by users: @Myles1 |
CLA should be signed now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for opening this PR @Myles1 !
Could you add a functional test for this case?
One place it could go initially is tests/functional/adapter/simple_seed/test_seed.py
(when this is reviewed by one of our software engineers, they might request to remove it to a different location).
I'm guessing the test could look something like this (which I didn't try out at all, so probably doesn't work as-is):
import Path
from tests.functional.adapter.simple_seed.seeds import (
seed__with_dots_csv,
)
class BaseTestAbsoluteSeedPaths:
@pytest.fixture(scope="class")
def project_config_update(self, project_root):
# Assign an absolute path to seed-paths
return {
"seed-paths": [Path(project_root, "seeds")],
}
@pytest.fixture(scope="class")
def seeds(self):
return {"my_seed.csv": seed__with_dots_csv}
def test_absolute_seeds_paths(self, project):
results = run_dbt(["seed"])
assert len(results) == 1
# Should not fail due to file being overwritten
results = run_dbt(["seed"])
index.html
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you restore the original index.html
(assuming this was an accidental change)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whoops. Sure: dc370bf
@@ -0,0 +1,6 @@ | |||
kind: Fixes | |||
body: Prevent dbt compile from overwriting files that are passed in as absolute paths |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wasn't able to get an error with dbt compile
, but I was with dbt seed
and dbt build
.
body: Prevent dbt compile from overwriting files that are passed in as absolute paths | |
body: Prevent `dbt seed` from overwriting files when `seed-path` contains an absolute path |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Happy to update it if needed. Lmk after giving this a shot: #10886 (comment)
@cla-bot check |
The cla-bot has been summoned, and re-checked this pull request! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #10887 +/- ##
==========================================
- Coverage 89.18% 79.31% -9.88%
==========================================
Files 183 183
Lines 23443 23443
==========================================
- Hits 20908 18593 -2315
- Misses 2535 4850 +2315
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Sure thing. Here's a (very) slightly modified version of your test that seems to do the trick: 9ae8aae |
Resolves #10886
Problem
Running
dbt compile
can overwrite source files with the newly compiled sqlSolution
Update path building to avoid issues with absolute paths.
The issue appears on this line:
dbt-core/core/dbt/contracts/graph/nodes.py
Line 267 in a0674db
Consider the case where
path
is an absolute path.For example, running this will give the absolute path as the return value, instead of the expected relative path that will be used as the output target file.
Following the change in this PR, we can expect this as the result:
Checklist