Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dbt setup #4011

Merged
merged 63 commits into from
Feb 22, 2025
Merged

Dbt setup #4011

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
bf40ffb
Add basic dbt setup
zschira Jan 9, 2025
9aac625
Update to dagster 1.9.7 & grpcio 1.67.1
zaneselvans Jan 10, 2025
415a113
Setup multiple dbt profiles
zschira Jan 10, 2025
ba32bd8
Merge remote-tracking branch 'refs/remotes/origin/dbt_setup' into dbt…
zaneselvans Jan 10, 2025
dc51c8f
Add all vcerare dbt tests
zschira Jan 10, 2025
590b02a
Add more example dbt tests
zschira Jan 13, 2025
63e663a
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zaneselvans Jan 13, 2025
d428b5d
Merge changes from main and revert to python 3.12
zaneselvans Jan 13, 2025
784cf96
Bump gdal to v3.10.1 bugfix release.
zaneselvans Jan 14, 2025
48a16e1
Merge branch 'main' into dbt_setup
zaneselvans Jan 14, 2025
6f45ba5
Merge branch 'main' into dbt_setup
zaneselvans Jan 15, 2025
ac41a41
Update to dagster 1.9.9
zaneselvans Jan 19, 2025
0ce1648
Merge branch 'main' into dbt_setup
zaneselvans Jan 20, 2025
c19cfd8
Merge branch 'main' into dbt_setup
zaneselvans Jan 20, 2025
6335e94
Reorganize dbt into multiple schema.yml files
zschira Jan 21, 2025
2585eca
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zschira Jan 21, 2025
e24af8c
Move dbt project to top level of repo
zschira Jan 22, 2025
1ed85b3
Only set parquet path in dbt project once
zschira Jan 30, 2025
e92f5be
Standardize dbt maning scheme
zschira Jan 30, 2025
5de9ebe
Add more detail to README
zschira Jan 30, 2025
da9ae93
Add script to generate dbt scaffolding and row count tests
zschira Feb 5, 2025
7461786
Add documentation for dbt helper script
zschira Feb 5, 2025
0d120c6
Add out_ferc1__yearly_steam_plants_fuel_by_plant_sched402 to yearly r…
zschira Feb 5, 2025
3666360
Add weighted quantile test (broken)
zschira Feb 5, 2025
a3579dc
Change row count test name
zschira Feb 5, 2025
c98219c
Update dbt initialization process
zschira Feb 5, 2025
f9b3fa7
Make dbt helper script work properly with non-yearly partitioned tables
zschira Feb 5, 2025
79e2153
Update dbt readme
zschira Feb 5, 2025
012ba4a
Regenerate ferc dbt schemas
zschira Feb 6, 2025
ff766b3
Merge branch 'main' into dbt_setup
zaneselvans Feb 10, 2025
94267a5
Improve dbt_helper command line usability
zschira Feb 10, 2025
8f660fd
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zschira Feb 10, 2025
70e6895
Flesh out test migration command
zschira Feb 13, 2025
389c540
Add test migration tutorial
zschira Feb 14, 2025
eb0765a
Merge branch 'main' into dbt_setup
zschira Feb 14, 2025
68040fd
Use duckdb for getting row counts per partition
zschira Feb 18, 2025
58aee27
Merge branch 'main' into 'dbt_setup'
zschira Feb 19, 2025
e6a0884
Add seperate etl full/fast row count checks
zschira Feb 19, 2025
b49ba78
Update quantile tests to work with both fast and full etl
zschira Feb 19, 2025
6bac07e
Update dbt readme
zschira Feb 19, 2025
4f34a58
Clarify dbt model documentation
zschira Feb 19, 2025
55d8160
Add no null cols test
zschira Feb 20, 2025
40448c3
Fix no_null_rows test
zschira Feb 20, 2025
5dfce94
Add tutorial for migrating a bespoke test
zschira Feb 20, 2025
0193c53
Merge branch 'main' into dbt_setup
zaneselvans Feb 21, 2025
16346fa
Avoid regressing changes on main
zaneselvans Feb 21, 2025
f17f14c
Revert to older mamba Docker image since 2.0.6 was yanked.
zaneselvans Feb 21, 2025
7d0523a
Re-lock dependencies since they've changed both here and on main.
zaneselvans Feb 21, 2025
0786bcf
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zschira Feb 21, 2025
c050114
Add better docstrings/help messages to dbt_helper script
zschira Feb 21, 2025
8ab821e
Update dbt readme
zschira Feb 21, 2025
4f71d1a
Merge branch 'main' into dbt_setup
zaneselvans Feb 21, 2025
3738507
Minor docs fixes
zschira Feb 21, 2025
89689d8
Remove option from dbt helper script
zschira Feb 21, 2025
f5da280
Add example to dbt helper command description
zschira Feb 21, 2025
31313e4
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zschira Feb 21, 2025
640b750
Update devtools/dbt_helper.py
zschira Feb 21, 2025
c41a3ef
Update devtools/dbt_helper.py
zschira Feb 21, 2025
244b4da
Update devtools/dbt_helper.py
zschira Feb 21, 2025
0244e2a
Update devtools/dbt_helper.py
zschira Feb 21, 2025
30de865
Update descriptions of row count dbt test
zschira Feb 21, 2025
b52577d
Merge branch 'dbt_setup' of github.com:catalyst-cooperative/pudl into…
zschira Feb 21, 2025
14cb119
Reflow README text; use 4 threads by default.
zaneselvans Feb 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 44 additions & 17 deletions environments/conda-linux-64.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading