Skip to content

Integrate the Census-based FIPS codes to replace addfips #4019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
Apr 10, 2025
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
fcdebb2
add censusfips metadata
cmgosnell Jan 9, 2025
d0b4c0c
Merge branch 'main' into fips-fix
e-belfer Jan 9, 2025
3b14928
change fips dataset name
cmgosnell Jan 9, 2025
e38b422
Update src/pudl/metadata/sources.py
cmgosnell Jan 14, 2025
22c2d50
Merge branch 'main' into fips-fix
cmgosnell Jan 14, 2025
cfa402e
Add extractor - currently using sandbox doi
cmgosnell Jan 14, 2025
6f18f82
Merge branch 'main' into fips-fix
e-belfer Jan 16, 2025
932ab34
remove the extraction step from this pr
cmgosnell Jan 17, 2025
5f09784
lol add the extractor stage back in - but this only works w/ the sand…
cmgosnell Jan 17, 2025
afa50fb
Update src/pudl/package_data/censuspep/__init__.py
cmgosnell Feb 3, 2025
a104c98
Update src/pudl/package_data/censuspep/column_maps/__init__.py
cmgosnell Feb 3, 2025
363a48a
Update src/pudl/extract/censuspep.py
cmgosnell Feb 3, 2025
0872082
Update src/pudl/metadata/sources.py
cmgosnell Feb 3, 2025
6b085e3
Merge branch 'main' into fips-integrate
cmgosnell Mar 5, 2025
abd1e28
attempt at full replacement of addfips
cmgosnell Mar 11, 2025
0ac799c
clean up extraction dtypes cruft
cmgosnell Mar 11, 2025
0f76ec1
add csv of territories, migrate adding the state into the _core asset…
cmgosnell Mar 13, 2025
bc750b0
omigosh args
cmgosnell Mar 13, 2025
900d133
add unit test for city/county and bedford situation
cmgosnell Mar 13, 2025
99f8b4a
Update src/pudl/transform/censuspep.py
cmgosnell Mar 18, 2025
5d9db3f
Merge branch 'main' into fips-integrate
cmgosnell Mar 18, 2025
f38c46e
add some docs fixes/updates and remove addfips as a dependency
cmgosnell Mar 18, 2025
0ea6930
force string types for codes
cmgosnell Mar 27, 2025
8c0bb86
update env files again wow
cmgosnell Mar 28, 2025
a856dd8
update row counts for 861 service territories - new rows bc we are no…
cmgosnell Mar 31, 2025
ab3049c
update the row counts with the new fips code rows
cmgosnell Apr 1, 2025
ec881f0
Merge branch 'main' into fips-integrate
cmgosnell Apr 1, 2025
951004a
oopsies update the fast test to have the correct year
cmgosnell Apr 1, 2025
5d41f04
Merge branch 'main' into fips-integrate
cmgosnell Apr 1, 2025
7cd68b5
update fast test row limits
cmgosnell Apr 2, 2025
217756b
Merge branch 'main' into fips-integrate
cmgosnell Apr 9, 2025
a2b6131
Update fast etl row count expectations
zaneselvans Apr 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
136 changes: 68 additions & 68 deletions dbt/seeds/etl_fast_row_counts.csv
Original file line number Diff line number Diff line change
Expand Up @@ -409,29 +409,29 @@ core_eia861__yearly_sales,2020,13280
core_eia861__yearly_sales,2021,13105
core_eia861__yearly_sales,2022,13215
core_eia861__yearly_sales,2023,13205
core_eia861__yearly_service_territory,2001,11636
core_eia861__yearly_service_territory,2002,11637
core_eia861__yearly_service_territory,2003,11660
core_eia861__yearly_service_territory,2004,11822
core_eia861__yearly_service_territory,2005,11721
core_eia861__yearly_service_territory,2001,11637
core_eia861__yearly_service_territory,2002,11638
core_eia861__yearly_service_territory,2003,11661
core_eia861__yearly_service_territory,2004,11823
core_eia861__yearly_service_territory,2005,11722
core_eia861__yearly_service_territory,2006,11812
core_eia861__yearly_service_territory,2007,11845
core_eia861__yearly_service_territory,2008,11813
core_eia861__yearly_service_territory,2009,11729
core_eia861__yearly_service_territory,2010,11681
core_eia861__yearly_service_territory,2011,11655
core_eia861__yearly_service_territory,2012,11654
core_eia861__yearly_service_territory,2013,11923
core_eia861__yearly_service_territory,2014,11908
core_eia861__yearly_service_territory,2015,11965
core_eia861__yearly_service_territory,2016,11945
core_eia861__yearly_service_territory,2017,11821
core_eia861__yearly_service_territory,2018,11820
core_eia861__yearly_service_territory,2019,11744
core_eia861__yearly_service_territory,2020,11786
core_eia861__yearly_service_territory,2021,11679
core_eia861__yearly_service_territory,2022,11775
core_eia861__yearly_service_territory,2023,11775
core_eia861__yearly_service_territory,2007,11846
core_eia861__yearly_service_territory,2008,11814
core_eia861__yearly_service_territory,2009,11730
core_eia861__yearly_service_territory,2010,11682
core_eia861__yearly_service_territory,2011,11656
core_eia861__yearly_service_territory,2012,11655
core_eia861__yearly_service_territory,2013,11924
core_eia861__yearly_service_territory,2014,11909
core_eia861__yearly_service_territory,2015,11966
core_eia861__yearly_service_territory,2016,11946
core_eia861__yearly_service_territory,2017,11822
core_eia861__yearly_service_territory,2018,11821
core_eia861__yearly_service_territory,2019,11745
core_eia861__yearly_service_territory,2020,11787
core_eia861__yearly_service_territory,2021,11680
core_eia861__yearly_service_territory,2022,11776
core_eia861__yearly_service_territory,2023,11776
core_eia861__yearly_short_form,2012,1124
core_eia861__yearly_short_form,2013,1122
core_eia861__yearly_short_form,2014,1121
Expand Down Expand Up @@ -785,52 +785,52 @@ out_eia860__yearly_emissions_control_equipment,2020,5712
out_eia860__yearly_emissions_control_equipment,2023,5504
out_eia860__yearly_ownership,2020,4832
out_eia860__yearly_ownership,2023,5378
out_eia861__yearly_balancing_authority_service_territory,2001,4957
out_eia861__yearly_balancing_authority_service_territory,2002,5806
out_eia861__yearly_balancing_authority_service_territory,2003,5665
out_eia861__yearly_balancing_authority_service_territory,2004,5946
out_eia861__yearly_balancing_authority_service_territory,2005,5873
out_eia861__yearly_balancing_authority_service_territory,2001,4958
out_eia861__yearly_balancing_authority_service_territory,2002,5807
out_eia861__yearly_balancing_authority_service_territory,2003,5666
out_eia861__yearly_balancing_authority_service_territory,2004,5947
out_eia861__yearly_balancing_authority_service_territory,2005,5874
out_eia861__yearly_balancing_authority_service_territory,2006,6003
out_eia861__yearly_balancing_authority_service_territory,2007,5861
out_eia861__yearly_balancing_authority_service_territory,2008,5934
out_eia861__yearly_balancing_authority_service_territory,2009,5933
out_eia861__yearly_balancing_authority_service_territory,2010,6003
out_eia861__yearly_balancing_authority_service_territory,2011,5069
out_eia861__yearly_balancing_authority_service_territory,2012,5140
out_eia861__yearly_balancing_authority_service_territory,2013,4762
out_eia861__yearly_balancing_authority_service_territory,2014,4486
out_eia861__yearly_balancing_authority_service_territory,2015,4435
out_eia861__yearly_balancing_authority_service_territory,2016,4451
out_eia861__yearly_balancing_authority_service_territory,2017,4487
out_eia861__yearly_balancing_authority_service_territory,2018,4486
out_eia861__yearly_balancing_authority_service_territory,2019,4488
out_eia861__yearly_balancing_authority_service_territory,2020,4491
out_eia861__yearly_balancing_authority_service_territory,2021,4434
out_eia861__yearly_balancing_authority_service_territory,2022,4432
out_eia861__yearly_balancing_authority_service_territory,2023,4435
out_eia861__yearly_utility_service_territory,2001,11589
out_eia861__yearly_utility_service_territory,2002,11585
out_eia861__yearly_utility_service_territory,2003,11588
out_eia861__yearly_utility_service_territory,2004,11688
out_eia861__yearly_utility_service_territory,2005,11542
out_eia861__yearly_balancing_authority_service_territory,2007,5862
out_eia861__yearly_balancing_authority_service_territory,2008,5935
out_eia861__yearly_balancing_authority_service_territory,2009,5934
out_eia861__yearly_balancing_authority_service_territory,2010,6004
out_eia861__yearly_balancing_authority_service_territory,2011,5070
out_eia861__yearly_balancing_authority_service_territory,2012,5141
out_eia861__yearly_balancing_authority_service_territory,2013,4763
out_eia861__yearly_balancing_authority_service_territory,2014,4487
out_eia861__yearly_balancing_authority_service_territory,2015,4436
out_eia861__yearly_balancing_authority_service_territory,2016,4452
out_eia861__yearly_balancing_authority_service_territory,2017,4488
out_eia861__yearly_balancing_authority_service_territory,2018,4487
out_eia861__yearly_balancing_authority_service_territory,2019,4489
out_eia861__yearly_balancing_authority_service_territory,2020,4492
out_eia861__yearly_balancing_authority_service_territory,2021,4435
out_eia861__yearly_balancing_authority_service_territory,2022,4433
out_eia861__yearly_balancing_authority_service_territory,2023,4436
out_eia861__yearly_utility_service_territory,2001,11590
out_eia861__yearly_utility_service_territory,2002,11586
out_eia861__yearly_utility_service_territory,2003,11589
out_eia861__yearly_utility_service_territory,2004,11689
out_eia861__yearly_utility_service_territory,2005,11543
out_eia861__yearly_utility_service_territory,2006,11661
out_eia861__yearly_utility_service_territory,2007,11665
out_eia861__yearly_utility_service_territory,2008,11720
out_eia861__yearly_utility_service_territory,2009,11671
out_eia861__yearly_utility_service_territory,2010,11625
out_eia861__yearly_utility_service_territory,2011,11584
out_eia861__yearly_utility_service_territory,2012,11518
out_eia861__yearly_utility_service_territory,2013,11769
out_eia861__yearly_utility_service_territory,2014,11790
out_eia861__yearly_utility_service_territory,2015,11838
out_eia861__yearly_utility_service_territory,2016,11816
out_eia861__yearly_utility_service_territory,2017,11760
out_eia861__yearly_utility_service_territory,2018,11751
out_eia861__yearly_utility_service_territory,2019,11674
out_eia861__yearly_utility_service_territory,2020,11716
out_eia861__yearly_utility_service_territory,2021,11679
out_eia861__yearly_utility_service_territory,2022,11720
out_eia861__yearly_utility_service_territory,2023,11716
out_eia861__yearly_utility_service_territory,2007,11666
out_eia861__yearly_utility_service_territory,2008,11721
out_eia861__yearly_utility_service_territory,2009,11672
out_eia861__yearly_utility_service_territory,2010,11626
out_eia861__yearly_utility_service_territory,2011,11585
out_eia861__yearly_utility_service_territory,2012,11519
out_eia861__yearly_utility_service_territory,2013,11770
out_eia861__yearly_utility_service_territory,2014,11791
out_eia861__yearly_utility_service_territory,2015,11839
out_eia861__yearly_utility_service_territory,2016,11817
out_eia861__yearly_utility_service_territory,2017,11761
out_eia861__yearly_utility_service_territory,2018,11752
out_eia861__yearly_utility_service_territory,2019,11675
out_eia861__yearly_utility_service_territory,2020,11717
out_eia861__yearly_utility_service_territory,2021,11680
out_eia861__yearly_utility_service_territory,2022,11721
out_eia861__yearly_utility_service_territory,2023,11717
out_eia923__boiler_fuel,2020,108635
out_eia923__boiler_fuel,2024,74563
out_eia923__fuel_receipts_costs,2020,34574
Expand Down Expand Up @@ -985,8 +985,8 @@ out_ferc714__hourly_planning_area_demand,2020,834384
out_ferc714__hourly_planning_area_demand,2022,17544
out_ferc714__hourly_planning_area_demand,2023,779571
out_ferc714__hourly_planning_area_demand,2024,47
out_ferc714__respondents_with_fips,2020,8988
out_ferc714__respondents_with_fips,2023,8957
out_ferc714__respondents_with_fips,2020,8995
out_ferc714__respondents_with_fips,2023,8964
out_ferc714__summarized_demand,2020,218
out_ferc714__summarized_demand,2023,218
out_gridpathratoolkit__hourly_available_capacity_factor,2007,227526
Expand Down
166 changes: 83 additions & 83 deletions dbt/seeds/etl_full_row_counts.csv
Original file line number Diff line number Diff line change
Expand Up @@ -920,29 +920,29 @@ core_eia861__yearly_sales,2020,13280
core_eia861__yearly_sales,2021,13105
core_eia861__yearly_sales,2022,13215
core_eia861__yearly_sales,2023,13205
core_eia861__yearly_service_territory,2001,11636
core_eia861__yearly_service_territory,2002,11637
core_eia861__yearly_service_territory,2003,11660
core_eia861__yearly_service_territory,2004,11822
core_eia861__yearly_service_territory,2005,11721
core_eia861__yearly_service_territory,2001,11637
core_eia861__yearly_service_territory,2002,11638
core_eia861__yearly_service_territory,2003,11661
core_eia861__yearly_service_territory,2004,11823
core_eia861__yearly_service_territory,2005,11722
core_eia861__yearly_service_territory,2006,11812
core_eia861__yearly_service_territory,2007,11845
core_eia861__yearly_service_territory,2008,11813
core_eia861__yearly_service_territory,2009,11729
core_eia861__yearly_service_territory,2010,11681
core_eia861__yearly_service_territory,2011,11655
core_eia861__yearly_service_territory,2012,11654
core_eia861__yearly_service_territory,2013,11923
core_eia861__yearly_service_territory,2014,11908
core_eia861__yearly_service_territory,2015,11965
core_eia861__yearly_service_territory,2016,11945
core_eia861__yearly_service_territory,2017,11821
core_eia861__yearly_service_territory,2018,11820
core_eia861__yearly_service_territory,2019,11744
core_eia861__yearly_service_territory,2020,11786
core_eia861__yearly_service_territory,2021,11679
core_eia861__yearly_service_territory,2022,11775
core_eia861__yearly_service_territory,2023,11775
core_eia861__yearly_service_territory,2007,11846
core_eia861__yearly_service_territory,2008,11814
core_eia861__yearly_service_territory,2009,11730
core_eia861__yearly_service_territory,2010,11682
core_eia861__yearly_service_territory,2011,11656
core_eia861__yearly_service_territory,2012,11655
core_eia861__yearly_service_territory,2013,11924
core_eia861__yearly_service_territory,2014,11909
core_eia861__yearly_service_territory,2015,11966
core_eia861__yearly_service_territory,2016,11946
core_eia861__yearly_service_territory,2017,11822
core_eia861__yearly_service_territory,2018,11821
core_eia861__yearly_service_territory,2019,11745
core_eia861__yearly_service_territory,2020,11787
core_eia861__yearly_service_territory,2021,11680
core_eia861__yearly_service_territory,2022,11776
core_eia861__yearly_service_territory,2023,11776
core_eia861__yearly_short_form,2012,1124
core_eia861__yearly_short_form,2013,1122
core_eia861__yearly_short_form,2014,1121
Expand Down Expand Up @@ -2111,52 +2111,52 @@ out_eia860__yearly_ownership,2020,4832
out_eia860__yearly_ownership,2021,5129
out_eia860__yearly_ownership,2022,5300
out_eia860__yearly_ownership,2023,5378
out_eia861__yearly_balancing_authority_service_territory,2001,4957
out_eia861__yearly_balancing_authority_service_territory,2002,5806
out_eia861__yearly_balancing_authority_service_territory,2003,5665
out_eia861__yearly_balancing_authority_service_territory,2004,5946
out_eia861__yearly_balancing_authority_service_territory,2005,5873
out_eia861__yearly_balancing_authority_service_territory,2001,4958
out_eia861__yearly_balancing_authority_service_territory,2002,5807
out_eia861__yearly_balancing_authority_service_territory,2003,5666
out_eia861__yearly_balancing_authority_service_territory,2004,5947
out_eia861__yearly_balancing_authority_service_territory,2005,5874
out_eia861__yearly_balancing_authority_service_territory,2006,6003
out_eia861__yearly_balancing_authority_service_territory,2007,5861
out_eia861__yearly_balancing_authority_service_territory,2008,5934
out_eia861__yearly_balancing_authority_service_territory,2009,5933
out_eia861__yearly_balancing_authority_service_territory,2010,6003
out_eia861__yearly_balancing_authority_service_territory,2011,5069
out_eia861__yearly_balancing_authority_service_territory,2012,5140
out_eia861__yearly_balancing_authority_service_territory,2013,4762
out_eia861__yearly_balancing_authority_service_territory,2014,4486
out_eia861__yearly_balancing_authority_service_territory,2015,4435
out_eia861__yearly_balancing_authority_service_territory,2016,4451
out_eia861__yearly_balancing_authority_service_territory,2017,4487
out_eia861__yearly_balancing_authority_service_territory,2018,4486
out_eia861__yearly_balancing_authority_service_territory,2019,4488
out_eia861__yearly_balancing_authority_service_territory,2020,4491
out_eia861__yearly_balancing_authority_service_territory,2021,4434
out_eia861__yearly_balancing_authority_service_territory,2022,4432
out_eia861__yearly_balancing_authority_service_territory,2023,4435
out_eia861__yearly_utility_service_territory,2001,11589
out_eia861__yearly_utility_service_territory,2002,11585
out_eia861__yearly_utility_service_territory,2003,11588
out_eia861__yearly_utility_service_territory,2004,11688
out_eia861__yearly_utility_service_territory,2005,11542
out_eia861__yearly_balancing_authority_service_territory,2007,5862
out_eia861__yearly_balancing_authority_service_territory,2008,5935
out_eia861__yearly_balancing_authority_service_territory,2009,5934
out_eia861__yearly_balancing_authority_service_territory,2010,6004
out_eia861__yearly_balancing_authority_service_territory,2011,5070
out_eia861__yearly_balancing_authority_service_territory,2012,5141
out_eia861__yearly_balancing_authority_service_territory,2013,4763
out_eia861__yearly_balancing_authority_service_territory,2014,4487
out_eia861__yearly_balancing_authority_service_territory,2015,4436
out_eia861__yearly_balancing_authority_service_territory,2016,4452
out_eia861__yearly_balancing_authority_service_territory,2017,4488
out_eia861__yearly_balancing_authority_service_territory,2018,4487
out_eia861__yearly_balancing_authority_service_territory,2019,4489
out_eia861__yearly_balancing_authority_service_territory,2020,4492
out_eia861__yearly_balancing_authority_service_territory,2021,4435
out_eia861__yearly_balancing_authority_service_territory,2022,4433
out_eia861__yearly_balancing_authority_service_territory,2023,4436
out_eia861__yearly_utility_service_territory,2001,11590
out_eia861__yearly_utility_service_territory,2002,11586
out_eia861__yearly_utility_service_territory,2003,11589
out_eia861__yearly_utility_service_territory,2004,11689
out_eia861__yearly_utility_service_territory,2005,11543
out_eia861__yearly_utility_service_territory,2006,11661
out_eia861__yearly_utility_service_territory,2007,11665
out_eia861__yearly_utility_service_territory,2008,11720
out_eia861__yearly_utility_service_territory,2009,11671
out_eia861__yearly_utility_service_territory,2010,11625
out_eia861__yearly_utility_service_territory,2011,11584
out_eia861__yearly_utility_service_territory,2012,11518
out_eia861__yearly_utility_service_territory,2013,11769
out_eia861__yearly_utility_service_territory,2014,11790
out_eia861__yearly_utility_service_territory,2015,11838
out_eia861__yearly_utility_service_territory,2016,11816
out_eia861__yearly_utility_service_territory,2017,11760
out_eia861__yearly_utility_service_territory,2018,11751
out_eia861__yearly_utility_service_territory,2019,11674
out_eia861__yearly_utility_service_territory,2020,11716
out_eia861__yearly_utility_service_territory,2021,11679
out_eia861__yearly_utility_service_territory,2022,11720
out_eia861__yearly_utility_service_territory,2023,11716
out_eia861__yearly_utility_service_territory,2007,11666
out_eia861__yearly_utility_service_territory,2008,11721
out_eia861__yearly_utility_service_territory,2009,11672
out_eia861__yearly_utility_service_territory,2010,11626
out_eia861__yearly_utility_service_territory,2011,11585
out_eia861__yearly_utility_service_territory,2012,11519
out_eia861__yearly_utility_service_territory,2013,11770
out_eia861__yearly_utility_service_territory,2014,11791
out_eia861__yearly_utility_service_territory,2015,11839
out_eia861__yearly_utility_service_territory,2016,11817
out_eia861__yearly_utility_service_territory,2017,11761
out_eia861__yearly_utility_service_territory,2018,11752
out_eia861__yearly_utility_service_territory,2019,11675
out_eia861__yearly_utility_service_territory,2020,11717
out_eia861__yearly_utility_service_territory,2021,11680
out_eia861__yearly_utility_service_territory,2022,11721
out_eia861__yearly_utility_service_territory,2023,11717
out_eia923__boiler_fuel,2008,77537
out_eia923__boiler_fuel,2009,79002
out_eia923__boiler_fuel,2010,81161
Expand Down Expand Up @@ -3587,23 +3587,23 @@ out_ferc714__hourly_planning_area_demand,2022,784261
out_ferc714__hourly_planning_area_demand,2023,779627
out_ferc714__hourly_planning_area_demand,2024,47
out_ferc714__respondents_with_fips,2006,9166
out_ferc714__respondents_with_fips,2007,9120
out_ferc714__respondents_with_fips,2008,9224
out_ferc714__respondents_with_fips,2009,9903
out_ferc714__respondents_with_fips,2010,10017
out_ferc714__respondents_with_fips,2011,9129
out_ferc714__respondents_with_fips,2012,9358
out_ferc714__respondents_with_fips,2013,9619
out_ferc714__respondents_with_fips,2014,9166
out_ferc714__respondents_with_fips,2015,8901
out_ferc714__respondents_with_fips,2016,8899
out_ferc714__respondents_with_fips,2017,8901
out_ferc714__respondents_with_fips,2018,8917
out_ferc714__respondents_with_fips,2019,8984
out_ferc714__respondents_with_fips,2020,8988
out_ferc714__respondents_with_fips,2021,8936
out_ferc714__respondents_with_fips,2022,8946
out_ferc714__respondents_with_fips,2023,8957
out_ferc714__respondents_with_fips,2007,9127
out_ferc714__respondents_with_fips,2008,9231
out_ferc714__respondents_with_fips,2009,9910
out_ferc714__respondents_with_fips,2010,10024
out_ferc714__respondents_with_fips,2011,9136
out_ferc714__respondents_with_fips,2012,9365
out_ferc714__respondents_with_fips,2013,9626
out_ferc714__respondents_with_fips,2014,9173
out_ferc714__respondents_with_fips,2015,8908
out_ferc714__respondents_with_fips,2016,8906
out_ferc714__respondents_with_fips,2017,8908
out_ferc714__respondents_with_fips,2018,8924
out_ferc714__respondents_with_fips,2019,8991
out_ferc714__respondents_with_fips,2020,8995
out_ferc714__respondents_with_fips,2021,8943
out_ferc714__respondents_with_fips,2022,8953
out_ferc714__respondents_with_fips,2023,8964
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️ ✔️ no 2006es 🏆

out_ferc714__summarized_demand,2006,218
out_ferc714__summarized_demand,2007,218
out_ferc714__summarized_demand,2008,218
Expand Down
3 changes: 1 addition & 2 deletions environments/conda-linux-64.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading