Integrate the Census-based FIPS codes to replace addfips

[cmgosnell]

Overview

Closes #3884.

What problem does this address?

What did you change?

Questions

• should we publish a transformed table as a _core or core table?? if core we may want to break it out into multiple tables bc really its many geographies all squished into one table
• rn i've called this table "geocodes" because that's what census calls it. sound okay?
• should i add in a lil manually compiled non-us state codes that addfips was including?

Documentation

Make sure to update relevant aspects of the documentation.

- [ ] Update the [release notes](https://catalystcoop-pudl.readthedocs.io/en/nightly/release_notes.html): reference the PR and related issues.
- [ ] Update relevant Data Source jinja templates (see `docs/data_sources/templates`).
- [ ] Update relevant table or source description metadata (see `src/metadata`).
- [ ] Review and update any other aspects of the documentation that might be affected by this PR.

Testing

How did you make sure this worked? How can a reviewer verify this?

# To-do list
- [ ] If updating analyses or data processing functions: make sure to update or write data validation tests (e.g. `test_minmax_rows()`).
- [ ] Run `make pytest-coverage` locally to ensure that the merge queue will accept your PR.
- [ ] Review the PR yourself and call out any questions or issues you have.
- [ ] For minor ETL changes or data additions, once `make pytest-coverage` passes, make sure you have a fresh full PUDL DB downloaded locally, materialize new/changed assets and all their downstream assets and [run relevant data validation tests](https://catalystcoop-pudl.readthedocs.io/en/nightly/dev/testing.html#data-validation) using `pytest` and `--live-dbs`.
- [ ] For bigger ETL or data changes run the full ETL locally and then [run the data validations](https://catalystcoop-pudl.readthedocs.io/en/nightly/dev/testing.html#data-validation) using `make pytest-validate`.
- [ ] Alternatively, run the `build-deploy-pudl` GitHub Action manually.

[krivard]

Re:Q1 - are any of the areas both a subdivision and a place/place and consolidated city/subdivision and consolidated city, or can we file them as an "aux code" "aux type" pair? if they're mutually exclusive then we could do either:

• one table, with a variable-length concatenated FIPS (3 chars for country, 5 for state, 8 for county, 13 for aux)
• 3 tables (country could just be hard coded): state, county, aux

Re:Q2 - 👍 "geocodes"

[cmgosnell]

hm @krivard I've been assuming we want to update the vintage of the fips codes (the addfips package's most recent data is from 2015). But I've already found one problem with this update. CT! all the county names have been.. idk boring-afied

the data table i've been testing with (core_eia861__yearly_service_territory) only has the old names in the data so all of the CT counties are not being labeled right now... which makes sense bc apparently these new regional counties are not fully implemented until 2024 according to wikipedia.

[cmgosnell]

okay since this revelation about different years having different info in there i decided to incorporate 2015 and 2023. I'm purposefully rn not merging the code using report_date. because even in this example the report_date from census don't line up. but now there are both options in the data. this seems to be working as intended.

[cmgosnell]

okay i also added in 2009 bc there were a small handful of counties that were mapped w/ addfips that weren't being mapped without an older year.

rn the county codes that don't match are only two and they are bc of name changes:

there are also 13 non-us state records that are not mapped

[cmgosnell]

oh nice okay i ran the validation tests before actually fixing the Bedford City vs County problem.... and then the row counts changed! because previously Bedford City was getting dropped because addfips wasn't labeling Bedford City and we were dropna-ing on the fips codes. so i am going to update the row counts!

[cmgosnell]

i didn't realize this was here until after i added these guys into the CSV. that feels like a better place for them anyway so i delete this guy.

[cmgosnell]

what has changed (i think) since last review:

• removed addfips and updated conda lock files
• make the fips codes to be strings in the non-census data tables (i was dev-ing on an already typed table before... silly me)
• .... long pause .... figure out how dbt works ty for your help
• update the row counts (see the comment above for why - tldr tiny addition of new fips == smol # of more rows)
• remove now duplicate territory codes

[krivard]

Assuming I read that right, good to go!

[krivard]

✔️ ✔️ no 2006es 🏆

[krivard]

Checking understanding: this change is for two reasons:

1. we needed county_col as a string, and sometimes it wasn't
2. _clean_area_name_col couldn't handle a Series so we have to run it one row at a time

if yes, all good, if no, help?

[zaneselvans]

I thought assign with a lambda function / any callable that takes a series still does its work vectorized. Is that not true? (df.apply() is row-wise and notoriously slow)