Open
Description
Description
It's been ~1 year since we've worked intensively on orchestration stuff, and Dagster has evolved since then. There are deprecation and design changes that we probably need to accommodate as the project continues to grow in complexity. Is there anything we want to bring in from #2386? Or can that be closed?
- [ ] #3161
- [ ] https://github.com/catalyst-cooperative/pudl/issues/3977
- [ ] Remove "experimental" warning from Parquet IO Manager
- [ ] Look at using Dagster's [tags](https://docs.dagster.io/concepts/metadata-tags/tags#tags) and hierarchical [key prefixes](https://docs.dagster.io/concepts/assets/software-defined-assets#multi-component-asset-keys) to better organize our DAG
- [ ] Switch to using Postgres instead of SQLite for local development.
- [ ] #3928
- [ ] #3913
- [ ] Investigate whether we can use Dagster's built-in partitioning to split up big jobs like EPA CEMS more simply.
- [ ] https://github.com/catalyst-cooperative/pudl/issues/2468
- [ ] https://github.com/catalyst-cooperative/pudl/issues/2470
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Backlog