Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(fct_transit_data_quality_issues): removes duplicate issues create… #3409

Merged
merged 1 commit into from
Aug 2, 2024

Conversation

vevetron
Copy link
Contributor

@vevetron vevetron commented Aug 2, 2024

…d due to unnesting by only selecting first dataset

Description

Previously the codebase unnested when multiple gtfs datasets (and services) were listed in an issue. Which could make sure we don't lose any information, but really just capturing the first gtfs dataset is sufficient.

Resolves #3407

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How has this been tested?

poetry run dbt run -s  +calitp_warehouse.mart.transit_database.fct_transit_data_quality_issues --full-refresh
then
SELECT count(*) FROM `cal-itp-data-infra-staging.vb_mart_transit_database.fct_transit_data_quality_issues` order by 

```-> Produced 284 issues up to issue 300, which is accurate at a snapshot of the airtable issue table
## Post-merge follow-ups

- [x] No action required

…d due to unnesting by only selecting first dataset
Copy link

github-actions bot commented Aug 2, 2024

Warehouse report 📦

DAG

Legend (in order of precedence)

Resource type Indicator Resolution
Large table-materialized model Orange Make the model incremental
Large model without partitioning or clustering Orange Add partitioning and/or clustering
View with more than one child Yellow Materialize as a table or incremental
Incremental Light green
Table Green
View White

@vevetron vevetron merged commit fd71594 into main Aug 2, 2024
4 checks passed
@vevetron vevetron deleted the remove_duplicate_issues branch August 2, 2024 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: duplicate entries showing up in the fct_transit_data_quality_issues table
2 participants