Fix errors on GTFS RT validation #3513
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
erikamov
force-pushed
the
mov/2780-gtfs-rt-validation
branch
from
abf1c4c
to
a2bf5eb
Compare
ohrite
force-pushed
the
mov/2780-gtfs-rt-validation
branch
from
b8b06c4
to
21263b2
Compare
|
Should the test be failing for this? |
Signed-off-by: Erika Pacheco <[email protected]>
Signed-off-by: Erika Pacheco <[email protected]>
* This simplifies the flow of control so that every command runs the same code Signed-off-by: Doc Ritezel <[email protected]>
Signed-off-by: Doc Ritezel <[email protected]>
* This commit resolves a race condition where all parse_and_validate calls shared the same temporary directory * That contention meant that processes would overwrite the existing GTFS schedule with the same name * This also resulted in an elevated number of skipped protobuf validations * The gtfs-realtime-validator skips protobufs with the same MD5 * The race condition caused elevated MD5 collisions for protobufs Signed-off-by: Doc Ritezel <[email protected]>
erikamov
force-pushed
the
mov/2780-gtfs-rt-validation
branch
from
21263b2
to
0d2a924
Compare
erikamov
force-pushed
the
mov/2780-gtfs-rt-validation
branch
from
0d2a924
to
6edb9a0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes some errors on GTFS RT validation that were found investigating the issue #2780:
The error message described in the issue
[Errno 2] No such file or directoryis caused by the file skipped (mentioned above). It could be treated as a message instead of an error.Type of change
How has this been tested?
Tested locally using
poetry run pytestwhich replicates running the validator tool on the command line, e.g.:Post-merge follow-ups
Monitoring next DAG runs and query
cal-itp-data-infra.staging.int_gtfs_quality__rt_validation_outcomesto see less errors happening.