Use explicit aliases in job search deleted outputs filter #21623
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
guerler
added
kind/enhancement
area/database
guerler
approved these changes
Jan 20, 2026
Member
Author
|
That's not it, failed on the third re-run |
mvdbeek
force-pushed
the
claude/fix-flaky-jobs-test-Lfb3h
branch
from
699bdba to
1ad4ed0
Compare
Member
|
Such an alluring theory though! |
The _exclude_jobs_with_deleted_outputs function was using unaliased model classes (e.g., model.HistoryDatasetAssociation) in its exists() subqueries, while the main query from _build_stmt_for_hdca uses aliased versions of these same tables (e.g., candidate_hda). While exists() subqueries should create independent table references, using the same model class without aliasing in both the main query and the exists() subquery could potentially cause ambiguity in PostgreSQL's query planner, leading to non-deterministic query behavior. This fix adds explicit aliases (job_output_collection_assoc, output_hdca, job_output_dataset_assoc, output_hda) to ensure the tables in the exists() subqueries are clearly distinct from any tables in the outer query. This may fix the flaky test_search_delete_hdca_output issue tracked in GitHub issue galaxyproject#21230.
The job_ids_cte in _filter_jobs was misleadingly named "job_ids_materialized_cte" but was not actually materialized. This CTE contains the complex HDCA signature matching logic. PostgreSQL 12+ can choose to inline CTEs (re-evaluate them as subqueries) rather than materialize them (evaluate once and store). This optimization decision is based on cost estimates which can vary between query executions. By explicitly materializing this CTE when supported (PostgreSQL 12+), we ensure the signature matching results are evaluated once and reused consistently throughout the query, reducing potential for non-deterministic behavior. This is a follow-up to the explicit aliases fix, both addressing the flaky test_search_delete_hdca_output (GitHub issue galaxyproject#21230).
Use unique_id in all is:published searches to avoid picking up published histories created by other tests running in parallel.
mvdbeek
force-pushed
the
claude/fix-flaky-jobs-test-Lfb3h
branch
from
f68f6ed to
cabc18b
Compare
Member
Author
|
OK, I ran the test for the 2 first commits 10 times on my fork and they passed ... we could try it. |
Member
Author
|
lol, first try failed, cool ... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The _exclude_jobs_with_deleted_outputs function was using unaliased model classes (e.g., model.HistoryDatasetAssociation) in its exists() subqueries, while the main query from _build_stmt_for_hdca uses aliased versions of these same tables (e.g., candidate_hda).
While exists() subqueries should create independent table references, using the same model class without aliasing in both the main query and the exists() subquery could potentially cause ambiguity in PostgreSQL's query planner, leading to non-deterministic query behavior.
This fix adds explicit aliases (job_output_collection_assoc, output_hdca, job_output_dataset_assoc, output_hda) to ensure the tables in the exists() subqueries are clearly distinct from any tables in the outer query.
This (maybe) addresses the flaky test_search_delete_hdca_output issue tracked in GitHub issue #21230.
How to test the changes?
(Select all options that apply)
License