Skip to content

Join through association #811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

camposandro
Copy link
Collaborator

The association tables may need to be repartitioned such that its pixels are different from the inner alignment between the primary and the join catalogs. If that's the case we need to recompute the alignment for the join.

This PR adds some logic to do that: align_catalog_with_association first aligns the primary catalog with the association catalog and then it aligns the result to the join catalog to obtain the pixels that need to be joined between the three.

This is a step toward #795. However, I am looking for feedback since this solution will introduce new assoc_Norder and assoc_Npix columns in the partition join info (apart from the already existing Norder, Npix and join_Norder, join_Npix).

Copy link

github-actions bot commented Jun 5, 2025

Before [279b559] After [80cd734] Ratio Benchmark (Parameter)
13.7±0.3ms 14.1±0.2ms 1.03 benchmarks.time_box_filter_on_partition
6.39±0.03s 6.49±0.05s 1.02 benchmarks.time_create_large_catalog
956±6ms 972±9ms 1.02 benchmarks.time_create_midsize_catalog
86.6±1ms 86.0±0.7ms 0.99 benchmarks.time_kdtree_crossmatch
28.9±0.6ms 28.5±0.6ms 0.99 benchmarks.time_polygon_search

Click here to view all benchmarks.

Copy link

codecov bot commented Jun 5, 2025

Codecov Report

Attention: Patch coverage is 93.75000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 97.38%. Comparing base (352a742) to head (2e4c871).
Report is 11 commits behind head on main.

Files with missing lines Patch % Lines
src/lsdb/dask/merge_catalog_functions.py 92.30% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #811      +/-   ##
==========================================
- Coverage   97.44%   97.38%   -0.06%     
==========================================
  Files          54       54              
  Lines        2463     2490      +27     
==========================================
+ Hits         2400     2425      +25     
- Misses         63       65       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant