Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CNDB-12308: Remove index-based sorting at CQL level #1488

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

adelapena
Copy link

@adelapena adelapena commented Jan 3, 2025

It was recently found that index-based sorting is done twice in the coordinator. The first sorting is done by StorageAttachedIndexQueryPlan#postProcessor, at the service level, which sorts a PartitionIterator. Then a second sorting is done by SortedRowsBuilder, at the CQL level, which sorts a simpler List<ByteBuffer>.

CNDB-11762 removed the entire StorageAttachedIndexQueryPlan#postProcessor method so sorting was not duplicated anymore. However, this was problematic for implementations of QueryInfoTracker.ReadTracker, which is applied after sorting with the postprocessor, but before sorting at the CQL level, here. Thus, that change was reverted to avoid breaking external trackers relying on the previous behaviour.

I think what we should do instead is to get rid of index-based sorting at the CQL level, so all sorting is done by the postprocessor. The postprocessor works at a lower level so, with some adjustments, it should be able to do all the sorting on its own without needing CQL intervention. That should remove the duplicated sorting and simplify CQL a bit.

Checklist before you submit for review

  • Make sure there is a PR in the CNDB project updating the Converged Cassandra version
  • Use NoSpamLogger for log lines that may appear frequently in the logs
  • Verify test results on Butler
  • Test coverage for new/modified code is > 80%
  • Proper code formatting
  • Proper title for each commit staring with the project-issue number, like CNDB-1234
  • Each commit has a meaningful description
  • Each commit is not very long and contains related changes
  • Renames, moves and reformatting are in distinct commits

@adelapena adelapena self-assigned this Jan 3, 2025
@adelapena adelapena marked this pull request as draft January 3, 2025 17:24
@adelapena adelapena marked this pull request as ready for review January 7, 2025 14:28
Do all coordinator-side sorting for index queries in StorageAttachedIndexQueryPlan#postProcessor.
@adelapena
Copy link
Author

@michaeljmarshall could you please review this one?

@cassci-bot
Copy link

❌ Build ds-cassandra-pr-gate/PR-1488 rejected by Butler


4 new test failure(s) in 4 builds
See build details here


Found 4 new test failures

Test Explanation Branch history Upstream history
...ToolEnableDisableBinaryTest.testMaybeChangeDocs regression 🔴🔵🔴🔴 🔵🔵🔵🔵🔵🔵🔵
...ap<float,float>,wide=true,scenario=MIXED_QUERY] regression 🔴🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵
...>,wide=false,scenario=POST_BUILD_QUERY] regression 🔴🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵
...n.ConnectionTest.testMessageDeliveryOnReconnect regression 🔴🔵🔵🔵 🔵🔵🔵🔵🔵🔵🔵

Found 10 known test failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants