Skip to content

Comments

feat(query): restore prefetch for spilled sort blocks#19409

Open
forsaken628 wants to merge 6 commits intodatabendlabs:mainfrom
forsaken628:sort-spill
Open

feat(query): restore prefetch for spilled sort blocks#19409
forsaken628 wants to merge 6 commits intodatabendlabs:mainfrom
forsaken628:sort-spill

Conversation

@forsaken628
Copy link
Collaborator

@forsaken628 forsaken628 commented Feb 5, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

  • Restore prefetch for spilled sort blocks.
  • Dynamically adjusts num_merge based on current memory usage.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 5, 2026
@forsaken628 forsaken628 added the ci-cloud Build docker image for cloud test label Feb 13, 2026
@github-actions
Copy link
Contributor

Docker Image for PR

  • tag: pr-19409-44d95f9-1770973358

note: this image tag is only available for internal use.

@forsaken628
Copy link
Collaborator Author

benchmark

DataSet: tpch sf100
SQL: select * from orders order by o_totalprice desc ignore_result;
version:
default: v1.2.881-nightly-1cc88fbd2d
pr-19409: v1.2.881-nightly-44d95f9ded

settings warehouse warehouse_size query_duration_s sort_cpu_s sort_io_s scan_cpu_s scan_io_s spill_write_mib spill_write_cnt spill_write_s spill_read_mib spill_read_cnt spill_read_s
default Small 8.992 82.069 0.000 28.664 120.315 0.00 0 0.000 0.00 0 0.000
pr-19409 Small 8.411 79.013 0.000 27.590 110.592 0.00 0 0.000 0.00 0 0.000
8GB memory,max_threads=16 default Small 129.949 191.875 435.848 24.951 129.753 8727.22 2576 271.944 8727.22 2576 245.662
8GB memory,max_threads=16 pr-19409 Small 103.968 171.155 458.787 23.912 118.767 9240.97 2592 271.162 9240.97 2592 252.046
8GB memory,max_threads=16,prefetch pr-19409 Small 94.509 172.547 278.685 25.427 149.108 9024.19 2618 273.412 9024.19 2618 268.342
8GB memory,max_threads=8 default Small 147.596 172.960 425.807 21.144 117.693 7626.20 2338 232.318 7626.20 2338 264.497
8GB memory,max_threads=8 pr-19409 Small 123.879 152.505 399.620 21.295 105.338 7499.10 2297 225.891 7499.10 2297 226.442
8GB memory,max_threads=8,prefetch pr-19409 Small 91.926 155.238 205.868 21.607 112.598 7517.94 2304 227.549 7517.94 2304 254.188
8GB memory,max_threads=4 default Small 213.574 168.825 416.782 20.946 104.166 7433.13 2281 227.004 7433.13 2281 258.523
8GB memory,max_threads=4 pr-19409 Small 197.512 150.545 415.796 20.503 105.156 7539.75 2317 233.391 7539.75 2317 235.309
8GB memory,max_threads=4,prefetch pr-19409 Small 133.019 154.509 214.640 20.888 107.092 7523.69 2315 238.795 7523.69 2315 261.771

@forsaken628 forsaken628 marked this pull request as ready for review February 15, 2026 07:06
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ed2c0f5c2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant