planner: bound the max out of range estimate | tidb-test=pr/2672

[terry1purcell]

What problem does this PR solve?

Issue Number: close #65294

Problem Summary:

What changed and how does it work?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[Copilot]

Pull request overview

This pull request aims to fix over-estimation issues in out-of-range row count calculations for index time ranges, addressing issue #65294 where estimates increased from ~52K to ~10M rows (193x). The changes refactor the OutOfRangeRowCount function in histogram.go to better bound maximum estimates and provide more accurate Min/Max estimate ranges.

Changes:

• Refactored oneValue calculation logic to handle low NDV cases more conservatively
• Restructured estRows and maxAddedRows calculations with separate logic for skew ratio scenarios
• Updated test expectations to include MinEst and MaxEst fields in cardinality estimation outputs

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
pkg/statistics/histogram.go	Refactors OutOfRangeRowCount function with new logic for calculating oneValue, estRows, minEst, and maxAddedRows to bound out-of-range estimates
tests/integrationtest/r/imdbload.result	Updates expected row count estimate from 1027.81 to 5283.37 for out-of-range query test case
pkg/planner/cardinality/testdata/cardinality_suite_out.json	Adds MinEst and MaxEst fields to test expectations for TestOutOfRangeEstimation cases

[codecov]

Codecov Report

❌ Patch coverage is 86.48649% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.9897%. Comparing base (995d6f9) to head (e77692e).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #65888        +/-   ##
================================================
+ Coverage   77.7546%   77.9897%   +0.2350%     
================================================
  Files          2001       1931        -70     
  Lines        545989     535036     -10953     
================================================
- Hits         424532     417273      -7259     
+ Misses       119795     117258      -2537     
+ Partials       1662        505      -1157     

Flag	Coverage Δ
integration	44.1886% <86.4864%> (-3.9973%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	56.7974% <ø> (ø)
parser	∅ <ø> (∅)
br	48.8204% <ø> (-12.1419%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pantheon-ai]

Hi @terry1purcell,

I noticed you mentioned me, but I couldn't find your Pantheon account linked to your GitHub account. Please connect your GitHub account to Pantheon first, and then try again.

Thank you!

_{Learn more about Pantheon AI}

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign terry1purcell for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS
• pkg/planner/OWNERS
• pkg/statistics/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

[ti-chi-bot]

@terry1purcell: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/mysql-test	e77692e	link	true	/test mysql-test
pull-unit-test-next-gen	e77692e	link	true	/test pull-unit-test-next-gen
idc-jenkins-ci-tidb/unit-test	e77692e	link	true	/test unit-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.