Skip to content

*: Collect client network info for topsql to pick topN sql with highest network traffic (#63796) #65962

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ti-chi-bot wants to merge 3 commits into
base: release-8.5
Choose a base branch
from
Open

*: Collect client network info for topsql to pick topN sql with highest network traffic (#63796) #65962

ti-chi-bot wants to merge 3 commits into from
+396 −86

Conversation

@ti-chi-bot
Copy link
Member

@ti-chi-bot ti-chi-bot commented Feb 2, 2026

This is an automated cherry-pick of #63796

What problem does this PR solve?

Issue Number: close #62920

Problem Summary:
Add client network in/out stats in topsql. And record topN network usage sqls adding to current topN cpu usage sqls. Note one sql will be added at most once.

What changed and how does it work?

  1. Add two new session variable "InPacketBytes", "OutPacketBytes" to record the network traffic from/to clients.
  2. When query finishes, record the network traffic info to topsql stmtstats
  3. Pick topN query with highest network traffic.

Check List

Sysbench Performance Test

The performance impact of this PR is not noticable.
Turn off topsql
img_v3_02ro_ba405d95-0fc4-4a72-a4d5-22025a97061g

Turn on topsql
img_v3_02ro_47c2617c-270e-415a-8df5-2ab18fa94a4g

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Run tpch benchmark, print topsql report data on tidb side, and check the network in/out data using sql digest manually to ensure network data make sense.
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Collects client network data for topsql to pick topN sql with highest network traffic.

@ti-chi-bot ti-chi-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/planner SIG: Planner size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. type/cherry-pick-for-release-8.5 This PR is cherry-picked to release-8.5 from a source PR. labels Feb 2, 2026
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 2, 2026
Merged
13 tasks
@ti-chi-bot
Copy link
Member Author

ti-chi-bot commented Feb 2, 2026

@yibin87 This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 2, 2026

@ti-chi-bot: ## If you want to know how to resolve it, please read the guide in TiDB Dev Guide.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot force-pushed the cherry-pick-63796-to-release-8.5 branch from d8a0a61 to 3367d2a Compare February 2, 2026 07:04
@yibin87
Copy link
Contributor

yibin87 commented Feb 2, 2026 •
edited
Loading

This is an automated cherry-pick of #63796

What problem does this PR solve?

Issue Number: close #62920

Problem Summary: Add client network in/out stats in topsql. And record topN network usage sqls adding to current topN cpu usage sqls. Note one sql will be added at most once.

What changed and how does it work?

  1. Add two new session variable "InPacketBytes", "OutPacketBytes" to record the network traffic from/to clients.
  2. When query finishes, record the network traffic info to topsql stmtstats
  3. Pick topN query with highest network traffic.

Check List

Sysbench Performance Test

The performance impact of this PR is not noticable. Turn off topsql img_v3_02ro_ba405d95-0fc4-4a72-a4d5-22025a97061g

Turn on topsql img_v3_02ro_47c2617c-270e-415a-8df5-2ab18fa94a4g

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
    Run tpch benchmark, print topsql report data on tidb side, and check the network in/out data using sql digest manually to ensure network data make sense.
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Collects client network data for topsql to pick topN sql with highest network traffic.

Merge the other cp into this one: #65688

@yibin87
Copy link
Contributor

yibin87 commented Feb 2, 2026

/test build

@yibin87
Copy link
Contributor

yibin87 commented Feb 2, 2026

/test unit-test

@tiprow
Copy link

tiprow bot commented Feb 2, 2026

@yibin87: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test fast_test_tiprow_for_release
Details

In response to this:

/test build

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tiprow
Copy link

tiprow bot commented Feb 2, 2026

@yibin87: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test fast_test_tiprow_for_release
Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@codecov
Copy link

codecov bot commented Feb 2, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 87.50000% with 15 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release-8.5@06bcd2d). Learn more about missing BASE report.

Additional details and impacted files 
@@               Coverage Diff                @@
##             release-8.5     #65962   +/-   ##
================================================
  Coverage               ?   54.9302%           
================================================
  Files                  ?       1815           
  Lines                  ?     650953           
  Branches               ?          0           
================================================
  Hits                   ?     357570           
  Misses                 ?     266757           
  Partials               ?      26626
Flag Coverage Δ
integration 38.2001% <15.0000%> (?)
unit 64.8687% <87.5000%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.9278% <0.0000%> (?)
parser ∅ <0.0000%> (?)
br 55.4672% <0.0000%> (?)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yibin87
Copy link
Contributor

yibin87 commented Feb 2, 2026

/test unit-test

@tiprow
Copy link

tiprow bot commented Feb 2, 2026

@yibin87: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test fast_test_tiprow_for_release
Details

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot bot added cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Feb 2, 2026
yibin87
yibin87 approved these changes Feb 3, 2026
View reviewed changes
Copy link
Contributor

@yibin87 yibin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Feb 3, 2026
@yibin87
Copy link
Contributor

yibin87 commented Feb 3, 2026

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 3, 2026
@yibin87
Copy link
Contributor

yibin87 commented Feb 3, 2026

/cc @zimulala

@ti-chi-bot ti-chi-bot bot requested a review from zimulala February 3, 2026 02:13
@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 3, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yibin87, zimulala
Once this PR has been reviewed and has the lgtm label, please assign yudongusa for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Feb 3, 2026
@ti-chi-bot
Copy link

ti-chi-bot bot commented Feb 3, 2026

[LGTM Timeline notifier]

Timeline:

  • 2026-02-03 02:12:28.295348598 +0000 UTC m=+146619.396747317: ☑️ agreed by yibin87.
  • 2026-02-03 08:38:44.39193593 +0000 UTC m=+169795.493334660: ☑️ agreed by zimulala.

@yibin87
Copy link
Contributor

yibin87 commented Feb 4, 2026

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zimulala zimulala zimulala approved these changes

+1 more reviewer

@yibin87 yibin87 yibin87 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@yibin87 yibin87

Labels

cherry-pick-approved Cherry pick PR approved by release team. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/planner SIG: Planner size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. type/cherry-pick-for-release-8.5 This PR is cherry-picked to release-8.5 from a source PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ti-chi-bot @yibin87 @zimulala