Skip to content

Conversation

@D3Hunter
Copy link
Contributor

@D3Hunter D3Hunter commented Nov 7, 2025

What problem does this PR solve?

Issue Number: ref #61702

Problem Summary:

What changed and how does it work?

depends on pingcap/metering_sdk#13

  • fix the issue that if we failed to write the data, previous data will be lost as we always cleanup existing data previously
  • move meter recorder register/unregister into DXF framework, we do it when create the step executor, so the DXF app can use it directly
  • reset the global metering instance when shutdown
  • refactor metering and recording part, not we always accumulate data inside the recorder, and let the meter to calculate the delta when flush
  • decouple meter write/flush part and recording part, so later we can choose directly write some metering data without go through recorder.
  • log the total metered data when we unregister the recorder, to help diagnose

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

run a simple import

[2025/11/07 20:03:00.023 +08:00] [INFO] [metering.go:243] ["succeed to write metering data"] [keyspaceName=SYSTEM] [component=meter] [duration=22.375333ms] [data="[{\"cluster_id\":\"SYSTEM\",\"read_bytes\":2,\"source_name\":\"dxf\",\"task_id\":1,\"task_type\":\"ImportInto\",\"version\":\"1\",\"write_bytes\":29}]"]
[2025/11/07 20:03:00.024 +08:00] [INFO] [metering.go:176] ["recorder unregistered and finished final flush"] [keyspaceName=SYSTEM] [component=meter] [accumulatedData="{taskID: 1, keyspace: SYSTEM, type: ImportInto, getReqs: 0, putReqs: 0, readBytes: 2, writeBytes: 29}"]

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Nov 7, 2025
@D3Hunter D3Hunter marked this pull request as draft November 7, 2025 12:25
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Nov 7, 2025
@ti-chi-bot
Copy link

ti-chi-bot bot commented Nov 7, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfzjywxk, d3hunter for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tiprow
Copy link

tiprow bot commented Nov 7, 2025

Hi @D3Hunter. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant