Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

repository: Merge Workload Repository into master #57148

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

wddevries
Copy link
Contributor

What problem does this PR solve?

Issue Number: ref #57147

Problem Summary:

This pull request merges in the Workload Repository code from the Serverless project. The project is not complete yet, and more testing remains to be done.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

xhebox and others added 10 commits November 5, 2024 13:58
* domain: add repository worker

Signed-off-by: xhe <[email protected]>

* sessionctx, domain: add function gate

Signed-off-by: xhe <[email protected]>

* repository: refine context usage

Signed-off-by: xhe <[email protected]>

* repository: add recover and session getter

Signed-off-by: xhe <[email protected]>

* sessionctx: hide variable

Signed-off-by: xhe <[email protected]>

* fix bazel

Signed-off-by: xhe <[email protected]>

* fix check

Signed-off-by: xhe <[email protected]>

* repository: refine owner management

Signed-off-by: xhe <[email protected]>

* repository: free memref

Signed-off-by: xhe <[email protected]>

* fix check

Signed-off-by: xhe <[email protected]>

---------

Signed-off-by: xhe <[email protected]>
* repository: sample and snapshot threads

Signed-off-by: xhe <[email protected]>

* fix

Signed-off-by: xhe <[email protected]>

* fix

Signed-off-by: xhe <[email protected]>

* address comments

Signed-off-by: xhe <[email protected]>

* address comments

Signed-off-by: xhe <[email protected]>

* address coments

Signed-off-by: xhe <[email protected]>

* address coments

Signed-off-by: xhe <[email protected]>

* address comments

Signed-off-by: xhe <[email protected]>

* address comments

Signed-off-by: xhe <[email protected]>

* address comments

Signed-off-by: xhe <[email protected]>

---------

Signed-off-by: xhe <[email protected]>
…als. (pingcap#1317)

* Add global variables to control sampling intervals in the Workload Repository.

* Return error when etc client is not available.

* Rework error messages.

* Fix required comments.

* Switch to using Ticker.Reset instead of restarting Fiber.

* Use swap instead of load and store.

* Make a couple of minor changes.

* Fix issues
* Wrap exit with a context.Context.

* Remove getGlobalVar from Worker.

* Seperate Repository from Domain.

* Use RegisterSysVar instead of defining varibles in SysVar.go

* Move samplingInterval and snapshotInterval to Worker.

* Convert setRepositoryDest to a method of Worker.

* Make Worker private to the repository package.

* Add error handling in SetupRepository and disable repository.

* Change to statically allocated worker class and enable repository.

* Don't wait on exit channel for domain.

* Stop the repository from cleanup in main.go.

* Fix nits and move the error check for etcd later in start().

* Add comments on public methods.

* Fix error message format.
@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Nov 6, 2024
Copy link

ti-chi-bot bot commented Nov 6, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign wjhuang2016 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 6, 2024
Copy link

tiprow bot commented Nov 6, 2024

Hi @wddevries. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

codecov bot commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 15.25680% with 561 lines in your changes missing coverage. Please review.

Project coverage is 56.8774%. Comparing base (865213c) to head (202a37b).
Report is 5 commits behind head on master.

Additional details and impacted files
@@                Coverage Diff                @@
##             master     #57148         +/-   ##
=================================================
- Coverage   72.9877%   56.8774%   -16.1103%     
=================================================
  Files          1657       1814        +157     
  Lines        457574     646015     +188441     
=================================================
+ Hits         333973     367437      +33464     
- Misses       103093     253339     +150246     
- Partials      20508      25239       +4731     
Flag Coverage Δ
integration 39.2014% <15.2568%> (?)
unit 72.2945% <50.0000%> (+0.0277%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.9478% <ø> (ø)
parser ∅ <ø> (∅)
br 62.6514% <ø> (+16.9291%) ⬆️

@hawkingrei
Copy link
Member

/ok-to-test

@ti-chi-bot ti-chi-bot bot added the ok-to-test Indicates a PR is ready to be tested. label Nov 6, 2024
@@ -10,6 +10,7 @@ go_library(
"//pkg/config",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some test cases for it?

Copy link

ti-chi-bot bot commented Nov 6, 2024

@wddevries: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
idc-jenkins-ci-tidb/build 202a37b link true /test build

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link

tiprow bot commented Nov 6, 2024

@wddevries: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
fast_test_tiprow 202a37b link true /test fast_test_tiprow

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link
Contributor

@henrybw henrybw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of the changes LGTM.

@@ -187,6 +187,8 @@ func (w *Worker) start(ctx context.Context) func() {
return func() {
w.owner = w.newOwner(ownerKey, promptKey)
ticker := time.NewTicker(time.Second)
w.owner.CampaignOwner()
defer w.owner.CampaignCancel()
Copy link
Contributor

@henrybw henrybw Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should w.owner.CampaignCancel() be in stop() instead? Seems like this would prematurely retire the owner while the worker is still active?

Copy link
Contributor

@xhebox xhebox Nov 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, house keeper need this. owner campaign is not a one-shot election, it is long-running.

Also we've assumed that newOwner will call CampaignOwner by default. And should not call CampaignOwner twice(implementation of owner has no guard for invoking twice).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants