ETCD-727: Rebase etcd 3.5.21 openshift 4.18 #325
+836
−240
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Backport tools/benchmark/cmd/txn_mixed.go from commit 79b2777, pull request etcd-io#13038. Co-authored-by: Ivan Valdes <[email protected]> Signed-off-by: Ivan Valdes <[email protected]>
Signed-off-by: GitHub <[email protected]>
…-txn-mixed [3.5] backport: tools: add mixed read-write performance evaluation scripts
add learner check to readyz
Signed-off-by: Benjamin Wang <[email protected]>
Signed-off-by: Benjamin Wang <[email protected]>
Signed-off-by: Benjamin Wang <[email protected]>
Signed-off-by: Benjamin Wang <[email protected]>
Signed-off-by: Benjamin Wang <[email protected]>
[3.5] Add more info into log for learner member operations
[3.5] Bump `go.mongodb.org/mongo-driver` and `github.com/golang/glog` to address two CVEs
Signed-off-by: Marcel Franca <[email protected]>
…e-3.5 [3.5] Update golang toolchain to 1.22.12
The compaction behavior is changed in commit [02635](etcd-io@0263597) and introduces a latency issue. To be more speicific, the `ticker.C` acts as a fixed timer that triggers every 10ms, regardless of how long each batch of compaction takes. This means that if a previous compaction batch takes longer than 10ms, the next batch starts immediately, making compaction a blocking operation for etcd. To fix the issue, this commit revert the compaction to the previous behavior which ensures a 10ms delay between each batch of compaction, allowing other read and write operations to proceed smoothly. Signed-off-by: Miancheng Lin <[email protected]>
…e-latency [release-3.5] Fix a performance regression due to uncertain compaction sleep interval
Signed-off-by: joshjms <[email protected]> change go directive to 1.23 Signed-off-by: joshjms <[email protected]>
… expiry Signed-off-by: huangzw <[email protected]>
…-3.5 [3.5] Update golang toolchain to 1.23.6
To address CVE-2025-22869. Signed-off-by: Ivan Valdes <[email protected]>
…org-x-crypto-to-0.35.0 [3.5] dependency: Bump golang.org/x/crypto from v0.32.0 to v0.35.0
Signed-off-by: Ivan Valdes <[email protected]>
Signed-off-by: Ivan Valdes <[email protected]>
….23.7 [3.5] Bump go toolchain to 1.23.7
…org-x-net-to-v0.36.0 [3.5] dependency: bump golang.org/x/net from v0.34.0 to v0.36.0
Signed-off-by: Ivan Valdes <[email protected]>
…ry-pick-19520-to-release-3.5 [3.5] Add verify release assets GitHub workflow
We should use WithCompactPhysical to wait for compaction to finish,
because 50ms sleep can't guarantee compaction is done.
Based on the log, the defragment is finished before compaction, which is
not expected. This patch is to make sure compaction should be finished
before assertation.
```
=========================== defragment ==============
logger.go:146: 2025-02-27T07:57:18.652Z INFO m0 finished defragment {"member": "m0"}
=====================================================
logger.go:146: 2025-02-27T07:57:18.652Z INFO m0 grpc service status changed {"member": "m0", "service": "", "status": "SERVING"}
logger.go:146: 2025-02-27T07:57:18.653Z INFO grpc [[core] [Channel etcd-io#1457]Channel Connectivity change to SHUTDOWN]
logger.go:146: 2025-02-27T07:57:18.653Z INFO grpc [[core] [Channel etcd-io#1457]Closing the name resolver]
logger.go:146: 2025-02-27T07:57:18.653Z INFO grpc [[core] [Channel etcd-io#1457]ccBalancerWrapper: closing]
logger.go:146: 2025-02-27T07:57:18.653Z INFO grpc [[core] [Channel etcd-io#1457 SubChannel etcd-io#1458]Subchannel Connectivity change to SHUTDOWN]
logger.go:146: 2025-02-27T07:57:18.653Z INFO grpc [[core] [Channel etcd-io#1457 SubChannel etcd-io#1458]Subchannel deleted]
logger.go:146: 2025-02-27T07:57:18.654Z INFO grpc [[transport] [client-transport 0xc00237bd48] Closing: rpc error: code = Canceled desc = grpc: the client connection is closing]
logger.go:146: 2025-02-27T07:57:18.654Z INFO grpc [[transport] [client-transport 0xc00237bd48] loopyWriter exiting with error: rpc error: code = Canceled desc = grpc: the client connection is closing]
logger.go:146: 2025-02-27T07:57:18.654Z INFO grpc [[transport] [server-transport 0xc0006bb520] Closing: EOF]
logger.go:146: 2025-02-27T07:57:18.654Z INFO grpc [[core] [Channel etcd-io#1457]Channel deleted]
logger.go:146: 2025-02-27T07:57:18.654Z INFO grpc [[transport] [server-transport 0xc0006bb520] loopyWriter exiting with error: transport closed by client]
hash.go:82:
Error Trace: /home/prow/go/src/github.com/etcd-io/etcd/server/storage/mvcc/testutil/hash.go:82
/home/prow/go/src/github.com/etcd-io/etcd/server/storage/mvcc/testutil/hash.go:44
Error: Not equal:
expected: testutil.KeyValueHash{Hash:0x94694091, CompactRevision:1278, Revision:2507}
actual : testutil.KeyValueHash{Hash:0x58af47dc, CompactRevision:2488, Revision:2507}
Diff:
--- Expected
+++ Actual
@@ -1,4 +1,4 @@
(testutil.KeyValueHash) {
- Hash: (uint32) 2489925777,
- CompactRevision: (int64) 1278,
+ Hash: (uint32) 1487882204,
+ CompactRevision: (int64) 2488,
Revision: (int64) 2507
Test: TestCompactionHash
Messages: hashes do not match on rev 2488
cluster.go:1423: ========= Cluster termination started =====================
logger.go:146: 2025-02-27T07:57:18.655Z INFO grpc [[core] [Channel etcd-io#1377]Channel Connectivity change to SHUTDOWN]
logger.go:146: 2025-02-27T07:57:18.655Z INFO grpc [[core] [Channel etcd-io#1377]Closing the name resolver]
logger.go:146: 2025-02-27T07:57:18.655Z INFO grpc [[core] [Channel etcd-io#1377]ccBalancerWrapper: closing]
logger.go:146: 2025-02-27T07:57:18.656Z INFO grpc [[core] [Channel etcd-io#1377 SubChannel etcd-io#1378]Subchannel Connectivity change to SHUTDOWN]
logger.go:146: 2025-02-27T07:57:18.656Z INFO grpc [[core] [Channel etcd-io#1377 SubChannel etcd-io#1378]Subchannel deleted]
logger.go:146: 2025-02-27T07:57:18.656Z INFO grpc [[transport] [client-transport 0xc0025f2248] Closing: rpc error: code = Canceled desc = grpc: the client connection is closing]
logger.go:146: 2025-02-27T07:57:18.656Z INFO grpc [[transport] [client-transport 0xc0025f2248] loopyWriter exiting with error: rpc error: code = Canceled desc = grpc: the client connection is closing]
logger.go:146: 2025-02-27T07:57:18.656Z INFO grpc [[transport] [server-transport 0xc000b56ea0] Closing: EOF]
=========================== compaction ==============
logger.go:146: 2025-02-27T07:57:18.656Z INFO m0 finished scheduled compaction {"member": "m0", "compact-revision": 2488, "took": "129.590742ms", "hash": 2489925777, "current-db-size-bytes": 40960, "current-db-size": "41 kB", "current-db-size-in-use-bytes": 40960, "current-db-size-in-use": "41 kB"}
=====================================================
```
Fixes: etcd-io#19497
Signed-off-by: Wei Fu <[email protected]>
Signed-off-by: Ivan Valdes <[email protected]>
…ry-pick-19538-to-release-3.5 [release-3.5] deflakey: TestCompactionHash in integration
Signed Signed-off-by: Gang Li <[email protected]>
caused by etcd-io#19557 Signed-off-by: Benjamin Wang <[email protected]>
[release-3.5] Add e2e test to verify etcd is able to automatically fix the issue
Signed-off-by: Ivan Valdes <[email protected]>
Addresses CVE-2025-30204. Signed-off-by: Ivan Valdes <[email protected]>
…com-golang-jwt-jwt-v4 [release-3.5] dependency: bump github.com/golang-jwt/jwt/v4 from 4.5.1 to 4.5.2
Addresses CVE-2025-22870 and CVE-2025-22872. Signed-off-by: Ivan Valdes <[email protected]>
…-net-to-v0.37.0 [release-3.5] dependency: bump golang.org/x/net from v0.36.0 to v0.38.0
Signed-off-by: Ivan Valdes <[email protected]>
…tore This PR will add the notion of cluster ID into the initial cluster discovery process. This allows us to automatically archive a data directory when we detect the cluster identifier changing. The cluster identifier will only change when we are running a restore operation. The detection requires that the revision.json (created by the revision monitor sidecar) contains the cluster id. The cluster identifier is also stored in the local WAL, which is much more expensive to parse. We're going to only fallback to it when we could not get the cluster id from the revision.json for any reason. Otherwise the WAL stays untouched, no repair operations are attempted when it is found corrupted. Signed-off-by: Thomas Jungblut <[email protected]>
Konflux is replacing RH's internal build system OSBS. OSBS supported a build-time dependency injection system called "cachito". Konflux replaces this with "cachi2" which works differently. REMOTE_SOURCES no longer need to be copied into place and there is no need to source cachito's environment information (Konflux automatically rewrites the Dockerfile to source cachi2/cachi2.env before running the original RUN commands). Additionally, cachito appears to have provided go.sum dependencies whereas cachi2 requires all build-time dependencies in go.mod. Missing dependencies are added to go.mod as // indirect in this change.
force-new-cluster seems to have similar watch cache issues as the ordinary snapshot restore. This PR introduces the already existing utl logic as a separate package into the server-side code. This will only introduce a revbump flag, but under the hood implement both rev bumping and compaction markers. Signed-off-by: Thomas Jungblut <[email protected]>
This adds the min and max TLS version support from etcd-io#13506 and etcd-io#15156 to the grpc proxy. Signed-off-by: Thomas Jungblut <[email protected]>
The compaction behavior is changed in commit [02635](etcd-io@0263597) and introduces a latency issue. To be more speicific, the `ticker.C` acts as a fixed timer that triggers every 10ms, regardless of how long each batch of compaction takes. This means that if a previous compaction batch takes longer than 10ms, the next batch starts immediately, making compaction a blocking operation for etcd. To fix the issue, this commit revert the compaction to the previous behavior which ensures a 10ms delay between each batch of compaction, allowing other read and write operations to proceed smoothly. Signed-off-by: Miancheng Lin <[email protected]>
…tcd-3.5.21-openshift-4.18
|
/payload 4.18 nightly informing |
|
/payload 4.18 nightly blocking |
|
@Elbehery: trigger 64 job(s) of type informing for the nightly release of OCP 4.18
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/f73eacc0-0d2b-11f0-9f90-1c0d9dcdba2b-0 |
|
@Elbehery: trigger 11 job(s) of type blocking for the nightly release of OCP 4.18
See details on https://pr-payload-tests.ci.openshift.org/runs/ci/fb5d0ea0-0d2b-11f0-835b-ada90e26dc73-0 |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Elbehery The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
Elbehery
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@Elbehery: This pull request references ETCD-727 which is a valid jira issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@Elbehery: This pull request references ETCD-727 which is a valid jira issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR rebases etcd 3.5.21 into openshift-4.18.
running
makelocallycc @openshift/openshift-team-etcd @sdodson