Skip to content

KEP-3243: Update milestone v1.33 to v1.34 and add the new feature gate to control the design change of TopologySpreadConstraint's matchLabelKeys #5205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
+35 −11
Open

KEP-3243: Update milestone v1.33 to v1.34 and add the new feature gate to control the design change of TopologySpreadConstraint's matchLabelKeys #5205

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
39ebbdc
Update milestone v1.33 to v1.34 and add feature gate for #129874
mochizuki875 Mar 21, 2025
d91c6ff
fix1
mochizuki875 Mar 24, 2025
c06d5c3
fix2
mochizuki875 Apr 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 30 additions & 9 deletions ...ig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update the implementation history?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for letting me know.
I've updated.

enhancements/keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/README.md

Line 1084 in c06d5c3

- 2025-04-07: Add a new feature flag `MatchLabelKeysInPodTopologySpreadSelectorMerge` and update milestone

Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ tags, and then generate with `hack/update-toc.sh`.
- [Possible misuse](#possible-misuse)
- [The update to labels specified at <code>matchLabelKeys</code> isn't supported](#the-update-to-labels-specified-at-matchlabelkeys-isnt-supported)
- [Design Details](#design-details)
- [[v1.33] design change and a safe upgrade path](#v133-design-change-and-a-safe-upgrade-path)
- [[v1.34] design change and a safe upgrade path](#v134-design-change-and-a-safe-upgrade-path)
- [Test Plan](#test-plan)
- [Prerequisite testing updates](#prerequisite-testing-updates)
- [Unit tests](#unit-tests)
Expand Down Expand Up @@ -401,25 +401,29 @@ kube-apiserver modifies the `labelSelector` like the following:
In addition, kube-scheduler will handle `matchLabelKeys` within the cluster-level default constraints
in the scheduler configuration in the future (see https://github.com/kubernetes/kubernetes/issues/129198).

Finally, the feature will be guarded by a new feature flag. If the feature is
Finally, the feature will be guarded by a new feature flag `MatchLabelKeysInPodTopologySpread`. If the feature is
disabled, the field `matchLabelKeys` and corresponding `labelSelector` are preserved
if it was already set in the persisted Pod object, otherwise new Pod with the field
creation will be rejected by kube-apiserver.
Also kube-scheduler will ignore `matchLabelKeys` in the cluster-level default constraints configuration.

### [v1.33] design change and a safe upgrade path
### [v1.34] design change and a safe upgrade path
Previously, kube-scheduler just internally handled `matchLabelKeys` before the calculation of scheduling results.
But, we changed the implementation design to the current form to make the design align with PodAffinity's `matchLabelKeys`.
(See the detailed discussion in [the alternative section](#implement-matchlabelkeys-in-only-either-the-scheduler-plugin-or-kube-apiserver))

However, this implementation change could break `matchLabelKeys` of unscheduled pods created before the upgrade
because kube-apiserver only handles `matchLabelKeys` at pods creation, that is,
it doesn't handle `matchLabelKeys` at existing unscheduled pods.
So, for a safe upgrade path from v1.32 to v1.33, kube-scheduler would handle not only `matchLabelKeys`
from the default constraints, but also all incoming pods during v1.33.
We're going to change kube-scheduler to only concern `matchLabelKeys` from the default constraints at v1.34 for efficiency,
So, for a safe upgrade path from v1.33 to v1.34, kube-scheduler would handle not only `matchLabelKeys`
from the default constraints, but also all incoming pods during v1.34.
We're going to change kube-scheduler to only concern `matchLabelKeys` from the default constraints at v1.35 for efficiency,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, are we planning to make the MatchLabelKeysInPodTopologySpreadSelectorMerge GA in v1.35? Without this, we won't be able to change the kube-scheduler's code this way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have not met the v1.33 lifecycle, I think every step should shift one version forward as below.

  • Merge this design change as beta and scheduler will handle all matchLabelKeys: v1.33 -> v1.34
  • Change scheduler to only concern matchLabelKeys from the default constraints: v1.34 -> v1.35
  • Change MatchLabelKeysInPodTopologySpread(including MatchLabelKeysInPodTopologySpreadSelectorMerge) to GA from beta: v1.35 -> v1.36

Therefore I think the change for scheduler was previously planned to take place during beta.
But if needed, we can change the plan for scheduler change from v1.35 to v1.36 to align with the GA of MatchLabelKeysInPodTopologySpreadSelectorMerge.

assuming kube-apiserver handles `matchLabelKeys` of all incoming pods.

Also, in case of bugs in this new design, users can disable this feature through a new feature flag,
`MatchLabelKeysInPodTopologySpreadSelectorMerge` (enabled by default).
(See more details in [Feature Enablement and Rollback](#feature-enablement-and-rollback))

### Test Plan

<!--
Expand Down Expand Up @@ -619,8 +623,8 @@ enhancement:

There's no version skew issue.

We changed the implementation design between v1.33 and v1.34, but we designed the change not to involve any version skew issue
as described at [[v1.33] design change and a safe upgrade path](#v133-design-change-and-a-safe-upgrade-path).
We changed the implementation design between v1.34 and v1.35, but we designed the change not to involve any version skew issue
as described at [[v1.34] design change and a safe upgrade path](#v134-design-change-and-a-safe-upgrade-path).

## Production Readiness Review Questionnaire

Expand Down Expand Up @@ -652,6 +656,19 @@ you need any help or guidance.
This section must be completed when targeting alpha to a release.
-->

- `MatchLabelKeysInPodTopologySpread` feature flag enables the `MatchLabelKeys` feature in `TopologySpreadConstraint`.
- `MatchLabelKeysInPodTopologySpreadSelectorMerge` feature flag enables the new design described at
[[v1.34] design change and a safe upgrade path](#v134-design-change-and-a-safe-upgrade-path).
- If `MatchLabelKeysInPodTopologySpreadSelectorMerge` is disabled while `MatchLabelKeysInPodTopologySpread` is enabled,
Kubernetes handles `MatchLabelKeys` with the classic design, kube-scheduler handles it.
However, that's basically not recommended unless you encounter a bug in a new design behavior.
- This flag cannot be enabled on its own, and has to be enabled together with `MatchLabelKeysInPodTopologySpread`.
Enabling `MatchLabelKeysInPodTopologySpreadSelectorMerge` alone has no effect, and `matchLabelKeys` will be ignored.

The `MatchLabelKeysInPodTopologySpreadSelectorMerge` feature flag has been added in v1.34 and enabled by default.
This flag can be disabled to revert [the implementation design change in v1.34](#v134-design-change-and-a-safe-upgrade-path)
and go back to the previous behavior in case of bug.

###### How can this feature be enabled / disabled in a live cluster?

<!--
Expand All @@ -667,6 +684,9 @@ well as the [existing list] of feature gates.
- [x] Feature gate (also fill in values in `kep.yaml`)
- Feature gate name: `MatchLabelKeysInPodTopologySpread`
- Components depending on the feature gate: `kube-scheduler`, `kube-apiserver`
- [x] Feature gate (also fill in values in `kep.yaml`)
- Feature gate name: `MatchLabelKeysInPodTopologySpreadSelectorMerge`
- Components depending on the feature gate: `kube-apiserver`

###### Does enabling the feature change any default behavior?

Expand Down Expand Up @@ -1061,6 +1081,7 @@ Major milestones might include:
- 2022-06-08: KEP merged
- 2023-01-16: Graduate to Beta
- 2025-01-23: Change the implementation design to be aligned with PodAffinity's `matchLabelKeys`
- 2025-04-07: Add a new feature flag `MatchLabelKeysInPodTopologySpreadSelectorMerge` and update milestone

## Drawbacks

Expand All @@ -1086,7 +1107,7 @@ and scheduler plugin shouldn't have special treatment for any labels/fields.
Technically, we can implement this feature within the PodTopologySpread plugin only;
merging the key-value labels corresponding to `MatchLabelKeys` into `LabelSelector` internally
within the plugin before calculating the scheduling results.
This is the actual implementation up to 1.32.
This is the actual implementation up to 1.33.
But, it may confuse users because this behavior would be different from PodAffinity's `MatchLabelKeys`.

Also, we cannot implement this feature only within kube-apiserver because it'd make it
Expand Down
7 changes: 5 additions & 2 deletions keps/sig-scheduling/3243-respect-pod-topology-spread-after-rolling-upgrades/kep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,13 @@ stage: beta
# The most recent milestone for which work toward delivery of this KEP has been
# done. This can be the current (upcoming) milestone, if it is being actively
# worked on.
latest-milestone: "v1.33"
latest-milestone: "v1.34"

# The milestone at which this feature was, or is targeted to be, at each stage.
milestone:
alpha: "v1.25"
beta: "v1.27"
stable: "v1.35"
stable: "v1.36"

# The following PRR answers are required at alpha release
# List the feature gate name and the components for which it must be enabled
Expand All @@ -37,6 +37,9 @@ feature-gates:
components:
- kube-apiserver
- kube-scheduler
- name: MatchLabelKeysInPodTopologySpreadSelectorMerge
components:
- kube-apiserver

disable-supported: true

Expand Down