Feat: improve VPA filtering and checkpoint garbage collection #7716

omerap12 · 2025-01-17T12:31:31Z

What type of PR is this?

/kind feature

What this PR does / why we need it:

Currently, the recommender's garbage collection for checkpoints iterates through all namespaces in the cluster, ignoring the vpaObjectNamespace and ignoredNamespaces flags. In large-scale clusters, this can lead to performance issues. This PR addresses the problem by ensuring that the garbage collection logic respects the specified flags.

Which issue(s) this PR fixes:

Fixes #7697

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Fixes VPA recommender garbage collection to respect `vpaObjectNamespace` and `ignoredNamespaces` flags, improving performance in large-scale clusters by limiting the scope of namespace checks.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Change flags description

k8s-ci-robot · 2025-01-17T12:31:33Z

@omerap12: The label(s) kind/featrue cannot be applied, because the repository doesn't have them.

In response to this:

What type of PR is this?

/kind featrue

What this PR does / why we need it:

Currently, the recommender's garbage collection for checkpoints iterates through all namespaces in the cluster, ignoring the vpaObjectNamespace and ignoredNamespaces flags. In large-scale clusters, this can lead to performance issues. This PR addresses the problem by ensuring that the garbage collection logic respects the specified flags.

Which issue(s) this PR fixes:

Fixes #7697

Special notes for your reviewer:

Does this PR introduce a user-facing change?
Fixes VPA recommender garbage collection to respect `vpaObjectNamespace` and `ignoredNamespaces` flags, improving performance in large-scale clusters by limiting the scope of namespace checks.
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:
Not sure where should I document this change

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-01-17T12:31:55Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: omerap12
Once this PR has been reviewed and has the lgtm label, please assign kgolab for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

vertical-pod-autoscaler/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

omerap12 · 2025-01-17T12:35:07Z

/kind feature

adrianmoisey · 2025-01-17T18:08:57Z

Not sure where should I document this change

I'm not sure if this is a good idea, but what about updating the flags:

autoscaler/vertical-pod-autoscaler/common/flags.go

Lines 45 to 46 in b8719d3

    
           flag.StringVar(&cf.VpaObjectNamespace, "vpa-object-namespace", apiv1.NamespaceAll, "Namespace to search for VPA objects. Empty means all namespaces will be used.") 
        
           flag.StringVar(&cf.IgnoredVpaObjectNamespaces, "ignored-vpa-object-namespaces", "", "Comma separated list of namespaces to ignore when searching for VPA objects. Empty means no namespaces will be ignored.")

?

omerap12 · 2025-01-17T20:22:58Z

Not sure where should I document this change

I'm not sure if this is a good idea, but what about updating the flags:

autoscaler/vertical-pod-autoscaler/common/flags.go

Lines 45 to 46 in b8719d3

flag.StringVar(&cf.VpaObjectNamespace, "vpa-object-namespace", apiv1.NamespaceAll, "Namespace to search for VPA objects. Empty means all namespaces will be used.")

flag.StringVar(&cf.IgnoredVpaObjectNamespaces, "ignored-vpa-object-namespaces", "", "Comma separated list of namespaces to ignore when searching for VPA objects. Empty means no namespaces will be ignored.")

?

sounds good

Signed-off-by: Omer Aplatony <[email protected]>

adrianmoisey · 2025-01-19T15:37:32Z

/lgtm

Thanks!

If you're up for it, an e2e test testing the flag would be cool, but I know we have an item in the backlog for that already.

Signed-off-by: Omer Aplatony <[email protected]>

omerap12 · 2025-01-21T08:24:30Z

Had to rebase so pinging again @adrianmoisey

adrianmoisey · 2025-01-21T19:35:17Z

/lgtm

adrianmoisey · 2025-01-28T11:49:12Z

/milestone vertical-pod-autoscaler-1.4.0

voelzmo

Thanks for the PR, I have some comments where code could be a bit simpler.

vertical-pod-autoscaler/pkg/recommender/input/cluster_feeder.go

voelzmo · 2025-01-31T10:13:17Z

vertical-pod-autoscaler/pkg/recommender/input/cluster_feeder.go

+	return feeder.vpaObjectNamespace == "" && len(feeder.ignoredNamespaces) == 0
+}
+
+func (feeder *clusterStateFeeder) cleanupCheckpointsForNamespace(namespace string, allVPAKeys map[model.VpaID]bool) {


We can make this function return err and have the caller handle it. This way, you can also simplify the if-else and the error messages, as they don't need to contain the context that this happened inside the garbage collection function. Adding context for the checkpoint listing with e.g. errors.Wrapf could still be useful, though.

Added in 86eb551

voelzmo · 2025-01-31T10:28:32Z

vertical-pod-autoscaler/pkg/recommender/input/cluster_feeder.go

@@ -354,7 +382,7 @@ func filterVPAs(feeder *clusterStateFeeder, allVpaCRDs []*vpa_types.VerticalPodA
 			}
 		}

-		if slices.Contains(feeder.ignoredNamespaces, vpaCRD.ObjectMeta.Namespace) {
+		if !feeder.shouldCleanupNamespace(vpaCRD.ObjectMeta.Namespace) {


If we're re-using shouldCleanupNamespace here, I think we should rename it. Currently, its name is tied pretty much to the garbage collection use-case. Maybe it is rather something like shouldIgnoreNamespace?

On the other hand, we already initialize the vpaLister with a single namespace if vpaObjectNamespace is set:

autoscaler/vertical-pod-autoscaler/pkg/recommender/main.go

Line 251 in 29b611d

VpaLister: vpa_api_util.NewVpasLister(vpa_clientset.NewForConfigOrDie(config), make(chan struct{}), commonFlag.VpaObjectNamespace),

so we probably don't need to do this here and filtering for the ignored namespaces is enough.

I suppose we can keep using slices.Contains as it was, but personally, I’m not a fan of duplicated code (since shouldCleanupNamespace does the same thing).

Do you have a preference? Should we rename the function, or are we okay with some duplication?

I'm fine with renaming the method and reduce duplication. We could also re-use the same method then for filtering the VPACheckpoints returned by the lister introduced in #6419

Added in 694cce8

vertical-pod-autoscaler/pkg/recommender/input/cluster_feeder.go

Signed-off-by: Omer Aplatony <[email protected]>

k8s-ci-robot · 2025-01-31T13:31:27Z

New changes are detected. LGTM label has been removed.

Signed-off-by: Omer Aplatony <[email protected]>

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jan 17, 2025

k8s-ci-robot requested a review from jbartosik January 17, 2025 12:31

k8s-ci-robot added the area/vertical-pod-autoscaler label Jan 17, 2025

k8s-ci-robot requested a review from voelzmo January 17, 2025 12:31

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jan 17, 2025

omerap12 force-pushed the recommender-namespaces branch from 90cb7a6 to 85370c3 Compare January 17, 2025 20:22

Feat: improve VPA filtering and checkpoint garbage collection

8117b14

Signed-off-by: Omer Aplatony <[email protected]>

omerap12 force-pushed the recommender-namespaces branch from 1e1fc7a to 8117b14 Compare January 19, 2025 08:29

k8s-ci-robot assigned adrianmoisey Jan 19, 2025

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jan 19, 2025

Merge branch 'master' into recommender-namespaces

9d450f8

Signed-off-by: Omer Aplatony <[email protected]>

k8s-ci-robot removed lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jan 21, 2025

Fixed typo

3d94088

Signed-off-by: Omer Aplatony <[email protected]>

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 21, 2025

k8s-ci-robot added this to the vertical-pod-autoscaler-1.4.0 milestone Jan 28, 2025

voelzmo suggested changes Jan 31, 2025

View reviewed changes

voelzmo mentioned this pull request Jan 31, 2025

list vpacheckpoint using vpacheckpoint lister #6419

Open

Simplify function

f4f67f5

Signed-off-by: Omer Aplatony <[email protected]>

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 31, 2025

omerap12 added 2 commits January 31, 2025 18:42

using wrapf errors

86eb551

Signed-off-by: Omer Aplatony <[email protected]>

Change function name

694cce8

Signed-off-by: Omer Aplatony <[email protected]>

omerap12 force-pushed the recommender-namespaces branch from 13fb008 to 694cce8 Compare February 4, 2025 20:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: improve VPA filtering and checkpoint garbage collection #7716

Feat: improve VPA filtering and checkpoint garbage collection #7716

omerap12 commented Jan 17, 2025 •

edited

Loading

k8s-ci-robot commented Jan 17, 2025

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Jan 17, 2025

omerap12 commented Jan 17, 2025

adrianmoisey commented Jan 17, 2025

omerap12 commented Jan 17, 2025

adrianmoisey commented Jan 19, 2025

omerap12 commented Jan 21, 2025

adrianmoisey commented Jan 21, 2025

adrianmoisey commented Jan 28, 2025

voelzmo left a comment

voelzmo Jan 31, 2025

omerap12 Jan 31, 2025

voelzmo Jan 31, 2025

omerap12 Jan 31, 2025

voelzmo Feb 3, 2025

omerap12 Feb 3, 2025

omerap12 Feb 4, 2025

k8s-ci-robot commented Jan 31, 2025

Feat: improve VPA filtering and checkpoint garbage collection #7716

Are you sure you want to change the base?

Feat: improve VPA filtering and checkpoint garbage collection #7716

Conversation

omerap12 commented Jan 17, 2025 • edited Loading

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Jan 17, 2025

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Jan 17, 2025

omerap12 commented Jan 17, 2025

adrianmoisey commented Jan 17, 2025

omerap12 commented Jan 17, 2025

adrianmoisey commented Jan 19, 2025

omerap12 commented Jan 21, 2025

adrianmoisey commented Jan 21, 2025

adrianmoisey commented Jan 28, 2025

voelzmo left a comment

Choose a reason for hiding this comment

voelzmo Jan 31, 2025

Choose a reason for hiding this comment

omerap12 Jan 31, 2025

Choose a reason for hiding this comment

voelzmo Jan 31, 2025

Choose a reason for hiding this comment

omerap12 Jan 31, 2025

Choose a reason for hiding this comment

voelzmo Feb 3, 2025

Choose a reason for hiding this comment

omerap12 Feb 3, 2025

Choose a reason for hiding this comment

omerap12 Feb 4, 2025

Choose a reason for hiding this comment

k8s-ci-robot commented Jan 31, 2025

omerap12 commented Jan 17, 2025 •

edited

Loading