feat!: upgrade to Helm 4 #4350

AustinAbro321 · 2025-11-07T17:25:51Z

Description

User facing changes:

Helm 4 brings Server Side Apply. If it is a first time chart install then server side apply will be used by default. Upgrades default to the apply method that was used in the latest release. To control the apply method for a specific packaged chart, a new field .Charts[x].ServerSideApply has been introduced with possible values true, false, auto . Auto will follow the default Helm behavior.
- A good example of how this could effect some charts can be seen in the init package registry deployment. Server side apply caught that replicas were set before even when they were controlled bythe autoscaler, which led to a conflict error. To solve this we conditionally set replicas when the autoscaler is not enabled.
Helm now uses slog style logging.
zarf tools helm is now a true copy of the Helm CLI. Previously, this command was a quasi fork of a subset of the Helm CLI and was not updated automatically. This functionality was made possible through a Zarf maintainer contribution #13617
Helm's now uses kstatus as a backend for it's wait logic, since Zarf already used kstatus the wait logic will be near identical to earlier Zarf versions. Zarf maintainers contributed this functionality to Helm in #13604
zarf dev find-images no longer includes images from Helm tests

We will have to be very careful about merging this as there may be impacts to charts that we do not test for in the migration from Helm 3 to Helm 4. We want the experience to be seamless for users. I intend to make a R.C. release from this branch before merging into main.

Note that vendoring in the entirety of Helm instead of the current quasi fork bring binary size up about 5MB from 173M on main to 178M on this branch

Something interesting is that server side apply default to true during new installs, but "auto" during upgrades and rollbacks meaning that existing charts will use the existing strategy. I decided to add a new addition to the schema, serverSideApply, which accepts the field "true", "false", and "auto".

zarf dev find-images and zarf dev inspect manifests no longer include files helm tests (from the chart/templates/test directory).`. I believe this is a good, since Zarf doesn't have a way to run these tests anyway so we now exclude unnecessary images. Would look to understand the reasoning

Related Issue

Fixes #2818
Fixes #2756
Fixes #4437

Checklist before merging

Test, docs, adr added or updated as needed
Contributor Guide Steps followed

Signed-off-by: Austin Abro <[email protected]>

netlify · 2025-11-07T17:25:59Z

✅ Deploy Preview for zarf-docs canceled.

Name	Link
🔨 Latest commit	`9b452c1`
🔍 Latest deploy log	https://app.netlify.com/projects/zarf-docs/deploys/696548a05516bc00083c335b

Signed-off-by: Austin Abro <[email protected]>

codecov · 2025-11-07T19:16:11Z

Codecov Report

❌ Patch coverage is 28.05430% with 159 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/internal/packager/helm/chart.go	1.17%	84 Missing ⚠️
src/internal/packager/helm/repo.go	40.00%	15 Missing ⚠️
src/internal/packager/helm/zarf.go	0.00%	11 Missing ⚠️
src/internal/packager/helm/template.go	33.33%	8 Missing and 2 partials ⚠️
src/internal/packager/helm/destroy.go	0.00%	9 Missing ⚠️
src/internal/packager/helm/common.go	52.94%	7 Missing and 1 partial ⚠️
src/pkg/packager/inspect.go	22.22%	6 Missing and 1 partial ⚠️
src/pkg/packager/deploy.go	0.00%	4 Missing ⚠️
src/internal/packager/helm/images.go	71.42%	2 Missing ⚠️
src/pkg/cluster/zarf.go	90.47%	2 Missing ⚠️
... and 6 more

Files with missing lines	Coverage Δ
src/api/v1alpha1/component.go	`19.14% <ø> (ø)`
src/internal/packager/helm/post-render.go	`0.00% <ø> (ø)`
src/pkg/packager/find_images.go	`55.00% <100.00%> (-0.94%)`	⬇️
src/cmd/helm.go	`76.92% <75.00%> (-5.43%)`	⬇️
src/cmd/initialize.go	`26.81% <50.00%> (+0.17%)`	⬆️
src/cmd/package.go	`37.38% <50.00%> (+0.01%)`	⬆️
src/cmd/zarf_tools.go	`33.40% <50.00%> (+0.06%)`	⬆️
src/pkg/packager/layout/assemble.go	`42.98% <80.00%> (ø)`
src/internal/packager/helm/images.go	`79.16% <71.42%> (ø)`
src/pkg/cluster/zarf.go	`20.15% <90.47%> (+2.94%)`	⬆️
... and 9 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 · 2026-01-09T15:04:36Z

packages/zarf-registry/chart/templates/deployment.yaml

+  {{ if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
+  {{ end }}


The reason this was added is because with server side apply if autoscaling is enabled then it owns the replicas field, so we will run into an error if we re-apply the chart with a set amount of replicas and auto-scaling.

While not a zarf issue directly (more k8s migration to SSA issue) - this will be exposed to through zarf to consumers.

Are there other known instances of this ownership issue and do we need to document any of them for users to discover?

Signed-off-by: Austin Abro <[email protected]>

brandtkeller

Looking over helm generically in this pass - continuing to dig into the integration. Couple minor questions for helm is isolation.

Excited to see helm vendored fully.

site/src/content/docs/commands/zarf_tools_helm_version.md

src/cmd/helm.go

Signed-off-by: Austin Abro <[email protected]>

brandtkeller

Review nearly complete - working through a few permutations of local testing. Otherwise I think you nailed this. My comments are largely either questions for my education or reinforcing improvements.

brandtkeller · 2026-01-13T21:19:26Z

src/internal/packager/helm/chart.go

+	useSSA := zarfChart.ServerSideApply != "false"
+	client.ServerSideApply = useSSA
+	// Only force conflicts if SSA is enabled and the option is set
+	client.ForceConflicts = useSSA && opts.ForceConflicts


I like this distinction for compatibility - given backwards CSA potential.

brandtkeller · 2026-01-13T21:21:24Z

src/internal/packager/helm/chart.go

+	}
+	client.ServerSideApply = zarfChart.ServerSideApply
+	// Only force conflicts if SSA is enabled and the option is set
+	useSSA := zarfChart.ServerSideApply != "false" && zarfChart.ServerSideApply != ""


Is the latter conditional possible given the preceding lines?

Suggested change

useSSA := zarfChart.ServerSideApply != "false" && zarfChart.ServerSideApply != ""

useSSA := zarfChart.ServerSideApply != "false"

brandtkeller · 2026-01-13T22:30:37Z

src/internal/packager/helm/chart.go

 }

-func rollbackChart(name string, version int, actionConfig *action.Configuration, timeout time.Duration) error {
+func rollbackChart(zarfChart v1alpha1.ZarfChart, version int, actionConfig *action.Configuration, timeout time.Duration) error {


Is there a permutation of scenarios where we need to set client.ForceConflicts on rollback?

brandtkeller · 2026-01-13T22:38:45Z

src/internal/packager/helm/chart.go

-func migrateDeprecatedAPIs(ctx context.Context, c *cluster.Cluster, actionConfig *action.Configuration, latestRelease *release.Release) error {
+func migrateDeprecatedAPIs(ctx context.Context, c *cluster.Cluster, actionConfig *action.Configuration, latestReleaser release.Releaser) error {
+	// We can re-evaluate handling this functionality for chart v3 / release v2 when available.
+	// Potentially we stop doing this functionality for future chart APIs, as it could be outside of the intended scope of Zarf.


I would agree - I believe this workflow also incurs a decent dependency footprint if I recall correctly.

brandtkeller · 2026-01-13T22:50:28Z

src/internal/packager/helm/repo.go

-	}(l)
-
-	saved, _, err := chartDownloader.DownloadTo(chartURL, pull.Version, temp)
+	saved, _, err := chartDownloader.DownloadToCache(chartURL, pull.Version)


neat improvement under-the-hood for zarf (caching).

AustinAbro321 added 2 commits November 6, 2025 16:12

helm 4

b462120

Signed-off-by: Austin Abro <[email protected]>

initial helm 4 changes

445da1e

Signed-off-by: Austin Abro <[email protected]>

github-project-automation bot added this to Zarf Nov 7, 2025

AustinAbro321 mentioned this pull request Nov 7, 2025

Bundled helm doesn't support all commands. #4074

Closed

AustinAbro321 added 2 commits November 7, 2025 13:15

chart path

955b1c4

Signed-off-by: Austin Abro <[email protected]>

fix unit tests

c62c18b

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 added 3 commits November 7, 2025 14:29

fix

c37e316

Signed-off-by: Austin Abro <[email protected]>

helm

68d8af4

Signed-off-by: Austin Abro <[email protected]>

force conflicts

ec58c93

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 changed the title ~~Helm 4~~ feat: upgrade to Helm 4 Nov 12, 2025

AustinAbro321 added 4 commits November 14, 2025 08:22

merge main

649d2bf

Signed-off-by: Austin Abro <[email protected]>

auto ssa

790c74a

Signed-off-by: Austin Abro <[email protected]>

WIP, many decision points

4b25cbd

Signed-off-by: Austin Abro <[email protected]>

notes

5007fa4

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 mentioned this pull request Dec 2, 2025

feat: move to flux cli utils #4426

Draft

2 tasks

AustinAbro321 and others added 13 commits December 3, 2025 12:44

update version of helm 4

2c2fe30

Signed-off-by: Austin Abro <[email protected]>

get rid of helm 3

c7d225f

Signed-off-by: Austin Abro <[email protected]>

go mod tidy

2ae67b8

Signed-off-by: Austin Abro <[email protected]>

adopt existing resources tied to force conflicts

a30a7d7

Signed-off-by: Austin Abro <[email protected]>

Merge branch 'main' into helm-4

f32fa56

Merge branch 'main' into helm-4

9c62670

Merge branch 'helm-4' of github.com:zarf-dev/zarf into helm-4

82eb8d8

add conflict force

28f069a

Signed-off-by: Austin Abro <[email protected]>

add cachepath

3073a6a

Signed-off-by: Austin Abro <[email protected]>

comment

4047ede

Signed-off-by: Austin Abro <[email protected]>

get rid of the managed fields

713a4de

Signed-off-by: Austin Abro <[email protected]>

merge

924f332

Signed-off-by: Austin Abro <[email protected]>

use enum

425c39f

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 and others added 13 commits January 7, 2026 13:18

remove force conflicts

a2020e2

Signed-off-by: Austin Abro <[email protected]>

cleanup + delete managed field removal

f79aed4

Signed-off-by: Austin Abro <[email protected]>

error message

cc8df7f

Signed-off-by: Austin Abro <[email protected]>

Merge branch 'main' into helm-4

4364877

reset helm chart zarf.yaml

26d7fed

Signed-off-by: Austin Abro <[email protected]>

Merge branch 'helm-4' of github.com:zarf-dev/zarf into helm-4

85455e1

fix releases

4357876

Signed-off-by: Austin Abro <[email protected]>

fix chart

639043e

Signed-off-by: Austin Abro <[email protected]>

merge main

5b170b4

Signed-off-by: Austin Abro <[email protected]>

merge main

ba4e379

Signed-off-by: Austin Abro <[email protected]>

add back zarf config tag

e271e8b

Signed-off-by: Austin Abro <[email protected]>

make docs and schema

4bd63ad

Signed-off-by: Austin Abro <[email protected]>

comments

4680612

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 commented Jan 9, 2026

View reviewed changes

AustinAbro321 added 5 commits January 9, 2026 10:09

make docs and schema

4f8eff6

Signed-off-by: Austin Abro <[email protected]>

rollback chart to follow ssa

84909ad

Signed-off-by: Austin Abro <[email protected]>

set debug logs

31d4233

Signed-off-by: Austin Abro <[email protected]>

comment

086a042

Signed-off-by: Austin Abro <[email protected]>

use release name

0b2ec33

Signed-off-by: Austin Abro <[email protected]>

AustinAbro321 changed the title ~~feat: upgrade to Helm 4~~ feat!: upgrade to Helm 4 Jan 9, 2026

AustinAbro321 marked this pull request as ready for review January 9, 2026 19:58

AustinAbro321 requested review from a team as code owners January 9, 2026 19:58

brandtkeller requested changes Jan 12, 2026

View reviewed changes

site/src/content/docs/commands/zarf_tools_helm_version.md Show resolved Hide resolved

src/cmd/helm.go Show resolved Hide resolved

github-project-automation bot moved this to In progress in Zarf Jan 12, 2026

AustinAbro321 added 4 commits January 12, 2026 13:32

kube client version

58fe262

Signed-off-by: Austin Abro <[email protected]>

better log message

b320f24

Signed-off-by: Austin Abro <[email protected]>

better message

c6fe763

Signed-off-by: Austin Abro <[email protected]>

no exit

9b452c1

Signed-off-by: Austin Abro <[email protected]>

brandtkeller reviewed Jan 13, 2026

View reviewed changes

	useSSA := zarfChart.ServerSideApply != "false" && zarfChart.ServerSideApply != ""
	useSSA := zarfChart.ServerSideApply != "false"

feat!: upgrade to Helm 4 #4350

Are you sure you want to change the base?

feat!: upgrade to Helm 4 #4350

Uh oh!

Conversation

AustinAbro321 commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Checklist before merging

Uh oh!

netlify bot commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for zarf-docs canceled.

Uh oh!

codecov bot commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

AustinAbro321 Jan 9, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

brandtkeller left a comment

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

brandtkeller Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AustinAbro321 commented Nov 7, 2025 •

edited

Loading

netlify bot commented Nov 7, 2025 •

edited

Loading

codecov bot commented Nov 7, 2025 •

edited

Loading