Skip to content

Conversation

@zamai
Copy link

@zamai zamai commented Oct 26, 2025

Description
Currently there is no way of using karpenter kwok image and helm chart outside of this repo.

This PR add two jobs to release.yaml:

  • publish-kwok-image: builds and publishes the KWOK container image to GHCR using ko to ghcr.io//karpenter-kwok:$TAG.
  • publish-kwok-chart: packages the KWOK Helm chart and pushes it to GHCR (OCI). Packages the chart using the release tag as the chart version.

There is a symlink file in karpenter/kwok/charts/crds/karpenter.kwok.sh_kwoknodeclasses.yaml helm build does not resolve symlinks, I've added copy of real file as a step in the action.

Proposed names and paths:

  • image: ghcr.io/kubernetes-sigs/karpenter/karpenter-kwok
  • chart: oci://ghcr.io/kubernetes-sigs/karpenter/charts/karpenter-kwok

How was this change tested?
In my repository - I've pushed images and chart, then tested them locally. Check them out here:
https://github.com/zamai?tab=packages&repo_name=karpenter

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 26, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @zamai. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 26, 2025
@zamai zamai force-pushed the publish-kwok-image-and-chart branch from 311669c to 83e0fc4 Compare October 26, 2025 13:43
Rename Helm chart name from "karpenter" to "karpenter-kwok" to identify 
the KWOK-specific provider distribution.

Add two jobs to release.yaml:
- publish-kwok-image: builds and publishes the KWOK container image to
  GHCR using ko  to ghcr.io/<repo>/karpenter-kwok:$TAG.
- publish-kwok-chart: packages the KWOK Helm chart and pushes it to
  GHCR (OCI). It ensures a CRD symlink is replaced with
  the real file when present, packages the chart using the release tag
  as the chart version, and pushes the resulting chart archive to the
  repository's charts OCI registry.

These changes automate publishing of the KWOK provider image and Helm
chart and align chart metadata with the KWOK integration.
@zamai zamai force-pushed the publish-kwok-image-and-chart branch from 83e0fc4 to 61a6158 Compare October 26, 2025 13:47
@coveralls
Copy link

coveralls commented Oct 26, 2025

Pull Request Test Coverage Report for Build 19194951759

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.01%) to 81.68%

Files with Coverage Reduction New Missed Lines %
pkg/controllers/disruption/drift.go 2 88.0%
Totals Coverage Status
Change from base Build 19111740812: -0.01%
Covered Lines: 11579
Relevant Lines: 14176

💛 - Coveralls

Copy link
Contributor

@DerekFrank DerekFrank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me, thanks for setting this up! I'll see if an approver can take a look at it sometime soon

@zamai zamai force-pushed the publish-kwok-image-and-chart branch 2 times, most recently from 9a858ed to 5f64c27 Compare November 8, 2025 11:55
@zamai zamai force-pushed the publish-kwok-image-and-chart branch from 5f64c27 to 80a4a3a Compare November 8, 2025 12:16
@zamai zamai requested a review from moko-poi November 8, 2025 12:19
Copy link
Contributor

@moko-poi moko-poi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot
Copy link
Contributor

@moko-poi: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@zamai
Copy link
Author

zamai commented Nov 8, 2025

@moko-poi the only question left if this repo's actions have permissions to publish packages.

 permissions:
      contents: read
      packages: write

@moko-poi
Copy link
Contributor

moko-poi commented Nov 9, 2025

@zamai
That's a good point to verify. From what I can see, most kubernetes-sigs repos (aws-ebs-csi-driver, aws-efs-csi-driver, etc.) currently publish to GitHub Pages rather than GHCR. There are several open issues requesting GHCR support (e.g., metrics-server #1527, descheduler #1470), but I couldn't find existing examples of GHCR publishing within kubernetes-sigs.

- name: Publish KWOK image
run: |
KO_DOCKER_REPO="ghcr.io/${{ github.repository }}/karpenter-kwok" ko publish ./kwok --bare -t "$VERSION"
Copy link
Contributor

@moko-poi moko-poi Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to switch to GitHub Pages.
#2592 (comment)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moko-poi so we can publish helm charts to pages, but not docker images, and we need kwok image to be published to use it in helm chart.

I don't know the structure and rules around kubernetes-sigs, I just propose the easy and transparent way to publish to ghcr directly from actions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zamai I agree that GHCR would be the simplest approach. However, similar proposals have been declined in other kubernetes-sigs projects (e.g., [metrics-server #1527](kubernetes-sigs/metrics-server#1527)).

Would registry.k8s.io be a viable alternative? It's the standard approach used by descheduler, metrics-server, and other kubernetes-sigs projects. It requires more setup (staging registry + promotion PRs), but aligns with existing conventions.

Alternatively, aws-ebs-csi-driver publishes to both registry.k8s.io and AWS ECR Public, which might be another option.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moko-poi registry.k8s.io - yes, would be great as well. Will take a look into those repos and try to match the existing setup to publish to registry.k8s.io.

Will tag you once done in the PR

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@moko-poi
After reading details on how to setup building and promoting images, from staging env to the registry.k8s.io I think that the effort isn't worth the reward for me.

KWOK image and helm chart are tools for testing not something official that k8s org is publishing, so I think it's okay to keep them in GHCR.

For now I'll use images from my fork, it's pretty easy to pull upstream changes.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: DerekFrank, zamai
Once this PR has been reviewed and has the lgtm label, please assign maciekpytel for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@zamai
Copy link
Author

zamai commented Nov 15, 2025

@engedaam I would not be updating PR to use registry.k8s.io, if you are okay keeping the build and artifacts to this repo - great lets merge, otherwise please close the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants