Skip to content

Skaffold doesn't respect kube-context for configuring platform #9144

@anthonyalayo

Description

@anthonyalayo

Expected behavior

If kubeContext is set in skaffold.yaml, skaffold should use this context to determine the build/deploy platform.

This is what I expect, but it only happens when ~.kube/config's context is matching skaffold.yaml.

DEBU[0000] CLI platforms provided: ""                    subtask=-1 task=DevLoop
DEBU[0000] getting client config for kubeContext: `docker-desktop`  subtask=-1 task=DevLoop
DEBU[0000] platforms detected from active kubernetes cluster nodes: "linux/arm64"  subtask=-1 task=DevLoop
DEBU[0000] platforms selected for artifact "server": "linux/arm64"  subtask=-1 task=DevLoop

Actual behavior

Skaffold will use the current cluster to determine it:

func GetClusterPlatforms(ctx context.Context, kContext string) (Matcher, error) {
client, err := kubernetesclient.Client(kContext)
if err != nil {
return Matcher{}, fmt.Errorf("failed to determine kubernetes cluster node platforms: %w", err)
}
nodes, err := client.CoreV1().Nodes().List(ctx, coreV1.ListOptions{})
if nodes == nil || err != nil {
return Matcher{}, fmt.Errorf("failed to determine kubernetes cluster node platforms: %w", err)
}
set := make(map[string]v1.Platform)
for _, n := range nodes.Items {
pl := v1.Platform{
Architecture: n.Status.NodeInfo.Architecture,
OS: n.Status.NodeInfo.OperatingSystem,
}
set[Format(pl)] = pl
}
keys := make([]string, 0, len(set))
for k := range set {
keys = append(keys, k)
}
sort.Strings(keys) // sort keys to have a deterministic selection
var m Matcher
for _, key := range keys {
m.Platforms = append(m.Platforms, set[key])
}
return m, nil
}

This is what actually happens. Skaffold ignores skaffold.yaml and uses the ~.kube/config's context.

DEBU[0001] CLI platforms provided: ""                    subtask=-1 task=DevLoop
DEBU[0001] getting client config for kubeContext: `test-context-amd-64`  subtask=-1 task=DevLoop
DEBU[0001] platforms detected from active kubernetes cluster nodes: "linux/amd64"  subtask=-1 task=DevLoop
DEBU[0001] platforms selected for artifact "server": "linux/amd64"  subtask=-1 task=DevLoop

Notice how it resolved amd64 instead of arm64, which caused issues for me.

Information

  • Skaffold version: v2.8.0
  • Operating system: macOS Ventura 13.2
  • Installed via: homebrew
  • Contents of skaffold.yaml:
apiVersion: skaffold/v4beta7
kind: Config
build:
  artifacts:
    - image: test
      docker:
        dockerfile: Dockerfile
  tagPolicy:
    inputDigest: {}
  local:
    push: false
    tryImportMissing: true
deploy:
  kubeContext: docker-desktop

Steps to reproduce the behavior

  1. Have 2 kubernetes contexts, one with amd64 and one with arm64
  2. Set your current context to the amd64 cluster
  3. Set your kubeContext to the arm64 cluster
  4. Run skaffold dev

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions