Kubernetes sanitycheck doesn't work

[Ben8t]

Describe the issue

@fdelbrayelle I don't think the sanity-check is an easy one and can be independant right?

It's failing on preview envs.:

Can you fix? If not possible, please remove the sanitycheck

Environment

• Kestra Version: develop

[fdelbrayelle]

@Ben8t I tried to launch a kind create cluster inside a Docker Run task, it works (the cluster is created) BUT the second task doesn't work because of a certificate issue.

I also tried to escalate with privileged: true to go further (see the YAML sanity check flow below - direct link on preview env) but again it fails because the preview env is protected from right escalation (which is safer!):

In the end the sanity check doesn't have any meaning if we cannot execute it on the preview env, so I think I'll delete the sanity check. In the meantime, I'll check in a last resort if we could access to the preview K8s cluster.

id: all_kubectl_test
namespace: sanitychecks.plugin-scripts.all_kubectl

tasks:
  - id: create_kubernetes_cluster
    type: io.kestra.plugin.docker.Run
    containerImage: kindest/node:v1.30.0
    privileged: true
    commands:
      - |
        set -e
        CLUSTER_NAME="kestra-kind-cluster"
        KUBECONFIG_PATH="/workingdir/kubeconfig_output.yaml"

        echo "Creating KIND cluster: ${CLUSTER_NAME}"
        kind create cluster --name ${CLUSTER_NAME}
        kind get kubeconfig --name ${CLUSTER_NAME} > ${KUBECONFIG_PATH}
        echo "Kubeconfig generated at ${KUBECONFIG_PATH}"

        echo "Extracting connection parameters from kubeconfig..."
        # Using yq (ensure it's v4+ syntax for 'eval', or adjust for older yq)
        # Server URL
        yq eval '.clusters[0].cluster.server' ${KUBECONFIG_PATH} > /workingdir/server_url.txt
        # CA Cert Data (already base64 encoded in kubeconfig)
        yq eval '.clusters[0].cluster."certificate-authority-data"' ${KUBECONFIG_PATH} > /workingdir/ca_cert_data.txt
        # Client Cert Data (already base64 encoded)
        yq eval '.users[0].user."client-certificate-data"' ${KUBECONFIG_PATH} > /workingdir/client_cert_data.txt
        # Client Key Data (already base64 encoded)
        yq eval '.users[0].user."client-key-data"' ${KUBECONFIG_PATH} > /workingdir/client_key_data.txt

        echo "Connection parameters extracted."
    outputFiles:
      - kubeconfig_output.yaml
      - server_url.txt
      - ca_cert_data.txt
      - client_cert_data.txt
      - client_key_data.txt

  - id: apply
    type: io.kestra.plugin.kubernetes.kubectl.Apply
    namespace: default
    connection:
      masterUrl: "{{ fileread(outputs.create_kubernetes_cluster.uris['server_url.txt']) | trim }}"
      caCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['ca_cert_data.txt']) | trim }}"
      clientCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_cert_data.txt']) | trim }}"
      clientKeyData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_key_data.txt']) | trim }}"
      trustCerts: true
    spec: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: my-deployment
        labels:
          app: myapp
      spec:
        replicas: 3
        selector:
          matchLabels:
            app: myapp
        template:
          metadata:
            labels:
              app: myapp
          spec:
            containers:
            - name: mycontainer
              image: nginx:latest
              ports:
              - containerPort: 80

  - id: get
    type: io.kestra.plugin.kubernetes.kubectl.Get
    namespace: default
    connection:
      masterUrl: "{{ fileread(outputs.create_kubernetes_cluster.uris['server_url.txt']) | trim }}"
      caCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['ca_cert_data.txt']) | trim }}"
      clientCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_cert_data.txt']) | trim }}"
      clientKeyData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_key_data.txt']) | trim }}"
      trustCerts: true
    resourceType: deployments
    resourcesNames:
      - my-deployment
    fetchType: FETCH_ONE

  - id: delete
    type: io.kestra.plugin.kubernetes.kubectl.Delete
    namespace: default
    connection:
      masterUrl: "{{ fileread(outputs.create_kubernetes_cluster.uris['server_url.txt']) | trim }}"
      caCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['ca_cert_data.txt']) | trim }}"
      clientCertData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_cert_data.txt']) | trim }}"
      clientKeyData: "{{ fileread(outputs.create_kubernetes_cluster.uris['client_key_data.txt']) | trim }}"
      trustCerts: true
    resourceType: deployments
    resourcesNames:
      - my-deployment