Separate Helm generator from core?

[metacosm]

Helm generation should perhaps be dealt with in a way similar to the OLM generation, via a separate extension module.

[mfvitale]

@metacosm Do you think is possible to evaluate the usage of https://docs.quarkiverse.io/quarkus-helm/dev/index.html for the helm generation?

[metacosm]

I think as far as I recall we did evaluate it but the issue was that that extension bundles the operator as a regular app without taking the operator-specific needs into account. But it might indeed be worth revisiting. Is that something you'd be interested in exploring?

[mfvitale]

I did some tests to see the differences and there are many. So lot of the effort will be to test and compare the differences to see if, with a certain set of configurations, we can get a helm chart as much as possible to the current one.

Unfortunately, I have no time, as of now, to work on it. I have opened an enhancement fromthe Debezium perspective to try this. I'll keep you posted about the results.

[metacosm]

Thank you!

[mfvitale]

Hi @metacosm I did an analysis. These are the main differences.

--- a/helm-comparison/qosdk/helm/templates/service.yaml
+++ b/helm-comparison/helm-ext/helm/templates/service.yaml
@@ -1,11 +1,13 @@
+---
 apiVersion: v1
 kind: Service
 metadata:
+  annotations:
+    app.quarkus.io/quarkus-version: 3.17.0
   labels:
-    app.kubernetes.io/name: {{ .Chart.Name }}
-    app.kubernetes.io/version: {{ .Chart.AppVersion }}
+    app.kubernetes.io/name: debezium-operator
     app.kubernetes.io/managed-by: quarkus
-  name: {{ .Chart.Name }}
+  name: {{ .Values.app.name }}
 spec:
   ports:
     - name: http
@@ -13,6 +15,5 @@ spec:
       protocol: TCP
       targetPort: 8080
   selector:
-    app.kubernetes.io/name: {{ .Chart.Name }}
-    app.kubernetes.io/version: {{ .Chart.AppVersion }}
+    app.kubernetes.io/name: debezium-operator
   type: ClusterIP

--- a/helm-comparison/qosdk/helm/templates/serviceaccount.yaml
+++ b/helm-comparison/helm-ext/helm/templates/serviceaccount.yaml
@@ -1,8 +1,10 @@
+---
 apiVersion: v1
 kind: ServiceAccount
 metadata:
+  annotations:
+    app.quarkus.io/quarkus-version: 3.17.0
   labels:
+    app.kubernetes.io/name: debezium-operator
     app.kubernetes.io/managed-by: quarkus
-    app.kubernetes.io/name: {{ .Chart.Name }}
-    app.kubernetes.io/version: {{ .Chart.AppVersion }}
-  name: {{ .Chart.Name }}
+  name: {{ .Values.app.name }}

--- a/helm-comparison/qosdk/helm/templates/deployment.yaml
+++ b/helm-comparison/helm-ext/helm/templates/deployment.yaml
@@ -1,69 +1,67 @@
 ---
-apiVersion: "apps/v1"
-kind: "Deployment"
+apiVersion: apps/v1
+kind: Deployment
 metadata:
   annotations:
-    app.quarkus.io/quarkus-version: "3.17.0"
+    app.quarkus.io/quarkus-version: 3.17.0
   labels:
-    app.kubernetes.io/name: "debezium-operator"
-    app.kubernetes.io/managed-by: "quarkus"
-  name: "debezium-operator"
+    app.kubernetes.io/name: debezium-operator
+    app.kubernetes.io/managed-by: quarkus
+  name: {{ .Values.app.name }}
:
-kind: "Deployment"
+apiVersion: apps/v1
+kind: Deployment
 metadata:
   annotations:
-    app.quarkus.io/quarkus-version: "3.17.0"
+    app.quarkus.io/quarkus-version: 3.17.0
   labels:
-    app.kubernetes.io/name: "debezium-operator"
-    app.kubernetes.io/managed-by: "quarkus"
-  name: "debezium-operator"
+    app.kubernetes.io/name: debezium-operator
+    app.kubernetes.io/managed-by: quarkus
+  name: {{ .Values.app.name }}
 spec:
   replicas: 1
   selector:
     matchLabels:
-      app.kubernetes.io/name: "debezium-operator"
+      app.kubernetes.io/name: debezium-operator
   template:
     metadata:
       annotations:
-        app.quarkus.io/quarkus-version: "3.17.0"
+        app.quarkus.io/quarkus-version: 3.17.0
       labels:
-        app.kubernetes.io/managed-by: "quarkus"
-        app.kubernetes.io/name: "debezium-operator"
+        app.kubernetes.io/managed-by: quarkus
+        app.kubernetes.io/name: debezium-operator
     spec:
       containers:
-      - env:
-        - name: "KUBERNETES_NAMESPACE"
-          valueFrom:
-            fieldRef:
-              fieldPath: "metadata.namespace"
-        - name: "QUARKUS_OPERATOR_SDK_CONTROLLERS_DEBEZIUMSERVER_NAMESPACES"
-          value: {{ .Values.watchNamespaces }}
-        image: {{ .Values.image }}
-        imagePullPolicy: "Always"
-        livenessProbe:
-          failureThreshold: 3
-          httpGet:
-            path: "/q/health/live"
-            port: 8080
-            scheme: "HTTP"
-          initialDelaySeconds: 5
-          periodSeconds: 10
-          successThreshold: 1
-          timeoutSeconds: 10
-        name: "debezium-operator"
-        ports:
-        - containerPort: 8080
-          name: "http"
-          protocol: "TCP"
-        readinessProbe:
-          failureThreshold: 3
-          httpGet:
-            path: "/q/health/ready"
-            port: 8080
-            scheme: "HTTP"
-          initialDelaySeconds: 5
-          periodSeconds: 10
-          successThreshold: 1
-          timeoutSeconds: 10
-        startupProbe:
-          failureThreshold: 3
-          httpGet:
-            path: "/q/health/started"
-            port: 8080
-            scheme: "HTTP"
-          initialDelaySeconds: 5
-          periodSeconds: 10
-          successThreshold: 1
-          timeoutSeconds: 10
-      serviceAccountName: "debezium-operator"
+        - env:
+            - name: KUBERNETES_NAMESPACE
+              valueFrom:
+                fieldRef:
+                  fieldPath: metadata.namespace
+          image: {{ .Values.app.image }}
+          imagePullPolicy: Always
+          livenessProbe:
+            failureThreshold: {{ .Values.app.livenessProbe.failureThreshold }}
+            httpGet:
+              path: {{ .Values.app.livenessProbe.httpGet.path }}
+              port: {{ .Values.app.ports.http }}
+              scheme: {{ .Values.app.livenessProbe.httpGet.scheme }}
+            initialDelaySeconds: {{ .Values.app.livenessProbe.initialDelaySeconds }}
+            periodSeconds: {{ .Values.app.livenessProbe.periodSeconds }}
+            successThreshold: {{ .Values.app.livenessProbe.successThreshold }}
+            timeoutSeconds: {{ .Values.app.livenessProbe.timeoutSeconds }}
+          name: debezium-operator
+          ports:
+            - containerPort: 8080
+              name: http
+              protocol: TCP
+          readinessProbe:
+            failureThreshold: {{ .Values.app.readinessProbe.failureThreshold }}
+            httpGet:
+              path: {{ .Values.app.readinessProbe.httpGet.path }}
+              port: {{ .Values.app.ports.http }}
+              scheme: {{ .Values.app.readinessProbe.httpGet.scheme }}
+            initialDelaySeconds: {{ .Values.app.readinessProbe.initialDelaySeconds }}
+            periodSeconds: {{ .Values.app.readinessProbe.periodSeconds }}
+            successThreshold: {{ .Values.app.readinessProbe.successThreshold }}
+            timeoutSeconds: {{ .Values.app.readinessProbe.timeoutSeconds }}
+          startupProbe:
+            failureThreshold: {{ .Values.app.startupProbe.failureThreshold }}
+            httpGet:
+              path: {{ .Values.app.startupProbe.httpGet.path }}
+              port: {{ .Values.app.ports.http }}
+              scheme: {{ .Values.app.startupProbe.httpGet.scheme }}
+            initialDelaySeconds: {{ .Values.app.startupProbe.initialDelaySeconds }}
+            periodSeconds: {{ .Values.app.startupProbe.periodSeconds }}
+            successThreshold: {{ .Values.app.startupProbe.successThreshold }}
+            timeoutSeconds: {{ .Values.app.startupProbe.timeoutSeconds }}
+      serviceAccountName: debezium-operator

--- a/helm-comparison/qosdk/helm/templates/debeziumserver-crd-cluster-role.yaml
+++ b/helm-comparison/helm-ext/helm/templates/clusterrole.yaml
@@ -1,112 +1,112 @@
 ---
-apiVersion: "rbac.authorization.k8s.io/v1"
-kind: "ClusterRole"
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
 metadata:
-  name: "debeziumserver-cluster-role"
+  name: {{ .Values.app.name }}
 rules:
-- apiGroups:
-  - "debezium.io"
-  resources:
-  - "debeziumservers"
-  - "debeziumservers/status"
-  - "debeziumservers/finalizers"
-  verbs:
-  - "get"
-  - "list"
-  - "watch"
-  - "patch"
-  - "update"
-  - "create"
-  - "delete"
-- apiGroups:
-  - ""
-  resources:
-  - "configmaps"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - "rbac.authorization.k8s.io"
-  resources:
-  - "rolebindings"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - ""
-  resources:
-  - "services"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - "apps"
-  resources:
-  - "deployments"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - ""
-  resources:
-  - "persistentvolumeclaims"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - ""
-  resources:
-  - "serviceaccounts"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - "rbac.authorization.k8s.io"
-  resources:
-  - "roles"
-  verbs:
-  - "create"
-  - "delete"
-  - "get"
-  - "list"
-  - "patch"
-  - "update"
-  - "watch"
-- apiGroups:
-  - ""
-  resources:
-  - "secrets"
-  verbs:
-  - "get"
-  - "list"
-  - "watch"
+  - apiGroups:
+      - debezium.io
+    resources:
+      - debeziumservers
+      - debeziumservers/status
+      - debeziumservers/finalizers
+    verbs:
+      - get
+      - list
+      - watch
+      - patch
+      - update
+      - create
+      - delete
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - rbac.authorization.k8s.io
+    resources:
+      - rolebindings
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+      - services
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - apps
+    resources:
+      - deployments
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+      - persistentvolumeclaims
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+      - serviceaccounts
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - rbac.authorization.k8s.io
+    resources:
+      - roles
+    verbs:
+      - create
+      - delete
+      - get
+      - list
+      - patch
+      - update
+      - watch
+  - apiGroups:
+      - ""
+    resources:
+      - secrets
+    verbs:
+      - get
+      - list
+      - watch

--- a/helm-comparison/qosdk/helm/templates/debeziumserver-crd-role-binding.yaml
+++ b/helm-comparison/helm-ext/helm/templates/rolebinding.yaml
@@ -1,57 +1,27 @@
-{{ if eq $.Values.watchNamespaces "JOSDK_WATCH_CURRENT" }}
+---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
 metadata:
-  name: debeziumserver-role-binding
-  namespace: {{ $.Release.Namespace }}
-  labels:
-    app.kubernetes.io/name: {{ .Chart.Name }}
-    app.kubernetes.io/version: {{ .Chart.AppVersion }}
-    app.kubernetes.io/managed-by: quarkus
-roleRef:
-  kind: ClusterRole
-  apiGroup: rbac.authorization.k8s.io
-  name: debeziumserver-cluster-role
-subjects:
-  - kind: ServiceAccount
-    name: {{ $.Chart.Name }}
-    namespace: {{ $.Release.Namespace }}
-{{ else if eq $.Values.watchNamespaces "JOSDK_ALL_NAMESPACES" }}
-apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRoleBinding
-metadata:
-  name: debeziumserver-role-binding
-  labels:
-    app.kubernetes.io/name: {{ .Chart.Name }}
-    app.kubernetes.io/version: {{ .Chart.AppVersion }}
-    app.kubernetes.io/managed-by: quarkus
+  name: {{ .Values.app.name }}
 roleRef:
   kind: ClusterRole
   apiGroup: rbac.authorization.k8s.io
   name: debeziumserver-cluster-role
 subjects:
   - kind: ServiceAccount
-    name: {{ $.Chart.Name }}
-    namespace: {{ $.Release.Namespace }}
-{{ else }}
-{{ range $anamespace := ( split "," $.Values.watchNamespaces ) }}
+    name: debezium-operator
+---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: RoleBinding
 metadata:
-  name: debeziumserver-role-binding
-  namespace: {{ $anamespace }}
   labels:
-    app.kubernetes.io/name: {{ $.Chart.Name }}
-    app.kubernetes.io/version: {{ $.Chart.AppVersion }}
-    app.kubernetes.io/managed-by: quarkus
+    app.kubernetes.io/name: debezium-operator
+    app.kubernetes.io/version: 3.2.0-nightly
+  name: {{ .Values.app.name }}
 roleRef:
   kind: ClusterRole
   apiGroup: rbac.authorization.k8s.io
-  name: debeziumserver-cluster-role
+  name: view
 subjects:
   - kind: ServiceAccount
-    name: {{ $.Chart.Name }}
-    namespace: {{ $.Release.Namespace }}
----
-{{- end }}
-{{- end }}
+    name: debezium-operator

Apart from these differences there are some (expected) file missing:

• generic-crd-cluster-role.yaml
• generic-crd-cluster-role-binding.yaml
• additional-crd-role-binding.yaml

With some tweaks on the quakus-helm extension configurations some of the difference can be resolved.

But the problem of missing file and mostly the rolebinding, this is the more critical since there is some if guided by watchNamespaces, are the ones that I think is not possible to solve.

I don't know if using the quarkus-helm for the basics and the adding to the generated file the missing ones could solve the problem but I don't have to much knowledge on the qosdk. Do you think that kind of approach would be feasible?

Our requirement is to have the possibility to ovverride some settings with the values.yaml. A basic requiriment for example is to add custom labels/annotations to the resources.

[metacosm]

Maybe we could try to see if the quarkus-helm extension could itself support extensions? Like maybe it could pick up implementations of a given interface that extension users could provide to augment the default generation?

[mfvitale]

There is no way to do this. As of now there is the possibility to add external files but it support a limited set.

The only way I think is to use the quakus-helm to generate the basics and then "attach" after its process to add the bits for the operator. Dunno if it is feasible to have an execution order of extension so that the quarkus-helm is executed first and then the QOSDK can add the operator bits.

[metacosm]

What I meant is that maybe we could modify the Helm extension to support that feature.

[mfvitale]

What I meant is that maybe we could modify the Helm extension to support that feature.

Yeah, this could be an option. But this will require a different effort.

[metacosm]

What I meant is that maybe we could modify the Helm extension to support that feature.

Yeah, this could be an option. But this will require a different effort.

To be honest, I don't know. It might be easier to add proper support in the Helm extension than figure out how to make things work in a way that would necessarily be more fragile…

[mfvitale]

But this will require a different effort.

This is also valid for any other options.

I was hoping that in some way, it could have been possible with just using the quarkus-helm extension with some config tweaks, unfortunately, this is not the case.

Maybe we could try to see if the quarkus-helm extension could itself support extensions? Like maybe it could pick up implementations of a given interface that extension users could provide to augment the default generation?

@Sgitario, what is your opinion on this?

[metacosm]

@xstefank would that be something you'd be interested to look into?

[Sgitario]

Can you clarify what is your intention and/or use case? I don't understand the statement about supporting extensions.

About the quarkus-helm extension:

• for adding existing resources, you need to add them via the Kubernetes extension: https://quarkus.io/guides/deploying-to-kubernetes#using-existing-resources
• then, you can manipulate these resources by replacing parts with Helm expressions: https://docs.quarkiverse.io/quarkus-helm/dev/index.html#helm-expressions
• for Helm resources, you can use the input-directory: https://docs.quarkiverse.io/quarkus-helm/dev/index.html#quarkus-helm_quarkus-helm-input-directory

If what you want is to run the operator sdk after the chart manifests is generated, then you need to add the class HelmChartBuildItem in your buildstep.

[metacosm]

Yes, that's what I was starting to look at (the HelmChartBuildItem option)… Thanks for the details, @Sgitario!