Invalid ClusterRole definition leads to error during installation and creation of the bootstrap secret

[huguesalary]

Versions

Client Version: v1.31.2
Kustomize Version: v5.4.2
Server Version: v1.31.5-gke.1023000

What happened?

An error while installing Pomerium with kubectl apply -k github.com/pomerium/ingress-controller/config/default\?ref=v0.27.2

Specifically, the Pod created by the job/pomerium-gen-secrets, fails with the following error:

Error: check existing secret: secrets "bootstrap" is forbidden: User "system:serviceaccount:pomerium:pomerium-gen-secrets" cannot get resource "secrets" in API group "" in the namespace "pomerium"
2025/02/20 01:14:49 check existing secret: secrets "bootstrap" is forbidden: User "system:serviceaccount:pomerium:pomerium-gen-secrets" cannot get resource "secrets" in API group "" in the namespace "pomerium"

What's the issue?

After investigating, it appears there are 2 issues contributing to this.

First, all tagged versions < v0.28.0 have the job/pomerium-gen-secrets spec.template.spec.container[0].image set to image: pomerium/ingress-controller:main instead of image: pomerium/ingress-controller:vx.y.z.

New behavior of pomerium-gen-secrets introduced on the main branch is not compatible with the Kubernetes configuration of tagged versions < v0.28.0. This is what the 2nd issue is. See below.

The clusterrole/pomerium-gen-secrets definition below is invalid for pomerium-gen-secrets >= v0.28.0 :

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/name: pomerium
  name: pomerium-gen-secrets
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create

The verbs: section is missing - get.

Since the process in the job/pomerium-gen-secrets is trying to list secrets, the proper definition should be:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/name: pomerium
  name: pomerium-gen-secrets
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - create
  - get        # <-------- missing get added here

What else?

This bug was apparently fixed yesterday e6e6c31#diff-fb5bfb98f469e2804ddb6212d6df46df13c44529fd68ede6a448fe65d7678b9d

However, anyone installing v0.28.0 and any older version is probably experiencing this issue. The job/pomerium-gen-secrets in versions <= v0.28.0 should be patched to use the proper image:

[jof]

🕥 👀 🤞🏽