Kubernetest 1.24 Compatibility

[eugenebnd]

Hello,

The operator does not work with kubernetes 1.24

The error is:
2022-08-30T20:02:58.968Z ERROR controller Reconciler error {"reconcilerGroup": "maupu.org", "reconcilerKind": "VaultSecret", "controller": "vaultsecret", "name": "external-dns", "namespace": "external-dns-system", "error": "No secret associated with the service account external-dns-system/default"}

It does look like it's happening since CHANGELOG
Secret API objects containing service account tokens are no longer auto-generated for every ServiceAccount

Affected code is here: resources.go

[nmaupu]

Hello,

Nice catch, I wasn't aware of this change in 1.24. However, I am not sure yet how to handle that to be retro-compatible with older k8s versions 🤔.

[nmaupu]

Can you still create an associated secret manually to get this working ?

[primeroz]

It does seem that you can create the secret indeed https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets

apiVersion: v1
kind: Secret
metadata:
  name: secret-sa-sample
  annotations:
    kubernetes.io/service-account.name: "sa-name"
type: kubernetes.io/service-account-token

should then get populated with the token for that SA by kubernetes

I think that the safe approach is to expect people to create this for now but somehow move the controller to use the TokenRequest API to fetch a token for a given SA ?
Not really sure though what the requirments are for the controller to be able to call the TokenRequest API on behalf of another SA are

[nmaupu]

Yes, that's the way to go. It should only be a matter of RBAC I suppose.

[primeroz]

My understandingis that the Secret creation for token is not deprecated just not automatic anymore so it is not super urgent to move to TokenRequest API

Since it seems that from 1.22+ this should be available maybe when the change is ready you can just cut a major version that is only compatible to 1.22+ and not worry too much about backward compatibility ?
This is the way of kubernetes after all :)

[nmaupu]

Yes, good idea indeed !

[sadovnikov]

Hi @primeroz, @nmaupu! An addition to the work-around above.

I got the secret created and populated by ca.crt and token. However, the vault operator still fails with "No secret associated with the service account argocd/default", also after restart.

Seems it's coming from this code:

vault-secret/pkg/k8sutils/resources.go

Lines 36 to 44 in 910b6a4

	saClient := &corev1.ServiceAccount{}
	err := cli.Get(context.TODO(), types.NamespacedName{Name: saName, Namespace: ns}, saClient)
	if err != nil && errors.IsNotFound(err) {
	return "", fmt.Errorf("Unable to retrieve service account, err=%v", err)
	}
	if len(saClient.Secrets) == 0 {
	return "", fmt.Errorf("No secret associated with the service account %s/%s", ns, saName)
	}

Although, the kubernetes.io/docs does not require this, I tried to link the created secret to the SA through secrets.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: xxxxxx
secrets:
- name: sa-default-token

Only after that, the Operator pod is able to discover the secret