Feature request: Watch for PolicyServers .spec.imagePullSecret changes to trigger rollout

[viccuad]

From Slack discussion.

Currently, we only support configuring policy-servers to pull via an imagePullSecret that is configured via dockerconfigjson.

This may fall short with auth workflows that expire and rotate the token. For example, AWS IAM using IRSA or PodIdentity, where the temporary token auth expires and needs to be refreshed.

While adding support for AWS IAM using IRSA may fall outside of this card, adding a way to automatically consume the refreshed tokens may simplify our users life.

Acceptance criteria

Add a Watch call to watch for Secret changes in the same namespace as the PolicyServer. If one of those is being used as the .spec.imagePullSecret and is updated, trigger a rollout of the resulting policy-server Deployment.

[flavio]

The credentials used to pull from a registry are injected into Polic yServer as a Kubernetes Secret. When the secret is updated, kubelet will automatically propagate the changes to the filesystem of the Pod running Policy Server.

The question is: do we read the container registry once, at boot time, or do we read them from the filesystem every time? If we read them from the filesystem every time, then there's nothing we have to do; we should be all set.

Admission Criteria

We have to do a quick investigation to understand if we read credentials on each pull OR if we read them once.