Support running in namespaced mode

[alexymantha]

By default, the manager runs with cluster scope and required cluster-scoped access to secrets. Changing the ClusterRole(Binding) to their namespaced equivalent will result in:

E0509 15:21:40.162416       1 reflector.go:166] "Unhandled Error" err="pkg/mod/k8s.io/client-go@v0.32.3/tools/cache/reflector.go:251: Failed to watch *v1.Secret: failed to list *v1.Secret: secrets is forbidden: User \"system:serviceaccount:argocd-dois:argocd-promoter-controller-manager\" cannot list resource \"secrets\" in API group \"\" at the cluster scope" logger="UnhandledError"

In our environment, I am not allowed to deploy a ClusterRole that can list all secrets. Instead, namespace owners need to opt-in by creating a namespaced role in their namespace with a binding to allow us to read the secrets. This means that I need to be able to specifically target the namespaces that should be watched.

Proposal

1. Support running the manager in namespaced mode, where a list of namespaces that should be watched can be provided. For example, a --watch-namespaces flag.
   • It should be pretty easy to do, see the examples here: https://book.kubebuilder.io/cronjob-tutorial/empty-main.html and here https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/
2. Provide a namespaced version of https://github.com/argoproj-labs/gitops-promoter/blob/main/dist/install.yaml that installs the promoter in namespaced mode and replaces the ClusterRoles and ClusterRoleBindings with their Role and RoleBinding equivalents.

[zachaller]

I think instead of using default controller-runtime for the watch namespace feature I would be curious what an implementation looks like using https://github.com/kubernetes-sigs/multicluster-runtime namespace provider feature