From 4cd702905a9ab91682d81eb481bf50c521082abf Mon Sep 17 00:00:00 2001 From: shahkv95 <35930270+shahkv95@users.noreply.github.com> Date: Sat, 25 Nov 2023 22:28:42 +0530 Subject: [PATCH 01/10] cert-manager installation using argocd Signed-off-by: shahkv95 <35930270+shahkv95@users.noreply.github.com> --- .../continuous-deployment-and-gitops.md | 143 ++++++++++++++++++ 1 file changed, 143 insertions(+) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index aaa3a07886c..de1d010c41f 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -112,3 +112,146 @@ Check the cert-manager logs for warnings and errors: ```bash kubectl logs -n cert-manager -l app.kubernetes.io/instance=cert-manager --prefix --all-containers ``` + + +## Using ArgoCD +Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. + +### Pre-requisites +Ensure, the following are in place before proceeding: +- A Kubernetes cluster +- ArgoCD deployed on the Kubernetes cluster: [installation guide](https://argo-cd.readthedocs.io/en/stable/getting_started/) +- Optional: A GitOps repository connected with ArgoCD: [setup guide](https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/) + +### Setting up cert-manager +1. Create an ArgoCD Application manifest file with the provided configuration to set up cert-manager. + + ```yaml + apiVersion: argoproj.io/v1alpha1 + kind: Application + metadata: + name: cert-manager + namespace: argocd + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + finalizers: + - resources-finalizer.argocd.argoproj.io + spec: + destination: + namespace: cert-manager + server: https://kubernetes.default.svc + project: default + source: + chart: cert-manager + repoURL: https://charts.jetstack.io + targetRevision: 1.10.1 + helm: + values: | + installCRDs: true + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=true + ``` +2. Commit the manifest file and sync the changes in ArgoCD. If a GitOps repository is not set up, use `kubectl apply -f ` to apply the manifest [installation guide for kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl). +3. ArgoCD will synchronize the Desired manifest and deploy cert-manager on Kubernetes based on the configuration provided. + + +### Troubleshooting + +#### Scenario 1: +Out-of-sync cert-manager in AKS(Azure Kubernetes Service) cluster + +##### Issue: +Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the Desired and Live manifest files. + +##### Potential Reasons +Multiple factors could cause the OutOfSync issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes. + +##### Example configuration differences +Below configurations are observed to be present in the Live manifest but not in the Desired manifest file. + +```yaml +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +... +webhooks: +- admissionReviewVersions: + namespaceSelector: + matchExpressions: + ... + ... + - key: control-plane + operator: NotIn + values: + - 'true' + - key: kubernetes.azure.com/managedby + operator: NotIn + values: + - aks +``` + +##### Root Cause Analysis +The discrepancy is rooted in how AKS manages admission controllers to protect internal services in the kube-system namespace. More details can be found [here](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces) + +##### Suggested Fix +It is also possible to ignore differences from fields owned by specific managers defined in `metadata.managedFields` in live resources. More details can be found [here](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#application-level-configuration) + +To resolve this issue, modify the cert-manager manifest file under spec to ignore specific differences: +``` +ignoreDifferences: + - group: admissionregistration.k8s.io + kind: ValidatingWebhookConfiguration + name: cert-manager-webhook + jqPathExpressions: + - .webhooks[].namespaceSelector.matchExpressions[] | select(.key == "control-plane") + - .webhooks[].namespaceSelector.matchExpressions[] | select(.key == "kubernetes.azure.com/managedby") +``` + +In that case, the updated cert-manager manifest would be as follows: + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: cert-manager + namespace: argocd + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + finalizers: + - resources-finalizer.argocd.argoproj.io +spec: + destination: + namespace: cert-manager + server: https://kubernetes.default.svc + project: default + source: + chart: cert-manager + repoURL: https://charts.jetstack.io + targetRevision: 1.10.1 + helm: + values: | + installCRDs: true + podLabels: + azure.workload.identity/use: "true" + serviceAccount: + labels: + azure.workload.identity/use: "true" + ignoreDifferences: + - group: admissionregistration.k8s.io + kind: ValidatingWebhookConfiguration + name: cert-manager-webhook + jqPathExpressions: + - .webhooks[].namespaceSelector.matchExpressions[] | select(.key == "control-plane") + - .webhooks[].namespaceSelector.matchExpressions[] | select(.key == "kubernetes.azure.com/managedby") + syncPolicy: + automated: + prune: true + selfHeal: true + syncOptions: + - CreateNamespace=true +``` + +Once ArgoCD syncs the updated manifest, the differences due to above 2 keys would be ignored and cert-manager would be in complete synchronization state. From adfe061058375c5433661e078d56143e80d25e94 Mon Sep 17 00:00:00 2001 From: shahkv95 <35930270+shahkv95@users.noreply.github.com> Date: Sat, 25 Nov 2023 22:36:13 +0530 Subject: [PATCH 02/10] removed unnecessary configurations Signed-off-by: shahkv95 <35930270+shahkv95@users.noreply.github.com> --- .../docs/installation/continuous-deployment-and-gitops.md | 5 ----- 1 file changed, 5 deletions(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index de1d010c41f..d67f5f1d1ac 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -234,11 +234,6 @@ spec: helm: values: | installCRDs: true - podLabels: - azure.workload.identity/use: "true" - serviceAccount: - labels: - azure.workload.identity/use: "true" ignoreDifferences: - group: admissionregistration.k8s.io kind: ValidatingWebhookConfiguration From ff060b8e4dd00bc7fb5ab58bb2518537dfb8cfb6 Mon Sep 17 00:00:00 2001 From: shahkv95 <35930270+shahkv95@users.noreply.github.com> Date: Sun, 26 Nov 2023 10:43:07 +0530 Subject: [PATCH 03/10] formatting updates for installing cert-manager via argocd Signed-off-by: shahkv95 <35930270+shahkv95@users.noreply.github.com> --- .../continuous-deployment-and-gitops.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index d67f5f1d1ac..5801c67fe2c 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -118,7 +118,7 @@ kubectl logs -n cert-manager -l app.kubernetes.io/instance=cert-manager --prefix Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. ### Pre-requisites -Ensure, the following are in place before proceeding: +Ensure the following are in place before proceeding: - A Kubernetes cluster - ArgoCD deployed on the Kubernetes cluster: [installation guide](https://argo-cd.readthedocs.io/en/stable/getting_started/) - Optional: A GitOps repository connected with ArgoCD: [setup guide](https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/) @@ -156,22 +156,22 @@ Ensure, the following are in place before proceeding: - CreateNamespace=true ``` 2. Commit the manifest file and sync the changes in ArgoCD. If a GitOps repository is not set up, use `kubectl apply -f ` to apply the manifest [installation guide for kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl). -3. ArgoCD will synchronize the Desired manifest and deploy cert-manager on Kubernetes based on the configuration provided. +3. ArgoCD will synchronize the `DESIRED MANIFEST` and deploy cert-manager on Kubernetes based on the provided configuration. ### Troubleshooting #### Scenario 1: -Out-of-sync cert-manager in AKS(Azure Kubernetes Service) cluster +Out-of-sync cert-manager in the AKS (Azure Kubernetes Service) cluster ##### Issue: -Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the Desired and Live manifest files. +Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. ##### Potential Reasons Multiple factors could cause the OutOfSync issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes. ##### Example configuration differences -Below configurations are observed to be present in the Live manifest but not in the Desired manifest file. +The below configurations are observed to be present in the `LIVE MANIFEST` but not in the `DESIRED MANIFEST` file. ```yaml apiVersion: admissionregistration.k8s.io/v1 @@ -194,10 +194,10 @@ webhooks: ``` ##### Root Cause Analysis -The discrepancy is rooted in how AKS manages admission controllers to protect internal services in the kube-system namespace. More details can be found [here](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces) +The discrepancy stems from how AKS manages admission controllers to protect internal services in the kube-system namespace. More details can be found in [Frequently Asked Questions about Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces) ##### Suggested Fix -It is also possible to ignore differences from fields owned by specific managers defined in `metadata.managedFields` in live resources. More details can be found [here](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#application-level-configuration) +It is also possible to ignore differences from fields owned by specific managers defined in `metadata.managedFields` in live resources. More details can be found in [(ArgoCD) Diffing Customization](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#application-level-configuration) To resolve this issue, modify the cert-manager manifest file under spec to ignore specific differences: ``` @@ -249,4 +249,4 @@ spec: - CreateNamespace=true ``` -Once ArgoCD syncs the updated manifest, the differences due to above 2 keys would be ignored and cert-manager would be in complete synchronization state. +Once ArgoCD syncs the updated manifest, the differences due to the above two keys will be ignored, and cert-manager will be in a complete synchronization state. From 115ae0bad58e61953601cdd620e24ebdefd57989 Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 16:50:01 +0000 Subject: [PATCH 04/10] Whitespace cleanup Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index 5801c67fe2c..6f703905586 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -164,7 +164,7 @@ Ensure the following are in place before proceeding: #### Scenario 1: Out-of-sync cert-manager in the AKS (Azure Kubernetes Service) cluster -##### Issue: +##### Issue: Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. ##### Potential Reasons From 9d47842e44b23f03c4de39ade1a3f11676e1ed8d Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 16:51:28 +0000 Subject: [PATCH 05/10] Fix spell check errors Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index 6f703905586..ccd2694a9a1 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -168,7 +168,7 @@ Out-of-sync cert-manager in the AKS (Azure Kubernetes Service) cluster Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. ##### Potential Reasons -Multiple factors could cause the OutOfSync issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes. +Multiple factors could cause the `OutOfSync` issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes. ##### Example configuration differences The below configurations are observed to be present in the `LIVE MANIFEST` but not in the `DESIRED MANIFEST` file. @@ -194,7 +194,7 @@ webhooks: ``` ##### Root Cause Analysis -The discrepancy stems from how AKS manages admission controllers to protect internal services in the kube-system namespace. More details can be found in [Frequently Asked Questions about Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces) +The discrepancy stems from how AKS manages admission controllers to protect internal services in the `kube-system` namespace. More details can be found in [Frequently Asked Questions about Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces) ##### Suggested Fix It is also possible to ignore differences from fields owned by specific managers defined in `metadata.managedFields` in live resources. More details can be found in [(ArgoCD) Diffing Customization](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#application-level-configuration) From 07aae2e042bb6b1094a095b677623c8148b4052a Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 16:59:56 +0000 Subject: [PATCH 06/10] Link to ArgoCD Application documentation Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index ccd2694a9a1..ceb8c2bad01 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -124,7 +124,7 @@ Ensure the following are in place before proceeding: - Optional: A GitOps repository connected with ArgoCD: [setup guide](https://argo-cd.readthedocs.io/en/stable/user-guide/private-repositories/) ### Setting up cert-manager -1. Create an ArgoCD Application manifest file with the provided configuration to set up cert-manager. +1. Create an [ArgoCD Application](https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications) manifest file with the provided configuration to set up cert-manager. ```yaml apiVersion: argoproj.io/v1alpha1 From 8b7f4aebeb30044d98029764994d77bd9c0c3215 Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 17:00:33 +0000 Subject: [PATCH 07/10] Suggest an application filename Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index ceb8c2bad01..d7b25f923de 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -127,6 +127,7 @@ Ensure the following are in place before proceeding: 1. Create an [ArgoCD Application](https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications) manifest file with the provided configuration to set up cert-manager. ```yaml + # application.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: @@ -155,7 +156,7 @@ Ensure the following are in place before proceeding: syncOptions: - CreateNamespace=true ``` -2. Commit the manifest file and sync the changes in ArgoCD. If a GitOps repository is not set up, use `kubectl apply -f ` to apply the manifest [installation guide for kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl). +2. Commit the manifest file and sync the changes in ArgoCD. If a GitOps repository is not set up, use `kubectl apply -f application.yaml` to apply the manifest [installation guide for kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl). 3. ArgoCD will synchronize the `DESIRED MANIFEST` and deploy cert-manager on Kubernetes based on the provided configuration. From 475248d3cfa59e719e6aa8f88d012f9c679dd50c Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 17:02:31 +0000 Subject: [PATCH 08/10] Use latest version in examples Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index d7b25f923de..980713400cc 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -145,7 +145,7 @@ Ensure the following are in place before proceeding: source: chart: cert-manager repoURL: https://charts.jetstack.io - targetRevision: 1.10.1 + targetRevision: [[VAR::cert_manager_latest_version]] helm: values: | installCRDs: true @@ -231,7 +231,7 @@ spec: source: chart: cert-manager repoURL: https://charts.jetstack.io - targetRevision: 1.10.1 + targetRevision: [[VAR::cert_manager_latest_version]] helm: values: | installCRDs: true From 756756c337291b4259c1e865d4aadd01a1fb5caa Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 17:05:03 +0000 Subject: [PATCH 09/10] Fix spelling of cert-manager Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index 980713400cc..475c1f9df95 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -166,7 +166,7 @@ Ensure the following are in place before proceeding: Out-of-sync cert-manager in the AKS (Azure Kubernetes Service) cluster ##### Issue: -Cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. +cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. ##### Potential Reasons Multiple factors could cause the `OutOfSync` issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes. From a59bfa74a945bdeb4e73390684057de7ba836cda Mon Sep 17 00:00:00 2001 From: Richard Wall Date: Thu, 14 Mar 2024 17:06:45 +0000 Subject: [PATCH 10/10] Consistent spelling of OutOfSync Signed-off-by: Richard Wall --- content/docs/installation/continuous-deployment-and-gitops.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/installation/continuous-deployment-and-gitops.md b/content/docs/installation/continuous-deployment-and-gitops.md index 475c1f9df95..3233dc68492 100644 --- a/content/docs/installation/continuous-deployment-and-gitops.md +++ b/content/docs/installation/continuous-deployment-and-gitops.md @@ -163,10 +163,10 @@ Ensure the following are in place before proceeding: ### Troubleshooting #### Scenario 1: -Out-of-sync cert-manager in the AKS (Azure Kubernetes Service) cluster +`OutOfSync` cert-manager in the AKS (Azure Kubernetes Service) cluster ##### Issue: -cert-manager in the AKS cluster remains out-of-sync due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. +cert-manager in the AKS cluster remains `OutOfSync` due to discrepancies between the `DESIRED MANIFEST` and `LIVE MANIFEST` files. ##### Potential Reasons Multiple factors could cause the `OutOfSync` issue; refer to [ArgoCD documentation](https://argo-cd.readthedocs.io/en/stable/user-guide/diffing/#diffing-customization) for potential causes.