DOCS-1273: Operator 6.0.0 Deploy/Upgrade docs, removing Console references #1284

ravindk89 · 2024-07-25T00:26:21Z

Addresses #1273

Summary

This pass does three things:

Updates all tutorials related to Operator/Tenant deployment for Kustomize and Helm
Removes references to Operator Console + updates to reference Kustomize/Helm wherever possible
Slightly tidies up old or dangling references

This pass does not do these things:

Link out heavily to Kubernetes docs (for later)
Clean up organization (singleplat build handles this)
Addresses OpenShift, Rancher, etc.

feorlen · 2024-07-25T11:34:02Z

Staged: http://192.241.195.202:9000/staging/DOCS-1273/k8s/index.html

feorlen · 2024-07-25T11:49:04Z

source/includes/k8s/deploy-operator.rst

-
-For Kubernetes 1.25.0 and later, MinIO supports deploying in environments with the :kube-docs:`Pod Security admission (PSA) <concepts/security/pod-security-admission>` ``restricted`` policy enabled.
-
+Starting with v5.0.0, MinIO **requires** Kubernetes 1.28.0 or later for both the infrastructure and the ``kubectl`` CLI tool.


Which release pushed the minimum required k8s to 1.28? Or was 1.21 incorrect already for 5.0.x?

1.21 is EOL and has been for a while. I think we just never got around to updating this.

1.28 will be EOL this year, but is at least still in-support (for now).

Yes, but EOL is different than our code requiring something that's in a specific version of K8s, as was the case with 5.0.0 and K8s 1.21. You cannot run Operator 5.0.0 on something older than 1.21. It doesn't work.

Does 6.0.0 require 1.28 because of something that's only in 1.28 and newer? Or is it just the "please don't use unsupported, old versions of software?"

From internal convos it sounds like even if we do technically support older K8s, we want to push users to always stay on latest stable.

I'm going to leave this for now and then see whether there is a way for us to pull this out of the code if we really want to keep a backstop in place.

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

feorlen · 2024-07-25T12:01:33Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

+      * - ``tenant.certificate.requestAutoCert``
+        - Enables or disables MinIO :ref:`automatic TLS certificate generation <minio-tls>`


are the accepted values here true or false?

Suggested change

* - ``tenant.certificate.requestAutoCert``

- Enables or disables MinIO :ref:`automatic TLS certificate generation <minio-tls>`

* - ``tenant.certificate.requestAutoCert``

- Enable or disable MinIO :ref:`automatic TLS certificate generation <minio-tls>`

Yes @feorlen

@ravindk89 @feorlen Are we using the imperative mood or simply describing the field (indicative mood)? It's not clear what the standard is.

My intent was to be imperative (I think), but not necessarily because it's our standard - it's just how I learned to write these sort of things.

In which case i think we'd leave this as-is. Maybe for docs to discuss.

The step (numbered line at ln144) is imperative. These are all descriptions, so should be indicative. I'd leave 156 as is to match the other field descriptions. Or you'll need to change all of the following descriptions as well.

I changed everything and lets see how we feel about it.

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

feorlen · 2024-07-25T12:20:49Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

+      * - ``tenant.certificate.certConfig``
+        - Controls the settings for :ref:`automatic TLS <minio-tls>`.
+          Requires ``spec.requestAutoCert: true``


"Controls" is a bit vague, is it worth mentioning the type of accepted values here? There is more info in the CRD, which is reasonable to presume the reader is looking at.

Also spec.requestAutoCert can be true, false, or simply not present. It is not a required field.

@allanrogerr iirc if `requestAutoCertisfalse`` then changes to this field do nothing, right?

Indeed omitting requestAutoCert defaults to true behavior so I could reverse this

Suggested change

* - ``tenant.certificate.certConfig``

- Controls the settings for :ref:`automatic TLS <minio-tls>`.

Requires ``spec.requestAutoCert: true``

* - ``tenant.certificate.certConfig``

- Controls the settings for :ref:`automatic TLS <minio-tls>`.

Only has affect if ``spec.requestAutoCert`` is not false.

Can we say

Suggested change

* - ``tenant.certificate.certConfig``

- Controls the settings for :ref:`automatic TLS <minio-tls>`.

Requires ``spec.requestAutoCert: true``

* - ``tenant.certificate.certConfig``

- Controls the settings for :ref:`automatic TLS <minio-tls>`.

Only has an effect if ``spec.requestAutoCert`` is set to ``true``.

Say that it must be "not false" sounds really awkward.

The easiest path here is to just state 'controls he settings...if enabled` and let the user infer that disabling one thing disables the othe.r

feorlen · 2024-07-25T13:26:19Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

@@ -182,8 +246,78 @@ This method may support easier pre-configuration of the Tenant compared to the :

   Each chart contains a ``values.yaml`` file you can customize to suit your needs.
   For details on the options available in the MinIO Tenant ``values.yaml``, see :ref:`minio-tenant-chart-values`.


Is this link still needed if we are now actually discussing the contents of values.yaml here?

I think so - values.yaml has a lot more gritty detail, this tutorial (for now) is very high level.

I'd suggest keeping the separate chart file to cover all of the options. Then the tutorials can focus on just the basics of getting something up and running.

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

feorlen · 2024-07-25T13:29:26Z

source/operations/install-deploy-manage/deploy-minio-tenant.rst


-:ref:`create-tenant-access-minio-operator-console`
+You can select a different base or pre-built template from the repository as your starting point, or build your own Kustomization resources using the MinIO Custom Resource Documentation.


Is there a good link for the DIY option?

Yeah, will do that in this pass

source/operations/install-deploy-manage/upgrade-minio-operator.rst

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

allanrogerr · 2024-07-25T17:41:28Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

-   Access the Tenant's :ref:`minio-console` by navigating to ``http://localhost:9443`` in a browser.
-   Log in to the Console with the default credentials ``myminio | minio123``.
-   If you modified these credentials in the ``values.yaml`` specify those values instead.
+      watch kubectl get all -n minio-tenant


Suggested change

watch kubectl get all -n minio-tenant

watch kubectl get all -n minio-tenant-1

Not sure if you want to use MINIO_TENANT_NAMESPACE and MINIO_TENANT_NAME nomenclature, but we should be consistent.

allanrogerr · 2024-07-25T17:45:39Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

-      To configure long term access to the pod, configure :kube-docs:`Ingress <concepts/services-networking/ingress/>` or similar network control components within Kubernetes to route traffic to and from the pod.
-      Configuring Ingress is out of the scope for this documentation.
+      helm install \
+        --namespace minio-tenant-1 \


Not sure if you want to use MINIO_TENANT_NAMESPACE and MINIO_TENANT_NAME nomenclature, but we should be consistent.

Hm. Yeah we kind of go back and forth on this. I'll migrate to the var-replace style.

source/operations/install-deploy-manage/upgrade-minio-operator-4.5.7-earlier.rst

…ences

Co-authored-by: Andrea Longo <[email protected]> Co-authored-by: Allan Roger Reid <[email protected]>

djwfyi

Do you have to be on 5.0.15 to upgrade to Operator 6.0.x?
The upgrade docs make that a little unclear.

Lots of suggestions here and there for clarity and whatnot, but otherwise what a lift to get this done! Nice work.

source/includes/common/common-install-operator-kustomize.rst

djwfyi · 2024-07-25T16:58:30Z

source/includes/k8s/deploy-operator.rst

-
-For Kubernetes 1.25.0 and later, MinIO supports deploying in environments with the :kube-docs:`Pod Security admission (PSA) <concepts/security/pod-security-admission>` ``restricted`` policy enabled.
-
+Starting with v5.0.0, MinIO **requires** Kubernetes 1.28.0 or later for both the infrastructure and the ``kubectl`` CLI tool.


Yes, but EOL is different than our code requiring something that's in a specific version of K8s, as was the case with 5.0.0 and K8s 1.21. You cannot run Operator 5.0.0 on something older than 1.21. It doesn't work.

Does 6.0.0 require 1.28 because of something that's only in 1.28 and newer? Or is it just the "please don't use unsupported, old versions of software?"

djwfyi · 2024-07-25T16:59:18Z

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

-   Do not use the ``kubectl krew`` or similar methods to update or manage the MinIO Tenant installation.
-   If you use Helm charts to deploy the Tenant, you must use Helm to manage that deployment.
+   If you use Helm to deploy a MinIO Tenant, you must use Helm to manage or upgrade that deployment.
+   Do not use ``kubectl krew``, Kustomize, or similar methods to manage or upgrade the MinIO Tenant.


We should drop krew here, right?

I think keep it because it was around at one point. We want to emphasize that even if someone was using or heard of or read of using krew, its no longer recommended.

source/includes/common/common-install-operator-kustomize.rst

djwfyi · 2024-07-25T19:46:11Z

source/operations/install-deploy-manage/deploy-operator-helm.rst

@@ -72,14 +72,16 @@ You can modify the Operator deployment after installation.

      NAME                            CHART VERSION   APP VERSION     DESCRIPTION                    
      minio-operator/minio-operator   4.3.7           v4.3.7          A Helm chart for MinIO Operator
-      minio-operator/operator         5.0.10          v5.0.10         A Helm chart for MinIO Operator
-      minio-operator/tenant           5.0.10          v5.0.10         A Helm chart for MinIO Operator
+      minio-operator/operator         6.0.1           v6.0.1          A Helm chart for MinIO Operator


Why specify the version number here and not use the variable?

djwfyi · 2024-07-25T19:53:25Z

source/operations/install-deploy-manage/upgrade-minio-operator-4.5.7-earlier.rst

+   - Your Kubernetes cluster runs 1.21.0 or later
+   - Your local host has ``kubectl`` installed and configured with access to the Kubernetes cluster
+
+This procedure upgrades the MinIO Operator from any 4.5.8 or later release to |operator-version-stable|.


Is it upgrading from 4.5.8 to 6.0.1? The heading says to 5.0.15 on ln 38.

source/operations/install-deploy-manage/upgrade-minio-operator-4.5.7-earlier.rst

source/operations/install-deploy-manage/upgrade-minio-operator.rst

Co-authored-by: Daryl White <[email protected]>

ravindk89 · 2024-07-25T20:40:59Z

source/_static/scss/includes/_misc.scss

+   font-weight: bold;
+}
+
+div.procedure ol.arabic li ul.simple li p {


I feel dumb having done this. @rushenn @kaankabalak can we make this cleaner? for some reason it cascades down w/o this.

Sure @ravindk89, I can take a look

ravindk89 · 2024-07-29T20:13:32Z

I tested upgrading from 5.0.15 to 6.0.1 with Helm 3.15.3, to see how the CRD would change.

I can see the change in CRD and the diff shows that it does appear to update. But online docs and internal discussion indicate that Helm does not deal with CRD updates cleanly, so I'm not sure what's going on here or how to validate Helm upgrades working as expected.

That said we can still push this forward to at least get the docs out, possibly leaving a note on Helm Operator install docs stating that Helm does not handle CRD upgrades cleanly and that additional work may be required - though I still do not know what work.

cc @cniackz @pjuarezd @dvaldivia @ramondeklein for clarity on the above

pjuarezd · 2024-07-29T20:22:17Z

I tested upgrading from 5.0.15 to 6.0.1 with Helm 3.15.3, to see how the CRD would change.

I can see the change in CRD and the diff shows that it does appear to update. But online docs and internal discussion indicate that Helm does not deal with CRD updates cleanly, so I'm not sure what's going on here or how to validate Helm upgrades working as expected.

That said we can still push this forward to at least get the docs out, possibly leaving a note on Helm Operator install docs stating that Helm does not handle CRD upgrades cleanly and that additional work may be required - though I still do not know what work.

cc @cniackz @pjuarezd @dvaldivia @ramondeklein for clarity on the above

CRD's are allways updated on the Operator helm chart update @ravindk89, we don't store them in the /crd directory, preciselly to ensure are always updated, otherwise what you mention as Helm does not handle CRD upgrades cleanly would happen, instead we store CRD's in the /templates and allways update it, as a regular helm chart resource.

At some point even the plan of provide a separated helm chart only for the CRDS was proposed, but got no traction (Helm does not handle CRD upgrades cleanly)

For now I think we only need to mention we allways update CRD's and are not stored in the /crd directory.

ravindk89 · 2024-07-29T20:39:47Z

Thanks @pjuarezd

From https://helm.sh/docs/chart_best_practices/custom_resource_definitions/

Another important point to consider in the discussion around CRD support is how the rendering of templates is handled. One of the distinct disadvantages of the crd-install method used in Helm 2 was the inability to properly validate charts due to changing API availability (a CRD is actually adding another available API to your Kubernetes cluster). If a chart installed a CRD, helm no longer had a valid set of API versions to work against. This is also the reason behind removing templating support from CRDs. With the new crds method of CRD installation, we now ensure that helm has completely valid information about the current state of the cluster.

It sounds like we're OK as long as we are building the charts with Helm 2, but Helm 3 removes the templating hook/approach we are currently using.

Is there any issue for the end user here to consider w.r.t. Helm 3, or can we safely assume that it's only baking the charts where it matters?

djwfyi

We need to clarify the upgrade path.
I think you have to get to 5.0.15 before upgrading to 6.0.1, but a few places seem to suggest that you can upgrade from 4.5.8 straight to 6.0.1.

Otherwise, another lot of minor fixes and nits I noted. And, in general, LGTM. I don't feel the need to see it again.

source/operations/install-deploy-manage/deploy-minio-tenant.rst

djwfyi · 2024-07-29T20:48:38Z

source/operations/install-deploy-manage/deploy-minio-tenant.rst

@@ -219,146 +216,165 @@ You can select a different base or pre-built template from the repository as you
 This procedure is not exhaustive of all possible configuration options available in the :ref:`Tenant CRD <minio-operator-crd>`.


Tenant CRD text is linked to the Operator CRD. Should be ref to the tenant CRD, correct?

Errr they're basically the same. the Operator CRD page includes the entire Tenant CRD definition.

there are other CRDs so technically at some point we should deal with that, but its not being done yet.

source/operations/install-deploy-manage/deploy-minio-tenant.rst

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst

djwfyi · 2024-07-29T21:15:15Z

source/operations/install-deploy-manage/expand-minio-tenant.rst


-#. Specify the following information for the new pool:
+Decommissioning a server pool involves three steps:


Four.
They'll also need to manually delete the decommissioned storage. See operator/PR#2203.

source/operations/install-deploy-manage/modify-minio-tenant.rst

djwfyi · 2024-07-29T21:17:45Z

source/operations/install-deploy-manage/modify-minio-tenant.rst

-Add Trusted Certificate Authorities
-   The MinIO Tenant validates the TLS certificate presented by each connecting client against the host system's trusted root certificate store.
-   The MinIO Operator can attach additional third-party Certificate Authorities (CA) to the Tenant to allow validation of client TLS certificates signed by those CAs.
+   .. tab-item:: Operator Console


Why are we keeping this tab?

Maybe call it kubectl instead of Operator Console?

Good question actually.

Oh the intent was to guide users away from it for existing installs, but that's handled in the upgrade docs.

djwfyi · 2024-07-29T21:19:44Z

source/operations/install-deploy-manage/modify-minio-tenant.rst

-.. include:: /includes/common-installation.rst
-   :start-after: start-pool-order-must-not-change
-   :end-before: end-pool-order-must-not-change
+      You can also use ``kubectl edit`` to 'live edit' Kubernetes objects.


The single quote marks look odd. I'd swap them for " or just drop the emphasis altogether.

djwfyi · 2024-07-29T21:23:03Z

source/operations/install-deploy-manage/upgrade-minio-operator.rst

Elsewhere, it seems like you can upgrade to 6.0.1 from 4.5.8.
However, lines 15 and 16 seem to imply that you have to be on 5.0.15 before upgrading to 6.0.1.

Which is it?

Also, line 19 should be 4.5.7 or earlier.

AFAIK we want users on 5.0.15 first before jumping. Changed this up.

Co-authored-by: Daryl White <[email protected]>

ravindk89 · 2024-07-30T19:39:45Z

@allanrogerr @cesnietor @pjuarezd final thoughts before I push tihs out? Staging updated

allanrogerr

Spelling

source/operations/install-deploy-manage/expand-minio-tenant.rst

Co-authored-by: Allan Roger Reid <[email protected]>

allanrogerr

PTAL

source/operations/install-deploy-manage/expand-minio-tenant.rst

source/operations/install-deploy-manage/modify-minio-tenant.rst

Co-authored-by: Allan Roger Reid <[email protected]>

allanrogerr

LGTM

ravindk89 added the WIP Work In Progress (Review Only If Requested) label Jul 25, 2024

ravindk89 requested review from feorlen, allanrogerr and djwfyi July 25, 2024 00:26

ravindk89 self-assigned this Jul 25, 2024

feorlen reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst Outdated Show resolved Hide resolved

feorlen reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst Outdated Show resolved Hide resolved

feorlen reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst Outdated Show resolved Hide resolved

feorlen reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst Outdated Show resolved Hide resolved

feorlen reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/upgrade-minio-operator.rst Outdated Show resolved Hide resolved

allanrogerr reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/deploy-minio-tenant-helm.rst Outdated Show resolved Hide resolved

allanrogerr reviewed Jul 25, 2024

View reviewed changes

source/operations/install-deploy-manage/upgrade-minio-operator-4.5.7-earlier.rst Outdated Show resolved Hide resolved

ravindk89 and others added 3 commits July 25, 2024 14:13

DOCS-1273: Operator 6.0.0 Deploy/Upgrade docs, removing Console refer…

5d8d59c

…ences

Apply suggestions from code review

aede057

Co-authored-by: Andrea Longo <[email protected]> Co-authored-by: Allan Roger Reid <[email protected]>

Further fixes

e282d01

ravindk89 force-pushed the DOCS-1273 branch from 3e55232 to e282d01 Compare July 25, 2024 18:13

djwfyi requested changes Jul 25, 2024

View reviewed changes

ravindk89 and others added 3 commits July 25, 2024 16:36

Apply suggestions from code review

46b6ae4

Co-authored-by: Daryl White <[email protected]>

Cleaning up, scss

861e84a

tiny fix

bc6bf81

ravindk89 added Review Assigned Reviewers Must LGTM To Close and removed WIP Work In Progress (Review Only If Requested) labels Jul 25, 2024

ravindk89 commented Jul 25, 2024

View reviewed changes

ravindk89 requested a review from djwfyi July 29, 2024 20:11

K8s floor check

3f55cf6

djwfyi approved these changes Jul 29, 2024

View reviewed changes

ravindk89 and others added 2 commits July 30, 2024 15:39

Apply suggestions from code review

158e285

Co-authored-by: Daryl White <[email protected]>

More fixups

a37fd53

allanrogerr requested changes Jul 30, 2024

View reviewed changes

Apply suggestions from code review

04689d7

Co-authored-by: Allan Roger Reid <[email protected]>

ravindk89 requested a review from allanrogerr July 30, 2024 20:01

allanrogerr requested changes Aug 1, 2024

View reviewed changes

Apply suggestions from code review

5348535

Co-authored-by: Allan Roger Reid <[email protected]>

ravindk89 requested a review from allanrogerr August 1, 2024 18:27

allanrogerr approved these changes Aug 1, 2024

View reviewed changes

djwfyi merged commit 23253dd into main Aug 1, 2024

djwfyi deleted the DOCS-1273 branch August 1, 2024 19:51


		For Kubernetes 1.25.0 and later, MinIO supports deploying in environments with the :kube-docs:`Pod Security admission (PSA) <concepts/security/pod-security-admission>` ``restricted`` policy enabled.

		Starting with v5.0.0, MinIO requires Kubernetes 1.28.0 or later for both the infrastructure and the ``kubectl`` CLI tool.

		* - ``tenant.certificate.requestAutoCert``
		- Enables or disables MinIO :ref:`automatic TLS certificate generation <minio-tls>`

		@@ -182,8 +246,78 @@ This method may support easier pre-configuration of the Tenant compared to the :

		Each chart contains a ``values.yaml`` file you can customize to suit your needs.
		For details on the options available in the MinIO Tenant ``values.yaml``, see :ref:`minio-tenant-chart-values`.


		:ref:`create-tenant-access-minio-operator-console`
		You can select a different base or pre-built template from the repository as your starting point, or build your own Kustomization resources using the MinIO Custom Resource Documentation.

	watch kubectl get all -n minio-tenant
	watch kubectl get all -n minio-tenant-1

		@@ -219,146 +216,165 @@ You can select a different base or pre-built template from the repository as you
		This procedure is not exhaustive of all possible configuration options available in the :ref:`Tenant CRD <minio-operator-crd>`.


		#. Specify the following information for the new pool:
		Decommissioning a server pool involves three steps:

DOCS-1273: Operator 6.0.0 Deploy/Upgrade docs, removing Console references #1284

DOCS-1273: Operator 6.0.0 Deploy/Upgrade docs, removing Console references #1284

Conversation

ravindk89 commented Jul 25, 2024

Summary

feorlen commented Jul 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

djwfyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravindk89 commented Jul 29, 2024

pjuarezd commented Jul 29, 2024 • edited Loading

ravindk89 commented Jul 29, 2024

djwfyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ravindk89 commented Jul 30, 2024

allanrogerr left a comment

Choose a reason for hiding this comment

allanrogerr left a comment

Choose a reason for hiding this comment

allanrogerr left a comment

Choose a reason for hiding this comment

pjuarezd commented Jul 29, 2024 •

edited

Loading