Updated structure for CD docs (#242)

* updated structure for CD docs

* added browser quickstart

* first deployments pages

* ran `yarn fix`

* first version of eployments section

* add docs for byok, architecture, etc

* add some more cluster docs

* add some more docs for cluster mgmt and add ons

* added deprecations page

* updated introduction + cleanup

* remaining cleanup

* added first cli quickstart and update browser quickstart for CD

* Fix babel vulnerability

GHSA-67hx-6x53-jw92

* Typo

* add github actions example

---------

Co-authored-by: Klink <[email protected]>
Co-authored-by: michaeljguarino <[email protected]>

Author: 3 people

Makefile

@@ -6,5 +6,5 @@ help:
 yarn-install: .PHONY
 	yarn
 
-web: yarn ## runs the docs site locally
+web: ## runs the docs site locally
 	yarn dev

pages/adding-new-application/background-app-install.md

@@ -6,7 +6,7 @@ description: >-
 
 In this guide we will lay out how your user provided values tie in with the deployment process as it relates to the configuration and templating of the app.
 
-When a user sets up a new Plural workspace in a git repository (we'll call that a *deployment repository* going forward) a `workspace.yaml` file is created that stores global values for that cluster such as the cloud account and region, the cluster and VPC name and what subdomain all the applications will be hosted under.
+When a user sets up a new Plural workspace in a git repository (we'll call that a _deployment repository_ going forward) a `workspace.yaml` file is created that stores global values for that cluster such as the cloud account and region, the cluster and VPC name and what subdomain all the applications will be hosted under.
 Next, the user can install an application bundle using the `plural bundle install <app_name> <bundle>` CLI command.
 
 > Most applications come with more than one bundle, one for each targeted cloud provider.
@@ -32,6 +32,7 @@ $ tree -L 1 .
 ```
 
 The `workspace.yaml` would look like this:
+
 ```yaml
 apiVersion: plural.sh/v1alpha1
 kind: ProjectManifest
@@ -40,7 +41,7 @@ metadata:
 spec:
   cluster: pluraldev
   bucket: pluraldevsh-tf-state
-  project: "123456765432"
+  project: '123456765432'
   provider: aws
   region: us-east-2
   owner:
@@ -53,6 +54,7 @@ spec:
 ```
 
 And the `context.yaml` like this. In the `spec.configuration` section you can see the recorded user input for each artifact.
+
 ```yaml
 apiVersion: plural.sh/v1alpha1
 kind: Context

pages/adding-new-application/index.md

@@ -9,4 +9,4 @@ description: >-
 If you want to contribute your own application to the Plural marketplace we invite you to create and publish your own artifacts.
 In this section we guide you through the ins and outs of Plural's application artifacts, how you can build your own with the tools that Plural provides you, and how you can publish them for others to use.
 
-> We are focused on properly compensating any contributions to the Plural platform, which includes a meaningful [bounty](https://github.com/pluralsh/plural-artifacts#plural-contributor-program).
+> We are focused on properly compensating any contributions to the Plural platform, which includes a meaningful [bounty](https://github.com/pluralsh/plural-artifacts#plural-contributor-program).

pages/adding-new-application/plural-artifact-structure.md

@@ -7,6 +7,7 @@ description: >-
 As mentioned in [Background on Application Installations](/adding-new-application/background-app-install),
 the Plural CLI creates a wrapper Helm chart and Terraform module for each installed application and inputs the user provided values for that installation.
 Some extra configuration files are necessary in the application's artifact for the Plural API to be able to understand:
+
 - the Helm charts and Terraform modules dependencies to run them through its templating engine
 - dependencies on other Plural artifacts
 - platform specific components and infrastructure configurations
@@ -83,8 +84,7 @@ $ tree .
 └── vendor_images.yaml
 ```
 
-Let's disect this artifact's structure.
-
+Let's dissect this artifact's structure.
 
 ## Helm
 
@@ -105,7 +105,8 @@ Plural leverages dependency tracking of applications to achieve a high degree of
 Dependencies between artifacts are defined in the recipe files (see below).
 Dependencies are also tracked between the Helm charts and Terraform modules of other applications in a dedicated `deps.yaml` file in each chart's or module's directory.
 For example, for Dagster's Helm chart you would list required dependencies on:
-- the `bootstrap` application's Helm chart 
+
+- the `bootstrap` application's Helm chart
 - the `ingress-nginx` application's Helm chart
 - the `postrges` operator application's Helm chart
 
@@ -120,33 +121,33 @@ metadata:
 spec:
   breaking: true
   dependencies:
-  - type: helm
-    name: bootstrap
-    repo: bootstrap
-    version: '>= 0.5.1'
-  - type: helm
-    name: ingress-nginx
-    repo: ingress-nginx
-    version: ">= 0.1.2"
-  - type: helm
-    name: postgres
-    repo: postgres
-    version: ">= 0.1.6"
-  - type: terraform
-    name: aws
-    repo: dagster
-    version: '>= 0.1.0'
-    optional: true
-  - type: terraform
-    name: azure
-    repo: dagster
-    version: '>= 0.1.0'
-    optional: true
-  - type: terraform
-    name: gcp
-    repo: dagster
-    version: '>= 0.1.0'
-    optional: true
+    - type: helm
+      name: bootstrap
+      repo: bootstrap
+      version: '>= 0.5.1'
+    - type: helm
+      name: ingress-nginx
+      repo: ingress-nginx
+      version: '>= 0.1.2'
+    - type: helm
+      name: postgres
+      repo: postgres
+      version: '>= 0.1.6'
+    - type: terraform
+      name: aws
+      repo: dagster
+      version: '>= 0.1.0'
+      optional: true
+    - type: terraform
+      name: azure
+      repo: dagster
+      version: '>= 0.1.0'
+      optional: true
+    - type: terraform
+      name: gcp
+      repo: dagster
+      version: '>= 0.1.0'
+      optional: true
 ```
 
 ## Terraform
@@ -157,9 +158,10 @@ Most commonly you'd find object storage alongside their IAM resources, or additi
 Sometimes it will also include Kubernetes resources like service accounts, if their deployment cannot be achieved through the artifact's Helm chart in a convenient manner.
 
 > One peculiarity about the Terraform modules is that, at the very least, they need to contain the Kubernetes namespace for your application.
-  This is because during a `plural deploy` with the Plural CLI the `terraform apply` will always run before the `helm install` step.
+> This is because during a `plural deploy` with the Plural CLI the `terraform apply` will always run before the `helm install` step.
 
 Just like the Helm chart, the Terraform modules also contain a `deps.yaml` file that inform the Plural API about dependencies on other modules.
+
 ```
 apiVersion: plural.sh/v1alpha1
 kind: Dependencies
@@ -176,7 +178,6 @@ spec:
   - aws
 ```
 
-
 ## Plural Files
 
 The `plural` directory contains the artifact's packaging information (`plural/recipes`), metadata (`plural/tags` and `plural/icons`), and application specific instructions (`plural/docs` and `plural/notes.tpl`).
@@ -185,6 +186,7 @@ On the top-level directory of each artifact you'll also find a`repository.yaml`.
 The `repository.yaml` and recipe YAMLs are an integral part of Plural's artifact packaging format.
 
 ### `repository.yaml`
+
 ```yaml
 name: dagster
 description: A data orchestration platform for the development, production, and observation of data assets.
@@ -199,53 +201,55 @@ oauthSettings:
   uriFormat: https://{domain}/oauth2/callback
   authMethod: POST
 tags:
-- tag: dag
-- tag: data
-- tag: data-pipelines
+  - tag: dag
+  - tag: data
+  - tag: data-pipelines
 ```
 
 The metadata in this file informs the Plural API about the application this artifact envelopes.
 It will store its name, category and description, where it can find the icon and docs to display in the marketplace,
 the notes template to prompt after installation, as well as links to any upstream git repository or homepage (if applicable).
 
 `oauthSettings` specifies the URI format for the OIDC callback address and informs the Plural API how to setup the OIDC endpoint for your application if you use it.
+
 > Behind the scenes, every `plural bundle install` triggers the OIDC client creation when you answer with `yes` on the OIDC prompt.
-  This happens, because every client needs to be created before a `plural build` which then inputs the client info into the helm chart.
+> This happens, because every client needs to be created before a `plural build` which then inputs the client info into the helm chart.
 
 The `private` flag controls whether the artifact's bundles are published publicly or privately on a `plural push` or `plural apply` (see [Publishing](/adding-new-application/publishing)).
 
 ### `plural/recipes/dagster-aws.yaml`
+
 ```yaml
 name: dagster-aws
 description: Installs dagster on an aws eks cluster
 provider: AWS
 primary: true
 dependencies:
-- repo: bootstrap
-  name: aws-k8s
-- repo: ingress-nginx
-  name: ingress-nginx-aws
-- repo: postgres
-  name: aws-postgres
+  - repo: bootstrap
+    name: aws-k8s
+  - repo: ingress-nginx
+    name: ingress-nginx-aws
+  - repo: postgres
+    name: aws-postgres
 oidcSettings:
   uriFormat: https://{domain}/oauth2/callback
   authMethod: POST
   domainKey: hostname
 sections:
-- name: dagster
-  configuration:
-  - name: dagsterBucket
-    type: BUCKET
-    documentation: s3 bucket for storing dagster logs
-    default: dagster
-  - name: hostname
-    type: DOMAIN
-    documentation: fqdn on which to deploy your dagster instance
-  items:
-  - type: TERRAFORM
-    name: aws
-  - type: HELM
-    name: dagster
+  - name: dagster
+    configuration:
+      - name: dagsterBucket
+        type: BUCKET
+        documentation: s3 bucket for storing dagster logs
+        default: dagster
+      - name: hostname
+        type: DOMAIN
+        documentation: fqdn on which to deploy your dagster instance
+    items:
+      - type: TERRAFORM
+        name: aws
+      - type: HELM
+        name: dagster
 ```
 
 The recipe file ties a bundle together, with one dedicated recipe per cloud provider.

pages/adding-new-application/publishing.md

@@ -11,16 +11,17 @@ To publish an artifact you first need to create a publisher with your Plural acc
 
 1. Click on your profile on the bottom left on `app.plural.sh`.
 2. Select `create publisher`.
-3. Enter your information. 
+3. Enter your information.
 4. Hit save.
 
 ## Publish an Artifact with the Plural CLI
 
-When you're done with implementing your artifact and you have created a publisher you're ready to publish it. 
+When you're done with implementing your artifact and you have created a publisher you're ready to publish it.
 
 1. Navigate to the directory of your artifact in your local copy of the [Plural artifacts git repository](https://github.com/pluralsh/plural-artifacts).
 2. Locate the `Pluralfile`.
 3. Enter the name of your publisher as the first argument in the `ATTRIBUTES` line. E.g. if your publisher is called `mypublisher`, it should look like this:
+
 ```
 REPO dagster
 ATTRIBUTES mypublisher repository.yaml
@@ -29,7 +30,9 @@ TF terraform/*
 HELM helm/*
 RECIPE plural/recipes/*
 ```
+
 4. Finally, publish your artifact with the command `plural apply -f Pluralfile`. E.g. for our dagster example
+
 ```console
 $ plural apply -f Pluralfile
 Setting attributes for dagster

pages/deployments/addons.md

@@ -0,0 +1,17 @@
+---
+title: Kubernetes Add-Ons
+description: Extend Kubernetes to support load balancing, dns, monitoring and security
+---
+
+Kubernetes natively is just a container orchestrator, but it's apis can be easily extended to support many other major concerns within the DevOps space. Some of the more common usecases we currently support are:
+
+- Networking - you need load balancers, dns, and cert management to build a full production service exposed to the outside world. We can deploy all the operators and controllers needed to get this set up in a fully self-service way.
+- Monitoring - inject agents from Datadog, New Relic, Prometheus and other providers to provide observability to your fleet
+- Security - tools like Trivy and Kubescape can continuously monitor clusters for vulnerabilities and enhance your security posture.
+- Cost Monitoring - deploy the kubecost agent to give you coherent cost dashboards for your entire fleet
+
+All of these will be constantly updated and maintained by us, you just need to perform the initial configuration via our wizards.
+
+You can see all the addons we support in the `Add Ons` tab of Plural Deployments, as seen here:
+
+![](/assets/deployments/addons.png)

pages/deployments/architecture.md

@@ -0,0 +1,35 @@
+---
+title: Plural CD Architecture
+description: Secure, Scalable Pull-Based Architecture
+---
+
+## Overview
+
+Plural CD is based on a scalable, secure, agent-based pull architecture. It doesn't require direct access to any of the clusters it deploys to, meaning it can manage workloads in any cloud, on-prem, on the edge, or even on a local laptop running KIND. Further since it doesn't require networking intensive kubernetes watch streams, central access to kubernetes, or relies on single-mastered operator control loops, it should scale to virtually any size kubernetes fleet. We also enhance your kubernetes setup with a auth proxy, allowing you to have full visibility to your workloads without compromising the network security of your setup or require the creation of complex multi-cloud networking setups.
+
+Here's a quick diagram of the setup:
+
+![](/assets/deployments/architecture.png)
+
+## Control Plane
+
+The Control Plane layer is a full-stack service deployable onto any kubernetes cluster you designate as your management cluster. It contains a few main components:
+
+- Horizontally scalable git cache - we should be able to ingest as many git repos as you'd like and auto-shard them throughout your cluster automatically and efficiently.
+- Configuration Management - supports re-configurable backends, but allows you to easily parameterize services with information like hostnames, docker image tags, and other secret and non-secret information.
+- Auth Proxy - this is a secure bidirectional grpc channel initiated by a deployment agent used to make kubernetes api calls no matter where a cluster may live and give you full dashboarding capabilities from the Plural CD UI.
+- Cluster API Providers - Plural CD natively integrates with cluster api and allows you to create and manage new clusters at scale and fork your own kubernetes cluster APIs on top of existing setups for services like EKS, AKS and GKE or on-prem solutions like Rancher
+
+We provide simple installers if you'd like to deploy the control plane layer to a kubernetes cluster already in your fleet, or you can use our own kubernetes setup in the standard plural install flow.
+
+## Deployment Agent
+
+A thin deployment agent is installed onto each cluster and perpetually managed by Plural CD from then on. It will perpetually poll the control plane for new services to apply and if there are any changes to make, apply them into your cluster. It also can do a few other things like:
+
+- establish the bidectional grpc channel for the auth proxy layer (this does require an ingress with websocket support)
+- execute integration tests
+
+There were a few design considerations involved in our agent that are worth understanding as well:
+
+- The expectation is the agent is deployed independently at scale, potentially hundreds of times for a large organization, so it must be maximally simple to reduce operational complexity to near 0.
+- It is designed to be extensible to arbitrarily many frameworks for defining kubernetes manifests. We aren't actually huge fans of helm or kustomize and want organizations to have the flexibility to ultimately use other toolchains for managing their kubernetes YAML codebase (or ultimately de-YAML themselves)

pages/deployments/browser-quickstart.md

@@ -0,0 +1,53 @@
+---
+title: Plural CD In-Browser Quickstart
+description: Deploying your services using the Plural Console UI.
+---
+
+## Overview
+
+This guide goes over how to deploy your services with Plural using the Plural Console UI. At the end of this tutorial, you will have:
+
+- Provisioned new clusters, and/or deployed the Plural Deployment Operator on your existing clusters.
+- Imported the Git repositories containing your code and manifests.
+- Deployed your code onto your clusters of choice.
+- Optionally updated any configurations and permissions for the clusters and services.
+
+in under 30 minutes, all within your browser. You can see the process in the video here or follow the instructions step-by-step:
+
+{% embed url="https://www.youtube.com/watch?v=jBlq45ntbBw" aspectRatio="16 / 9" /%}
+
+## Onboard to Plural and install the Plural Console
+
+If you haven't already, you'll need to follow the Plural guide to install Console. You can use the guide for the [in-browser Cloud Shell](/getting-started/cloud-shell-quickstart) or the [CLI](/getting-started/quickstart) to get started.
+
+## Log in to Console and Add Clusters
+
+Once you've logged in to Plural Console, you can click the "Continuous deployment" tab to set up your clusters. By default, you should see the cluster that your Console is deployed on.
+
+To deploy additional clusters, click the "Create cluster" button in the top right. To spin up a new cluster, fill in the required fields:
+
+![](/assets/deployments/create-cluster.png)
+
+To import an existing cluster, select the "Import existing" toggle and add a name, handle, and any tags. You then need to run the following command on the cluster itself to install the Plural Deploy Operator:
+
+```
+plural cd install --url https://<CONSOLE_URL>/ext/gql --token <TOKEN>
+```
+
+Make sure your kubeconfig points to the desired cluster for import. More information is available in the guide for [existing clusters](/deployments/existing-cluster).
+
+Once you've added the required clusters, you should see them populated on the "Clusters" subtab within your Console.
+
+## Import Git Repositories and Deploy services
+
+You'll need to then import the Git repository containing your service and the associated Kubernetes manifests. Click the "Git repositories" subtab, and hit the "Import Git" button in the top right.
+
+![](/assets/deployments/import-git.png)
+
+Add the url of the repo and authorization if necessary, and hit import. Once your service is pullable, you can deploy it to your clusters.
+
+Navigate to the "Services" subtab and click "Deploy service". Add a name and namespace and select the cluster you want to deploy to. You can then select the relevant repository and add any secrets necessary to parameterize the deployment.
+
+Your service should populate initially as `Stale` until your components are Ready, at which point they will flip to `Healthy`.
+
+Congratulations! You've deployed your first service with Plural.