Skip to content

Commit

Permalink
Document how to restore from k8up backup
Browse files Browse the repository at this point in the history
  • Loading branch information
HappyTetrahedron committed Aug 19, 2024
1 parent a1ba184 commit 40f7a18
Show file tree
Hide file tree
Showing 4 changed files with 153 additions and 0 deletions.
138 changes: 138 additions & 0 deletions docs/modules/ROOT/pages/how-tos/restore-from-backup.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
= Restore from Backup

When configuring component-vault with `backup.enabled = true`, the component sets up regular backups using k8up.
This how-to explains how Vault can be restored from such a backup.

== Information on the Vault Unseal Key and Root Token

include::partial$vault-warning.adoc[]

== Prerequisites

* `restic` - command line tool to access backups made by k8up
* `vault` - command line tool to interact with Vault
* `kubectl`
* Write access to the cluster's tenant repository
* Read access to the restic repository in which k8up stored the Vault backups
* The Vault instance's *unseal key* and *root token* - these must be backed up manually; they are not part of the automated k8up backup.

== Procedure

=== 1. Set up new Vault instance

* Add the `vault` application to your cluster configuration
* Initially disable backups by setting `.backups.enabled` to `false`
* Compile and push the cluster config and wait for Vault to start.

=== 2. Set up permissions for restoring the Vault instance

* Access the Vault UI
+
[source,shell]
----
export VAULT_INSTANCE_NAME="my-vault-instance"
# Get root token to log in
kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
----
* Open your browser at http://localhost:8200 and log in with the `Token` method and the root token retreived in the previous step
* Navigate to `Policies` and open the `backup` policy
* Edit the policy to the following:
+
[source,hcl]
----
path "sys/storage/raft/snapshot-force" {
capabilities = ["read", "create", "update", "sudo"]
}
----
* Save the new policy

NOTE: Editing this policy is unproblematic, as it will be reset once the backup is restored.

=== 3. Retrieve the Vault snapshot

* Set up the restic credentials (values correspond to the component parameters `backup.bucket` and `backup.password`)
+
[source,shell]
----
export AWS_ACCESS_KEY_ID="S3_KEY backup.bucket.accesskey"
export AWS_SECRET_ACCESS_KEY="S3_SECRET backup.bucket.secretkey"
export RESTIC_REPOSITORY="s3:https://path.to.my/bucket"
export RESTIC_PASSWORD="RESTIC_REPO_KEY backup.password"
----
* Retrieve the latest Vault snapshot to your local disk
+
[source,shell]
----
mkdir restore
restic restore --target restore/ latest
----
* Verify the snapshot file
+
[source,shell]
----
ls restore
# This should show a file named "[instance name]-backup.snapshot"
----

=== 4. Restore the snapshot

* Expose the Vault pod
+
[source,shell]
----
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
----
* In a separate terminal, prepare the environment to access Vault
+
[source,shell]
----
# Get root token to log in
export VAULT_TOKEN="$(kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d)"
export VAULT_ADDR="http://127.0.0.1:8200"
----
* Restore the backup
+
[source,shell]
----
vault operator raft snapshot restore -force restore/${VAULT_INSTANCE_NAME}-backup.snapshot
----

=== 5. Unseal Vault

If you were logged into the Vault UI, you should have gotten logged out now.
This is expected.

* Open your browser at http://localhost:8200
* Use the *Vault Unseal Key* of the Vault instance you've just restored to unseal Vault
* Use the *Vault root token* of the Vault instance you've just restored to log in with the `Token` method
* Verify that the restore worked, and secrets are now restored in Vault.

[IMPORTANT]
====
The unseal key and root token of the Vault instance you're restoring need to have been stored separately.
Without them, the restore procedure cannot be completed.
====

=== 6. Update the Vault Secret

NOTE: Without this step, your Vault instance will not be able to auto-unseal.

* Encode the Vault credentials
+
[source,shell]
----
export VAULT_UNSEAL_KEY="OLD_UNSEAL_KEY"
export VAULT_ROOT_TOKEN="OLD_ROOT_TOKEN"
echo -n "$VAULT_UNSEAL_KEY" | base64 -w0
echo -n "$VAULT_ROOT_TOKEN" | base64 -w0
----
* Update the Vault secret
+
[source,shell]
----
kubectl edit secret -n ${VAULT_INSTANCE_NAME} ${VAULT_INSTANCE_NAME}-seal
----
* Update the `vault-root` and `vault-unseal-0` keys to reflect the values you have just encoded
* Save the secret
6 changes: 6 additions & 0 deletions docs/modules/ROOT/pages/references/parameters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,12 @@ default:: `true`

Whether to do backups using k8up.

[WARNING]
.Manual Setup Required
====
include::partial$vault-warning.adoc[]
====

== `backup.schedule`

[horizontal]
Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/partials/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
* xref:index.adoc[Home]
* xref:tutorials/install.adoc[Installation]
* xref:how-tos/lieutenant.adoc[Lieutenant Integration]
* xref:how-tos/restore-from-backup.adoc[Restore Vault from Backup]
* xref:references/parameters.adoc[Parameters]
8 changes: 8 additions & 0 deletions docs/modules/ROOT/partials/vault-warning.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
`component-vault` leverages k8up's application specific backups to create Vault snapshots.
Restoring such a snapshot is only possible by providing the Vault unseal key and root token.
As these are sensitive secrets, they are not backed up as part of the component's automated backup process.

When setting up Vault, you must store the Vault unseal key and root token in a safe location in order to be able to restore backups.

In a running Vault instance, both of these secrets can be found in the `[instance name]-seal` secret in the Vault instance's namespace.
In order to have a reliable Vault backup, this secret must be backed up separately to a safe location.

0 comments on commit 40f7a18

Please sign in to comment.