-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Document how to restore from k8up backup
- Loading branch information
1 parent
a1ba184
commit 40f7a18
Showing
4 changed files
with
153 additions
and
0 deletions.
There are no files selected for viewing
138 changes: 138 additions & 0 deletions
138
docs/modules/ROOT/pages/how-tos/restore-from-backup.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
= Restore from Backup | ||
|
||
When configuring component-vault with `backup.enabled = true`, the component sets up regular backups using k8up. | ||
This how-to explains how Vault can be restored from such a backup. | ||
|
||
== Information on the Vault Unseal Key and Root Token | ||
|
||
include::partial$vault-warning.adoc[] | ||
|
||
== Prerequisites | ||
|
||
* `restic` - command line tool to access backups made by k8up | ||
* `vault` - command line tool to interact with Vault | ||
* `kubectl` | ||
* Write access to the cluster's tenant repository | ||
* Read access to the restic repository in which k8up stored the Vault backups | ||
* The Vault instance's *unseal key* and *root token* - these must be backed up manually; they are not part of the automated k8up backup. | ||
|
||
== Procedure | ||
|
||
=== 1. Set up new Vault instance | ||
|
||
* Add the `vault` application to your cluster configuration | ||
* Initially disable backups by setting `.backups.enabled` to `false` | ||
* Compile and push the cluster config and wait for Vault to start. | ||
|
||
=== 2. Set up permissions for restoring the Vault instance | ||
|
||
* Access the Vault UI | ||
+ | ||
[source,shell] | ||
---- | ||
export VAULT_INSTANCE_NAME="my-vault-instance" | ||
# Get root token to log in | ||
kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d | ||
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200 | ||
---- | ||
* Open your browser at http://localhost:8200 and log in with the `Token` method and the root token retreived in the previous step | ||
* Navigate to `Policies` and open the `backup` policy | ||
* Edit the policy to the following: | ||
+ | ||
[source,hcl] | ||
---- | ||
path "sys/storage/raft/snapshot-force" { | ||
capabilities = ["read", "create", "update", "sudo"] | ||
} | ||
---- | ||
* Save the new policy | ||
|
||
NOTE: Editing this policy is unproblematic, as it will be reset once the backup is restored. | ||
|
||
=== 3. Retrieve the Vault snapshot | ||
|
||
* Set up the restic credentials (values correspond to the component parameters `backup.bucket` and `backup.password`) | ||
+ | ||
[source,shell] | ||
---- | ||
export AWS_ACCESS_KEY_ID="S3_KEY backup.bucket.accesskey" | ||
export AWS_SECRET_ACCESS_KEY="S3_SECRET backup.bucket.secretkey" | ||
export RESTIC_REPOSITORY="s3:https://path.to.my/bucket" | ||
export RESTIC_PASSWORD="RESTIC_REPO_KEY backup.password" | ||
---- | ||
* Retrieve the latest Vault snapshot to your local disk | ||
+ | ||
[source,shell] | ||
---- | ||
mkdir restore | ||
restic restore --target restore/ latest | ||
---- | ||
* Verify the snapshot file | ||
+ | ||
[source,shell] | ||
---- | ||
ls restore | ||
# This should show a file named "[instance name]-backup.snapshot" | ||
---- | ||
|
||
=== 4. Restore the snapshot | ||
|
||
* Expose the Vault pod | ||
+ | ||
[source,shell] | ||
---- | ||
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200 | ||
---- | ||
* In a separate terminal, prepare the environment to access Vault | ||
+ | ||
[source,shell] | ||
---- | ||
# Get root token to log in | ||
export VAULT_TOKEN="$(kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d)" | ||
export VAULT_ADDR="http://127.0.0.1:8200" | ||
---- | ||
* Restore the backup | ||
+ | ||
[source,shell] | ||
---- | ||
vault operator raft snapshot restore -force restore/${VAULT_INSTANCE_NAME}-backup.snapshot | ||
---- | ||
|
||
=== 5. Unseal Vault | ||
|
||
If you were logged into the Vault UI, you should have gotten logged out now. | ||
This is expected. | ||
|
||
* Open your browser at http://localhost:8200 | ||
* Use the *Vault Unseal Key* of the Vault instance you've just restored to unseal Vault | ||
* Use the *Vault root token* of the Vault instance you've just restored to log in with the `Token` method | ||
* Verify that the restore worked, and secrets are now restored in Vault. | ||
|
||
[IMPORTANT] | ||
==== | ||
The unseal key and root token of the Vault instance you're restoring need to have been stored separately. | ||
Without them, the restore procedure cannot be completed. | ||
==== | ||
|
||
=== 6. Update the Vault Secret | ||
|
||
NOTE: Without this step, your Vault instance will not be able to auto-unseal. | ||
|
||
* Encode the Vault credentials | ||
+ | ||
[source,shell] | ||
---- | ||
export VAULT_UNSEAL_KEY="OLD_UNSEAL_KEY" | ||
export VAULT_ROOT_TOKEN="OLD_ROOT_TOKEN" | ||
echo -n "$VAULT_UNSEAL_KEY" | base64 -w0 | ||
echo -n "$VAULT_ROOT_TOKEN" | base64 -w0 | ||
---- | ||
* Update the Vault secret | ||
+ | ||
[source,shell] | ||
---- | ||
kubectl edit secret -n ${VAULT_INSTANCE_NAME} ${VAULT_INSTANCE_NAME}-seal | ||
---- | ||
* Update the `vault-root` and `vault-unseal-0` keys to reflect the values you have just encoded | ||
* Save the secret |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
* xref:index.adoc[Home] | ||
* xref:tutorials/install.adoc[Installation] | ||
* xref:how-tos/lieutenant.adoc[Lieutenant Integration] | ||
* xref:how-tos/restore-from-backup.adoc[Restore Vault from Backup] | ||
* xref:references/parameters.adoc[Parameters] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
`component-vault` leverages k8up's application specific backups to create Vault snapshots. | ||
Restoring such a snapshot is only possible by providing the Vault unseal key and root token. | ||
As these are sensitive secrets, they are not backed up as part of the component's automated backup process. | ||
|
||
When setting up Vault, you must store the Vault unseal key and root token in a safe location in order to be able to restore backups. | ||
|
||
In a running Vault instance, both of these secrets can be found in the `[instance name]-seal` secret in the Vault instance's namespace. | ||
In order to have a reliable Vault backup, this secret must be backed up separately to a safe location. |