Skip to content

Commit 40f7a18

Browse files
HappyTetrahedronHappyTetrahedron
committed
Document how to restore from k8up backup
1 parent a1ba184 commit 40f7a18

File tree

4 files changed

+153
-0
lines changed

4 files changed

+153
-0
lines changed

‎docs/modules/ROOT/pages/how-tos/restore-from-backup.adoc

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
= Restore from Backup
2+
3+
When configuring component-vault with `backup.enabled = true`, the component sets up regular backups using k8up.
4+
This how-to explains how Vault can be restored from such a backup.
5+
6+
== Information on the Vault Unseal Key and Root Token
7+
8+
include::partial$vault-warning.adoc[]
9+
10+
== Prerequisites
11+
12+
* `restic` - command line tool to access backups made by k8up
13+
* `vault` - command line tool to interact with Vault
14+
* `kubectl`
15+
* Write access to the cluster's tenant repository
16+
* Read access to the restic repository in which k8up stored the Vault backups
17+
* The Vault instance's *unseal key* and *root token* - these must be backed up manually; they are not part of the automated k8up backup.
18+
19+
== Procedure
20+
21+
=== 1. Set up new Vault instance
22+
23+
* Add the `vault` application to your cluster configuration
24+
* Initially disable backups by setting `.backups.enabled` to `false`
25+
* Compile and push the cluster config and wait for Vault to start.
26+
27+
=== 2. Set up permissions for restoring the Vault instance
28+
29+
* Access the Vault UI
30+
+
31+
[source,shell]
32+
----
33+
export VAULT_INSTANCE_NAME="my-vault-instance"
34+
# Get root token to log in
35+
kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d
36+
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
37+
----
38+
* Open your browser at http://localhost:8200 and log in with the `Token` method and the root token retreived in the previous step
39+
* Navigate to `Policies` and open the `backup` policy
40+
* Edit the policy to the following:
41+
+
42+
[source,hcl]
43+
----
44+
path "sys/storage/raft/snapshot-force" {
45+
capabilities = ["read", "create", "update", "sudo"]
46+
}
47+
----
48+
* Save the new policy
49+
50+
NOTE: Editing this policy is unproblematic, as it will be reset once the backup is restored.
51+
52+
=== 3. Retrieve the Vault snapshot
53+
54+
* Set up the restic credentials (values correspond to the component parameters `backup.bucket` and `backup.password`)
55+
+
56+
[source,shell]
57+
----
58+
export AWS_ACCESS_KEY_ID="S3_KEY backup.bucket.accesskey"
59+
export AWS_SECRET_ACCESS_KEY="S3_SECRET backup.bucket.secretkey"
60+
export RESTIC_REPOSITORY="s3:https://path.to.my/bucket"
61+
export RESTIC_PASSWORD="RESTIC_REPO_KEY backup.password"
62+
----
63+
* Retrieve the latest Vault snapshot to your local disk
64+
+
65+
[source,shell]
66+
----
67+
mkdir restore
68+
restic restore --target restore/ latest
69+
----
70+
* Verify the snapshot file
71+
+
72+
[source,shell]
73+
----
74+
ls restore
75+
# This should show a file named "[instance name]-backup.snapshot"
76+
----
77+
78+
=== 4. Restore the snapshot
79+
80+
* Expose the Vault pod
81+
+
82+
[source,shell]
83+
----
84+
kubectl port-forward -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-0 8200
85+
----
86+
* In a separate terminal, prepare the environment to access Vault
87+
+
88+
[source,shell]
89+
----
90+
# Get root token to log in
91+
export VAULT_TOKEN="$(kubectl get secret -n $VAULT_INSTANCE_NAME ${VAULT_INSTANCE_NAME}-seal -ojsonpath='{.data.vault-root}' | base64 -d)"
92+
export VAULT_ADDR="http://127.0.0.1:8200"
93+
----
94+
* Restore the backup
95+
+
96+
[source,shell]
97+
----
98+
vault operator raft snapshot restore -force restore/${VAULT_INSTANCE_NAME}-backup.snapshot
99+
----
100+
101+
=== 5. Unseal Vault
102+
103+
If you were logged into the Vault UI, you should have gotten logged out now.
104+
This is expected.
105+
106+
* Open your browser at http://localhost:8200
107+
* Use the *Vault Unseal Key* of the Vault instance you've just restored to unseal Vault
108+
* Use the *Vault root token* of the Vault instance you've just restored to log in with the `Token` method
109+
* Verify that the restore worked, and secrets are now restored in Vault.
110+
111+
[IMPORTANT]
112+
====
113+
The unseal key and root token of the Vault instance you're restoring need to have been stored separately.
114+
Without them, the restore procedure cannot be completed.
115+
====
116+
117+
=== 6. Update the Vault Secret
118+
119+
NOTE: Without this step, your Vault instance will not be able to auto-unseal.
120+
121+
* Encode the Vault credentials
122+
+
123+
[source,shell]
124+
----
125+
export VAULT_UNSEAL_KEY="OLD_UNSEAL_KEY"
126+
export VAULT_ROOT_TOKEN="OLD_ROOT_TOKEN"
127+
128+
echo -n "$VAULT_UNSEAL_KEY" | base64 -w0
129+
echo -n "$VAULT_ROOT_TOKEN" | base64 -w0
130+
----
131+
* Update the Vault secret
132+
+
133+
[source,shell]
134+
----
135+
kubectl edit secret -n ${VAULT_INSTANCE_NAME} ${VAULT_INSTANCE_NAME}-seal
136+
----
137+
* Update the `vault-root` and `vault-unseal-0` keys to reflect the values you have just encoded
138+
* Save the secret

‎docs/modules/ROOT/pages/references/parameters.adoc

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,12 @@ default:: `true`
252252

253253
Whether to do backups using k8up.
254254

255+
[WARNING]
256+
.Manual Setup Required
257+
====
258+
include::partial$vault-warning.adoc[]
259+
====
260+
255261
== `backup.schedule`
256262

257263
[horizontal]

‎docs/modules/ROOT/partials/nav.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
* xref:index.adoc[Home]
22
* xref:tutorials/install.adoc[Installation]
33
* xref:how-tos/lieutenant.adoc[Lieutenant Integration]
4+
* xref:how-tos/restore-from-backup.adoc[Restore Vault from Backup]
45
* xref:references/parameters.adoc[Parameters]

‎docs/modules/ROOT/partials/vault-warning.adoc

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
`component-vault` leverages k8up's application specific backups to create Vault snapshots.
2+
Restoring such a snapshot is only possible by providing the Vault unseal key and root token.
3+
As these are sensitive secrets, they are not backed up as part of the component's automated backup process.
4+
5+
When setting up Vault, you must store the Vault unseal key and root token in a safe location in order to be able to restore backups.
6+
7+
In a running Vault instance, both of these secrets can be found in the `[instance name]-seal` secret in the Vault instance's namespace.
8+
In order to have a reliable Vault backup, this secret must be backed up separately to a safe location.

0 commit comments

Comments
 (0)