Expired Token on refreshed AWS keys #3713
Description
I am using Hashicorp Vault to rotate my AWS keys automatically, however after they are rotated, I was able to verify the keys were updated but external-dns is still showing they were expired, requiring me to restart my pod in order to get external-dns to access AWS again
I'm using helm to install external-dns from here https://kubernetes-sigs.github.io/external-dns v1.13.0
My values look like this
env:
- name: AWS_SHARED_CREDENTIALS_FILE
value: "/.aws/credentials"
domainFilters:
- <my domain>.com
podAnnotations:
vault.hashicorp.com/agent-inject: 'true'
vault.hashicorp.com/role: 'vault'
vault.hashicorp.com/agent-inject-secret-credentials: '/aws/creds/external-dns'
vault.hashicorp.com/secret-volume-path: '/.aws'
vault.hashicorp.com/agent-inject-template-credentials: |
{{ with secret "/aws/creds/external-dns" }}
[default]
aws_access_key_id = {{ .Data.access_key }}
aws_secret_access_key = {{ .Data.secret_key }}
aws_session_token = {{ .Data.security_token }}
{{ end }}
vault is refreshing them in the .aws/credentials, im not sure why external-dns is unable to see the new keys
my workaround was to add this which tells vault to kill the process every time vault refreshes the keys
shareProcessNamespace: true
podAnnotations:
vault.hashicorp.com/agent-inject-command-credentials: |
kill -TERM $(pidof external-dns) || true