Skip to content
This repository was archived by the owner on Dec 14, 2023. It is now read-only.

Commit 2f863cd

Browse files
matthieuJacquot-voodoomatthieuJacquot-voodoo
committed
feat: add dev-env
1 parent 89887bb commit 2f863cd

22 files changed

+549
-70
lines changed

‎.gitignore

-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
_dev-env
21
irsa-operator
32

43
# Binaries for programs and plugins

‎_dev-env/.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
k8s-pki
2+
webhook

‎_dev-env/Makefile

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
start_kind:
2+
sudo rm -rf ./k8s-pki
3+
mkdir ./k8s-pki
4+
kind create cluster --config ./kind-config.yml
5+
sudo chmod 644 ./k8s-pki/sa.*
6+
7+
start_docker_compose:
8+
USER_ID=$(shell id -u) GROUP_ID=$(shell id -g) docker-compose up -d
9+
10+
register_oidc:
11+
AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_REGION=us-east-1 aws --endpoint-url=http://localhost:4566 iam create-open-id-connect-provider --url https://hydra.local:4444 --client-id-list sts.amazonaws.com --thumbprint-list $(shell ./get-hydra-thumbprint.sh)
12+
13+
check:
14+
AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test AWS_REGION=us-east-1 PAGER= aws --no-cli-pager --endpoint-url=http://localhost:4566 iam list-open-id-connect-providers
15+
16+
wait_for_localstack:
17+
./wait-for-localstack.sh
18+
echo "localstack ready"
19+
20+
start: start_kind start_docker_compose wait_for_localstack register_oidc
21+
22+
tear_down:
23+
kind delete clusters irsa-operator
24+
USER_ID=$(shell id -u) GROUP_ID=$(shell id -g) docker-compose down
25+
sudo rm -rf ./k8s-pki

‎_dev-env/README.md

+130
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# dev env
2+
3+
## caveats
4+
- localstack (community edition) doesn't enforce IAM
5+
- k8s version compatibility issue with
6+
7+
## clean up
8+
```
9+
sudo rm -rf ./k8s-pki
10+
mkdir ./k8s-pki
11+
```
12+
13+
## start the k8s cluster
14+
15+
```
16+
kind create cluster --config ./kind-config.yml
17+
sudo chmod 644 ./k8s-pki/sa.*
18+
```
19+
20+
- it will create the kubernetes cluster, the `kind` docker network we'll join later, populate the `./k8s-pki/` folder with all the kubernetes pki keys.
21+
- `kubectl get nodes` should return a `Ready` node.
22+
23+
24+
## start the other services
25+
26+
we'll start 3 other services :
27+
- aws localstack to fake aws
28+
- hydra to have an oidc provider
29+
- a local container registry (accessible from the outside at `localhost:5000`, from inside the `kind` network at `local-registry:5000`)
30+
31+
```
32+
docker-compose up -d
33+
```
34+
35+
2 short-lived containers will :
36+
- setup hydra's sqlite
37+
- load the serviceaccount `sa` keys in hydra
38+
39+
### check
40+
41+
a `docker ps` should only return only 3 containers : `hydra`, `aws-localstack` & `kind`
42+
43+
if you see one of the 2 other ones restarting, they have a problem, check their logs :
44+
- `hydra-db-migrate` logs should print `Successfully applied migrations!`
45+
- `hydra-add-keys` logs should print `JSON Web Key Set successfully imported!`
46+
47+
```
48+
curl https://localhost:4444/.well-known/openid-configuration -k
49+
curl https://localhost:4444/.well-known/jwks.json -k
50+
```
51+
52+
should return no error
53+
54+
## register the oidc provider on aws
55+
56+
register hydra as an oidc provider
57+
58+
```
59+
export AWS_ACCESS_KEY_ID=test
60+
export AWS_SECRET_ACCESS_KEY=test
61+
export AWS_REGION=us-east-1
62+
aws --endpoint-url=http://localhost:4566 iam create-open-id-connect-provider --url https://hydra.local:4444 --client-id-list sts.amazonaws.com --thumbprint-list $(./get-hydra-thumbprint.sh)
63+
```
64+
65+
NB : with set the client-id used by AWS to a value provided to the api-server (see ./kind-config.yml)
66+
67+
### check
68+
```
69+
aws --endpoint-url=http://localhost:4566 iam list-open-id-connect-providers
70+
```
71+
should return
72+
73+
```
74+
{
75+
"OpenIDConnectProviderList": [
76+
{
77+
"Arn": "arn:aws:iam::000000000000:oidc-provider/hydra.local:4444"
78+
}
79+
]
80+
}
81+
```
82+
83+
you can also get details using
84+
```
85+
aws --endpoint-url=http://localhost:4566 iam get-open-id-connect-provider --open-id-connect-provider-arn arn:aws:iam::000000000000:oidc-provider/hydra.local:4444
86+
```
87+
88+
## aws setup
89+
create : s3 bucket, upload this README, full-access to s3 bucket policy, role with the oidc provider, attach policy to role
90+
91+
```
92+
aws --endpoint-url=http://localhost:4566 s3api create-bucket --bucket irsa-test
93+
aws --endpoint-url=http://localhost:4566 s3 cp ./README.md s3://irsa-test
94+
aws --endpoint-url=http://localhost:4566 iam create-policy --policy-name my-test-policy --policy-document file://./test/policy.json
95+
aws --endpoint-url=http://localhost:4566 iam create-role --role-name my-app-role --assume-role-policy-document file://./test/trust-role.json
96+
aws --endpoint-url=http://localhost:4566 iam attach-role-policy --role-name my-app-role --policy-arn arn:aws:iam::000000000000:policy/my-test-policy
97+
```
98+
99+
## setup the webhook
100+
101+
```
102+
cd ./webhook
103+
./deploy.sh
104+
cd ..
105+
```
106+
107+
## deploy irsa-tester
108+
```
109+
kubectl create -f ./test/irsa-tester.yml
110+
```
111+
112+
### check
113+
```
114+
k exec irsa-tester -- env | grep AWS
115+
```
116+
117+
should return
118+
```
119+
AWS_ROLE_ARN=arn:aws:iam::000000000000:role/my-app-role
120+
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
121+
```
122+
123+
124+
## resources
125+
126+
https://blog.mikesir87.io/2020/09/eks-pod-identity-webhook-deep-dive/
127+
128+
https://www.eksworkshop.com/beginner/110_irsa/
129+
130+
https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html

‎_dev-env/docker-compose.yml

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
version: '3.7'
2+
3+
services:
4+
local-registry:
5+
image: registry:2
6+
ports:
7+
- "5000:5000"
8+
restart: unless-stopped
9+
10+
# AWS
11+
aws-local:
12+
image: localstack/localstack:0.12.12
13+
ports:
14+
- "4566:4566"
15+
environment:
16+
- SERVICES=iam,s3,sts
17+
- DEBUG=1
18+
19+
# OIDC
20+
hydra.local:
21+
image: oryd/hydra:v1.9.0-alpha.3-sqlite
22+
ports:
23+
- "4444:4444" # Public port
24+
- "4445:4445" # Admin port
25+
- "5555:5555" # Port for hydra token user
26+
environment:
27+
- DSN=sqlite:///var/lib/sqlite/db.sqlite?_fk=true
28+
- SERVE_TLS_KEY_PATH=/etc/config/certs/hydra.local.key
29+
- SERVE_TLS_CERT_PATH=/etc/config/certs/hydra.local.crt
30+
user: "${USER_ID}:${GROUP_ID}"
31+
command:
32+
serve -c /etc/config/hydra.yml all
33+
volumes:
34+
- type: volume
35+
source: hydra-sqlite
36+
target: /var/lib/sqlite
37+
read_only: false
38+
- type: bind
39+
source: ./oidc-provider/hydra.yml
40+
target: /etc/config/hydra.yml
41+
- type: bind
42+
source: ./oidc-provider/tls
43+
target: /etc/config/certs
44+
restart: unless-stopped
45+
depends_on:
46+
- hydra-migrate-db
47+
48+
hydra-migrate-db:
49+
image: oryd/hydra:v1.9.0-alpha.3-sqlite
50+
environment:
51+
- DSN=sqlite:///var/lib/sqlite/db.sqlite?_fk=true
52+
user: "${USER_ID}:${GROUP_ID}"
53+
command:
54+
migrate -c /etc/config/hydra.yml sql -e --yes
55+
volumes:
56+
- type: volume
57+
source: hydra-sqlite
58+
target: /var/lib/sqlite
59+
read_only: false
60+
- type: bind
61+
source: ./oidc-provider/hydra.yml
62+
target: /etc/config/hydra.yml
63+
restart: on-failure
64+
65+
hydra-add-keys:
66+
image: oryd/hydra:v1.9.0-alpha.3-sqlite
67+
environment:
68+
- DSN=sqlite:///var/lib/sqlite/db.sqlite?_fk=true
69+
- HYDRA_ADMIN_URL=https://hydra.local:4445
70+
user: "${USER_ID}:${GROUP_ID}"
71+
command:
72+
keys import my-set /etc/pki/sa.key /etc/pki/sa.pub --skip-tls-verify
73+
volumes:
74+
- type: bind
75+
source: ./k8s-pki
76+
target: /etc/pki
77+
restart: on-failure
78+
depends_on:
79+
- hydra.local
80+
81+
82+
volumes:
83+
hydra-sqlite:
84+
85+
networks:
86+
default:
87+
external:
88+
name: kind

‎_dev-env/get-hydra-thumbprint.sh

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
openssl s_client -connect localhost:4444 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin | sed 's/.*=\|://g'

‎_dev-env/kind-config.yml

+24
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
kind: Cluster
2+
apiVersion: kind.x-k8s.io/v1alpha4
3+
name: irsa-operator
4+
kubeadmConfigPatches:
5+
- |
6+
kind: ClusterConfiguration
7+
apiServer:
8+
extraArgs:
9+
service-account-issuer: "https://hydra.local:4444"
10+
service-account-key-file: "/etc/kubernetes/pki/sa.pub"
11+
service-account-signing-key-file: "/etc/kubernetes/pki/sa.key"
12+
api-audiences: "sts.amazonaws.com"
13+
14+
containerdConfigPatches:
15+
- |-
16+
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
17+
endpoint = ["http://local-registry:5000"]
18+
19+
nodes:
20+
- role: control-plane
21+
image: kindest/node:v1.20.7
22+
extraMounts:
23+
- hostPath: ./k8s-pki/
24+
containerPath: /etc/kubernetes/pki

‎_dev-env/oidc-provider/hydra.yml

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
serve:
2+
cookies:
3+
same_site_mode: Lax
4+
5+
urls:
6+
self:
7+
issuer: https://hydra.local:4444
8+
9+
secrets:
10+
system:
11+
- youReallyNeedToChangeThis
12+
13+
oidc:
14+
subject_identifiers:
15+
supported_types:
16+
- pairwise
17+
- public
18+
pairwise:
19+
salt: youReallyNeedToChangeThis
20+
21+
webfinger:
22+
oidc_discovery:
23+
supported_claims:
24+
- sub
25+
- iss

‎_dev-env/oidc-provider/tls/README.md

+14
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
key & cert used for ory tls (mandatory to be added as an oidc provider on aws)
2+
3+
4+
```
5+
openssl genrsa -out hydra.local.key 4096
6+
openssl req -new -key hydra.local -out hydra.local.csr
7+
8+
openssl req -new -x509 -sha256 -key key.pem -out cert.crt -days 365 -subj "/CN=hydra"
9+
```
10+
11+
(old school, should use SAN instead [https://geekflare.com/san-ssl-certificate/](https://geekflare.com/san-ssl-certificate/) )
12+
13+
## todo
14+
CN should include port ? (`4444`)

‎_dev-env/oidc-provider/tls/hydra.local.crt

+31
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
-----BEGIN CERTIFICATE-----
2+
MIIFPTCCAyUCFGapjFo1S6WfW2F+Ldv8EZlXj64SMA0GCSqGSIb3DQEBCwUAMFsx
3+
CzAJBgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRl
4+
cm5ldCBXaWRnaXRzIFB0eSBMdGQxFDASBgNVBAMMC2h5ZHJhLmxvY2FsMB4XDTIw
5+
MTIyMzIwMjEzMFoXDTIxMTIyMzIwMjEzMFowWzELMAkGA1UEBhMCQVUxEzARBgNV
6+
BAgMClNvbWUtU3RhdGUxITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0
7+
ZDEUMBIGA1UEAwwLaHlkcmEubG9jYWwwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAw
8+
ggIKAoICAQDD1xNXxGDUhO03+hHm3XKcu5WldW5LWUw75z/0qzXrjPUfDSyieCi6
9+
9YcU5jstWU9zApvIU0pw75MZgdoh+KRELfrprnAJIkNtnAN2AHUVqplTE9uyvkGc
10+
trMVHzZe6GZJdosSKFZvwzEhbBek1JLHFB+1FCydVhxzagK3SK1YzW9ZoMP3e58f
11+
Bbg6UvKFgJe+h17jbarbLCvxj5+HdPV6QI4+pJSZYU3jPlxhTGG4c9p39BMv24SD
12+
9Kkx0GM/4gW+pM+GZgYpDyr3nJi8wV8/Cv3kBy8hRwpKy0vqoa2kZ669PoUKO0Mc
13+
mLFMCvvMPkuL3/40/Qo3BUmRx0exqC/C/bKg3uhZ8Zm5q2gO5SAjeWrO5xJfdd6w
14+
E/pAjdTQ8Syqp3DKbY7Roz9VQOKtoLJdcVaozUOQ5ET0ESOZVVgSeP6MzCb52RQk
15+
h/JAtSFnd1xopjquVGJUm9K9FGiyufI2Uv7e18Yeq74Yh/HEI1pWVmtO/niEmwui
16+
HODElQV5aRCX0BRLcYegFlFlnFp5ti+wxH7KgGalVvykc68fXO1NKf2qWw5G/mJ2
17+
mKWm5pOpPLLYjhowHc9nDOQaehhNVlA3ZRaYRJwmPPpAVkjwr8pCnkvI28Im5ZQX
18+
wGBAB4sbelIUyDn5/Jd72ZsGT5QGuaTLT07pBtVzmCSKkk3rqOla0wIDAQABMA0G
19+
CSqGSIb3DQEBCwUAA4ICAQC2v9hbOvrU4yj5lXrpcZIyWDHOg1jjMuolVIWLnWkp
20+
io2FwuAAAzu87WDLaS4xHHveWFI5KgAK3MPJvewPZqhxOdp8MlcGKQTpc2OlXbcQ
21+
dMUHw1rqJaip4nr6uBy3qp1rJz+luPCqAcC50AUb3F7EyIbIFD/OuR36ZkdVN2+R
22+
CxBnstQyRLigvq3juAE5wDw6io1062Y4/3lEqIBLybKZft/WR4BnCcamCY0Wo/w6
23+
7y05JQ3knkCos8SZ+OLW4tK8jlALiB51fKtZdkPpK4wA5KgcuJ2aYIW7iCwK31sU
24+
DnwYyHrBUWS91d15MnmgYtpiKlHDrWaUqO+2FmbtN12nyc2fFFlESwGQSInZuzZ/
25+
Z9eTYeq9cSIa1vOlmGDcunHOvDnRqYbNTHlGXdQ13B5RjtQQTliIQ1DZHuyrpJIi
26+
Yb/QZRvm0C6+ZI7N1I9sxwL6mZoTBEggU621XYfC7J4mjWGEsg2/WYe69pWMaOmV
27+
v0XUS0SnnmsJtllvLY3mbgNWz7kWW+JQeHi3x7HDSNhj9ZE3VuY9mjZAsa7kRrkW
28+
OoWT1TH9tNWkqjTQU2fto3rQFl/DbaEvRnXNhx5jngm7I5i0MP1dM2XCEBs3vMkm
29+
zdTtmADjuMmk6fgBz0C5dPklVzOTkhvanMzLY0vaa8jBfih3AUcxILl+V3XbsQxt
30+
2Q==
31+
-----END CERTIFICATE-----

‎_dev-env/oidc-provider/tls/hydra.local.csr

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
-----BEGIN CERTIFICATE REQUEST-----
2+
MIIEoDCCAogCAQAwWzELMAkGA1UEBhMCQVUxEzARBgNVBAgMClNvbWUtU3RhdGUx
3+
ITAfBgNVBAoMGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDEUMBIGA1UEAwwLaHlk
4+
cmEubG9jYWwwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDD1xNXxGDU
5+
hO03+hHm3XKcu5WldW5LWUw75z/0qzXrjPUfDSyieCi69YcU5jstWU9zApvIU0pw
6+
75MZgdoh+KRELfrprnAJIkNtnAN2AHUVqplTE9uyvkGctrMVHzZe6GZJdosSKFZv
7+
wzEhbBek1JLHFB+1FCydVhxzagK3SK1YzW9ZoMP3e58fBbg6UvKFgJe+h17jbarb
8+
LCvxj5+HdPV6QI4+pJSZYU3jPlxhTGG4c9p39BMv24SD9Kkx0GM/4gW+pM+GZgYp
9+
Dyr3nJi8wV8/Cv3kBy8hRwpKy0vqoa2kZ669PoUKO0McmLFMCvvMPkuL3/40/Qo3
10+
BUmRx0exqC/C/bKg3uhZ8Zm5q2gO5SAjeWrO5xJfdd6wE/pAjdTQ8Syqp3DKbY7R
11+
oz9VQOKtoLJdcVaozUOQ5ET0ESOZVVgSeP6MzCb52RQkh/JAtSFnd1xopjquVGJU
12+
m9K9FGiyufI2Uv7e18Yeq74Yh/HEI1pWVmtO/niEmwuiHODElQV5aRCX0BRLcYeg
13+
FlFlnFp5ti+wxH7KgGalVvykc68fXO1NKf2qWw5G/mJ2mKWm5pOpPLLYjhowHc9n
14+
DOQaehhNVlA3ZRaYRJwmPPpAVkjwr8pCnkvI28Im5ZQXwGBAB4sbelIUyDn5/Jd7
15+
2ZsGT5QGuaTLT07pBtVzmCSKkk3rqOla0wIDAQABoAAwDQYJKoZIhvcNAQELBQAD
16+
ggIBAI6wLFUBfAqIkGrvFIWhy7PeoDKSK4wSrBgAxa8rvnLdRluiYNKITW56ay0h
17+
WRyntGjmR/4JJ9PZXQSDpZAvajtoO8UOkTjxZgc1IvS3GTbM0BIrl2sADWba9kSm
18+
HjNd9qemzkJ4JwWBq8k0GpwK5uWckEXKtPaDpiNnerqsge9p5e7hLCjL41n+aGVQ
19+
0LjzwUm/nvzxMEx6elHrxREhVZPxnqUzU7LQO4DrizbCJZ5p2WlbX8P7Xbm4mUtz
20+
s0NPW/TYmJH8NIVIzd6+6A75KRQrMtNSuIWIgfokFy7/fEJc9L+COFMAQGTKGiGO
21+
BHcXhvcNVRm+h10q7WwR0KdeAC60/QtgAl763G2zS1/QkN3Oe2eCSfEW1L3Bi3cA
22+
czL1E4iXH2G2YiAEfRe2UbSMcq1ydppMipUs9aXg4XQ88pgSOwqw7Pphz8zKZGjl
23+
+fVcgdMPQRYUs+xpmHZ2BMP/hesUzdp43+EY3kFf5sez6r/uw7DvGL/ojk6A7tBT
24+
uhF4Ok0ocR5PmXMijaSQvi9k/wnSJbMaJRXOavicCShw7gDqrBTyDoSUIX39IqXl
25+
BigpRXuxCEFNqgiKbR8R1647tCLMoqRtiuDKfQXnyBb/3ik9n93Tv+lZtQLbY1oC
26+
B32HQAvftNpAS0DZij1FyBl2Mj9raaW8mI9RR3GIUOCzx5D+
27+
-----END CERTIFICATE REQUEST-----

0 commit comments

Comments
 (0)