feat(charts/authentik): add blueprints-sidecar to collect from cluster

[genofire]

fix:

• Documentate unoffice kubernetes Operatoren authentik#4558
• Allow defining Providers and Applications via Kubernetes Resources authentik#4119

Helm values:

serviceAccount:
  create: true
sidecar:
  blueprints:
    enabled: true
    namespace: "ALL" <1>

<1> normally just current namespace

Collects ConfigMaps and Secrets based on label and put them into goauthentik, here an example ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: goauthentik-blueprint-sidecar-test
  namespace: "default"
  labels:
    goauthentik_blueprint: "1"
data:
  test.yaml: |-
    version: 1
    metadata:
      name: sidecar-test
    entries: []

[genofire]

@BeryJu please review

[genofire]

@BeryJu i do not know where this CI error cames from:

Error: INSTALLATION FAILED: serviceaccounts "authentik-q63g50s5tb" already exists

[genofire]

solved

[benedikt-bartscher]

#156

[genofire]

thanks @benedikt-bartscher i will add the sidecar to all component

[genofire]

done - lets review again

[holmesb]

I can vouch that this sidecar works as expected. Nice work @genofire. New configmaps with label goauthentik_blueprint: "1" and:

data:
  blueprint_test.yaml: |
    <blueprint yaml>

(or any other yaml filename) result in blueprint in Authentik. Deletion of configmap removes from Authentik. Be great to get this merged. I've not tried the operator, but this seems a lower-effort way to deliver gitops blueprints. CRD & controller will have to be kept up-to-date with the blueprint spec. This is lower maintenance.

[holmesb]

Only downside is blueprints containing secrets referred to using env var tag eg !Env <app>_client_secret don't get updated. When these env vars are mounted using helm chart value: worker.envFrom[0].secretRef.name: <my_k8_secret_containing_env_vars_to_mount_in_authentik_worker> and this secret is updated (eg an oauth client_secret is changed), the env vars mounted don't change until worker restart, nor does the blueprint get reloaded and update the Authentik construct defined in the blueprint. I guess the solution would be for this sidecar to watch for changes to the worker.envFrom[0].secretRef.name secret, and reload blueprints when it does. Happy to break this out into a separate issue, as sidecar needs to be implemented first!

[genofire]

you could store the blueprint in a kubernetes Secret eigther (instatt of ConfigMap).

So your secrets are stored secure.

For your problem, maybe there could this operator works reloader

[holmesb]

Good suggestions. Long story, but need to stick with configMaps and !Env <my_env_var>. Reloader now rolling-restarts my worker pods when the secret that defines env vars changes, and I'm patching the worker deployment so it runs ak apply_blueprint to apply sidecar blueprints at startup (which have access to new env vars). When this PR is merged, I'll create a new PR, as I think applying blueprints at startup will be useful as an option beyond this use-case to prevent drift. Maybe worker.applySidecarBlueprintsAtStartup: true\false, with default of false?

Nearly gitops Authentik :-) Just need resources created by deleted blueprints to be cleaned up.

[x-0D]

It's mid of 2025 and this sidecar method works like a charm! Now able to get rid of keycloak and terraform mess to have fully-featured GitOps authentik instance managed by argocd continuously deployed configmaps. Yes, it still forces me to delete resources by hand, but it's better than nothing.

[DrummyFloyd]

It's mid of 2025 and this sidecar method works like a charm! Now able to get rid of keycloak and terraform mess to have fully-featured GitOps authentik instance managed by argocd continuously deployed configmaps. Yes, it still forces me to delete resources by hand, but it's better than nothing.

would be great if blueprint, could have a mecanism to delete related object if there not present anymore ( without the need of cahnge the state: absent

[x-0D]

would be great if blueprint, could have a mecanism to delete related object if there not present anymore ( without the need of change the state: absent

technically, there's yq e '.entries[].state = "absent"' file.yaml > result.yaml exists.

and, kiwigrid sidecar has ability to run shebang! scripts after configmaps and secrets change via path/to/script.{sh, py} declared in SCRIPT environment variable.

So, maybe something like this can be used to handle blueprints properly

(DO NOT USE IN PRODUCTION, CODE NOT TESTED)

#!/bin/python
import os
import yaml
import hashlib
import json
import copy

def hash_identification(identification):
    id_str = json.dumps(identification, sort_keys=True)
    return hashlib.sha256(id_str.encode()).hexdigest()

blueprints_dir = os.getenv("FOLDER", "/tmp/blueprints/")
sidecar_dir = os.getenv("BLUEPRINTS_FOLDER", "/blueprints/sidecar/")
state_file = os.getenv("PERSISTENT_STATE_FILE", "/tmp/blueprints_state.json") 

# Load previous state if exists
if os.path.exists(state_file):
    with open(state_file) as f:
        previous_state = json.load(f)
else:
    previous_state = {}

# Scan for YAML files
yaml_files = [f for f in os.listdir(blueprints_dir) if f.endswith(('.yaml', '.yml'))]

# Load YAML contents
yaml_contents = {}
for file in yaml_files:
    with open(os.path.join(blueprints_dir, file)) as f:
        yaml_contents[file] = yaml.safe_load(f)

# Build current state from YAML
state = {}
for filename, content in yaml_contents.items():
    state[filename] = {}
    for entry in content.get('entries', []):
        entry_id = entry.get('id') or hash_identification(entry.get('identification', {}))
        state[filename][entry_id] = entry

# Update state: add new, mark missing as absent
updated_state = copy.deepcopy(previous_state)
for filename, entries in state.items():
    if filename not in updated_state:
        updated_state[filename] = {}
    # Mark entries not in YAML as absent
    for prev_id in list(updated_state[filename].keys()):
        if prev_id not in entries:
            updated_state[filename][prev_id]['state'] = 'absent'
    # Add/update entries from YAML
    for entry_id, entry in entries.items():
        updated_state[filename][entry_id] = entry

# Prepare updated YAML contents for writing
updated_yaml_contents = copy.deepcopy(yaml_contents)
for filename, entries in updated_state.items():
    current_entries = {e.get('id') or hash_identification(e.get('identification', {})) for e in yaml_contents.get(filename, {}).get('entries', [])}
    for entry_id, entry in entries.items():
        if entry_id not in current_entries:
            if filename not in updated_yaml_contents:
                updated_yaml_contents[filename] = {'entries': []}
            updated_yaml_contents[filename]['entries'].append(entry)

# Write updated YAML files to sidecar directory
os.makedirs(sidecar_dir, exist_ok=True)
for filename, content in updated_yaml_contents.items():
    with open(os.path.join(sidecar_dir, filename), "w") as f:
        yaml.safe_dump(content, f, sort_keys=False)

# Save updated state
with open(state_file, "w") as f:
    json.dump(updated_state, f, indent=2)

Please, publish updated version of the code here if you have luck to make it kinda working. if code still needs improvements to be used in production, hide it under details spolier.

[DrummyFloyd]

would be great if blueprint, could have a mecanism to delete related object if there not present anymore ( without the need of change the state: absent

technically, there's yq e '.entries[].state = "absent"' file.yaml > result.yaml exists.

and, kiwigrid sidecar has ability to run shebang! scripts after configmaps and secrets change via path/to/script.{sh, py} declared in SCRIPT environment variable.

So, maybe something like this can be used to handle blueprints properly
(DO NOT USE IN PRODUCTION, CODE NOT TESTED)

Please, publish updated version of the code here if you have luck to make it kinda working. if code still needs improvements to be used in production, hide it under details spolier.

from what it think it shoudl not be handle by sidecar, but by Authentik itself like

new blueprint => set it in DB => if do not exist anymore => remove from DB
maybe be based on HASH of something to not overload jobs from DB .. dunno

[x-0D]

from what it think it shoudl not be handle by sidecar, but by Authentik itself like

new blueprint => set it in DB => if do not exist anymore => remove from DB maybe be based on HASH of something to not overload jobs from DB .. dunno

I limited by technology i can understand. Making changes in authentik logic should break someone's production, and making changes safe to "not break the things" are difficult without examination of the entire codebase. But scripts on sidecar are independent from the authentik version, that's the point why we can use it as sidecar now, and integrate into main codebase later.