A template repository for building Nebari Software Packs - Kubernetes applications that deploy on the Nebari platform with optional routing, TLS, and OIDC authentication.
- What is a Nebari Software Pack?
- Prerequisites
- Getting Started
- Repository Structure
- The NebariApp CRD
- Example 1: Vanilla YAML (Plain Manifests)
- Example 2: Kustomize (Nginx)
- Example 3: Helm - Basic Pack (Nginx)
- Example 4: Helm - Auth-Aware Pack (FastAPI)
- Example 5: Helm - Wrapping an Existing Chart (Podinfo)
- How Authentication Works
- Local Development
- CI/CD Pipeline
- Deploying to a Nebari Cluster
- Customizing for Your Own Application
- Troubleshooting
A software pack is any Kubernetes deployment that includes a NebariApp custom resource. The NebariApp tells the nebari-operator to auto-configure:
- Routing - Creates an HTTPRoute on the shared Envoy Gateway
- TLS - Provisions a certificate via cert-manager
- Authentication - Sets up Keycloak OIDC via an Envoy Gateway SecurityPolicy
The NebariApp CRD is the integration point between your application and the Nebari platform. How you deploy the rest of your application is up to you - Helm charts, Kustomize overlays, and plain YAML manifests are all supported. All three are first-class deployment methods in ArgoCD.
graph LR
User -->|HTTPS| EG[Envoy Gateway]
EG -->|No session?| KC[Keycloak]
KC -->|Auth code| EG
EG -->|IdToken cookie| HR[HTTPRoute]
HR --> SVC[Service]
SVC --> Pod
style EG fill:#e1f5fe
style KC fill:#fff3e0
style HR fill:#e8f5e9
Depending on your deployment method:
For local development (optional):
-
Use this template - Click "Use this template" on GitHub to create your own repo
-
Clone your new repo
git clone https://github.com/YOUR-ORG/YOUR-REPO.git cd YOUR-REPO -
Pick an example that matches your use case:
No tooling dependencies:
examples/vanilla-yaml/- Plain YAML manifests, justkubectl applyexamples/kustomize-nginx/- Kustomize overlays for per-environment config
Helm-based:
examples/basic-nginx/- Simplest possible Helm chartexamples/auth-fastapi/- Custom app that reads auth tokensexamples/wrap-existing-chart/- Wrapping an existing Helm chart (most common for Helm)
-
Search and replace
my-packwith your pack name:# Preview changes grep -r "my-pack" examples/vanilla-yaml/ # Replace (using your pack name) find . -type f -name "*.yaml" -o -name "*.tpl" -o -name "*.txt" -o -name "Makefile" | \ xargs sed -i 's/my-pack/your-pack-name/g'
-
Deploy locally to test:
# Vanilla YAML (simplest) kubectl apply -f examples/vanilla-yaml/deployment.yaml \ -f examples/vanilla-yaml/service.yaml kubectl port-forward svc/my-pack 8080:80 # Or with Helm helm install test examples/basic-nginx/chart/ kubectl port-forward svc/test-my-pack 8080:80 # Open http://localhost:8080
nebari-software-pack-template/
.github/workflows/
build-image.yaml # Build + publish auth-fastapi image to GHCR
lint.yaml # Manifest validation (all examples)
test.yaml # Integration tests on kind cluster
test-integration.yaml # NebariApp integration tests (full stack)
release.yaml # Helm package + GitHub release + gh-pages index
examples/
vanilla-yaml/ # Example 1: Plain Kubernetes manifests
deployment.yaml # nginx Deployment
service.yaml # ClusterIP Service
nebariapp.yaml # NebariApp CRD resource
README.md
kustomize-nginx/ # Example 2: Kustomize-based pack
base/
kustomization.yaml # References base resources
deployment.yaml
service.yaml
nebariapp.yaml
overlays/
dev/ # Dev overlay: dev hostname, no auth
kustomization.yaml
nebariapp-patch.yaml
production/ # Prod overlay: prod hostname, auth + groups
kustomization.yaml
nebariapp-patch.yaml
README.md
basic-nginx/ # Example 3: Simplest Helm chart
chart/
Chart.yaml
values.yaml
templates/
_helpers.tpl # Name, label, selector helpers
nebariapp.yaml # NebariApp CRD (conditional)
deployment.yaml # Kubernetes Deployment
service.yaml # ClusterIP Service
NOTES.txt # Post-install instructions
README.md
auth-fastapi/ # Example 4: Custom app reading IdToken
app/
main.py # FastAPI reading IdToken cookie
requirements.txt
templates/index.html # User info display
Dockerfile
chart/ # Same structure as basic-nginx
README.md
wrap-existing-chart/ # Example 5: Wrapping podinfo via Helm
chart/
Chart.yaml # Has podinfo as a dependency
Chart.lock
values.yaml # Podinfo overrides + NebariApp config
templates/
_helpers.tpl
nebariapp.yaml # Points to podinfo's service
NOTES.txt
README.md
dev/
Makefile # Local dev with full Nebari stack on kind
.cache/ # (gitignored) Cloned nebari-operator scripts
docs/
nebariapp-crd-reference.md # Full NebariApp field reference
auth-flow.md # Authentication flow details
.gitignore
.editorconfig
LICENSE # Apache 2.0
README.md # This file
The NebariApp custom resource is the integration point between your pack and the Nebari platform. When you create a NebariApp, the nebari-operator watches for it and automatically configures routing, TLS, and authentication.
Here's a fully annotated example:
apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
metadata:
name: my-pack
spec:
# The domain where your app will be accessible
hostname: my-pack.nebari.example.com
# The Kubernetes Service that should receive traffic
service:
name: my-pack # Service name in the same namespace
port: 80 # Service port (1-65535)
# Optional: path-based routing rules
routing:
routes:
- pathPrefix: / # Match all paths (default behavior)
pathType: PathPrefix # PathPrefix or Exact
tls:
enabled: true # Auto-provision TLS certificate (default: true)
# Optional: OIDC authentication
auth:
enabled: true # Require login (default: false)
provider: keycloak # keycloak or generic-oidc
provisionClient: true # Auto-create Keycloak client (default: true)
redirectURI: / # OAuth callback path
scopes: # OIDC scopes to request
- openid
- profile
- email
groups: # Restrict to specific groups (optional)
- admin
enforceAtGateway: true # Create SecurityPolicy at gateway (default: true)
# Which gateway to use: "public" (default) or "internal"
gateway: publicThe NebariApp is just a Kubernetes resource. It can live in a plain YAML file, a Kustomize base, or a Helm template. In Helm charts, you can make the NebariApp conditional so the chart works both standalone and on Nebari:
{{- if .Values.nebariapp.enabled }}
apiVersion: reconcilers.nebari.dev/v1
kind: NebariApp
...
{{- end }}With plain YAML or Kustomize, the NebariApp manifest is always present. When deploying standalone, simply skip that file or exclude it from your apply command.
For the complete field reference, see docs/nebariapp-crd-reference.md.
The simplest possible pack. Plain Kubernetes manifests with no tooling
dependencies beyond kubectl.
What it demonstrates:
- Lowest barrier to entry
- NebariApp as a plain YAML file alongside Deployment and Service
- No templating or tooling required
# Deploy standalone (skip the NebariApp)
kubectl apply -f examples/vanilla-yaml/deployment.yaml \
-f examples/vanilla-yaml/service.yaml
kubectl port-forward svc/my-pack 8080:80
# Open http://localhost:8080
# Deploy on Nebari (edit nebariapp.yaml hostname first)
kubectl apply -f examples/vanilla-yaml/See examples/vanilla-yaml/README.md for the full walkthrough.
Uses Kustomize overlays to manage environment-specific NebariApp configuration. Same nginx app as the vanilla example, but with structured per-environment patches.
What it demonstrates:
- Kustomize base with overlays for dev and production
- Patching hostname and auth settings per environment
- No Helm dependency
# Preview the dev overlay
kubectl kustomize examples/kustomize-nginx/overlays/dev/
# Deploy the dev overlay on Nebari
kubectl apply -k examples/kustomize-nginx/overlays/dev/
# Deploy the production overlay (auth enabled, group-restricted)
kubectl apply -k examples/kustomize-nginx/overlays/production/See examples/kustomize-nginx/README.md for the full walkthrough.
The simplest possible Helm-based pack. Deploys a stock nginx container with optional Nebari integration via a conditional NebariApp template.
What it demonstrates:
- Minimum viable Helm chart structure
- Conditional NebariApp template (
nebariapp.enabledtoggle) - Toggling between standalone and Nebari modes
# Deploy standalone
helm install test-basic examples/basic-nginx/chart/
kubectl port-forward svc/test-basic-my-pack 8080:80
# Open http://localhost:8080
# Deploy on Nebari
helm install my-pack examples/basic-nginx/chart/ \
--set nebariapp.enabled=true \
--set nebariapp.hostname=my-pack.nebari.example.com
# Deploy on Nebari with auth
helm install my-pack examples/basic-nginx/chart/ \
--set nebariapp.enabled=true \
--set nebariapp.hostname=my-pack.nebari.example.com \
--set nebariapp.auth.enabled=trueSee examples/basic-nginx/README.md for the full walkthrough.
A custom Python app that reads the IdToken cookie set by Envoy Gateway after Keycloak authentication. Shows how to consume authenticated user identity.
What it demonstrates:
- Building a custom container image
- Reading the IdToken cookie to get user claims
- Rendering user info (username, email, groups)
The key code in app/main.py:
def get_id_token(request: Request) -> str | None:
"""Extract IdToken from Envoy Gateway's OIDC filter cookies.
Envoy Gateway sets a cookie named IdToken-<suffix> where <suffix>
is an 8-char hex string derived from the SecurityPolicy UID.
"""
for name, value in request.cookies.items():
if name.startswith("IdToken-"):
return value
return None# Run locally (shows "Not Authenticated" - no IdToken cookie without Envoy Gateway)
docker run -p 8000:8000 ghcr.io/nebari-dev/nebari-software-pack-template/auth-fastapi-example:latest
# Deploy on Nebari with auth
helm install my-pack examples/auth-fastapi/chart/ \
--set nebariapp.enabled=true \
--set nebariapp.hostname=my-pack.nebari.example.comSee examples/auth-fastapi/README.md for the full walkthrough.
This is the most realistic Helm use case. Most Helm-based packs wrap existing software - you don't write your own Deployment or Service. You add the upstream chart as a dependency and create a NebariApp that points to its service.
What it demonstrates:
- Chart.yaml dependency on an existing chart
- Overriding upstream values
- NebariApp pointing to the upstream service
- No custom Deployment or Service templates needed
# Chart.yaml - just add the dependency
dependencies:
- name: podinfo
version: 6.10.1
repository: oci://ghcr.io/stefanprodan/chartsThe only template you write is nebariapp.yaml, which points to podinfo's service:
spec:
service:
name: {{ .Release.Name }}-podinfo # Upstream service
port: 9898You don't rewrite the app. You just connect it to Nebari.
# Build dependencies
helm dependency update examples/wrap-existing-chart/chart/
# Deploy standalone
helm install test-podinfo examples/wrap-existing-chart/chart/
kubectl port-forward svc/test-podinfo-podinfo 9898:9898
# Deploy on Nebari
helm install my-pack examples/wrap-existing-chart/chart/ \
--set nebariapp.enabled=true \
--set nebariapp.hostname=my-pack.nebari.example.comSee examples/wrap-existing-chart/README.md for the full walkthrough.
When a NebariApp has auth.enabled: true, the nebari-operator creates an Envoy
Gateway SecurityPolicy that handles the full OIDC flow:
1. User visits my-pack.nebari.example.com
2. Envoy Gateway checks for a valid session cookie
- No cookie? Redirect to Keycloak login page
3. User authenticates with Keycloak
4. Keycloak redirects back with an authorization code
5. Envoy Gateway exchanges the code for tokens
6. Envoy Gateway sets cookies:
- IdToken-<suffix> (JWT with user claims)
- AccessToken-<suffix>
- RefreshToken-<suffix>
(<suffix> is an 8-char hex derived from the SecurityPolicy UID)
7. Request (now with cookies) is forwarded to your app
What the operator automates:
- Creates a Keycloak OIDC client (when
provisionClient: true) - Stores client credentials in a Kubernetes Secret
- Creates an Envoy Gateway SecurityPolicy with the OIDC configuration
- Creates an HTTPRoute directing traffic to your service
- Provisions a TLS certificate via cert-manager
What your app can do:
- Read the
IdToken-*cookies to get the JWT (see Example 4) - Decode the JWT payload to extract claims:
preferred_username,email,groups - The JWT signature is already verified by Envoy Gateway - you only need to base64-decode the payload
If your app handles OAuth natively (like Grafana), set enforceAtGateway: false.
The operator will still provision the OIDC client and store credentials in a Secret,
but won't create a SecurityPolicy. Your app reads the credentials from the Secret
and handles the OAuth flow itself.
For more details, see docs/auth-flow.md.
The dev/ directory provides a Makefile for local development with
kind. Running any up-* target automatically
creates a kind cluster with the full Nebari infrastructure stack - MetalLB,
Envoy Gateway, cert-manager, Keycloak, and the nebari-operator - so every
example deploys with NebariApp enabled, routing, TLS, and authentication
working just like a real Nebari cluster.
The first make up-* run takes ~5-10 minutes (cluster and infrastructure
setup). Subsequent runs reuse the existing cluster and are fast.
cd dev
# Deploy vanilla YAML example
make up-vanilla
# Deploy kustomize example (dev overlay)
make up-kustomize
# Deploy Helm nginx example
make up-basic
# Deploy podinfo Helm example
make up-podinfo
# Deploy FastAPI Helm example (auth enabled, uses pre-built GHCR image)
make up-fastapi
# Update /etc/hosts with NebariApp hostnames
make update-hosts
# Delete the kind cluster
make downEach up-* target deploys with NebariApp enabled at https://my-pack.nebari.local,
waits for the NebariApp Ready condition, and updates /etc/hosts so you can access
the app in your browser.
The local dev environment does not include ArgoCD. If you need to develop or test the ArgoCD Application that wraps your software pack, you'll need to set that up separately. In the future, Nebari will support pointing at a local Git repo (and creating a temporary one if none is provided) so ArgoCD-based workflows can be tested locally without an external repository.
Runs on every push and PR. Validates all examples:
kubectl apply --dry-run=clientfor the vanilla YAML examplekubectl kustomizefor each Kustomize overlayhelm lintandhelm templatefor each Helm chart (both NebariApp enabled and disabled)
Runs on pushes to main that modify examples/auth-fastapi/app/ or the
Dockerfile, plus manual dispatch. Builds and publishes the auth-fastapi
example image to ghcr.io/nebari-dev/nebari-software-pack-template/auth-fastapi-example.
Runs on every push and PR. Standalone integration tests on a kind cluster:
- Creates a kind cluster
- Deploys each example with
nebariapp.enabled=false(no operator required) - Waits for pods and runs HTTP health checks via port-forward
- Validates that each example works as a standalone Kubernetes deployment
Runs on pushes to main and PRs that modify examples/, dev/, or the workflow
file. Tests each example with nebariapp.enabled=true on a full Nebari
infrastructure stack:
- Creates a kind cluster with MetalLB, Envoy Gateway, cert-manager, and Keycloak
- Installs the nebari-operator from the latest published release
- Deploys each example with NebariApp enabled and a
*.nebari.localhostname - Verifies NebariApp reaches
Readycondition (HTTPRoute created, TLS configured) - For auth-enabled examples (kustomize production, auth-fastapi), verifies SecurityPolicy is created
This catches bugs in NebariApp configuration, operator compatibility, and routing setup that the standalone test cannot detect.
Manual dispatch. Packages and releases a Helm chart:
- Packages the selected chart as a
.tgz - Creates a GitHub release with the package
- Updates the
gh-pagesbranch with a Helm repo index
ArgoCD supports all three deployment methods. Set the source section based on
your pack type:
ArgoCD with Helm:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-pack
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/YOUR-ORG/YOUR-REPO.git
targetRevision: main
path: examples/basic-nginx/chart # or your chart path
helm:
valuesObject:
nebariapp:
enabled: true
hostname: my-pack.nebari.example.com
auth:
enabled: true
destination:
server: https://kubernetes.default.svc
namespace: my-pack
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=trueArgoCD with Kustomize:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-pack
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/YOUR-ORG/YOUR-REPO.git
targetRevision: main
path: examples/kustomize-nginx/overlays/production
# ArgoCD auto-detects kustomization.yaml
destination:
server: https://kubernetes.default.svc
namespace: my-pack
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=trueArgoCD with plain YAML (directory):
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: my-pack
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/YOUR-ORG/YOUR-REPO.git
targetRevision: main
path: examples/vanilla-yaml
directory:
recurse: false
destination:
server: https://kubernetes.default.svc
namespace: my-pack
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true# Edit nebariapp.yaml with your hostname first
kubectl apply -f examples/vanilla-yaml/ \
--namespace my-packkubectl apply -k examples/kustomize-nginx/overlays/production/ \
--namespace my-packhelm install my-pack ./chart/ \
--namespace my-pack \
--create-namespace \
--set nebariapp.enabled=true \
--set nebariapp.hostname=my-pack.nebari.example.com \
--set nebariapp.auth.enabled=true# Check the NebariApp status
kubectl get nebariapp -n my-pack
# Check conditions (should all be True when ready)
kubectl describe nebariapp my-pack -n my-pack
# Expected conditions:
# RoutingReady: True - HTTPRoute created
# TLSReady: True - Certificate provisioned
# AuthReady: True - SecurityPolicy created (if auth enabled)
# Ready: True - All components ready| Token | Replace with | Where |
|---|---|---|
my-pack |
Your pack name (lowercase, hyphenated) | All YAML files, chart files, Makefile |
OWNER/REPO or YOUR-ORG/YOUR-REPO |
Your GitHub org/repo | Workflows, README |
The placeholder my-pack is valid YAML/Helm syntax, so linting passes on the
template repo as-is.
In values.yaml (Helm) or directly in deployment.yaml (vanilla/Kustomize):
# Helm values.yaml
image:
repository: your-registry/your-image
tag: "1.0.0"# Plain YAML or Kustomize deployment.yaml
containers:
- name: your-app
image: your-registry/your-image:1.0.0Common additions:
- ConfigMap - Configuration files mounted into pods
- Secret - Credentials (in Helm, use
lookup()for ArgoCD safety) - PersistentVolumeClaim - Persistent storage
- ServiceAccount - Pod identity for RBAC
For Helm charts, add these to templates/. For Kustomize, add them to base/
and reference them in kustomization.yaml. For vanilla YAML, add them as
additional files.
If your app serves multiple paths:
# In the NebariApp spec (any deployment method)
routing:
routes:
- pathPrefix: /api
pathType: PathPrefix
- pathPrefix: /dashboard
pathType: PathPrefix# In the NebariApp spec (any deployment method)
auth:
enabled: true
groups:
- admin
- data-science-teamThe namespace needs the label that opts it in for nebari-operator processing:
kubectl label namespace my-pack nebari.dev/managed=trueThe NebariApp's spec.service.name doesn't match any Service in the namespace.
Check the service name:
kubectl get svc -n my-packFor Helm-based wrapped charts, the service name follows the upstream chart's
naming convention (usually <release>-<chart-name>).
- Check that
auth.enabledistruein the NebariApp spec - Check that the nebari-operator is running:
kubectl get pods -n nebari-system -l app=nebari-operator
- Check the NebariApp conditions:
Look for
kubectl describe nebariapp my-pack -n my-pack
AuthReadycondition.
- Check cert-manager is running:
kubectl get pods -n cert-manager
- Check the Certificate resource:
kubectl get certificate -n my-pack kubectl describe certificate my-pack-tls -n my-pack
- Ensure you're accessing through the configured hostname (not via port-forward)
- Check that the SecurityPolicy was created:
kubectl get securitypolicy -n my-pack
- Check Envoy Gateway logs:
kubectl logs -n envoy-gateway-system -l app=envoy-gateway
For OCI-based dependencies, ensure Helm 3.8+ is installed:
helm version
helm dependency update examples/wrap-existing-chart/chart/Apache 2.0 - see LICENSE.