Home Ops

Bringing my Cloud Ops skills to self-hosting at home.

Overview

Welcome to my home infrastructure repository. It is currently the starting point of my next adventure into self-hosting, homelab and home automation. After moving houses, my lab was dismantled so what better time to start fresh and document the journey.

Where possible I will be embracing Infrastructure as Code (IaC) and GitOps principles to manage my home infrastructure. This will include using tools such as Flux, Pulumi, Ansible, Renovate, and GitHub Actions.

Architecture

While most of the home operations repos will focus primarily on Kubernetes, you need to start somewhere and having a solid management plane is key.

Diagram coming soon...

Management Plane

TrueNAS Scale

See the Truenas README for details on the TrueNAS Scale setup.

Portainer

Portainer BE is used to manage the management plane services on TrueNAS Scale. See the Portainer README for details on how Portainer stacks are managed using Pulumi.

Cloudflare

See the Cloudflare README for details on how Cloudflare resources are managed using Pulumi. Resources include DNS zones, Zero Trust tunnels, and WebFinger services.

Tailscale

See the Tailscale README for details on how I have configured my Tailscale tailnet so far. Currently Tailscale configuration is managed via click-ops in the Tailscale admin console, but eventually Tailscale ACLs, DNS settings, and other configurations will be managed via Pulumi as code.

Plans

• Run Portainer BE on TrueNAS Scale for management services
  • Management only pocket-id for OIDC authentication
  • tailscale (management network subnet router)
  • netbox for documenting network and IPAM to IaC
  • dnsmasq as proxy DHCP server for PXE booting
  • matchbox for easier iPXE provisioning
  • Sidero Omni for Talos cluster creation and management
  • Gatus to monitor management plane and Kubernetes control planes later (keep it simple initially)
• On Portainer, deploy small number of services critical for other uses at home (migrate to Kubernetes later)
  • Main pocket-id for OIDC authentication
  • cloudflared to expose main pocket-id for Tailnet OIDC
  • tailscale (home network subnet router)
  • tiny-auth to protect services that do not support OIDC
  • home-assistant
  • code-server
  • seafile
• Manage network resources with Pulumi
  • unifi network (UCG Max)
  • tailscale ACLs etc
  • Cloudflare tunnels (maybe)
• Install incus-os on Dell Optiplex 7040 SFF for staging environment
  • Testing PXE booting
  • Testing Talos cluster formation
  • Testing kubernetes changes via Flux
  • WoL and power down when not needed
• Production Kubernetes cluster
  • Mix of Intel NUCs, Raspberry Pi 4s and other physical hosts and VMs
  • Talos Linux deployed via PXE booting using Sidero Omni and matchbox
  • GitOps management via flux
  • cilium and multus for CNI and load balancing
  • tailscale operator for cluster access
  • traefik for ingress
  • cert-manager for TLS certificates
  • external-dns for DNS management
  • external-secrets for secrets
  • topolvm for local storage
  • rook for volumes and S3 compatible object storage
  • democratic-csi for bulk storage on the NAS (NFS)
  • grafana stack (Mimir, Loki, Alloy) for monitoring and logging, VictoriaMetrics/Logs will also be assessed
  • spegel for image cache
  • All the applications (including those above marked for later migration)
• Experiments
  • OCI flux source
  • CDK8s to build sources to put in an OCI

Hardware

TBC

Cloud Services

TBC

DNS

TBC