pal-e-platform

Self-hosted Internal Developer Platform on k3s, managed entirely through OpenTofu.

A single tofu apply provisions a complete IDP: git hosting, CI/CD pipelines, container registry, object storage, SSO, PostgreSQL, GPU inference, and full observability -- all running on a single-node k3s cluster with Tailscale mesh networking for zero-config ingress and TLS.

Architecture

Woodpecker CI --> Harbor Registry --> ArgoCD --> k3s
     |                                           |
  Forgejo (git push)                    Tailscale Funnels (TLS)

12 OpenTofu modules compose the platform:

Module	What it deploys
networking	Tailscale operator, ACL policies, subnet router, ingress funnels
monitoring	Prometheus, Grafana, Loki, Blackbox exporter, DORA metrics exporter
forgejo	Self-hosted git forge (Gitea fork)
ci	Woodpecker CI server + CNPG PostgreSQL cluster
harbor	CNCF container registry with vulnerability scanning
storage	MinIO (S3-compatible), buckets, IAM policies
keycloak	SSO/OIDC identity provider with custom themes
database	CloudNativePG operator, shared postgres namespace, backup verification
ops	NVIDIA GPU plugin, Ollama, embedding worker, tofu state backup CronJobs
admin	SSO landing page / admin dashboard
staging	Pre-production validation namespace
hetzner-edge	Reverse proxy VPS for custom domains (Caddy + Tailscale)

Documentation

Doc	Description
Architecture	Module structure, providers, state management, dependency graph
Monitoring	Prometheus, Grafana, Loki, DORA exporter, alerting
Database	CNPG operator, shared postgres namespace, backup verification
Networking	Tailscale funnels, network policies, namespace isolation
Secrets	Salt GPG pipeline, make tofu-secrets, secret inventory
Hetzner Edge Node	Reverse proxy architecture for custom domains
Keycloak SMTP	Per-realm email configuration
CI Bootstrap Resilience	Spike: admin bypass, clone retry, recovery patterns

Quick Start

make tofu-secrets    # Render secrets from Salt pillar (requires sudo)
make tofu-plan       # Plan changes (auto-validates secrets first)
make tofu-apply      # Apply changes
make tofu-fmt        # Format all .tf files recursively

Project Layout

terraform/
  main.tf                  # Root orchestrator -- wires 12 modules
  modules/{12 domains}/    # One module per platform concern
  network-policies.tf      # Default-deny ingress per namespace
  dashboards/              # Grafana dashboard JSON (8 dashboards)
  providers.tf             # 5 providers: kubernetes, helm, tailscale, minio, hcloud
  versions.tf              # Provider version constraints + kubernetes backend
salt/
  pillar/secrets/          # GPG-encrypted secret values
  states/                  # Host configuration (firewall, kernel, packages, SSH)
Makefile                   # tofu-secrets, apply, plan, salt shortcuts

Related Repositories

Repo	Purpose
pal-e-services	Service onboarding terraform (consumes this platform)
pal-e-deployments	Kustomize overlays for ArgoCD
pal-e-docs	Knowledge base and project management API
pal-e-app	SvelteKit frontend