229 lines
8.6 KiB
Markdown
Executable File
229 lines
8.6 KiB
Markdown
Executable File
# Homelab — k3s Cluster
|
||
|
||
2-node k3s cluster (1 manager, 1 worker) running a self-hosted homelab stack on `ratboo.me`.
|
||
|
||
## Architecture
|
||
|
||
### Nodes
|
||
|
||
| Node | Role | OS | IP | Runtime |
|
||
|------|------|----|----|---------|
|
||
| **dogbox** | control-plane | Fedora 40 Server | `10.0.1.2` | k3s server + containerd |
|
||
| **mac-worker** | worker | Ubuntu 25.10 (OrbStack VM) | `192.168.139.12` | k3s agent + containerd |
|
||
|
||
### Overview
|
||
|
||
```
|
||
Internet
|
||
│
|
||
Cloudflare DNS
|
||
*.ratboo.me
|
||
│
|
||
┌──────────────────────────┼──────────────────────────┐
|
||
│ dogbox (manager) │
|
||
│ Fedora 40 · 10.0.1.2 │
|
||
│ │
|
||
│ ┌─────────────────┐ ┌──────────────────────┐ │
|
||
│ │ k3s server │ │ Traefik (k3s) │ │
|
||
│ │ control-plane │ │ :443 websecure │ │
|
||
│ └─────────────────┘ │ Let's Encrypt + CF │ │
|
||
│ └──────────┬───────────┘ │
|
||
│ ┌─────────────────┐ │ │
|
||
│ │ traefik-internal │ Routes to pods across │
|
||
│ │ :80 LB 10.0.1.250│ both nodes via CNI │
|
||
│ │ (MetalLB L2) │ │ │
|
||
│ └─────────────────┘ │ │
|
||
│ Longhorn │ │
|
||
└──────────────┬─────────────────────┼─────────────────┘
|
||
│ │
|
||
NFS /dogstore k3s cluster
|
||
│ │
|
||
┌──────────────┴─────────────────────┼─────────────────┐
|
||
│ mac-worker (worker) │
|
||
│ Ubuntu 25.10 · OrbStack VM │
|
||
│ 192.168.139.12 │
|
||
│ │
|
||
│ Longhorn · workload pods │
|
||
└──────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
### Networking
|
||
|
||
**Public ingress** — k3s bundles Traefik, configured via `HelmChartConfig` in `traefik-config`. TLS terminates at Traefik using Let's Encrypt with Cloudflare DNS-01 challenge. HTTP automatically redirects to HTTPS.
|
||
|
||
| Public hostname | Service |
|
||
|-----------------|---------|
|
||
| `plex.ratboo.me` | Plex |
|
||
| `sonarr.ratboo.me` | Sonarr |
|
||
| `radarr.ratboo.me` | Radarr |
|
||
| `paperless.ratboo.me` | Paperless-ngx |
|
||
| `mealie.ratboo.me` | Mealie |
|
||
| `watch.ratboo.me` | Seerr |
|
||
|
||
**Internal ingress** — A separate Traefik instance (`traefik-internal`) listens on `10.0.1.250:80`, served by MetalLB L2. A DNS rewrite points `*.internal` to that IP. Internal services use Traefik `IngressRoute` CRDs with `ingressClass: traefik-internal`.
|
||
|
||
| Internal hostname | Service |
|
||
|-------------------|---------|
|
||
| `homepage.rat` | Homepage |
|
||
| `glance.rat` | Glance |
|
||
| `headlamp.dog` | Headlamp |
|
||
|
||
**Cluster-only (no ingress):** Prowlarr, Bazarr, qBittorrent, Zerobyte.
|
||
|
||
### Storage
|
||
|
||
| Mechanism | Use |
|
||
|-----------|-----|
|
||
| **Longhorn** (`storageClass: longhorn`, replica count 2) | Small config/state PVCs — Traefik ACME (128Mi), app configs (1–20Gi), Paperless Postgres/Redis, Mealie data, Seerr, Zerobyte |
|
||
| **NFS via hostPath `/dogstore`** | Large/shared data — Plex media + transcode, Sonarr/Radarr/qBittorrent/unpackerr data trees, Paperless documents, Homepage/Glance configs |
|
||
|
||
### Secrets
|
||
|
||
SOPS + age encryption. All secrets live in `secrets/secrets.enc.yaml`, encrypted at rest. The age key lives at `/etc/sops/age/keys.txt` on each node. Referenced secrets include Cloudflare API tokens, database passwords, Plex claim tokens, and application API keys.
|
||
|
||
## Namespaces
|
||
|
||
| Namespace | Contents |
|
||
|-----------|----------|
|
||
| `kube-system` | k3s Traefik, `traefik-config` (HelmChartConfig + redirect middleware) |
|
||
| `longhorn-system` | Longhorn storage |
|
||
| `media` | Plex, Sonarr, Radarr, Bazarr, Prowlarr, qBittorrent, unpackerr |
|
||
| `apps` | Mealie, Homepage, Glance, Headlamp, Seerr, Zerobyte, Paperless-ngx + Postgres + Redis |
|
||
|
||
## Services
|
||
|
||
| Chart | Namespace | Services | Notes |
|
||
|-------|-----------|----------|-------|
|
||
| traefik-config | kube-system | Traefik HelmChartConfig overlay | Cloudflare DNS-01, ACME on Longhorn |
|
||
| traefik-internal | — | Internal Traefik instance | LB via MetalLB at `10.0.1.250` |
|
||
| metallb | — | MetalLB L2 pool | Single-IP pool for internal LB |
|
||
| media | media | Plex, Sonarr, Radarr, Bazarr, Prowlarr, qBittorrent, unpackerr | Media stack with `/dogstore` data paths |
|
||
| paperless | apps | Paperless-ngx, Redis, PostgreSQL | Postgres 15, Redis 7 |
|
||
| mealie | apps | Mealie (v3.14.0) | Gemini API integration for recipes |
|
||
| dashboards | apps | Homepage, Glance | Internal-only via `traefik-internal` |
|
||
| headlamp | apps | Headlamp | K8s dashboard, internal-only via `traefik-internal` |
|
||
| utils | apps | Seerr, Zerobyte | Seerr public, Zerobyte cluster-only |
|
||
|
||
|
||
## Prerequisites
|
||
|
||
- Two Linux machines with NFS `/dogstore` mounted on both
|
||
- `curl`, `helm`, `kubectl`, `sops`, `age` installed
|
||
|
||
## Bootstrap
|
||
|
||
### 1. Install k3s server (manager node)
|
||
|
||
```bash
|
||
./scripts/bootstrap.sh server
|
||
```
|
||
|
||
This prints the worker join command at the end.
|
||
|
||
### 2. Install k3s agent (worker node)
|
||
|
||
```bash
|
||
K3S_URL="https://<manager-ip>:6443" K3S_TOKEN="<token>" ./scripts/bootstrap.sh agent
|
||
```
|
||
|
||
### 3. Install Longhorn
|
||
|
||
```bash
|
||
./scripts/bootstrap.sh longhorn
|
||
```
|
||
|
||
### 4. Set up SOPS encryption
|
||
|
||
Generate an age keypair (run on each node):
|
||
|
||
```bash
|
||
./scripts/bootstrap.sh sops-keygen
|
||
```
|
||
|
||
Copy the public key into `.sops.yaml`, replacing the placeholder. Then encrypt your secrets:
|
||
|
||
```bash
|
||
# Edit secrets/secrets.enc.yaml — replace REPLACE_WITH_* placeholders with real values
|
||
sops -e -i secrets/secrets.enc.yaml
|
||
```
|
||
|
||
### 5. Apply secrets
|
||
|
||
```bash
|
||
./scripts/bootstrap.sh apply-secrets
|
||
```
|
||
|
||
### 6. Deploy all charts
|
||
|
||
```bash
|
||
./scripts/bootstrap.sh deploy
|
||
```
|
||
|
||
Or deploy individually:
|
||
|
||
```bash
|
||
helm upgrade --install metallb charts/metallb -n kube-system --wait
|
||
helm upgrade --install traefik-internal charts/traefik-internal -n kube-system --wait
|
||
# Traefik config goes in kube-system (managed by k3s)
|
||
helm upgrade --install traefik-config charts/traefik-config -n kube-system
|
||
|
||
kubectl create namespace apps
|
||
helm upgrade --install headlamp charts/headlamp -n apps
|
||
|
||
helm upgrade --install dashboards charts/dashboards -n apps
|
||
helm upgrade --install paperless charts/paperless -n apps
|
||
helm upgrade --install mealie charts/mealie -n apps
|
||
|
||
kubectl create namespace media
|
||
helm upgrade --install media charts/media -n media
|
||
|
||
helm upgrade --install utils charts/utils -n apps
|
||
|
||
|
||
```
|
||
|
||
## Verifying
|
||
|
||
```bash
|
||
# Check all pods
|
||
kubectl get pods -A
|
||
|
||
# Check ingress routes
|
||
kubectl get ingress -A
|
||
|
||
# Test a specific service
|
||
curl -I https://mealie.ratboo.me
|
||
```
|
||
|
||
## Secret Rotation
|
||
|
||
1. Decrypt: `sops secrets/secrets.enc.yaml` (opens in `$EDITOR`)
|
||
2. Change the values
|
||
3. Save and close (SOPS re-encrypts automatically)
|
||
4. Apply: `./scripts/bootstrap.sh apply-secrets`
|
||
5. Restart affected pods: `kubectl rollout restart deployment/<name> -n <namespace>`
|
||
|
||
## Repo Structure
|
||
|
||
```
|
||
homelab/
|
||
├── README.md
|
||
├── AGENTS.md
|
||
├── .sops.yaml
|
||
├── scripts/
|
||
│ └── bootstrap.sh
|
||
├── charts/
|
||
│ ├── traefik-config/ # k3s Traefik overrides (HelmChartConfig)
|
||
│ ├── traefik-internal/ # Separate internal Traefik instance
|
||
│ ├── metallb/ # MetalLB L2 for internal LB IP
|
||
│ ├── media/ # Plex, *arr stack, qBittorrent, unpackerr
|
||
│ ├── paperless/ # Paperless-ngx + Postgres + Redis
|
||
│ ├── mealie/ # Mealie recipe manager
|
||
│ ├── dashboards/ # Homepage + Glance (internal only)
|
||
│ ├── headlamp/ # Headlamp K8s dashboard (internal only)
|
||
│ └── utils/ # Seerr + Zerobyte
|
||
└── secrets/
|
||
└── secrets.enc.yaml
|
||
```
|
||
|