❯ apps · Running

Outillage CLI personnalisé pour la gestion du homelab Custom CLI Tooling for Homelab Management

Construction d'un CLI Node.js et d'un dashboard web pour automatiser les opérations courantes du homelab — mode maintenance des nœuds, health checks et rapports d'état du cluster. Building a custom Node.js CLI and web dashboard to automate common homelab operations — node maintenance mode, health checks, and cluster state reporting.

· 6 min de lecture 6 min read ·
k3s-clustertoolingclidevtools
🇫🇷 Lire cet article en français

The first time I put a node into maintenance by hand, kubectl drain hung for long minutes without printing anything — the time it took Longhorn to evacuate its replicas in silence. Enough to think the cluster had crashed at every update, until I understood what it was actually waiting for. That kind of anxious silence, repeated at every maintenance operation, eventually justified a real tool rather than a mental checklist.

The need for dedicated tooling

Some cluster operations are repetitive and error-prone when run manually:

  • Putting a node into maintenance mode (drain + cordon) before an update
  • Checking the overall state of the cluster in a single command
  • Restoring a node after maintenance (uncordon + verification)

A bash script would work, but a structured Node.js tool is more maintainable, testable and extensible. The result: homelab-admin, a CLI + web interface deployed on the cluster.

Project structure

apps/homelab-admin/
├── src/
│   ├── commands/
│   │   ├── node-maintenance.ts   # drain/cordon/uncordon
│   │   ├── health-check.ts       # overall cluster state
│   │   └── storage-report.ts     # Longhorn state
│   ├── lib/
│   │   ├── kubernetes.ts         # kubectl wrapper client
│   │   └── longhorn.ts           # Longhorn API
│   └── index.ts                  # CLI entrypoint
└── web/                          # Web dashboard (optional)

The maintenance command

Putting a node into maintenance is a multi-step procedure:

homelab-admin node maintain worker-1

# What it does:
# 1. kubectl cordon worker-1           (blocks new pods)
# 2. kubectl drain worker-1 --ignore-daemonsets --delete-emptydir-data
# 3. Disable Longhorn scheduling on the node
# 4. Wait for the Longhorn replicas to be evicted
# 5. Print: "worker-1 is in maintenance. Safe to reboot."

Handling step 4 (waiting for the Longhorn eviction) was the trickiest part — the tool polls the Longhorn API every 10 seconds until no active replica remains on the node.

Health check

$ homelab-admin health

Cluster: HEALTHY
Nodes:   6/6 Ready
Pods:    142/142 Running (0 pending, 0 failed)
Storage: 10 volumes Healthy, 0 Degraded
Ingress: 12 active routes

⚠  worker-3: memory 87% (1.74Gi/2Gi)

The health check aggregates several sources: the Kubernetes API, the Longhorn API, and the VictoriaMetrics metrics. The terminal format is optimised for a quick read.

Deployment on the cluster

homelab-admin is deployed as a Kubernetes pod with a ServiceAccount holding the necessary RBAC permissions:

rules:
- apiGroups: [""]
  resources: ["nodes", "pods", "namespaces"]
  verbs: ["get", "list", "patch"]
- apiGroups: [""]
  resources: ["pods/eviction"]
  verbs: ["create"]
- apiGroups: ["apps"]
  resources: ["deployments", "daemonsets"]
  verbs: ["get", "list"]

The web interface is reachable at homelab-admin.olig-homelab-2025.win — a minimalist dashboard showing the cluster state in real time, with no need for kubectl.

Lessons learned

Minimal RBAC: grant only the strictly necessary permissions. An administration tool should not hold cluster-admin even though it would be simpler — the attack surface in case of a compromise is far too wide.

Generous timeouts for the drain: kubectl drain can take several minutes on nodes hosting pods with PodDisruptionBudgets or active Longhorn volumes. Configure a 10-minute timeout in the tool, with a progress display.

k3s-clustertoolingclidevtools