Hostityourself/plan.md

HostItYourself — Heroku Clone MVP Plan

A self-hosted PaaS running on a Raspberry Pi that auto-deploys apps from GitHub pushes.

---

Goals

• Single operator, multiple apps (target: 5–15 small services)
• git push → live in under 2 minutes
• Subdomain routing with automatic HTTPS
• Central dashboard for logs, env vars, status
• Low idle resource footprint (Raspberry Pi 4, 4–8 GB RAM)

---

Architecture Overview

                        Internet
                           │
                     Cloudflare DNS
                           │
                    ┌──────▼──────┐
                    │   Caddy     │  ← Reverse proxy + auto TLS
                    │  (port 443) │
                    └──────┬──────┘
                           │  routes by subdomain
            ┌──────────────┼──────────────┐
            │              │              │
      ┌─────▼──┐     ┌─────▼──┐    ┌─────▼──┐
      │ App A  │     │ App B  │    │Control │
      │(Docker)│     │(Docker)│    │ Plane  │
      └────────┘     └────────┘    └───┬────┘
                                       │
                              ┌────────▼────────┐
                              │   Build Engine  │
                              │ (clone→build→   │
                              │  run container) │
                              └────────┬────────┘
                                       │
                              ┌────────▼────────┐
                              │  GitHub Webhook │
                              │    Listener     │
                              └─────────────────┘

---

Components

1. Reverse Proxy — Caddy

• Automatic TLS via Let's Encrypt (ACME DNS challenge through Cloudflare)
• Dynamic config: each deployed app gets a <appname>.yourdomain.com subdomain
• Caddy reloads config via API on each deploy; no restart needed
• Runs as a systemd service, not inside Docker (avoids bootstrapping issues)

2. Control Plane — hiy-server

A small Go or Node.js HTTP API + web UI. Responsibilities:

• CRUD for apps (name, GitHub repo URL, branch, env vars, port)
• Trigger manual deploys
• Proxy log streams from running containers
• Store state in a local SQLite database (~/.hiy/db.sqlite)
• Exposes REST API consumed by the dashboard and CLI

Dashboard pages:

• App list with status badges (running / building / stopped / crashed)
• Per-app: logs, env vars editor, deploy history, resource sparklines
• System overview: CPU, RAM, disk on the Pi

3. GitHub Webhook Listener

• Receives push events from GitHub (configured per-repo in GitHub settings)
• Validates HMAC signature (X-Hub-Signature-256)
• Filters on configured branch (default: main)
• Enqueues a deploy job; responds 200 immediately
• Can also be triggered via the dashboard or CLI

4. Build Engine — hiy-builder

Sequential steps run in a build container:

1. git clone --depth=1 <repo> /build/<appname>/<sha>
2. Detect build strategy:
   a. Dockerfile present → docker build
   b. Procfile + recognised language → buildpack (Cloud Native Buildpacks via `pack`)
   c. static/ directory → Caddy file server image
3. docker build -t hiy/<appname>:<sha> .
4. docker stop hiy-<appname> (if running)
5. docker run -d --name hiy-<appname> \
       --env-file /etc/hiy/<appname>.env \
       --restart unless-stopped \
       --network hiy-net \
       hiy/<appname>:<sha>
6. Update Caddy upstream to new container
7. Prune old images (keep last 2)

Build logs are streamed to the control plane and stored per-deploy.

5. App Runtime — Docker

• Each app runs in its own container on the hiy-net bridge network
• Containers are not port-exposed to host; only Caddy reaches them
• Resource limits set at start: default 512 MB RAM, 0.5 CPU (configurable per app)
• --restart unless-stopped handles crash recovery
• Persistent data: apps that need storage mount named volumes (hiy-<appname>-data)

6. CLI — hiy

Thin shell script or Go binary for operator convenience:

hiy apps                        # list apps
hiy create myapp --repo <url>   # register new app
hiy deploy myapp                # trigger manual deploy
hiy logs myapp -f               # tail logs
hiy env:set myapp KEY=value     # set env var
hiy env:get myapp               # list env vars
hiy restart myapp
hiy destroy myapp

---

Data Model (SQLite)

apps (
  id          TEXT PRIMARY KEY,   -- slug e.g. "my-api"
  repo_url    TEXT NOT NULL,
  branch      TEXT DEFAULT 'main',
  port        INTEGER NOT NULL,   -- internal container port
  created_at  DATETIME,
  updated_at  DATETIME
)

deploys (
  id          TEXT PRIMARY KEY,   -- uuid
  app_id      TEXT REFERENCES apps(id),
  sha         TEXT,               -- git commit sha
  status      TEXT,               -- queued|building|success|failed
  log         TEXT,               -- full build log
  triggered_by TEXT,             -- webhook|manual|cli
  started_at  DATETIME,
  finished_at DATETIME
)

env_vars (
  app_id      TEXT REFERENCES apps(id),
  key         TEXT,
  value       TEXT,               -- encrypted at rest (age or libsodium)
  PRIMARY KEY (app_id, key)
)

---

Generic Infrastructure (Non-Functional Requirements)

Security

Layer	Mechanism
SSH access	Key-only, disable password auth, non-standard port
Firewall	ufw: allow only 22, 80, 443 inbound
Fail2ban	Bans IPs with repeated SSH/HTTP failures
Webhook auth	HMAC-SHA256 signature verification
Env var encryption	Encrypted at rest with age; decrypted into container env at deploy time
Container isolation	No --privileged, no host network, drop capabilities
Dashboard auth	HTTP Basic Auth behind Caddy (or simple JWT session)
Secrets never in logs	Build logs redact env var values
OS updates	unattended-upgrades for security patches

Monitoring

• Metrics: Netdata (single binary, ~50 MB RAM) — CPU, RAM, disk, network, per-container stats
• Uptime checks: Gatus — HTTP health checks per app, alerts via email/Telegram
• Alerting thresholds: disk > 80%, RAM > 85%, any app down > 2 min
• Dashboard link: Netdata and Gatus UIs accessible via subdomains on the Pi

Logging

• Container stdout/stderr captured by Docker logging driver
• hiy logs <app> tails docker logs
• Control plane stores last 10,000 lines per app in SQLite (ring buffer)
• Optional: ship to a free Loki/Grafana Cloud tier for persistence

Backups

Daily cron job:

1. sqlite3 ~/.hiy/db.sqlite .dump > backup.sql
2. Tar all env var files from /etc/hiy/
3. Copy to an attached USB drive or remote (rclone to S3/Backblaze B2)
4. Retain 30 days of backups

Deployment of the Platform Itself

The platform (Caddy, hiy-server, hiy-builder) is managed via a single docker-compose.yml at ~/hiy-platform/. Upgrade process:

git pull origin main
docker compose up -d --build

Caddy runs as a systemd service outside Compose (avoids chicken-and-egg on port 443).

---

Repository Layout

hostityourself/
├── plan.md                  ← this file
├── server/                  ← control plane API + dashboard
│   ├── main.go (or index.ts)
│   ├── db/
│   ├── routes/
│   └── ui/                  ← simple HTML/HTMX dashboard
├── builder/                 ← build engine scripts
│   ├── build.sh
│   └── detect-strategy.sh
├── cli/                     ← hiy CLI tool
│   └── hiy.sh
├── proxy/
│   └── Caddyfile.template
├── infra/
│   ├── docker-compose.yml   ← platform compose file
│   ├── ufw-setup.sh
│   ├── fail2ban/
│   └── backup.sh
└── docs/
    ├── setup.md             ← Pi OS bootstrap guide
    └── add-app.md

---

MVP Milestones

M1 — Foundation (Week 1–2)

• Pi OS setup: Raspberry Pi OS Lite, Docker, Caddy, ufw, fail2ban
• hiy-server skeleton: SQLite, REST endpoints for apps + deploys
• Manual deploy via CLI: clone → build → run container → Caddy config reload

M2 — Auto-Deploy (Week 3)

• GitHub webhook listener integrated into server
• HMAC validation
• Build queue (single worker, sequential builds)
• Build log streaming to dashboard

M3 — Dashboard (Week 4)

• App list + status
• Log viewer (last 500 lines, live tail via SSE)
• Env var editor
• Deploy history

M4 — Hardening (Week 5)

• Env var encryption at rest
• Resource limits on containers
• Netdata + Gatus setup
• Backup cron job
• Dashboard auth

M5 — Polish (Week 6)

• Buildpack detection (Dockerfile / Node / Python / static)
• hiy CLI binary
• docs/setup.md end-to-end bootstrap guide
• Smoke test: deploy a real app end-to-end from a GitHub push

---

Hardware Recommendation

Component	Recommendation
Pi model	Raspberry Pi 4 or 5, 4 GB RAM minimum, 8 GB preferred
Storage	64 GB+ A2-rated microSD or USB SSD (much faster builds)
Network	Wired Ethernet; configure static IP or DHCP reservation
DNS	Cloudflare — free, supports ACME DNS challenge for wildcard TLS
Domain	Any registrar; point *.yourdomain.com → home IP + Cloudflare proxy
Power	Official Pi USB-C PSU + UPS hat (PiJuice or Geekworm X120)

---

Key Design Decisions & Trade-offs

Decision	Rationale
Docker over bare processes	Isolation, restart policy, easy image rollback
Caddy over Nginx/Traefik	Automatic HTTPS, simple API-driven config, single binary
SQLite over Postgres	Zero-ops, sufficient for one operator and dozens of apps
Sequential build queue	Avoids saturating the Pi CPU/RAM; builds rarely overlap for one person
Buildpacks as optional	Most personal projects have a Dockerfile; buildpacks add complexity for MVP
No Kubernetes	Massive overkill for a single Pi and one person
Age encryption for env vars	Simple, modern, no daemon needed vs. Vault