# HostItYourself — Heroku Clone MVP Plan A self-hosted PaaS running on a Raspberry Pi that auto-deploys apps from GitHub pushes. --- ## Goals - Single operator, multiple apps (target: 5–15 small services) - `git push` → live in under 2 minutes - Subdomain routing with automatic HTTPS - Central dashboard for logs, env vars, status - Low idle resource footprint (Raspberry Pi 4, 4–8 GB RAM) --- ## Architecture Overview ``` Internet │ Cloudflare DNS │ ┌──────▼──────┐ │ Caddy │ ← Reverse proxy + auto TLS │ (port 443) │ └──────┬──────┘ │ routes by subdomain ┌──────────────┼──────────────┐ │ │ │ ┌─────▼──┐ ┌─────▼──┐ ┌─────▼──┐ │ App A │ │ App B │ │Control │ │(Docker)│ │(Docker)│ │ Plane │ └────────┘ └────────┘ └───┬────┘ │ ┌────────▼────────┐ │ Build Engine │ │ (clone→build→ │ │ run container) │ └────────┬────────┘ │ ┌────────▼────────┐ │ GitHub Webhook │ │ Listener │ └─────────────────┘ ``` --- ## Components ### 1. Reverse Proxy — Caddy - Automatic TLS via Let's Encrypt (ACME DNS challenge through Cloudflare) - Dynamic config: each deployed app gets a `.yourdomain.com` subdomain - Caddy reloads config via API on each deploy; no restart needed - Runs as a systemd service, not inside Docker (avoids bootstrapping issues) ### 2. Control Plane — `hiy-server` A small Go or Node.js HTTP API + web UI. Responsibilities: - CRUD for apps (name, GitHub repo URL, branch, env vars, port) - Trigger manual deploys - Proxy log streams from running containers - Store state in a local SQLite database (`~/.hiy/db.sqlite`) - Exposes REST API consumed by the dashboard and CLI Dashboard pages: - App list with status badges (running / building / stopped / crashed) - Per-app: logs, env vars editor, deploy history, resource sparklines - System overview: CPU, RAM, disk on the Pi ### 3. GitHub Webhook Listener - Receives `push` events from GitHub (configured per-repo in GitHub settings) - Validates HMAC signature (`X-Hub-Signature-256`) - Filters on configured branch (default: `main`) - Enqueues a deploy job; responds 200 immediately - Can also be triggered via the dashboard or CLI ### 4. Build Engine — `hiy-builder` Sequential steps run in a build container: ``` 1. git clone --depth=1 /build// 2. Detect build strategy: a. Dockerfile present → docker build b. Procfile + recognised language → buildpack (Cloud Native Buildpacks via `pack`) c. static/ directory → Caddy file server image 3. docker build -t hiy/: . 4. docker stop hiy- (if running) 5. docker run -d --name hiy- \ --env-file /etc/hiy/.env \ --restart unless-stopped \ --network hiy-net \ hiy/: 6. Update Caddy upstream to new container 7. Prune old images (keep last 2) ``` Build logs are streamed to the control plane and stored per-deploy. ### 5. App Runtime — Docker - Each app runs in its own container on the `hiy-net` bridge network - Containers are **not** port-exposed to host; only Caddy reaches them - Resource limits set at start: default 512 MB RAM, 0.5 CPU (configurable per app) - `--restart unless-stopped` handles crash recovery - Persistent data: apps that need storage mount named volumes (`hiy--data`) ### 6. CLI — `hiy` Thin shell script or Go binary for operator convenience: ```bash hiy apps # list apps hiy create myapp --repo # register new app hiy deploy myapp # trigger manual deploy hiy logs myapp -f # tail logs hiy env:set myapp KEY=value # set env var hiy env:get myapp # list env vars hiy restart myapp hiy destroy myapp ``` --- ## Data Model (SQLite) ```sql apps ( id TEXT PRIMARY KEY, -- slug e.g. "my-api" repo_url TEXT NOT NULL, branch TEXT DEFAULT 'main', port INTEGER NOT NULL, -- internal container port created_at DATETIME, updated_at DATETIME ) deploys ( id TEXT PRIMARY KEY, -- uuid app_id TEXT REFERENCES apps(id), sha TEXT, -- git commit sha status TEXT, -- queued|building|success|failed log TEXT, -- full build log triggered_by TEXT, -- webhook|manual|cli started_at DATETIME, finished_at DATETIME ) env_vars ( app_id TEXT REFERENCES apps(id), key TEXT, value TEXT, -- encrypted at rest (age or libsodium) PRIMARY KEY (app_id, key) ) ``` --- ## Generic Infrastructure (Non-Functional Requirements) ### Security | Layer | Mechanism | |---|---| | SSH access | Key-only, disable password auth, non-standard port | | Firewall | `ufw`: allow only 22, 80, 443 inbound | | Fail2ban | Bans IPs with repeated SSH/HTTP failures | | Webhook auth | HMAC-SHA256 signature verification | | Env var encryption | Encrypted at rest with `age`; decrypted into container env at deploy time | | Container isolation | No `--privileged`, no host network, drop capabilities | | Dashboard auth | HTTP Basic Auth behind Caddy (or simple JWT session) | | Secrets never in logs | Build logs redact env var values | | OS updates | `unattended-upgrades` for security patches | ### Monitoring - **Metrics**: [Netdata](https://github.com/netdata/netdata) (single binary, ~50 MB RAM) — CPU, RAM, disk, network, per-container stats - **Uptime checks**: [Gatus](https://github.com/TwiN/gatus) — HTTP health checks per app, alerts via email/Telegram - **Alerting thresholds**: disk > 80%, RAM > 85%, any app down > 2 min - **Dashboard link**: Netdata and Gatus UIs accessible via subdomains on the Pi ### Logging - Container stdout/stderr captured by Docker logging driver - `hiy logs ` tails `docker logs` - Control plane stores last 10,000 lines per app in SQLite (ring buffer) - Optional: ship to a free Loki/Grafana Cloud tier for persistence ### Backups Daily cron job: 1. `sqlite3 ~/.hiy/db.sqlite .dump > backup.sql` 2. Tar all env var files from `/etc/hiy/` 3. Copy to an attached USB drive or remote (rclone to S3/Backblaze B2) 4. Retain 30 days of backups ### Deployment of the Platform Itself The platform (Caddy, hiy-server, hiy-builder) is managed via a single `docker-compose.yml` at `~/hiy-platform/`. Upgrade process: ```bash git pull origin main docker compose up -d --build ``` Caddy runs as a systemd service outside Compose (avoids chicken-and-egg on port 443). --- ## Repository Layout ``` hostityourself/ ├── plan.md ← this file ├── server/ ← control plane API + dashboard │ ├── main.go (or index.ts) │ ├── db/ │ ├── routes/ │ └── ui/ ← simple HTML/HTMX dashboard ├── builder/ ← build engine scripts │ ├── build.sh │ └── detect-strategy.sh ├── cli/ ← hiy CLI tool │ └── hiy.sh ├── proxy/ │ └── Caddyfile.template ├── infra/ │ ├── docker-compose.yml ← platform compose file │ ├── ufw-setup.sh │ ├── fail2ban/ │ └── backup.sh └── docs/ ├── setup.md ← Pi OS bootstrap guide └── add-app.md ``` --- ## MVP Milestones ### M1 — Foundation (Week 1–2) - [ ] Pi OS setup: Raspberry Pi OS Lite, Docker, Caddy, ufw, fail2ban - [ ] `hiy-server` skeleton: SQLite, REST endpoints for apps + deploys - [ ] Manual deploy via CLI: clone → build → run container → Caddy config reload ### M2 — Auto-Deploy (Week 3) - [ ] GitHub webhook listener integrated into server - [ ] HMAC validation - [ ] Build queue (single worker, sequential builds) - [ ] Build log streaming to dashboard ### M3 — Dashboard (Week 4) - [ ] App list + status - [ ] Log viewer (last 500 lines, live tail via SSE) - [ ] Env var editor - [ ] Deploy history ### M4 — Hardening (Week 5) - [x] Env var encryption at rest (AES-256-GCM via `HIY_SECRET_KEY`; transparent plaintext passthrough for migration) - [x] Resource limits on containers (per-app `memory_limit` + `cpu_limit`; defaults 512m / 0.5 CPU) - [x] Netdata + Gatus setup (`monitoring` compose profile; `infra/gatus.yml`) - [x] Backup cron job (`infra/backup.sh` — SQLite dump + env files + git repos; local + rclone remote) - [x] Dashboard auth (multi-user sessions, bcrypt, API keys — done in earlier milestone) ### M5 — Polish (Week 6) - [ ] Buildpack detection (Dockerfile / Node / Python / static) - [ ] `hiy` CLI binary - [ ] `docs/setup.md` end-to-end bootstrap guide - [ ] Smoke test: deploy a real app end-to-end from a GitHub push --- ## Hardware Recommendation | Component | Recommendation | |---|---| | Pi model | Raspberry Pi 4 or 5, 4 GB RAM minimum, 8 GB preferred | | Storage | 64 GB+ A2-rated microSD **or** USB SSD (much faster builds) | | Network | Wired Ethernet; configure static IP or DHCP reservation | | DNS | Cloudflare — free, supports ACME DNS challenge for wildcard TLS | | Domain | Any registrar; point `*.yourdomain.com` → home IP + Cloudflare proxy | | Power | Official Pi USB-C PSU + UPS hat (PiJuice or Geekworm X120) | --- ## Key Design Decisions & Trade-offs | Decision | Rationale | |---|---| | Docker over bare processes | Isolation, restart policy, easy image rollback | | Caddy over Nginx/Traefik | Automatic HTTPS, simple API-driven config, single binary | | SQLite over Postgres | Zero-ops, sufficient for one operator and dozens of apps | | Sequential build queue | Avoids saturating the Pi CPU/RAM; builds rarely overlap for one person | | Buildpacks as optional | Most personal projects have a Dockerfile; buildpacks add complexity for MVP | | No Kubernetes | Massive overkill for a single Pi and one person | | Age encryption for env vars | Simple, modern, no daemon needed vs. Vault |