Deployment

Production runbook for the backend — build it, migrate the stores, run each process, and harden for production traffic. The dashboard has its own build and deploy; see Dashboard.

Before deploying: meet all Requirements and set every variable in Configuration. The make commands below run from the backend repo root.

1. Build

You can ship the backend as binaries (systemd, bare processes) or container images (Kubernetes, Compose). Pick one.

Option A — binaries

make build

make build compiles the following into bin/. bin/pug is the unified CLI; every other binary is a single-purpose build of one process — an alternative to the matching pug sub-command for deployment tools that prefer one binary per unit.

There is no make migrate or make dev target. Migrations run via the pug CLI or the standalone migrate binaries (below); local all-in-one runs via the pug dev command. See the CLI reference.

Option B — container images

The repo ships one parameterized Dockerfile that builds a right-sized image for any role. Two build args select the role:

• --build-arg CMD=<path under ./cmd> — which main to compile.
• --target <app|worker|migrate> — the runtime shape (what, besides the binary, lands in the image).

The three targets exist because the roles need different files at runtime:

Each image’s entrypoint is that single-purpose binary, so you run the container with no arguments. Images are multi-arch (amd64 + arm64, cross-compiled — no QEMU) and run as distroless nonroot.

# API server
docker build --build-arg CMD=server            --target app     -t pug-server .
 
# a worker (swap the CMD path for any worker below)
docker build --build-arg CMD=workers/events    --target worker  -t pug-worker-events .
 
# a migration runner
docker build --build-arg CMD=migrate/postgres  --target migrate -t pug-migrate-postgres .

CMD paths, relative to ./cmd:

2. Migrations

Run migrations once against each store before the first boot, and again on every upgrade. They are idempotent — re-running them on an up-to-date store is safe.

# unified CLI
./bin/pug postgres migrate
./bin/pug nats migrate
./bin/pug clickhouse migrate
 
# or the standalone binaries / migrate images
./bin/pug-migrate-postgres
./bin/pug-migrate-nats
./bin/pug-migrate-clickhouse

All migration commands read connection details from the environment (DATABASE_URL, NATS_URL, CLICKHOUSE_URL). postgres migrate and clickhouse migrate default to applying all pending migrations up (--direction up); pass --direction down --num N to roll back N steps. nats migrate initializes the JetStream streams and consumers.

3. Run

Production topology: one process per role, each as its own replica. Do not use pug dev in production — it runs every component in a single process with no independent scaling or restart boundaries.

API server

./bin/pug server        # or ./bin/pug-server, or the `app` image

Listens on :3000 (override with PUG_SERVER_PORT). Place a TLS-terminating load balancer in front — see Production hardening.

Workers

There are 6 worker processes in a normal deployment (the demo worker is only for public demos). Run each as its own replica, using whichever form fits your tooling:

Scale the events worker horizontally for high ingest — run multiple replicas (see Horizontal scaling):

./bin/pug worker events        # or ./bin/pug-worker-events

Running a single small node? pug dev (alias pug start) runs the server and all workers in one process. It’s built for local dev, but it’s a legitimate way to stand up a low-volume single-box instance — you just give up independent scaling and restarts. See the CLI reference.

4. Production hardening

TLS termination

pug server listens on plain HTTP (:3000). Terminate TLS at a load balancer (nginx, Caddy, AWS ALB, Cloudflare) and proxy to port 3000. Do not expose port 3000 directly to the internet — the SDKs and dashboard send credentials (x-api-key, Authorization JWT), so TLS is required.

Secrets management

Do not put secrets in unencrypted files. Inject the following via your secrets manager (Vault, AWS Secrets Manager, Kubernetes Secrets, etc.):

CORS origins

Set PUG_CORS_ORIGINS to the exact origin(s) your dashboard runs on (e.g. https://app.example.com). Using * disables credentialed requests, which breaks dashboard sign-in.

NATS JetStream HA

Run a 3-node NATS cluster with JetStream on each node — a single node is a single point of failure for all ingestion. Start each with --jetstream --store_dir /data/nats, configure --cluster routes, and use a stream replication factor ≥ 3 so one node’s loss doesn’t lose data.

ClickHouse HA and storage

A single ClickHouse node is fine for low-to-medium volumes. For higher volumes or fault tolerance:

• Replication — run replicated tables backed by ClickHouse Keeper (or ZooKeeper) so a node loss doesn’t lose data.
• Tiered storage — an S3-compatible object store as the “cold” tier for high-retention deployments.
• ulimits — ensure nofile ≥ 262144 on all ClickHouse nodes (required).

Horizontal scaling

The events worker is the primary bottleneck at high ingest rates. Each replica is a stateless NATS JetStream consumer and scales linearly — run multiple. Other workers process lower-volume streams; start with one replica each and scale as needed.

Monitoring

Point OTEL_EXPORTER_OTLP_ENDPOINT at your collector (Grafana, Datadog, Honeycomb, …) to export traces, metrics, and logs. With it unset, Pug logs to stdout as text. See Configuration → Telemetry.

Health checks

Both the server and every worker expose Kubernetes-style probes. Wire your orchestrator’s liveness/readiness probes (or load-balancer health checks) to these:

The worker probe address is PUG_WORKER_HEALTH_ADDR (default :8090; set to off to disable). When several worker binaries share a host, give each a distinct port to avoid a bind clash. pug dev disables worker probes (it’s for local use).

Backing services keep their own health endpoints — probe those for infrastructure liveness:

Further reading

• Dashboard — build and serve the UI
• Connect your SDKs — point your apps at this instance
• CLI reference — every pug command and flag
• Configuration — all environment variables