maaz519
6fc03fa4c3
- API guide §5.11 Notifications: vapid-public-key / subscribe / unsubscribe / thread mute endpoints, the focus_thread presence signal, the three gates, and the engine flow diagram; SDK reference (registerPush/muteThread/focus); VAPID env; tests 205. - DEPLOYMENT: VAPID env + the in-memory-presence → Redis (multi-replica) caveat. - CEO overview: counts (205 tests, 54 tables / 20 migrations) + notifications in the "P9 has begun" note. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
122 lines
6.9 KiB
Markdown
122 lines
6.9 KiB
Markdown
# Deploying IIOS
|
||
|
||
IIOS deploys as **one runtime service** (`@insignia/iios-service`) plus **one datastore**
|
||
(Postgres). Everything else in the repo is a **library** (SDKs) or a **dev demo**, not a
|
||
deployed service.
|
||
|
||
| Artifact | Type | How it ships |
|
||
|---|---|---|
|
||
| `@insignia/iios-service` | **Runtime service** | Docker image (this guide) |
|
||
| `iios-contracts`, `iios-adapter-sdk`, `iios-kernel-client`, `iios-testkit` | Library | published to npm; imported by host apps |
|
||
| `iios-message-web`, `iios-inbox-web`, `iios-support-web`, `iios-ai-web`, `iios-meeting-web`, `iios-community-web` | Web SDK (library) | published to npm; **bundled into host-app frontends** (optionally one CDN widget) |
|
||
| `apps/*` (message-demo, ai-studio, route-admin…) | Dev demo | not deployed |
|
||
|
||
The service is a **modular monolith**: a single NestJS process serves the HTTP API, the
|
||
WebSocket (socket.io) gateway, the outbox relay, and the projectors together.
|
||
|
||
## Build the image
|
||
|
||
Build context is the **monorepo root** (the image needs the workspace + lockfile):
|
||
|
||
```bash
|
||
docker build -f packages/iios-service/Dockerfile -t iios-service:latest .
|
||
```
|
||
|
||
The multi-stage build compiles the service + its workspace deps, generates the Prisma
|
||
client, and prunes to a self-contained prod bundle (`pnpm deploy --legacy`). The runtime
|
||
image runs as the non-root `node` user and starts via `docker-entrypoint.sh`, which runs
|
||
`prisma migrate deploy` then `node dist/main.js`.
|
||
|
||
## Run it
|
||
|
||
```bash
|
||
docker run --rm -p 3200:3200 \
|
||
-e DATABASE_URL='postgresql://USER:PASS@HOST:5432/iios?schema=public' \
|
||
-e APP_SECRETS='{"portal-demo":"<secret>"}' \
|
||
-e IIOS_DEV_TOKENS=0 \
|
||
iios-service:latest
|
||
```
|
||
|
||
Health check: `GET /health` → `200`. Metrics/ops: `GET /metrics` (relay lag, projection
|
||
cursors, retention snapshot counts).
|
||
|
||
## Configuration & secrets contract
|
||
|
||
Every knob is an environment variable — see [`packages/iios-service/.env.example`](../packages/iios-service/.env.example)
|
||
for the full, commented list. Highlights:
|
||
|
||
- **Secrets** (inject from a vault, never bake into the image): `DATABASE_URL`,
|
||
`APP_SECRETS` (per-app HS256 JWT keys), `ADAPTER_SECRETS` (webhook HMAC keys),
|
||
`MEDIA_SECRET` (signs presigned media upload/download URLs).
|
||
- **Auth (real IdP):** set `SUPABASE_URL` or `AUTH_ISSUERS` to trust real OIDC access
|
||
tokens, verified against the issuer's public **JWKS** (ES256) — **no secret to store**.
|
||
`AUTH_ISSUERS` is a JSON registry `[{url, appId, orgId?}]` routing many issuers → isolated
|
||
app scopes. The dev HS256 path (`APP_SECRETS`) stays for local/tests.
|
||
- **Media storage:** `MEDIA_DIR` + `PUBLIC_URL` configure the **dev** local-disk store; for
|
||
prod, bind the `StoragePort` to object storage (see topology) — the API/SDK don't change.
|
||
- **Notifications (Web Push):** set `VAPID_PUBLIC_KEY` / `VAPID_PRIVATE_KEY` / `VAPID_SUBJECT`
|
||
to enable push (unset → the engine no-ops). ⚠️ Presence (the "are you viewing this thread?"
|
||
gate) is **in-memory / single-instance** today — with N>1 replicas, back `PresenceService`
|
||
with Redis so the projector on one replica sees focus from another.
|
||
- **⚠️ `IIOS_DEV_TOKENS` MUST be `0`/unset in production.** It exposes `/v1/dev/*`
|
||
(unauthenticated token minting, webhook injection, chaos, retention sweep). This is the
|
||
single most important prod-hardening flag.
|
||
- **`IIOS_CELL_ID`** tags which physical cell this instance serves — the hook for splitting
|
||
noisy/regulated tenants into isolated cells later, with no code change.
|
||
- **Worker timers** (`IIOS_RELAY_INTERVAL_MS`, `IIOS_RETENTION_SWEEP_INTERVAL_MS`) — see
|
||
scaling notes below.
|
||
|
||
## Topology
|
||
|
||
```
|
||
host apps ──HTTPS/WSS──► [ LB / ingress ] ──► iios-service ×N ──► Postgres (pgvector)
|
||
(embed SDKs) (WS + sticky) (API+relay+ │
|
||
projectors) (+ Redis, + OPA/CMP/MDM
|
||
as external ports — later)
|
||
```
|
||
|
||
- **Postgres** — managed (RDS / Cloud SQL / Neon). One logical DB, tenant-isolated by scope.
|
||
- **Redis** — **not required for a single instance.** Set **`REDIS_URL`** when you run
|
||
**multiple replicas with live chat**: the built-in [socket.io Redis adapter](../packages/iios-service/src/realtime/redis-io.adapter.ts)
|
||
fans room emits across all instances, so a client on replica B sees messages emitted by
|
||
replica A. Without `REDIS_URL` the in-memory adapter is used (single instance). Verified by
|
||
`scripts/smoke-realtime-cluster.mjs` (two instances + one Redis; cross-instance delivery
|
||
fails without Redis, passes with it).
|
||
> Delivery is **at-least-once** across instances — with N>1 the inline send-emit and the
|
||
> relay's outbox-bridge emit can both fire for the same message, so clients should **dedupe
|
||
> by message `id`** (the payload always carries one).
|
||
- **Platform ports** (OPA policy, CMP consent, MDM, CRRE, SAS) — today in-process permissive
|
||
stubs (`LocalDevPorts`). For production, point these at real external services; the service
|
||
already calls them **fail-closed**. The **session** port already verifies real OIDC tokens
|
||
(Supabase/JWKS) when configured.
|
||
- **Media storage** (`StoragePort`) — dev = **local disk** (`MEDIA_DIR`). ⚠️ Local disk is
|
||
**single-instance and non-durable**; with N>1 replicas or for persistence, bind it to shared
|
||
**object storage** (S3/R2/Supabase Storage). Bytes always flow **client ↔ storage directly**
|
||
via presigned URLs — they never transit the service — so this scales independently.
|
||
|
||
## Scaling & release strategy
|
||
|
||
**Horizontal scaling is safe** because the event core was hardened for it:
|
||
|
||
- The outbox relay claims rows with `FOR UPDATE SKIP LOCKED` → each event is relayed by
|
||
exactly one replica; its co-located projectors process it once (idempotent `claim()` +
|
||
the projection-cursor + idempotency-command ledgers give exactly-once effects).
|
||
- The **DLQ + replay** path means a bad deploy loses nothing — fix and replay.
|
||
|
||
**Rolling / blue-green deploys:**
|
||
|
||
1. **Migrations:** run `prisma migrate deploy` as a **one-off pre-deploy job**, then set
|
||
`IIOS_SKIP_MIGRATE=1` on the replicas so N pods don't race the migration. (For a single
|
||
instance, the default boot-time migration is fine.)
|
||
2. Use **expand-contract (backward-compatible) schema changes** so old and new pods can run
|
||
against the same schema during the roll — this is the prerequisite for zero-downtime.
|
||
3. Roll replicas with a `/health` readiness gate; drain WebSocket connections on `SIGTERM`.
|
||
4. **WebSocket ingress** needs sticky sessions (and the Redis socket.io adapter once N>1).
|
||
|
||
## Not yet built (prod-readiness gaps)
|
||
|
||
- **CI/CD pipeline** to build/push this image and run migrations.
|
||
- **Real platform-port services** (OPA/CMP/MDM) — today permissive stubs.
|
||
- **Secrets manager** integration (currently plain env).
|
||
- A **schema-compatibility gate** in CI to enforce expand-contract migrations.
|