Feat/s3 storage #2

Closed
maaz519 wants to merge 30 commits from feat/s3-storage into main
3 changed files with 58 additions and 4 deletions
Showing only changes of commit 6fc03fa4c3 - Show all commits
+4
View File
@@ -54,6 +54,10 @@ for the full, commented list. Highlights:
app scopes. The dev HS256 path (`APP_SECRETS`) stays for local/tests. app scopes. The dev HS256 path (`APP_SECRETS`) stays for local/tests.
- **Media storage:** `MEDIA_DIR` + `PUBLIC_URL` configure the **dev** local-disk store; for - **Media storage:** `MEDIA_DIR` + `PUBLIC_URL` configure the **dev** local-disk store; for
prod, bind the `StoragePort` to object storage (see topology) — the API/SDK don't change. prod, bind the `StoragePort` to object storage (see topology) — the API/SDK don't change.
- **Notifications (Web Push):** set `VAPID_PUBLIC_KEY` / `VAPID_PRIVATE_KEY` / `VAPID_SUBJECT`
to enable push (unset → the engine no-ops). ⚠️ Presence (the "are you viewing this thread?"
gate) is **in-memory / single-instance** today — with N>1 replicas, back `PresenceService`
with Redis so the projector on one replica sees focus from another.
- **⚠️ `IIOS_DEV_TOKENS` MUST be `0`/unset in production.** It exposes `/v1/dev/*` - **⚠️ `IIOS_DEV_TOKENS` MUST be `0`/unset in production.** It exposes `/v1/dev/*`
(unauthenticated token minting, webhook injection, chaos, retention sweep). This is the (unauthenticated token minting, webhook injection, chaos, retention sweep). This is the
single most important prod-hardening flag. single most important prod-hardening flag.
+51 -1
View File
@@ -243,6 +243,54 @@ sequenceDiagram
--- ---
### 5.11 Notifications (push, presence-gated)
The engine reacts to `message.sent` and pushes to **absent** recipients through a swappable
**NotificationPort** (Web Push/VAPID now; email/FCM later). The DB holds only push
subscriptions + a per-thread mute flag; the reusable "who has an unread" feed stays `IiosInboxItem`.
| Method | Path | Body | Returns |
|---|---|---|---|
| GET | `/v1/notifications/vapid-public-key` | — | `{key}` — the public VAPID key the client needs to subscribe |
| POST | `/v1/notifications/subscribe` | `{kind?, endpoint, keys:{p256dh, auth}, userAgent?}` | `{ok:true}` — store/refresh the caller's push subscription |
| DELETE | `/v1/notifications/subscribe` | `{endpoint}` | `{ok:true}` |
| POST | `/v1/threads/:id/mute` · `/unmute` | — | `{threadId, muted}` — per-thread notification mute for the caller |
**Presence (socket):** the app emits `focus_thread` `{threadId | null}` on the `/message`
namespace whenever the foreground conversation changes (or the tab blurs). This is how the
engine knows you're *viewing* a thread — **room membership ≠ viewing**, because the sidebar
joins every thread room for live updates. `GET /v1/threads` returns each thread's `muted`.
**The three gates (fail-closed, in the notification policy — not the kernel):**
1. **Policy** — DM always; a group message only if you were `@`-mentioned **or** it's a reply to you.
2. **Presence** — skip if you're currently focused on that thread (you saw it live).
3. **Mute** — skip if you muted the thread.
A dead subscription (Web Push `404/410`) is pruned. DM-vs-group is read from the opaque
`membership` thread attribute here in the *notification policy*; the kernel never branches on it.
```mermaid
flowchart TB
SENT["message.sent (outbox → bus)"] --> PROJ["NotificationProjector"]
PROJ --> LOOP{{"for each recipient (never the sender)"}}
LOOP --> G1{"POLICY: DM? · @mention? · reply-to-you?"}
G1 -->|no| X1["skip"]
G1 -->|yes| G2{"PRESENCE: focused on this thread?"}
G2 -->|yes| X2["skip — seen live"]
G2 -->|no| G3{"MUTE: thread muted?"}
G3 -->|yes| X3["skip"]
G3 -->|no| SUBS["load subscriptions"] --> PORT["NotificationPort.deliver()"]
PORT --> WP["Web Push (VAPID)"]
WP -->|sent| OUT["→ browser push service → service worker → OS notification"]
WP -->|"gone (404/410)"| PRUNE["prune dead subscription"]
```
**Client (SDK layer, `lib/notifications.ts` → belongs in `@insignia/iios-kernel-client`):**
`registerPush()` (permission → register service worker → `PushManager.subscribe` with the
VAPID key → POST the subscription), `muteThread(threadId, muted)`, and `MessageSocket.focus(threadId)`.
A service worker renders the OS notification on `push` and deep-links to the thread on click.
---
## 6. SDK reference ## 6. SDK reference
Two layers: a low-level `RestClient` (+ socket) and per-domain React hook packages. Two layers: a low-level `RestClient` (+ socket) and per-domain React hook packages.
@@ -256,6 +304,7 @@ Methods (all return typed promises):
- **Messaging:** `listThreads()`, `createThread({membership?, creatorRole?, subject?})`, `addParticipant(threadId, userId, role?)`, `getThreadMessages(threadId)`, `sendMessage(threadId, content, {attachment?, parentInteractionId?, mentions?, idempotencyKey?})` - **Messaging:** `listThreads()`, `createThread({membership?, creatorRole?, subject?})`, `addParticipant(threadId, userId, role?)`, `getThreadMessages(threadId)`, `sendMessage(threadId, content, {attachment?, parentInteractionId?, mentions?, idempotencyKey?})`
- **Reactions / pins / saves (annotations):** `MessageSocket.react(threadId, interactionId, emoji)` · `pin(...)` · `save(...)` (generic `annotate` under the hood); `listMyAnnotated('save')` for a cross-thread saved list. Subscribe to the `annotation` event for live updates. - **Reactions / pins / saves (annotations):** `MessageSocket.react(threadId, interactionId, emoji)` · `pin(...)` · `save(...)` (generic `annotate` under the hood); `listMyAnnotated('save')` for a cross-thread saved list. Subscribe to the `annotation` event for live updates.
- **Media:** `uploadMedia(file, {onProgress?}) → {contentRef, mimeType, sizeBytes, checksumSha256, kind}` (presign → PUT-with-progress → normalized ref); `mediaUrl(contentRef) → signed view URL` (cached, short-lived). *This plumbing is identical for every app, so it lives in the SDK; the app only renders by `kind`.* - **Media:** `uploadMedia(file, {onProgress?}) → {contentRef, mimeType, sizeBytes, checksumSha256, kind}` (presign → PUT-with-progress → normalized ref); `mediaUrl(contentRef) → signed view URL` (cached, short-lived). *This plumbing is identical for every app, so it lives in the SDK; the app only renders by `kind`.*
- **Notifications:** `registerPush()` (service worker + `PushManager.subscribe` + POST subscription), `muteThread(threadId, muted)`, `MessageSocket.focus(threadId)` (presence signal). The engine (projector + Web Push port) is server-side; the SDK/app own registration + rendering.
- **Inbox:** `listInboxItems(state?)`, `patchInboxItem(id, {state, reason?})` - **Inbox:** `listInboxItems(state?)`, `patchInboxItem(id, {state, reason?})`
- **Support:** `createTicket({subject, priority?, threadId?})`, `escalate(threadId, subject?)`, `listTickets('mine'|'assigned')`, `patchTicket(id, state)`, `requestCallback({...})`, `createQueue(name)`, `joinQueue(id)`, `joinDefaultQueue()`, `setAvailability(state)` - **Support:** `createTicket({subject, priority?, threadId?})`, `escalate(threadId, subject?)`, `listTickets('mine'|'assigned')`, `patchTicket(id, state)`, `requestCallback({...})`, `createQueue(name)`, `joinQueue(id)`, `joinDefaultQueue()`, `setAvailability(state)`
- **Routing:** `createBinding(input)`, `listBindings()`, `simulateRoute({interactionId, originChannelType, originRef?})`, `listRouteDecisions(state?)`, `approveDecision(id)`, `denyDecision(id)` - **Routing:** `createBinding(input)`, `listBindings()`, `simulateRoute({interactionId, originChannelType, originRef?})`, `listRouteDecisions(state?)`, `approveDecision(id)`, `denyDecision(id)`
@@ -333,7 +382,7 @@ Run against a live service (from `packages/iios-service`, `node scripts/<name>`)
| `smoke-capability.mjs` | governed egress + real HTTP provider (needs `IIOS_PROVIDER_URL_EMAIL`) | | `smoke-capability.mjs` | governed egress + real HTTP provider (needs `IIOS_PROVIDER_URL_EMAIL`) |
| `smoke-tenant.mjs` | cross-tenant 403 + list isolation | | `smoke-tenant.mjs` | cross-tenant 403 + list isolation |
Automated unit/integration suite: `pnpm test` (192 tests). Import-boundary check: `pnpm boundary`. Automated unit/integration suite: `pnpm test` (205 tests). Import-boundary check: `pnpm boundary`.
--- ---
@@ -358,6 +407,7 @@ Automated unit/integration suite: `pnpm test` (192 tests). Import-boundary check
| `MEDIA_DIR` | `<tmp>/iios-media` | local media storage dir (dev `StoragePort`) | | `MEDIA_DIR` | `<tmp>/iios-media` | local media storage dir (dev `StoragePort`) |
| `MEDIA_SECRET` | `dev-media-secret` | signs media upload/download URLs | | `MEDIA_SECRET` | `dev-media-secret` | signs media upload/download URLs |
| `PUBLIC_URL` | `http://localhost:$PORT` | base used to build presigned media URLs | | `PUBLIC_URL` | `http://localhost:$PORT` | base used to build presigned media URLs |
| `VAPID_PUBLIC_KEY` / `VAPID_PRIVATE_KEY` / `VAPID_SUBJECT` | — | Web Push (notifications). Unset → push disabled (engine no-ops). Generate once: `node -e "console.log(require('web-push').generateVAPIDKeys())"` |
| `IIOS_DEV_TOKENS` | `0` | set `1` to enable `/v1/dev/*` | | `IIOS_DEV_TOKENS` | `0` | set `1` to enable `/v1/dev/*` |
| `ADAPTER_SECRETS` | `{}` | per-channel HMAC secrets (default `dev-adapter-secret`) | | `ADAPTER_SECRETS` | `{}` | per-channel HMAC secrets (default `dev-adapter-secret`) |
| `IIOS_OUTBOUND_LIMIT` / `_WINDOW_MS` | `5` / `60000` | per-(channel,target) rate limit | | `IIOS_OUTBOUND_LIMIT` / `_WINDOW_MS` | `5` / `60000` | per-(channel,target) rate limit |
+3 -3
View File
@@ -159,9 +159,9 @@ Each product SDK is a handful of React hooks — a front-end dev wires the UI, t
| Calendar/Zoom providers | Simulated sync | P9 | | Calendar/Zoom providers | Simulated sync | P9 |
| Multi-tenant scale, retention, SLOs | Not yet | P9 | | Multi-tenant scale, retention, SLOs | Not yet | P9 |
**Proof it works:** 192 automated tests pass; every capability has a runnable demo and an end-to-end smoke script; the layer-boundary check enforces the architecture. **Proof it works:** 205 automated tests pass; every capability has a runnable demo and an end-to-end smoke script; the layer-boundary check enforces the architecture.
**P9 has begun (real providers).** A production chat app (`chat-web`) now runs on IIOS with **real Supabase login** (the session port verifies real OIDC tokens via JWKS — the first stubbed port turned real), plus richer chat built generically on the kernel: **emoji reactions, pinned & saved messages, @mentions → inbox, and media sharing** (images/video/audio/docs on a swappable storage port). Each was a thin, generic addition — no chat-specific logic in the kernel — which is the reuse thesis paying off. **P9 has begun (real providers).** A production chat app (`chat-web`) now runs on IIOS with **real Supabase login** (the session port verifies real OIDC tokens via JWKS — the first stubbed port turned real), plus richer chat built generically on the kernel: **emoji reactions, pinned & saved messages, @mentions → inbox, media sharing** (images/video/audio/docs on a swappable storage port), and **presence-gated push notifications** (Web Push via a swappable notification port). Each was a thin, generic addition — no chat-specific logic in the kernel — which is the reuse thesis paying off.
--- ---
@@ -177,4 +177,4 @@ Narrative arc for the CEO: **one engine → chat → inbox → support → chann
--- ---
*Appendix — repo facts: 11 packages, 6 demo apps, one NestJS service, 53 Postgres tables across 19 migrations (kernel → messaging → inbox → support → adapters → routing → ai → calendar → annotations/mentions → media). Boundary-enforced dependency law; 192 passing tests; 8+ end-to-end smoke scripts. First real-provider swap live: Supabase auth (JWKS-verified) + a media storage port.* *Appendix — repo facts: 11 packages, 6 demo apps, one NestJS service, 54 Postgres tables across 20 migrations (kernel → messaging → inbox → support → adapters → routing → ai → calendar → annotations/mentions → media → notifications). Boundary-enforced dependency law; 205 passing tests; 8+ end-to-end smoke scripts. First real-provider swaps live: Supabase auth (JWKS-verified), a media storage port, and Web Push notifications.*