From cea0a27118ca38c9fac8d41d5160bc8b5ded4d11 Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:18:26 +0530 Subject: [PATCH 01/14] docs(templates): email/message template module plan Design + 8-task TDD plan for the IIOS template module. Self-reviewed: provenance-write path, nullable-scope unique index, PII minimization. Co-Authored-By: Claude Opus 4.8 --- docs/email-template-module-plan.md | 277 +++++++++++++++++++++++++++++ 1 file changed, 277 insertions(+) create mode 100644 docs/email-template-module-plan.md diff --git a/docs/email-template-module-plan.md b/docs/email-template-module-plan.md new file mode 100644 index 0000000..ef5d87e --- /dev/null +++ b/docs/email-template-module-plan.md @@ -0,0 +1,277 @@ +# Email / Message Template Module — Implementation Plan + +**Owner:** Maaz · **Repo:** `iios` (`packages/iios-service`) · **Target:** first templates sending by Friday go-live. + +## Goal + +A reusable template module so **any** message the system sends — welcome, payment receipt, +onboarding reminders, drip — is produced from a stored (or ad-hoc) template with variables filled +in, then handed to the **existing** outbound pipeline. One render path, one send path, provenance +recorded for every send. + +From the July 16 meeting: *"template का एक पूरा module बनाना है… template में value भरोगे, और वो +outbound क्यू में डाल दोगे।"* + +## What already exists (the substrate — do NOT rebuild) + +- `OutboundService.send(channelType, target, payload, idempotencyKey?, scopeId?, purpose?)` — + idempotency + per-target/per-tenant rate limits + delivery ledger (`IiosOutboundCommand`). +- `CapabilityBroker` — policy gate + obligations + provider selection for egress. +- `EMAIL` is a registered channel; `EmailProvider` exists (**HTTP**, not SMTP — see Out of Scope). +- `EMAIL` is a first-class `IiosInteractionKind`; `IiosMessagePartKind` has `HTML`/`TEXT`. +- `IiosActorKind` includes `SERVICE`/`BOT` (system sender is first-class). +- `InboxModule` uses `OnModuleInit` — copy that pattern for the template seeder. + +The module adds **content/rendering** in front of this. It never talks to a provider directly. + +## Design decisions (locked) + +| # | Decision | Rationale | +|---|---|---| +| 1 | **Channel-generic**: one template per `(key, channel)`; channels `EMAIL` / `SMS` / `INTERNAL` | Vivek wants the same confirmation on email *and* SMS; INTERNAL = app-to-app, no SMTP | +| 2 | **Seed-from-files, DB-is-truth**: templates are files in the repo, seeded into the DB on boot if absent | Copy is version-controlled + code-reviewed; a later admin UI can edit the DB with no deploy; Friday needs no UI | +| 3 | **Handlebars** rendering | Auto-escapes HTML (customer names go into email → XSS risk), logic-less, no code execution, one dep | +| 4 | **Global default + scope override**: `scopeId` nullable — `NULL` = platform default, set = tenant override; resolve scoped-first-else-global | Seeds cleanly at boot (IIOS scopes are created lazily, so a boot seeder has no scope to seed into); leaves room for white-label | +| 5 | **Provenance on `IiosOutboundCommand`** (4 columns), not a new `template_snapshot` table | Rendered content is already in `payload`; only provenance is missing. Honours the SOT's intent at 4 columns | +| 6 | **`TemplateSource` = stored `{key,version?}` OR `{inline:{subject,html,text}}`** | Marketing hands over finished HTML (*"Maaz, HTML भेज सकते हैं"*); inline still renders vars + records provenance | +| 7 | **Integration pattern B**: pure `render()` + thin `sendTemplated()` composer | Single entry point for callers; provenance guaranteed by construction; `OutboundService` stays content-agnostic | +| 8 | **Caller = a service** (be-crm's system token); **recipient = a `target` address, never a login** | System mail has no user on the sending side; the recipient may have no account yet | + +## Caller & auth model + +Every send is authenticated as the **calling app/service** (e.g. be-crm via its `APP_SECRETS` +entry), verified by `SessionVerifier` like every other IIOS endpoint. The recipient is a plain +`target` string — **not** an IIOS principal and **not** required to be logged in or registered. +Sending a welcome email to an anonymous payer is the normal case: the app is the sender, the +address is data. `scopeId` for the send is derived from the caller's principal (`org/app/tenant`). + +## Data model + +### New table — `IiosMessageTemplate` + +```prisma +enum IiosTemplateChannel { + EMAIL + SMS + INTERNAL +} + +/// The template SOURCE. DB is runtime truth; platform defaults are seeded from repo files on boot. +/// Versions are immutable: a change writes a new (higher) version, never edits in place. +model IiosMessageTemplate { + id String @id @default(cuid()) + /// NULL = platform default (seeded). Set = a tenant scope's override of the same key. + scopeId String? + scope IiosScope? @relation(fields: [scopeId], references: [id], onDelete: Cascade) + key String // "welcome", "payment.receipt", "onboarding.reminder" + channel IiosTemplateChannel + locale String @default("en") + version Int @default(1) + subject String? // EMAIL only + bodyHtml String? // EMAIL / INTERNAL + bodyText String? // SMS, and EMAIL plaintext fallback + /// Declared variable names — render throws if a declared var is missing (fail loud). + variables Json? + active Boolean @default(true) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([scopeId, key, channel, locale, version]) + @@index([key, channel, locale, active]) +} +``` + +`IiosScope` needs the back-relation `messageTemplates IiosMessageTemplate[]`. + +**Resolution:** `resolve(key, channel, locale, scopeId?)` returns the highest-`version` `active` +row, preferring `scopeId = ` and falling back to `scopeId IS NULL`. No match → 404. + +### Provenance columns on `IiosOutboundCommand` + +```prisma + templateKey String? + templateVersion Int? + templateLocale String? + renderedHash String? // sha256 of the rendered content — replay/audit +``` + +All nullable: non-templated sends (if any) leave them null. + +**How provenance is written (review finding):** `OutboundService.send` **creates the command row +itself** (both the RATE_LIMITED and PENDING paths), so a caller cannot stamp provenance after the +fact without a racy update. Therefore `send()` gains one optional trailing arg — +`provenance?: { templateKey; templateVersion; templateLocale; renderedHash }` — written into the +same `create()` in both paths. `OutboundService` stays content-agnostic: it does not render or +resolve templates, it only persists four opaque strings it is handed. This is what makes +decision 7's "provenance guaranteed by construction" true. **This is a change to an existing file** +(`adapters/outbound.service.ts`) and its spec — call it out in the PR. + +### Migrations (hand-written, non-destructive) + +1. `add_message_template` — the enum + table + `IiosScope` back-relation. **Plus a partial unique + index for global rows (review finding):** the `@@unique([scopeId, key, channel, locale, version])` + does **not** prevent duplicate *platform-default* rows, because Postgres treats `NULL` scopeId as + distinct (`NULL != NULL`) — the same footgun handled in the inbox idempotency migration. Add: + `CREATE UNIQUE INDEX "IiosMessageTemplate_global_key" ON "IiosMessageTemplate"("key","channel","locale","version") WHERE "scopeId" IS NULL;` + so a double-seed or race cannot create two defaults for the same key. +2. `add_outbound_template_provenance` — the 4 nullable columns on `IiosOutboundCommand`. + +## Module structure + +``` +packages/iios-service/src/templates/ + template.channel.ts // IiosTemplateChannel re-export/helpers if needed + template.model.ts // TemplateSource, RenderedContent, CreateSendInput types + template.repository.ts // resolve(): scoped ?? global, highest active version + template.renderer.ts // Handlebars compile+render; escaping; missing-var throw + template.service.ts // render(source, vars) — pure; resolve + renderer + templated-sender.ts // sendTemplated(): render -> OutboundService.send -> stamp provenance + template.seeder.ts // OnModuleInit: seed file defaults into DB if (key,channel,locale,version) absent + template.controller.ts // POST /v1/templates/send, POST /v1/templates/preview + template.dto.ts // SendTemplateDto, PreviewTemplateDto (class-validator) + template.module.ts + seeds/ + welcome.email.ts // { key, channel, locale, version, subject, html, text, variables } + payment-receipt.email.ts + payment-receipt.sms.ts + onboarding-reminder.email.ts + templates.spec.ts // TDD, real Postgres (localhost:5434), mirrors inbox.spec.ts +``` + +**Scope boundary:** `render()` is channel-generic (it can render an `INTERNAL` template to +`{subject,html,text}`). `sendTemplated()` covers **external** channels (`EMAIL`/`SMS`) via +`OutboundService`. `INTERNAL` *delivery* (render → create an in-app `Interaction`, no SMTP) reuses +`render()` but is wired by the messaging/mirror spec — this module never imports the messaging layer. + +## Contract + +```ts +type TemplateSource = + | { key: string; version?: number } + | { inline: { subject?: string; html?: string; text?: string } }; + +interface RenderedContent { subject?: string; html?: string; text?: string } + +// template.renderer.ts — PURE (no I/O): given a template's raw strings + vars, produce content. +// template.service.ts render() — resolves the template from the DB (I/O), then calls the pure renderer. +// For an inline source there is no DB row, so declared-variable validation is skipped — inline +// content is the caller's responsibility; only stored templates enforce their declared `variables`. +render(source: TemplateSource, vars: Record, opts: { channel: IiosTemplateChannel; locale?: string; scopeId?: string }): Promise + +// templated-sender.ts — external egress +sendTemplated(input: { + source: TemplateSource; + channel: 'EMAIL' | 'SMS'; + target: string; // email address / phone + vars: Record; + scopeId?: string; + idempotencyKey: string; // REQUIRED — e.g. "receipt:" + purpose?: string; +}): Promise +``` + +**HTTP** (`SessionVerifier`-auth'd; `scopeId` from the caller's principal): +- `POST /v1/templates/send` → `sendTemplated`. OPA action `iios.template.send`; + inline source additionally gated on `iios.template.send.inline` (arbitrary HTML to customers). +- `POST /v1/templates/preview` → `render` only, returns `RenderedContent`. No send. For marketing/QA. + +## Task-by-task (TDD) + +Each task: write the failing test → run it (confirm red) → implement → run (green) → commit. + +**T1 — Renderer (`template.renderer.ts`)** +- Tests: substitutes `{{firstName}}`; **escapes `' }, + ); + expect(out.html).toBe('

<script>alert(1)</script>

'); + expect(out.html).not.toContain(''); // plaintext is not HTML → not escaped + }); + + it('supports {{#if}} and {{#each}}', () => { + const out = renderTemplate( + { bodyText: '{{#if companyName}}Company: {{companyName}}\n{{/if}}{{#each items}}- {{this}}\n{{/each}}', variables: [] }, + { companyName: 'Acme', items: ['a', 'b'] }, + ); + expect(out.text).toBe('Company: Acme\n- a\n- b\n'); + }); + + it('throws when a declared variable is missing (fail loud)', () => { + expect(() => renderTemplate({ bodyText: 'Hi {{firstName}}', variables: ['firstName'] }, {})).toThrow(MissingTemplateVariableError); + }); + + it('renders only the parts that are present', () => { + const out = renderTemplate({ bodyText: 'code {{code}}', variables: ['code'] }, { code: '123' }); + expect(out.text).toBe('code 123'); + expect(out.subject).toBeUndefined(); + expect(out.html).toBeUndefined(); + }); +}); diff --git a/packages/iios-service/src/templates/template.renderer.ts b/packages/iios-service/src/templates/template.renderer.ts new file mode 100644 index 0000000..5668f59 --- /dev/null +++ b/packages/iios-service/src/templates/template.renderer.ts @@ -0,0 +1,46 @@ +import Handlebars from 'handlebars'; + +/** The raw, unrendered strings of a template (from the DB row or an inline source). */ +export interface RawTemplate { + subject?: string | null; + bodyHtml?: string | null; + bodyText?: string | null; + /** Declared variable names. Every one MUST be supplied at render time (fail loud). */ + variables?: string[] | null; +} + +export interface RenderedContent { + subject?: string; + html?: string; + text?: string; +} + +/** A declared variable was not supplied — we refuse to send a half-rendered "Hi ," message. */ +export class MissingTemplateVariableError extends Error { + constructor(public readonly missing: string[]) { + super(`missing required template variable(s): ${missing.join(', ')}`); + this.name = 'MissingTemplateVariableError'; + } +} + +/** + * Pure render: interpolate `vars` into a template's strings. HTML output is auto-escaped (values + * like a customer name go into an email body — XSS risk); subject and plaintext are NOT escaped. + * No I/O — the caller supplies the raw template. Throws if any declared variable is absent. + */ +export function renderTemplate(tpl: RawTemplate, vars: Record): RenderedContent { + const declared = tpl.variables ?? []; + const missing = declared.filter((name) => vars[name] === undefined || vars[name] === null); + if (missing.length > 0) throw new MissingTemplateVariableError(missing); + + const out: RenderedContent = {}; + if (tpl.subject != null) out.subject = compile(tpl.subject, false)(vars); + if (tpl.bodyHtml != null) out.html = compile(tpl.bodyHtml, true)(vars); + if (tpl.bodyText != null) out.text = compile(tpl.bodyText, false)(vars); + return out; +} + +/** `escape=true` → HTML-escape interpolated values (for the html body); false → verbatim. */ +function compile(src: string, escape: boolean): Handlebars.TemplateDelegate { + return Handlebars.compile(src, { noEscape: !escape, strict: false }); +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 5159988..32beda2 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -364,6 +364,9 @@ importers: dotenv: specifier: ^16.4.7 version: 16.6.1 + handlebars: + specifier: ^4.7.9 + version: 4.7.9 ioredis: specifier: ^5.11.1 version: 5.11.1 @@ -2224,6 +2227,11 @@ packages: graceful-fs@4.2.11: resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} + handlebars@4.7.9: + resolution: {integrity: sha512-4E71E0rpOaQuJR2A3xDZ+GM1HyWYv1clR58tC8emQNeQe3RH7MAzSbat+V0wG78LQBo6m6bzSG/L4pBuCsgnUQ==} + engines: {node: '>=0.4.7'} + hasBin: true + has-flag@4.0.0: resolution: {integrity: sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==} engines: {node: '>=8'} @@ -3052,6 +3060,11 @@ packages: ufo@1.6.4: resolution: {integrity: sha512-JFNbkD1Svwe0KvGi8GOeLcP4kAWQ609twvCdcHxq1oSL8svv39ZuSvajcD8B+5D0eL4+s1Is2D/O6KN3qcTeRA==} + uglify-js@3.19.3: + resolution: {integrity: sha512-v3Xu+yuwBXisp6QYTcH4UbH+xYJXqnq2m/LtQVWKWzYc1iehYnLixoQDN9FH6/j9/oybfd6W9Ghwkl8+UMKTKQ==} + engines: {node: '>=0.8.0'} + hasBin: true + uid2@1.0.0: resolution: {integrity: sha512-+I6aJUv63YAcY9n4mQreLUt0d4lvwkkopDNmpomkAUz0fAkEMV9pRWxN0EjhW1YfRhcuyHg2v3mwddCDW1+LFQ==} engines: {node: '>= 4.0.0'} @@ -3243,6 +3256,9 @@ packages: engines: {node: '>=8'} hasBin: true + wordwrap@1.0.0: + resolution: {integrity: sha512-gvVzJFlPycKc5dZN4yPkP8w7Dc37BtP1yczEneOb4uq34pXZcvrtRTmWV8W+Ume+XCxKgbjM+nevkyFPMybd4Q==} + wrap-ansi@6.2.0: resolution: {integrity: sha512-r6lPcBGxZXlIcymEu7InxDMhdW0KDxpLgoFLcguasxCaJ/SOIZwINatK9KY/tf+ZrlywOKU0UDj3ATXUBfxJXA==} engines: {node: '>=8'} @@ -5013,6 +5029,15 @@ snapshots: graceful-fs@4.2.11: {} + handlebars@4.7.9: + dependencies: + minimist: 1.2.8 + neo-async: 2.6.2 + source-map: 0.6.1 + wordwrap: 1.0.0 + optionalDependencies: + uglify-js: 3.19.3 + has-flag@4.0.0: {} has-symbols@1.1.0: {} @@ -5821,6 +5846,9 @@ snapshots: ufo@1.6.4: {} + uglify-js@3.19.3: + optional: true + uid2@1.0.0: {} uid@2.0.2: @@ -6008,6 +6036,8 @@ snapshots: siginfo: 2.0.0 stackback: 0.0.2 + wordwrap@1.0.0: {} + wrap-ansi@6.2.0: dependencies: ansi-styles: 4.3.0 From b3afd44d0024a89fbcbfc84f0d212ad04fb0bb9c Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:23:44 +0530 Subject: [PATCH 03/14] feat(templates): T2 IiosMessageTemplate table + resolve() MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Schema + migration for the template source table, and TemplateRepository.resolve: highest active version for (key,channel,locale), scoped override preferred over the global (scopeId NULL) default. Migration adds a PARTIAL unique index WHERE scopeId IS NULL so two platform defaults for the same key can't coexist (Postgres treats NULL as distinct under a plain UNIQUE — the scoped constraint alone wouldn't catch it). 6 tests. Co-Authored-By: Claude Opus 4.8 --- .../migration.sql | 39 +++++++++++ packages/iios-service/prisma/schema.prisma | 34 ++++++++++ .../src/templates/template.repository.spec.ts | 65 +++++++++++++++++++ .../src/templates/template.repository.ts | 42 ++++++++++++ 4 files changed, 180 insertions(+) create mode 100644 packages/iios-service/prisma/migrations/20260718100000_add_message_template/migration.sql create mode 100644 packages/iios-service/src/templates/template.repository.spec.ts create mode 100644 packages/iios-service/src/templates/template.repository.ts diff --git a/packages/iios-service/prisma/migrations/20260718100000_add_message_template/migration.sql b/packages/iios-service/prisma/migrations/20260718100000_add_message_template/migration.sql new file mode 100644 index 0000000..6052957 --- /dev/null +++ b/packages/iios-service/prisma/migrations/20260718100000_add_message_template/migration.sql @@ -0,0 +1,39 @@ +-- Reusable message templates (email/SMS/in-app). DB is runtime truth; platform defaults seeded +-- from repo files on boot. Versions are immutable (a change = a new higher version). +CREATE TYPE "IiosTemplateChannel" AS ENUM ('EMAIL', 'SMS', 'INTERNAL'); + +CREATE TABLE "IiosMessageTemplate" ( + "id" TEXT NOT NULL, + "scopeId" TEXT, + "key" TEXT NOT NULL, + "channel" "IiosTemplateChannel" NOT NULL, + "locale" TEXT NOT NULL DEFAULT 'en', + "version" INTEGER NOT NULL DEFAULT 1, + "subject" TEXT, + "bodyHtml" TEXT, + "bodyText" TEXT, + "variables" JSONB, + "active" BOOLEAN NOT NULL DEFAULT true, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + "updatedAt" TIMESTAMP(3) NOT NULL, + CONSTRAINT "IiosMessageTemplate_pkey" PRIMARY KEY ("id") +); + +ALTER TABLE "IiosMessageTemplate" + ADD CONSTRAINT "IiosMessageTemplate_scopeId_fkey" + FOREIGN KEY ("scopeId") REFERENCES "IiosScope"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- Uniqueness for SCOPED rows (scopeId NOT NULL). +CREATE UNIQUE INDEX "IiosMessageTemplate_scopeId_key_channel_locale_version_key" + ON "IiosMessageTemplate" ("scopeId", "key", "channel", "locale", "version"); + +-- Uniqueness for GLOBAL rows (scopeId IS NULL). Postgres treats NULL as distinct under a normal +-- UNIQUE, so the constraint above does NOT stop two platform defaults for the same key — this +-- partial index does. (Same footgun handled in the inbox idempotency migration.) +CREATE UNIQUE INDEX "IiosMessageTemplate_global_key" + ON "IiosMessageTemplate" ("key", "channel", "locale", "version") + WHERE "scopeId" IS NULL; + +-- Resolution lookup: by key/channel/locale among active rows. +CREATE INDEX "IiosMessageTemplate_key_channel_locale_active_idx" + ON "IiosMessageTemplate" ("key", "channel", "locale", "active"); diff --git a/packages/iios-service/prisma/schema.prisma b/packages/iios-service/prisma/schema.prisma index 9b4aa91..59b2b1a 100644 --- a/packages/iios-service/prisma/schema.prisma +++ b/packages/iios-service/prisma/schema.prisma @@ -96,6 +96,12 @@ enum IiosInboxState { STALE } +enum IiosTemplateChannel { + EMAIL + SMS + INTERNAL +} + enum IiosTicketState { NEW OPEN @@ -310,6 +316,7 @@ model IiosScope { tickets IiosTicket[] callbacks IiosCallbackRequest[] notificationSubscriptions IiosNotificationSubscription[] + messageTemplates IiosMessageTemplate[] @@index([orgId, appId, tenantId]) } @@ -677,6 +684,33 @@ model IiosInboxItem { @@index([ownerActorId, state, priority]) } +/// A reusable message template (email/SMS/in-app). DB is runtime truth; platform defaults are +/// seeded from repo files on boot. Versions are immutable — a change writes a new (higher) version, +/// never edits in place, so a rendered send can always be traced to the exact source it used. +/// scopeId NULL = a platform default; set = a tenant scope's override of the same key. Resolution +/// prefers the scoped row, else the global default. (The global-uniqueness of NULL-scope rows is +/// enforced by a partial unique index added in the migration — Postgres treats NULL as distinct.) +model IiosMessageTemplate { + id String @id @default(cuid()) + scopeId String? + scope IiosScope? @relation(fields: [scopeId], references: [id], onDelete: Cascade) + key String + channel IiosTemplateChannel + locale String @default("en") + version Int @default(1) + subject String? + bodyHtml String? + bodyText String? + /// Declared variable names — render throws if a declared var is missing (fail loud). + variables Json? + active Boolean @default(true) + createdAt DateTime @default(now()) + updatedAt DateTime @updatedAt + + @@unique([scopeId, key, channel, locale, version]) + @@index([key, channel, locale, active]) +} + /// Audit trail of inbox-item state transitions. model IiosInboxItemStateHistory { id String @id @default(cuid()) diff --git a/packages/iios-service/src/templates/template.repository.spec.ts b/packages/iios-service/src/templates/template.repository.spec.ts new file mode 100644 index 0000000..1e10040 --- /dev/null +++ b/packages/iios-service/src/templates/template.repository.spec.ts @@ -0,0 +1,65 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { PrismaClient } from '@prisma/client'; +import { resetDb } from '../test-utils/reset-db'; +import { TemplateNotFoundError, TemplateRepository } from './template.repository'; +import type { PrismaService } from '../prisma/prisma.service'; + +const url = process.env.DATABASE_URL ?? 'postgresql://iios:iios@localhost:5434/iios?schema=public'; +const prisma = new PrismaClient({ datasources: { db: { url } } }); +const repo = new TemplateRepository(prisma as unknown as PrismaService); + +async function scope(): Promise { + const s = await prisma.iiosScope.create({ data: { orgId: 'org_demo', appId: 'crm-web' } }); + return s.id; +} +async function seed(data: Partial[0]['data']> & { key: string }) { + return prisma.iiosMessageTemplate.create({ + data: { channel: 'EMAIL', locale: 'en', version: 1, bodyText: 't', ...data } as never, + }); +} + +beforeAll(async () => { await prisma.$connect(); }); +afterAll(async () => { await prisma.$disconnect(); }); +beforeEach(async () => { await resetDb(prisma); }); + +describe('TemplateRepository.resolve', () => { + it('returns the global default when no scope override exists', async () => { + await seed({ key: 'welcome', bodyText: 'global' }); + const t = await repo.resolve('welcome', 'EMAIL', 'en', await scope()); + expect(t.bodyText).toBe('global'); + expect(t.scopeId).toBeNull(); + }); + + it('prefers the scoped override over the global default', async () => { + const scopeId = await scope(); + await seed({ key: 'welcome', bodyText: 'global' }); // scopeId NULL + await seed({ key: 'welcome', bodyText: 'scoped', scopeId }); // override + const t = await repo.resolve('welcome', 'EMAIL', 'en', scopeId); + expect(t.bodyText).toBe('scoped'); + expect(t.scopeId).toBe(scopeId); + }); + + it('returns the highest active version', async () => { + await seed({ key: 'welcome', version: 1, bodyText: 'v1' }); + await seed({ key: 'welcome', version: 3, bodyText: 'v3' }); + await seed({ key: 'welcome', version: 2, bodyText: 'v2' }); + const t = await repo.resolve('welcome', 'EMAIL', 'en'); + expect(t.version).toBe(3); + }); + + it('ignores inactive versions (a retracted v3 falls back to v2)', async () => { + await seed({ key: 'welcome', version: 2, bodyText: 'v2' }); + await seed({ key: 'welcome', version: 3, bodyText: 'v3', active: false }); + const t = await repo.resolve('welcome', 'EMAIL', 'en'); + expect(t.version).toBe(2); + }); + + it('throws TemplateNotFoundError for an unknown key', async () => { + await expect(repo.resolve('nope', 'EMAIL', 'en')).rejects.toBeInstanceOf(TemplateNotFoundError); + }); + + it('does not cross channels (an EMAIL template is not an SMS template)', async () => { + await seed({ key: 'welcome', channel: 'EMAIL' }); + await expect(repo.resolve('welcome', 'SMS', 'en')).rejects.toBeInstanceOf(TemplateNotFoundError); + }); +}); diff --git a/packages/iios-service/src/templates/template.repository.ts b/packages/iios-service/src/templates/template.repository.ts new file mode 100644 index 0000000..b34d988 --- /dev/null +++ b/packages/iios-service/src/templates/template.repository.ts @@ -0,0 +1,42 @@ +import { Injectable, NotFoundException } from '@nestjs/common'; +import { IiosMessageTemplate, IiosTemplateChannel } from '@prisma/client'; +import { PrismaService } from '../prisma/prisma.service'; + +export class TemplateNotFoundError extends NotFoundException { + constructor(key: string, channel: string, locale: string) { + super(`no active template for key="${key}" channel=${channel} locale=${locale}`); + this.name = 'TemplateNotFoundError'; + } +} + +@Injectable() +export class TemplateRepository { + constructor(private readonly prisma: PrismaService) {} + + /** + * Resolve the template to use: the highest active version for (key, channel, locale), preferring + * a row owned by `scopeId` (a tenant override) and falling back to the global default (scopeId + * NULL). Throws if neither exists. + */ + async resolve( + key: string, + channel: IiosTemplateChannel, + locale = 'en', + scopeId?: string, + ): Promise { + if (scopeId) { + const scoped = await this.highest({ key, channel, locale, scopeId }); + if (scoped) return scoped; + } + const global = await this.highest({ key, channel, locale, scopeId: null }); + if (global) return global; + throw new TemplateNotFoundError(key, channel, locale); + } + + private highest(where: { key: string; channel: IiosTemplateChannel; locale: string; scopeId: string | null }) { + return this.prisma.iiosMessageTemplate.findFirst({ + where: { ...where, active: true }, + orderBy: { version: 'desc' }, + }); + } +} From 8ce647552ad9604168ddf0cd3fecd67746281424 Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:25:27 +0530 Subject: [PATCH 04/14] feat(templates): T3 render() service (resolve+render, inline, version pin) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TemplateService.render(source, vars, opts) → { content, provenance }. Stored key → resolve+render; inline → render without a DB row (key/version null); explicit version pins. Provenance = {templateKey, version, locale, sha256(content)}. 10 tests (svc + repo). Co-Authored-By: Claude Opus 4.8 --- .../src/templates/template.model.ts | 16 ++++++ .../src/templates/template.repository.ts | 15 +++-- .../src/templates/template.service.spec.ts | 55 +++++++++++++++++++ .../src/templates/template.service.ts | 55 +++++++++++++++++++ 4 files changed, 137 insertions(+), 4 deletions(-) create mode 100644 packages/iios-service/src/templates/template.model.ts create mode 100644 packages/iios-service/src/templates/template.service.spec.ts create mode 100644 packages/iios-service/src/templates/template.service.ts diff --git a/packages/iios-service/src/templates/template.model.ts b/packages/iios-service/src/templates/template.model.ts new file mode 100644 index 0000000..20bf6a6 --- /dev/null +++ b/packages/iios-service/src/templates/template.model.ts @@ -0,0 +1,16 @@ +import type { IiosTemplateChannel } from '@prisma/client'; + +/** What to render: a STORED template (by key, optionally pinned to a version) or INLINE content. */ +export type TemplateSource = + | { key: string; version?: number } + | { inline: { subject?: string; html?: string; text?: string; variables?: string[] } }; + +export interface RenderOptions { + channel: IiosTemplateChannel; + locale?: string; + scopeId?: string; +} + +export function isInlineSource(s: TemplateSource): s is { inline: { subject?: string; html?: string; text?: string; variables?: string[] } } { + return 'inline' in s; +} diff --git a/packages/iios-service/src/templates/template.repository.ts b/packages/iios-service/src/templates/template.repository.ts index b34d988..248ad0a 100644 --- a/packages/iios-service/src/templates/template.repository.ts +++ b/packages/iios-service/src/templates/template.repository.ts @@ -23,19 +23,26 @@ export class TemplateRepository { channel: IiosTemplateChannel, locale = 'en', scopeId?: string, + version?: number, ): Promise { + // Scoped override first, then the global default — for both the pinned and highest-active paths. if (scopeId) { - const scoped = await this.highest({ key, channel, locale, scopeId }); + const scoped = await this.pick({ key, channel, locale, scopeId }, version); if (scoped) return scoped; } - const global = await this.highest({ key, channel, locale, scopeId: null }); + const global = await this.pick({ key, channel, locale, scopeId: null }, version); if (global) return global; throw new TemplateNotFoundError(key, channel, locale); } - private highest(where: { key: string; channel: IiosTemplateChannel; locale: string; scopeId: string | null }) { + /** A pinned `version` fetches that exact version (the caller was explicit); otherwise the highest + * active version — a retracted (inactive) newest version falls back to the last good one. */ + private pick( + where: { key: string; channel: IiosTemplateChannel; locale: string; scopeId: string | null }, + version?: number, + ) { return this.prisma.iiosMessageTemplate.findFirst({ - where: { ...where, active: true }, + where: version != null ? { ...where, version } : { ...where, active: true }, orderBy: { version: 'desc' }, }); } diff --git a/packages/iios-service/src/templates/template.service.spec.ts b/packages/iios-service/src/templates/template.service.spec.ts new file mode 100644 index 0000000..e3b171d --- /dev/null +++ b/packages/iios-service/src/templates/template.service.spec.ts @@ -0,0 +1,55 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { PrismaClient } from '@prisma/client'; +import { resetDb } from '../test-utils/reset-db'; +import { TemplateRepository } from './template.repository'; +import { TemplateService } from './template.service'; +import { TemplateNotFoundError } from './template.repository'; +import type { PrismaService } from '../prisma/prisma.service'; + +const url = process.env.DATABASE_URL ?? 'postgresql://iios:iios@localhost:5434/iios?schema=public'; +const prisma = new PrismaClient({ datasources: { db: { url } } }); +const svc = new TemplateService(new TemplateRepository(prisma as unknown as PrismaService)); + +beforeAll(async () => { await prisma.$connect(); }); +afterAll(async () => { await prisma.$disconnect(); }); +beforeEach(async () => { await resetDb(prisma); }); + +describe('TemplateService.render', () => { + it('resolves a stored template and renders it, with provenance', async () => { + await prisma.iiosMessageTemplate.create({ + data: { key: 'welcome', channel: 'EMAIL', locale: 'en', version: 2, subject: 'Hi {{firstName}}', bodyHtml: '

{{firstName}}

', variables: ['firstName'] }, + }); + const { content, provenance } = await svc.render({ key: 'welcome' }, { firstName: 'Dana' }, { channel: 'EMAIL' }); + expect(content.subject).toBe('Hi Dana'); + expect(content.html).toBe('

Dana

'); + expect(provenance).toMatchObject({ templateKey: 'welcome', templateVersion: 2, templateLocale: 'en' }); + expect(provenance.renderedHash).toMatch(/^[a-f0-9]{64}$/); + }); + + it('renders an inline source without a DB row (key/version null)', async () => { + const { content, provenance } = await svc.render( + { inline: { subject: 'Ad-hoc {{x}}', html: '{{x}}', variables: ['x'] } }, + { x: 'Y' }, + { channel: 'EMAIL' }, + ); + expect(content.subject).toBe('Ad-hoc Y'); + expect(content.html).toBe('Y'); + expect(provenance.templateKey).toBeNull(); + expect(provenance.templateVersion).toBeNull(); + expect(provenance.renderedHash).toMatch(/^[a-f0-9]{64}$/); + }); + + it('pins to an explicit version when asked', async () => { + await prisma.iiosMessageTemplate.create({ data: { key: 'k', channel: 'EMAIL', locale: 'en', version: 1, bodyText: 'v1' } }); + await prisma.iiosMessageTemplate.create({ data: { key: 'k', channel: 'EMAIL', locale: 'en', version: 2, bodyText: 'v2' } }); + const latest = await svc.render({ key: 'k' }, {}, { channel: 'EMAIL' }); + const pinned = await svc.render({ key: 'k', version: 1 }, {}, { channel: 'EMAIL' }); + expect(latest.content.text).toBe('v2'); + expect(pinned.content.text).toBe('v1'); + expect(pinned.provenance.templateVersion).toBe(1); + }); + + it('throws for an unknown stored key', async () => { + await expect(svc.render({ key: 'missing' }, {}, { channel: 'EMAIL' })).rejects.toBeInstanceOf(TemplateNotFoundError); + }); +}); diff --git a/packages/iios-service/src/templates/template.service.ts b/packages/iios-service/src/templates/template.service.ts new file mode 100644 index 0000000..4ed9bc5 --- /dev/null +++ b/packages/iios-service/src/templates/template.service.ts @@ -0,0 +1,55 @@ +import { Injectable } from '@nestjs/common'; +import { createHash } from 'node:crypto'; +import { TemplateRepository } from './template.repository'; +import { renderTemplate, type RawTemplate, type RenderedContent } from './template.renderer'; +import { isInlineSource, type RenderOptions, type TemplateSource } from './template.model'; + +/** The provenance of a rendered send — carried onto the outbound command for replay/audit. */ +export interface RenderProvenance { + templateKey: string | null; // null for inline sources + templateVersion: number | null; + templateLocale: string | null; + renderedHash: string; // sha256 of the rendered content +} + +export interface RenderResult { + content: RenderedContent; + provenance: RenderProvenance; +} + +@Injectable() +export class TemplateService { + constructor(private readonly repo: TemplateRepository) {} + + /** + * Resolve (if a stored key) and render a template with `vars`, returning the content plus the + * provenance to record on the send. Inline sources skip the DB and carry null key/version — the + * caller owns inline content, so no declared-variable contract is enforced for it. + */ + async render(source: TemplateSource, vars: Record, opts: RenderOptions): Promise { + const locale = opts.locale ?? 'en'; + + if (isInlineSource(source)) { + const raw: RawTemplate = { + subject: source.inline.subject, + bodyHtml: source.inline.html, + bodyText: source.inline.text, + variables: source.inline.variables ?? [], + }; + const content = renderTemplate(raw, vars); + return { content, provenance: this.provenance(content, null, null, null) }; + } + + const tpl = await this.repo.resolve(source.key, opts.channel, locale, opts.scopeId, source.version); + const content = renderTemplate( + { subject: tpl.subject, bodyHtml: tpl.bodyHtml, bodyText: tpl.bodyText, variables: (tpl.variables as string[] | null) ?? [] }, + vars, + ); + return { content, provenance: this.provenance(content, tpl.key, tpl.version, tpl.locale) }; + } + + private provenance(content: RenderedContent, key: string | null, version: number | null, locale: string | null): RenderProvenance { + const hash = createHash('sha256').update(JSON.stringify(content)).digest('hex'); + return { templateKey: key, templateVersion: version, templateLocale: locale, renderedHash: hash }; + } +} From 10f3545a2557cad627a42ea242d75c0acb7f158d Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:28:28 +0530 Subject: [PATCH 05/14] feat(templates): T4 sendTemplated() + outbound provenance MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - IiosOutboundCommand gains templateKey/version/locale/renderedHash (migration). - OutboundService.send accepts an optional opaque `provenance` and writes it into the command on BOTH the PENDING and RATE_LIMITED create paths — the only way to stamp provenance by construction, since send() creates the row itself (review finding). OutboundService still neither renders nor resolves templates. - TemplatedSender.sendTemplated: render -> OutboundService.send -> provenance, one entry point. EMAIL payload = {subject,html,text}; SMS = {text}. Idempotent per key. Tests: 4 sender + 16 adapters regression green. Co-Authored-By: Claude Opus 4.8 --- .../migration.sql | 9 +++ packages/iios-service/prisma/schema.prisma | 5 ++ .../src/adapters/outbound.service.ts | 14 +++- .../src/capability/capability.registry.ts | 2 +- .../src/templates/templated-sender.spec.ts | 80 +++++++++++++++++++ .../src/templates/templated-sender.ts | 55 +++++++++++++ 6 files changed, 162 insertions(+), 3 deletions(-) create mode 100644 packages/iios-service/prisma/migrations/20260718110000_add_outbound_template_provenance/migration.sql create mode 100644 packages/iios-service/src/templates/templated-sender.spec.ts create mode 100644 packages/iios-service/src/templates/templated-sender.ts diff --git a/packages/iios-service/prisma/migrations/20260718110000_add_outbound_template_provenance/migration.sql b/packages/iios-service/prisma/migrations/20260718110000_add_outbound_template_provenance/migration.sql new file mode 100644 index 0000000..99beb7f --- /dev/null +++ b/packages/iios-service/prisma/migrations/20260718110000_add_outbound_template_provenance/migration.sql @@ -0,0 +1,9 @@ +-- Template provenance on the outbound command: which template (key/version/locale) produced this +-- send, plus a hash of the rendered content. The rendered content itself already lives in `payload`; +-- these columns answer "which template version produced this send?" for replay/audit. All nullable — +-- non-templated sends leave them null. +ALTER TABLE "IiosOutboundCommand" + ADD COLUMN "templateKey" TEXT, + ADD COLUMN "templateVersion" INTEGER, + ADD COLUMN "templateLocale" TEXT, + ADD COLUMN "renderedHash" TEXT; diff --git a/packages/iios-service/prisma/schema.prisma b/packages/iios-service/prisma/schema.prisma index 59b2b1a..bda1038 100644 --- a/packages/iios-service/prisma/schema.prisma +++ b/packages/iios-service/prisma/schema.prisma @@ -868,6 +868,11 @@ model IiosOutboundCommand { idempotencyKey String @unique providerRef String? consentReceiptRef String? // CMP consent receipt this send went out under (P9) + // Template provenance (which source produced this send) — null for non-templated sends. + templateKey String? + templateVersion Int? + templateLocale String? + renderedHash String? // sha256 of the rendered content, for replay/audit createdAt DateTime @default(now()) attempts IiosDeliveryAttempt[] diff --git a/packages/iios-service/src/adapters/outbound.service.ts b/packages/iios-service/src/adapters/outbound.service.ts index c0370c8..a6ea69a 100644 --- a/packages/iios-service/src/adapters/outbound.service.ts +++ b/packages/iios-service/src/adapters/outbound.service.ts @@ -34,8 +34,18 @@ export class OutboundService { idempotencyKey?: string, scopeId?: string, purpose?: string, + /** Opaque template provenance recorded on the command. OutboundService neither renders nor + * resolves templates — it only persists the four strings it is handed (guaranteed by the + * templated sender, never by caller convention). */ + provenance?: { templateKey?: string | null; templateVersion?: number | null; templateLocale?: string | null; renderedHash?: string | null }, ) { const key = idempotencyKey ?? randomUUID(); + const prov = { + templateKey: provenance?.templateKey ?? null, + templateVersion: provenance?.templateVersion ?? null, + templateLocale: provenance?.templateLocale ?? null, + renderedHash: provenance?.renderedHash ?? null, + }; // The actual send. The inner findUnique stays as a backstop so a FAILED-then-retried // command returns the existing row instead of colliding on idempotencyKey @unique. @@ -46,14 +56,14 @@ export class OutboundService { // Per-target rate limit AND per-tenant quota (a noisy tenant can't starve shared egress). if (!(await this.allow(channelType, target)) || !(await this.allowTenant(scopeId))) { const cmd = await this.prisma.iiosOutboundCommand.create({ - data: { channelType, target, scopeId, payload: payload as Prisma.InputJsonValue, status: 'RATE_LIMITED', idempotencyKey: key }, + data: { channelType, target, scopeId, payload: payload as Prisma.InputJsonValue, status: 'RATE_LIMITED', idempotencyKey: key, ...prov }, }); await this.prisma.iiosDeliveryAttempt.create({ data: { commandId: cmd.id, attemptNo: 1, status: 'RATE_LIMITED' } }); return cmd; } const cmd = await this.prisma.iiosOutboundCommand.create({ - data: { channelType, target, scopeId, payload: payload as Prisma.InputJsonValue, status: 'PENDING', idempotencyKey: key }, + data: { channelType, target, scopeId, payload: payload as Prisma.InputJsonValue, status: 'PENDING', idempotencyKey: key, ...prov }, }); let result; diff --git a/packages/iios-service/src/capability/capability.registry.ts b/packages/iios-service/src/capability/capability.registry.ts index 4e4161d..c055fc0 100644 --- a/packages/iios-service/src/capability/capability.registry.ts +++ b/packages/iios-service/src/capability/capability.registry.ts @@ -4,7 +4,7 @@ import { SandboxProvider } from './sandbox.provider'; import { HttpProvider } from './http.provider'; import { EmailProvider } from './email.provider'; -const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'WHATSAPP', 'PORTAL']; +const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'SMS', 'WHATSAPP', 'PORTAL']; /** * Maps a channelType to the provider that executes egress for it. Sandbox by diff --git a/packages/iios-service/src/templates/templated-sender.spec.ts b/packages/iios-service/src/templates/templated-sender.spec.ts new file mode 100644 index 0000000..4c05745 --- /dev/null +++ b/packages/iios-service/src/templates/templated-sender.spec.ts @@ -0,0 +1,80 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { PrismaClient } from '@prisma/client'; +import { makeFakePorts } from '@insignia/iios-testkit'; +import { resetDb } from '../test-utils/reset-db'; +import { OutboundService } from '../adapters/outbound.service'; +import { CapabilityBroker } from '../capability/capability.broker'; +import { CapabilityProviderRegistry } from '../capability/capability.registry'; +import { IdempotencyService } from '../idempotency/idempotency.service'; +import { TemplateRepository } from './template.repository'; +import { TemplateService } from './template.service'; +import { TemplatedSender } from './templated-sender'; +import type { PrismaService } from '../prisma/prisma.service'; + +const url = process.env.DATABASE_URL ?? 'postgresql://iios:iios@localhost:5434/iios?schema=public'; +const prisma = new PrismaClient({ datasources: { db: { url } } }); +const asService = prisma as unknown as PrismaService; + +function sender(): TemplatedSender { + const outbound = new OutboundService(asService, new CapabilityBroker(makeFakePorts(), new CapabilityProviderRegistry()), new IdempotencyService(asService)); + const templates = new TemplateService(new TemplateRepository(asService)); + return new TemplatedSender(templates, outbound); +} + +async function seedReceipt() { + await prisma.iiosMessageTemplate.create({ + data: { key: 'payment.receipt', channel: 'EMAIL', locale: 'en', version: 1, subject: 'Thanks {{firstName}}', bodyHtml: '

{{amount}}

', bodyText: '{{amount}}', variables: ['firstName', 'amount'] }, + }); +} + +beforeAll(async () => { await prisma.$connect(); }); +afterAll(async () => { await prisma.$disconnect(); }); +beforeEach(async () => { await resetDb(prisma); }); + +describe('TemplatedSender.sendTemplated', () => { + it('renders a stored template, queues an outbound command, and stamps provenance', async () => { + await seedReceipt(); + const cmd = await sender().sendTemplated({ + source: { key: 'payment.receipt' }, channel: 'EMAIL', target: 'dana@acme.com', + vars: { firstName: 'Dana', amount: '$2,000' }, idempotencyKey: 'receipt:cs_1', + }); + expect(cmd.channelType).toBe('EMAIL'); + expect(cmd.target).toBe('dana@acme.com'); + expect((cmd.payload as { subject: string }).subject).toBe('Thanks Dana'); + expect(cmd.templateKey).toBe('payment.receipt'); + expect(cmd.templateVersion).toBe(1); + expect(cmd.templateLocale).toBe('en'); + expect(cmd.renderedHash).toMatch(/^[a-f0-9]{64}$/); + }); + + it('is idempotent per key: a replay yields ONE command', async () => { + await seedReceipt(); + const s = sender(); + const input = { source: { key: 'payment.receipt' }, channel: 'EMAIL' as const, target: 'dana@acme.com', vars: { firstName: 'Dana', amount: '$1' }, idempotencyKey: 'receipt:cs_dup' }; + const a = await s.sendTemplated(input); + const b = await s.sendTemplated(input); + expect(b.id).toBe(a.id); + expect(await prisma.iiosOutboundCommand.count({ where: { idempotencyKey: 'receipt:cs_dup' } })).toBe(1); + }); + + it('sends inline content with null template key but a hash', async () => { + const cmd = await sender().sendTemplated({ + source: { inline: { subject: 'Hi {{n}}', html: '{{n}}', variables: ['n'] } }, + channel: 'EMAIL', target: 'x@y.com', vars: { n: 'Z' }, idempotencyKey: 'inline:1', + }); + expect((cmd.payload as { subject: string }).subject).toBe('Hi Z'); + expect(cmd.templateKey).toBeNull(); + expect(cmd.renderedHash).toMatch(/^[a-f0-9]{64}$/); + }); + + it('shapes an SMS send as text only', async () => { + await prisma.iiosMessageTemplate.create({ + data: { key: 'payment.receipt', channel: 'SMS', locale: 'en', version: 1, bodyText: 'Paid {{amount}}', variables: ['amount'] }, + }); + const cmd = await sender().sendTemplated({ + source: { key: 'payment.receipt' }, channel: 'SMS', target: '+1415', vars: { amount: '$2,000' }, idempotencyKey: 'sms:1', + }); + expect(cmd.channelType).toBe('SMS'); + expect(cmd.payload).toEqual({ text: 'Paid $2,000' }); + }); +}); diff --git a/packages/iios-service/src/templates/templated-sender.ts b/packages/iios-service/src/templates/templated-sender.ts new file mode 100644 index 0000000..4b4373f --- /dev/null +++ b/packages/iios-service/src/templates/templated-sender.ts @@ -0,0 +1,55 @@ +import { Injectable } from '@nestjs/common'; +import { OutboundService } from '../adapters/outbound.service'; +import { TemplateService } from './template.service'; +import type { RenderedContent } from './template.renderer'; +import type { TemplateSource } from './template.model'; + +/** External-egress channels only. INTERNAL (in-app, no SMTP) delivery is the messaging module's job. */ +export type ExternalChannel = 'EMAIL' | 'SMS'; + +export interface SendTemplatedInput { + source: TemplateSource; + channel: ExternalChannel; + target: string; // email address / phone number + vars: Record; + scopeId?: string; + locale?: string; + idempotencyKey: string; // REQUIRED — e.g. "receipt:" + purpose?: string; +} + +/** + * The single entry point for a templated external send: render → OutboundService.send → provenance + * stamped in one place (by construction, not caller convention). Rendering stays pure/testable; + * OutboundService stays content-agnostic (it only records the provenance it is handed). + */ +@Injectable() +export class TemplatedSender { + constructor( + private readonly templates: TemplateService, + private readonly outbound: OutboundService, + ) {} + + async sendTemplated(input: SendTemplatedInput) { + const { content, provenance } = await this.templates.render(input.source, input.vars, { + channel: input.channel, + locale: input.locale, + scopeId: input.scopeId, + }); + return this.outbound.send( + input.channel, + input.target, + this.toPayload(input.channel, content), + input.idempotencyKey, + input.scopeId, + input.purpose, + provenance, + ); + } + + /** Shape rendered content into the channel's egress envelope (EmailProvider reads subject/text/html). */ + private toPayload(channel: ExternalChannel, content: RenderedContent): Record { + if (channel === 'EMAIL') return { subject: content.subject, html: content.html, text: content.text }; + return { text: content.text ?? content.subject ?? '' }; // SMS: text only + } +} From 37407cb0af58db25479a7e7bb914edfb0b65f2bb Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:33:12 +0530 Subject: [PATCH 06/14] feat(templates): T5 boot seeder + default seeds MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TemplateSeeder (OnModuleInit) seeds repo file defaults as global (scopeId NULL) templates if absent. Idempotent per (key,channel,locale,version): re-boot inserts nothing; a bumped version adds a new row and keeps the old for audit. Seeds: welcome (email), payment.receipt (email+sms), onboarding.reminder (email) — placeholder copy for marketing to replace. 3 tests. Co-Authored-By: Claude Opus 4.8 --- .../iios-service/src/templates/seeds/index.ts | 79 +++++++++++++++++++ .../src/templates/template.seeder.spec.ts | 42 ++++++++++ .../src/templates/template.seeder.ts | 49 ++++++++++++ 3 files changed, 170 insertions(+) create mode 100644 packages/iios-service/src/templates/seeds/index.ts create mode 100644 packages/iios-service/src/templates/template.seeder.spec.ts create mode 100644 packages/iios-service/src/templates/template.seeder.ts diff --git a/packages/iios-service/src/templates/seeds/index.ts b/packages/iios-service/src/templates/seeds/index.ts new file mode 100644 index 0000000..848b0f9 --- /dev/null +++ b/packages/iios-service/src/templates/seeds/index.ts @@ -0,0 +1,79 @@ +import type { IiosTemplateChannel } from '@prisma/client'; + +/** A platform-default template shipped in the repo and seeded at boot (scopeId NULL). */ +export interface TemplateSeed { + key: string; + channel: IiosTemplateChannel; + locale: string; + version: number; + subject?: string; + bodyHtml?: string; + bodyText?: string; + variables: string[]; +} + +// Placeholder copy — marketing (Nikita/Himanshu) owns the words, Gautam the HTML. These are +// functional defaults so the pipeline works end-to-end before final copy lands; bump `version` +// when the content changes (the old version stays for audit/replay). + +const WELCOME_EMAIL: TemplateSeed = { + key: 'welcome', + channel: 'EMAIL', + locale: 'en', + version: 1, + subject: 'Welcome to the Founders Club, {{firstName}}', + bodyHtml: + '

Hi {{firstName}},

' + + '

Thanks for joining the Founders Club. Your account is ready to set up.

' + + '

Create your account

' + + '

We are launching soon — we will keep you posted.

', + bodyText: + 'Hi {{firstName}},\n\nThanks for joining the Founders Club. Create your account: {{signupLink}}\n\nWe are launching soon.', + variables: ['firstName', 'signupLink'], +}; + +const PAYMENT_RECEIPT_EMAIL: TemplateSeed = { + key: 'payment.receipt', + channel: 'EMAIL', + locale: 'en', + version: 1, + subject: 'Thanks for your payment, {{firstName}}', + bodyHtml: + '

Hi {{firstName}},

' + + '

We have received your payment of {{amount}} for {{licenseCount}} license(s).

' + + '

A separate tax receipt from our payment processor will follow.

', + bodyText: + 'Hi {{firstName}},\n\nWe have received your payment of {{amount}} for {{licenseCount}} license(s).\nA separate tax receipt will follow.', + variables: ['firstName', 'amount', 'licenseCount'], +}; + +const PAYMENT_RECEIPT_SMS: TemplateSeed = { + key: 'payment.receipt', + channel: 'SMS', + locale: 'en', + version: 1, + bodyText: 'Thanks {{firstName}}! We received your payment of {{amount}}. A receipt is on its way to your email.', + variables: ['firstName', 'amount'], +}; + +const ONBOARDING_REMINDER_EMAIL: TemplateSeed = { + key: 'onboarding.reminder', + channel: 'EMAIL', + locale: 'en', + version: 1, + subject: 'Looks like you have not finished setting up', + bodyHtml: + '

Hi {{firstName}},

' + + '

You paid but have not created your account yet. It only takes a minute.

' + + '

Finish creating your account

', + bodyText: + 'Hi {{firstName}},\n\nYou paid but have not created your account yet. Finish here: {{signupLink}}', + variables: ['firstName', 'signupLink'], +}; + +export const TEMPLATE_SEEDS: TemplateSeed[] = [ + WELCOME_EMAIL, + PAYMENT_RECEIPT_EMAIL, + PAYMENT_RECEIPT_SMS, + ONBOARDING_REMINDER_EMAIL, +]; diff --git a/packages/iios-service/src/templates/template.seeder.spec.ts b/packages/iios-service/src/templates/template.seeder.spec.ts new file mode 100644 index 0000000..3459a5c --- /dev/null +++ b/packages/iios-service/src/templates/template.seeder.spec.ts @@ -0,0 +1,42 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { PrismaClient } from '@prisma/client'; +import { resetDb } from '../test-utils/reset-db'; +import { TemplateSeeder } from './template.seeder'; +import { TEMPLATE_SEEDS, type TemplateSeed } from './seeds'; +import type { PrismaService } from '../prisma/prisma.service'; + +const url = process.env.DATABASE_URL ?? 'postgresql://iios:iios@localhost:5434/iios?schema=public'; +const prisma = new PrismaClient({ datasources: { db: { url } } }); +const seeder = new TemplateSeeder(prisma as unknown as PrismaService); + +beforeAll(async () => { await prisma.$connect(); }); +afterAll(async () => { await prisma.$disconnect(); }); +beforeEach(async () => { await resetDb(prisma); }); + +describe('TemplateSeeder', () => { + it('seeds the file defaults as global (scopeId NULL) templates', async () => { + const n = await seeder.seed(TEMPLATE_SEEDS); + expect(n).toBe(TEMPLATE_SEEDS.length); + const rows = await prisma.iiosMessageTemplate.findMany(); + expect(rows).toHaveLength(TEMPLATE_SEEDS.length); + expect(rows.every((r) => r.scopeId === null)).toBe(true); + const welcome = rows.find((r) => r.key === 'welcome' && r.channel === 'EMAIL'); + expect(welcome?.variables).toEqual(['firstName', 'signupLink']); + }); + + it('is idempotent — re-seeding inserts nothing and creates no duplicates', async () => { + await seeder.seed(TEMPLATE_SEEDS); + const second = await seeder.seed(TEMPLATE_SEEDS); + expect(second).toBe(0); + expect(await prisma.iiosMessageTemplate.count()).toBe(TEMPLATE_SEEDS.length); + }); + + it('a bumped version inserts a new row and leaves the old one', async () => { + await seeder.seed(TEMPLATE_SEEDS); + const bumped: TemplateSeed = { key: 'welcome', channel: 'EMAIL', locale: 'en', version: 2, subject: 'v2', bodyText: 'v2', variables: [] }; + const n = await seeder.seed([bumped]); + expect(n).toBe(1); + const welcomes = await prisma.iiosMessageTemplate.findMany({ where: { key: 'welcome', channel: 'EMAIL' }, orderBy: { version: 'asc' } }); + expect(welcomes.map((w) => w.version)).toEqual([1, 2]); + }); +}); diff --git a/packages/iios-service/src/templates/template.seeder.ts b/packages/iios-service/src/templates/template.seeder.ts new file mode 100644 index 0000000..8412edf --- /dev/null +++ b/packages/iios-service/src/templates/template.seeder.ts @@ -0,0 +1,49 @@ +import { Injectable, Logger, OnModuleInit } from '@nestjs/common'; +import { Prisma } from '@prisma/client'; +import { PrismaService } from '../prisma/prisma.service'; +import { TEMPLATE_SEEDS, type TemplateSeed } from './seeds'; + +/** + * Seeds platform-default templates (scopeId NULL) from the repo files on boot — DB is runtime truth, + * files are the version-controlled source. Idempotent: a seed is inserted only if its exact + * (key, channel, locale, version) default is absent, so re-boots never duplicate and a bumped + * version adds a new row while leaving the old one for audit/replay. + */ +@Injectable() +export class TemplateSeeder implements OnModuleInit { + private readonly log = new Logger(TemplateSeeder.name); + + constructor(private readonly prisma: PrismaService) {} + + async onModuleInit(): Promise { + const inserted = await this.seed(TEMPLATE_SEEDS); + if (inserted > 0) this.log.log(`seeded ${inserted} default template(s)`); + } + + /** Returns how many rows were newly inserted. */ + async seed(seeds: TemplateSeed[]): Promise { + let inserted = 0; + for (const s of seeds) { + const exists = await this.prisma.iiosMessageTemplate.findFirst({ + where: { scopeId: null, key: s.key, channel: s.channel, locale: s.locale, version: s.version }, + select: { id: true }, + }); + if (exists) continue; + await this.prisma.iiosMessageTemplate.create({ + data: { + scopeId: null, + key: s.key, + channel: s.channel, + locale: s.locale, + version: s.version, + subject: s.subject, + bodyHtml: s.bodyHtml, + bodyText: s.bodyText, + variables: s.variables as Prisma.InputJsonValue, + }, + }); + inserted++; + } + return inserted; + } +} From b44f795ba403d7f9a5f6c9600277a038eba79f7b Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:39:04 +0530 Subject: [PATCH 07/14] feat(templates): T6 controller + DTOs + module wiring MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POST /v1/templates/send (render→queue) and /preview (render only). OPA-gated in the sender: iios.template.send, and iios.template.send.inline for ad-hoc HTML (arbitrary markup to a customer is riskier than a reviewed stored template). Scope resolved from the caller's principal. TemplateModule registered in AppModule. Verified over HTTP against a local boot: preview + send of the seeded payment.receipt (SENT via sandbox, provenance persisted), XSS name escaped, idempotent replay → 1 row, unknown key 404, bad channel/no-source/no-auth 400. Co-Authored-By: Claude Opus 4.8 --- packages/iios-service/src/app.module.ts | 2 + .../src/templates/template.controller.ts | 62 +++++++++++++++++++ .../src/templates/template.dto.ts | 38 ++++++++++++ .../src/templates/template.module.ts | 21 +++++++ .../src/templates/templated-sender.spec.ts | 2 +- .../src/templates/templated-sender.ts | 13 +++- 6 files changed, 135 insertions(+), 3 deletions(-) create mode 100644 packages/iios-service/src/templates/template.controller.ts create mode 100644 packages/iios-service/src/templates/template.dto.ts create mode 100644 packages/iios-service/src/templates/template.module.ts diff --git a/packages/iios-service/src/app.module.ts b/packages/iios-service/src/app.module.ts index 29129e2..a1345a2 100644 --- a/packages/iios-service/src/app.module.ts +++ b/packages/iios-service/src/app.module.ts @@ -10,6 +10,7 @@ import { OutboxModule } from './outbox/outbox.module'; import { ThreadsModule } from './threads/threads.module'; import { MessageModule } from './messaging/message.module'; import { InboxModule } from './inbox/inbox.module'; +import { TemplateModule } from './templates/template.module'; import { MediaModule } from './media/media.module'; import { NotificationModule } from './notifications/notification.module'; import { SupportModule } from './support/support.module'; @@ -37,6 +38,7 @@ import { DevController } from './dev/dev.controller'; ThreadsModule, MessageModule, InboxModule, + TemplateModule, MediaModule, NotificationModule, SupportModule, diff --git a/packages/iios-service/src/templates/template.controller.ts b/packages/iios-service/src/templates/template.controller.ts new file mode 100644 index 0000000..bfa5bda --- /dev/null +++ b/packages/iios-service/src/templates/template.controller.ts @@ -0,0 +1,62 @@ +import { BadRequestException, Body, Controller, Headers, Post } from '@nestjs/common'; +import { IiosTemplateChannel } from '@prisma/client'; +import { SessionVerifier } from '../platform/session.verifier'; +import { ActorResolver, type MessagePrincipal } from '../identity/actor.resolver'; +import { TemplatedSender, type ExternalChannel } from './templated-sender'; +import { TemplateService } from './template.service'; +import { PreviewTemplateDto, SendTemplateDto } from './template.dto'; +import type { TemplateSource } from './template.model'; + +@Controller('v1/templates') +export class TemplateController { + constructor( + private readonly sender: TemplatedSender, + private readonly templates: TemplateService, + private readonly session: SessionVerifier, + private readonly actors: ActorResolver, + ) {} + + /** Render a stored/inline template and queue it for external delivery (EMAIL/SMS). */ + @Post('send') + async send(@Body() body: SendTemplateDto, @Headers('authorization') authorization?: string) { + const principal = this.principal(authorization); + const scope = await this.actors.resolveScope(principal); + return this.sender.sendTemplated({ + source: this.source(body), + channel: body.channel as ExternalChannel, + target: body.target, + vars: body.vars ?? {}, + scopeId: scope.id, + ...(body.locale ? { locale: body.locale } : {}), + idempotencyKey: body.idempotencyKey, + ...(body.purpose ? { purpose: body.purpose } : {}), + }); + } + + /** Render only — no send. For marketing/QA to eyeball copy before it goes out. */ + @Post('preview') + async preview(@Body() body: PreviewTemplateDto, @Headers('authorization') authorization?: string) { + const principal = this.principal(authorization); + const scope = await this.actors.resolveScope(principal); + const { content } = await this.templates.render(this.source(body), body.vars ?? {}, { + channel: body.channel as IiosTemplateChannel, + ...(body.locale ? { locale: body.locale } : {}), + scopeId: scope.id, + }); + return content; + } + + /** Map the DTO's key|inline into a TemplateSource (exactly one must be present). */ + private source(body: { key?: string; version?: number; inline?: { subject?: string; html?: string; text?: string; variables?: string[] } }): TemplateSource { + if (body.inline && body.key) throw new BadRequestException('provide either "key" or "inline", not both'); + if (body.inline) return { inline: body.inline }; + if (body.key) return body.version != null ? { key: body.key, version: body.version } : { key: body.key }; + throw new BadRequestException('one of "key" or "inline" is required'); + } + + private principal(authorization?: string): MessagePrincipal { + const token = (authorization ?? '').replace(/^Bearer\s+/i, ''); + if (!token) throw new BadRequestException('Authorization bearer token is required'); + return this.session.verify(token); + } +} diff --git a/packages/iios-service/src/templates/template.dto.ts b/packages/iios-service/src/templates/template.dto.ts new file mode 100644 index 0000000..cd14284 --- /dev/null +++ b/packages/iios-service/src/templates/template.dto.ts @@ -0,0 +1,38 @@ +import { Type } from 'class-transformer'; +import { IsIn, IsInt, IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator'; + +const EXTERNAL_CHANNELS = ['EMAIL', 'SMS'] as const; +const ALL_CHANNELS = ['EMAIL', 'SMS', 'INTERNAL'] as const; + +/** Inline template content — an ad-hoc source (e.g. finished HTML from marketing). */ +export class InlineTemplateDto { + @IsOptional() @IsString() subject?: string; + @IsOptional() @IsString() html?: string; + @IsOptional() @IsString() text?: string; + @IsOptional() @IsString({ each: true }) variables?: string[]; +} + +/** Body of POST /v1/templates/send. Provide EITHER `key` (stored) OR `inline` (ad-hoc). */ +export class SendTemplateDto { + @IsOptional() @IsString() @IsNotEmpty() key?: string; + @IsOptional() @IsInt() version?: number; + @IsOptional() @ValidateNested() @Type(() => InlineTemplateDto) inline?: InlineTemplateDto; + + @IsIn(EXTERNAL_CHANNELS) channel!: (typeof EXTERNAL_CHANNELS)[number]; + @IsString() @IsNotEmpty() target!: string; + @IsOptional() @IsObject() vars?: Record; + @IsOptional() @IsString() locale?: string; + @IsString() @IsNotEmpty() idempotencyKey!: string; + @IsOptional() @IsString() purpose?: string; +} + +/** Body of POST /v1/templates/preview — render only, no send. INTERNAL allowed (preview only). */ +export class PreviewTemplateDto { + @IsOptional() @IsString() @IsNotEmpty() key?: string; + @IsOptional() @IsInt() version?: number; + @IsOptional() @ValidateNested() @Type(() => InlineTemplateDto) inline?: InlineTemplateDto; + + @IsIn(ALL_CHANNELS) channel!: (typeof ALL_CHANNELS)[number]; + @IsOptional() @IsObject() vars?: Record; + @IsOptional() @IsString() locale?: string; +} diff --git a/packages/iios-service/src/templates/template.module.ts b/packages/iios-service/src/templates/template.module.ts new file mode 100644 index 0000000..d8e13e0 --- /dev/null +++ b/packages/iios-service/src/templates/template.module.ts @@ -0,0 +1,21 @@ +import { Module } from '@nestjs/common'; +import { AdaptersModule } from '../adapters/adapters.module'; +import { TemplateRepository } from './template.repository'; +import { TemplateService } from './template.service'; +import { TemplatedSender } from './templated-sender'; +import { TemplateSeeder } from './template.seeder'; +import { TemplateController } from './template.controller'; + +/** + * Reusable message-template module: render a stored/inline template → hand it to the existing + * outbound pipeline (via AdaptersModule's OutboundService). Content lives here; delivery stays in + * the outbound/capability layers. SessionVerifier, ActorResolver (IdentityModule) and PLATFORM_PORTS + * (PlatformModule) are global. + */ +@Module({ + imports: [AdaptersModule], + controllers: [TemplateController], + providers: [TemplateRepository, TemplateService, TemplatedSender, TemplateSeeder], + exports: [TemplateService, TemplatedSender], +}) +export class TemplateModule {} diff --git a/packages/iios-service/src/templates/templated-sender.spec.ts b/packages/iios-service/src/templates/templated-sender.spec.ts index 4c05745..b1f52e2 100644 --- a/packages/iios-service/src/templates/templated-sender.spec.ts +++ b/packages/iios-service/src/templates/templated-sender.spec.ts @@ -18,7 +18,7 @@ const asService = prisma as unknown as PrismaService; function sender(): TemplatedSender { const outbound = new OutboundService(asService, new CapabilityBroker(makeFakePorts(), new CapabilityProviderRegistry()), new IdempotencyService(asService)); const templates = new TemplateService(new TemplateRepository(asService)); - return new TemplatedSender(templates, outbound); + return new TemplatedSender(templates, outbound, makeFakePorts()); } async function seedReceipt() { diff --git a/packages/iios-service/src/templates/templated-sender.ts b/packages/iios-service/src/templates/templated-sender.ts index 4b4373f..b7a96b6 100644 --- a/packages/iios-service/src/templates/templated-sender.ts +++ b/packages/iios-service/src/templates/templated-sender.ts @@ -1,8 +1,11 @@ -import { Injectable } from '@nestjs/common'; +import { Inject, Injectable } from '@nestjs/common'; +import type { IiosPlatformPorts } from '@insignia/iios-contracts'; import { OutboundService } from '../adapters/outbound.service'; +import { PLATFORM_PORTS } from '../platform/platform-ports'; +import { decideOrThrow } from '../platform/fail-closed'; import { TemplateService } from './template.service'; import type { RenderedContent } from './template.renderer'; -import type { TemplateSource } from './template.model'; +import { isInlineSource, type TemplateSource } from './template.model'; /** External-egress channels only. INTERNAL (in-app, no SMTP) delivery is the messaging module's job. */ export type ExternalChannel = 'EMAIL' | 'SMS'; @@ -28,9 +31,15 @@ export class TemplatedSender { constructor( private readonly templates: TemplateService, private readonly outbound: OutboundService, + @Inject(PLATFORM_PORTS) private readonly ports: IiosPlatformPorts, ) {} async sendTemplated(input: SendTemplatedInput) { + // Inline HTML to a customer is more dangerous than a reviewed stored template — gate it apart, + // so policy can permit stored sends while restricting ad-hoc ones. Fail-closed. + const action = isInlineSource(input.source) ? 'iios.template.send.inline' : 'iios.template.send'; + await decideOrThrow(this.ports, { action, scopeId: input.scopeId, channel: input.channel }); + const { content, provenance } = await this.templates.render(input.source, input.vars, { channel: input.channel, locale: input.locale, From 65023ce404c6f046b5ab2b59b4b3220d7f0057fa Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 12:56:22 +0530 Subject: [PATCH 08/14] feat(templates): T8 PII redaction of outbound commands (fast-follow) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An email/SMS command holds PII: the recipient address (target) and the rendered body (payload). RetentionService now snapshots outbound commands and, once aged, redacts target->'[redacted]' and payload->{redacted:true} in place while KEEPING the template provenance (key/version/locale/hash) — so 'which template version did we send?' stays answerable after the PII is gone. - ensureOutboundSnapshots: one snapshot per command, dataClass 'outbound', archiveAfter==deleteAfter (PII goes straight to redact, no archive phase). Nullable command scope uses an 'unscoped' tag so the global sweep still reaches it. - applySweep redact branch switches on targetType; compliance holds honored; audit 'retention.redacted' resourceType 'outbound_command'. Closes lever #2 of the PII-minimization plan. 3 new + 6 regression tests. Co-Authored-By: Claude Opus 4.8 --- .../src/retention/retention.service.spec.ts | 56 +++++++++++++++++ .../src/retention/retention.service.ts | 61 ++++++++++++++++--- 2 files changed, 109 insertions(+), 8 deletions(-) diff --git a/packages/iios-service/src/retention/retention.service.spec.ts b/packages/iios-service/src/retention/retention.service.spec.ts index 3c51f75..40d88d1 100644 --- a/packages/iios-service/src/retention/retention.service.spec.ts +++ b/packages/iios-service/src/retention/retention.service.spec.ts @@ -117,3 +117,59 @@ describe('RetentionService (P9 — retention sweep)', () => { expect(await bodyOf(b.interactionId)).toBe('tenant b'); // scope B untouched }); }); + +// T8: an outbound email/SMS command holds PII (the recipient address, and the rendered body with +// their name). Once aged, redact those while KEEPING the template provenance for audit/replay. +describe('RetentionService — outbound command PII (T8)', () => { + const setOutboundSnapshot = (targetId: string, data: { archiveAfter?: Date; deleteAfter?: Date }) => + prisma.iiosRetentionPolicySnapshot.update({ where: { targetType_targetId: { targetType: 'outbound_command', targetId } }, data }); + + async function command(target: string): Promise<{ id: string; scopeId: string }> { + const scope = await prisma.iiosScope.create({ data: { orgId: 'org_demo', appId: 'crm-web' } }); + const cmd = await prisma.iiosOutboundCommand.create({ + data: { + channelType: 'EMAIL', target, scopeId: scope.id, status: 'SENT', idempotencyKey: `k-${randomUUID()}`, + payload: { subject: 'Hi Dana', html: '

secret body

' }, + templateKey: 'welcome', templateVersion: 1, templateLocale: 'en', renderedHash: 'abc123', + }, + }); + return { id: cmd.id, scopeId: scope.id }; + } + + it('ensureOutboundSnapshots captures one snapshot per outbound command', async () => { + const { id } = await command('dana@acme.com'); + const n = await svc().ensureOutboundSnapshots(); + expect(n).toBe(1); + const snap = await prisma.iiosRetentionPolicySnapshot.findUniqueOrThrow({ where: { targetType_targetId: { targetType: 'outbound_command', targetId: id } } }); + expect(snap.targetType).toBe('outbound_command'); + expect(snap.dataClass).toBe('outbound'); + }); + + it('redacts target + payload of an aged command but keeps template provenance', async () => { + const { id } = await command('dana@acme.com'); + await svc().ensureOutboundSnapshots(); + await setOutboundSnapshot(id, { archiveAfter: past, deleteAfter: past }); + + const res = await svc().applySweep(); + expect(res.redacted).toBe(1); + const after = await prisma.iiosOutboundCommand.findUniqueOrThrow({ where: { id } }); + expect(after.target).toBe('[redacted]'); + expect(after.payload).toEqual({ redacted: true }); + // Provenance survives — you can still answer "which template version did we send?" + expect(after.templateKey).toBe('welcome'); + expect(after.templateVersion).toBe(1); + expect(after.renderedHash).toBe('abc123'); + expect(await prisma.iiosAuditLink.count({ where: { action: 'retention.redacted', resourceType: 'outbound_command' } })).toBe(1); + }); + + it('an active compliance hold blocks the command sweep', async () => { + const { id, scopeId } = await command('held@acme.com'); + await svc().ensureOutboundSnapshots(); + await setOutboundSnapshot(id, { archiveAfter: past, deleteAfter: past }); + await prisma.iiosComplianceHold.create({ data: { scopeId, targetType: 'outbound_command', targetId: id, holdReason: 'legal' } }); + + const res = await svc().applySweep(); + expect(res.skippedHeld).toBe(1); + expect((await prisma.iiosOutboundCommand.findUniqueOrThrow({ where: { id } })).target).toBe('held@acme.com'); + }); +}); diff --git a/packages/iios-service/src/retention/retention.service.ts b/packages/iios-service/src/retention/retention.service.ts index b5831a7..f88b409 100644 --- a/packages/iios-service/src/retention/retention.service.ts +++ b/packages/iios-service/src/retention/retention.service.ts @@ -72,6 +72,41 @@ export class RetentionService implements OnModuleInit, OnModuleDestroy { return created; } + /** + * Capture a retention snapshot for any outbound command that lacks one. An email/SMS command + * holds PII (the recipient address, and the rendered body with their name), so it gets a single + * window and goes STRAIGHT to redact (archiveAfter == deleteAfter — no archive phase). Unscoped + * system sends use an 'unscoped' partition tag so the global sweep still reaches them. + */ + async ensureOutboundSnapshots(scopeId?: string): Promise { + const commands = await this.prisma.iiosOutboundCommand.findMany({ + where: scopeId ? { scopeId } : {}, + select: { id: true, scopeId: true, createdAt: true }, + }); + let created = 0; + for (const c of commands) { + const exists = await this.prisma.iiosRetentionPolicySnapshot.findUnique({ + where: { targetType_targetId: { targetType: 'outbound_command', targetId: c.id } }, + }); + if (exists) continue; + const deleteAt = new Date(c.createdAt.getTime() + this.windowDays('outbound', 'DELETE') * DAY_MS); + await this.prisma.iiosRetentionPolicySnapshot.create({ + data: { + policyKey: 'outbound:pii:v1', + scopeSnapshotId: c.scopeId ?? 'unscoped', + targetType: 'outbound_command', + targetId: c.id, + dataClass: 'outbound', + archiveAfter: deleteAt, + deleteAfter: deleteAt, + sourceVersion: process.env.IIOS_RETENTION_POLICY_VERSION ?? 'v1', + }, + }); + created++; + } + return created; + } + /** Act on due snapshots: archive, or delete (redact-in-place), honoring compliance holds. */ async applySweep(scopeId?: string): Promise { const now = new Date(); @@ -90,13 +125,22 @@ export class RetentionService implements OnModuleInit, OnModuleDestroy { } if (s.deleteAfter <= now) { - await this.prisma.$transaction([ - this.prisma.iiosMessagePart.updateMany({ where: { interactionId: s.targetId }, data: { bodyText: '[redacted]', contentRef: null } }), - this.prisma.iiosInboundRawEvent.updateMany({ where: { interactionId: s.targetId }, data: { payload: { redacted: true } } }), - this.prisma.iiosInteraction.update({ where: { id: s.targetId }, data: { status: 'REDACTED' } }), - this.prisma.iiosRetentionPolicySnapshot.update({ where: { id: s.id }, data: { status: 'REDACTED' } }), - ]); - await recordAudit(this.prisma, { action: 'retention.redacted', resourceType: 'interaction', resourceId: s.targetId, scopeId: s.scopeSnapshotId }); + if (s.targetType === 'outbound_command') { + // Redact the recipient address + rendered body; KEEP the template provenance columns. + await this.prisma.$transaction([ + this.prisma.iiosOutboundCommand.update({ where: { id: s.targetId }, data: { target: '[redacted]', payload: { redacted: true } } }), + this.prisma.iiosRetentionPolicySnapshot.update({ where: { id: s.id }, data: { status: 'REDACTED' } }), + ]); + await recordAudit(this.prisma, { action: 'retention.redacted', resourceType: 'outbound_command', resourceId: s.targetId, scopeId: s.scopeSnapshotId }); + } else { + await this.prisma.$transaction([ + this.prisma.iiosMessagePart.updateMany({ where: { interactionId: s.targetId }, data: { bodyText: '[redacted]', contentRef: null } }), + this.prisma.iiosInboundRawEvent.updateMany({ where: { interactionId: s.targetId }, data: { payload: { redacted: true } } }), + this.prisma.iiosInteraction.update({ where: { id: s.targetId }, data: { status: 'REDACTED' } }), + this.prisma.iiosRetentionPolicySnapshot.update({ where: { id: s.id }, data: { status: 'REDACTED' } }), + ]); + await recordAudit(this.prisma, { action: 'retention.redacted', resourceType: 'interaction', resourceId: s.targetId, scopeId: s.scopeSnapshotId }); + } result.redacted++; } else if (s.status === 'ACTIVE') { await this.prisma.iiosRetentionPolicySnapshot.update({ where: { id: s.id }, data: { status: 'ARCHIVED' } }); @@ -107,9 +151,10 @@ export class RetentionService implements OnModuleInit, OnModuleDestroy { return result; } - /** Full sweep: capture missing snapshots, then act on due ones. */ + /** Full sweep: capture missing snapshots (interactions + outbound commands), then act on due ones. */ async sweep(scopeId?: string): Promise { await this.ensureSnapshots(scopeId); + await this.ensureOutboundSnapshots(scopeId); return this.applySweep(scopeId); } From 7d7c75915a943b75cb67d2c77c64fd140dbf82b9 Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 13:25:37 +0530 Subject: [PATCH 09/14] docs(smtp): SMTP provider plan Co-Authored-By: Claude Opus 4.8 --- docs/smtp-provider-plan.md | 139 +++++++++++++++++++++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 docs/smtp-provider-plan.md diff --git a/docs/smtp-provider-plan.md b/docs/smtp-provider-plan.md new file mode 100644 index 0000000..298df07 --- /dev/null +++ b/docs/smtp-provider-plan.md @@ -0,0 +1,139 @@ +# SMTP Provider — Implementation Plan + +**Owner:** Maaz · **Repo:** `iios` (`packages/iios-service`) · **Purpose:** make external email *actually leave the building* (welcome / receipt), the critical path for Friday. + +## Goal + +Add an `SmtpProvider` so the `EMAIL` channel delivers via real SMTP (`accounts@lynkeduppro.com`, +fallback `ceo@lynkeduppro.com`) instead of the sandbox. The template module already renders and +queues to the `EMAIL` channel; this is the one piece between "queued (SENT via sandbox)" and "the +customer receives it." From the meeting: *"जो पहला जा रहा है, वो SMTP से जा रहा है, क्योंकि हमें तुरंत चाहिए."* + +## What already exists (do NOT rebuild) + +- `CapabilityProvider { name, channelTypes, capabilities, send(req) }` — the seam. +- `CapabilityProviderRegistry` binds a provider per channel: **sandbox by default**; `EmailProvider` + (HTTP) when `IIOS_PROVIDER_URL_EMAIL` is set. Unknown channels fail closed. +- `OutboundService.send` → `CapabilityBroker` (policy + obligations) → the bound provider. Idempotency, + rate limits, ledger, provenance all upstream — untouched. +- `req.payload` for EMAIL is `{ subject, text, html, inReplyTo }` (from `TemplatedSender`). + +## Design decisions (locked) + +| # | Decision | Why | +|---|---|---| +| 1 | New `SmtpProvider implements CapabilityProvider`, `channelTypes=['EMAIL']`, via **nodemailer** | The established provider pattern; nodemailer is the standard SMTP client | +| 2 | **Env-driven activation**, like `IIOS_PROVIDER_URL_EMAIL` | Off by default (sandbox); flip on by setting SMTP env — no code change to enable | +| 3 | **Registry precedence for EMAIL: SMTP > HTTP > sandbox** | SMTP is the intended prod path; HTTP relay stays available; sandbox is the safe default | +| 4 | **Transporter is injected** (constructor takes a `Transporter` or a factory) | SMTP is untestable against a live server in CI; inject a stub/`jsonTransport` to assert the envelope | +| 5 | **Optional fallback sender** (`accounts@` primary → `ceo@` on failure) | The meeting's fallback: if the primary mailbox send fails, retry once via the fallback identity | +| 6 | **Never throw** — a transport error returns `{ outcome: 'FAILED', errorCode }` | Adapter doctrine; the command is marked FAILED, the caller isn't broken | + +## Config (env) + +``` +IIOS_SMTP_HOST=smtp. # e.g. smtp.gmail.com (Google Workspace) +IIOS_SMTP_PORT=587 +IIOS_SMTP_SECURE=false # true for 465, false for 587/STARTTLS +IIOS_SMTP_USER=accounts@lynkeduppro.com +IIOS_SMTP_PASS= # Workspace App Password, NOT the account password +IIOS_SMTP_FROM="LynkedUp Pro " # defaults to USER +# optional fallback identity used only if the primary send FAILS +IIOS_SMTP_FALLBACK_USER=ceo@lynkeduppro.com +IIOS_SMTP_FALLBACK_PASS= +IIOS_SMTP_FALLBACK_FROM="Justin Johnson " +``` + +Activation rule: `SmtpProvider` is bound for `EMAIL` iff `IIOS_SMTP_HOST` + `IIOS_SMTP_USER` + +`IIOS_SMTP_PASS` are all set. Fallback transporter built only if the `_FALLBACK_*` trio is set. + +## Files + +``` +src/capability/smtp.provider.ts # new — the provider +src/capability/smtp.provider.spec.ts # new — injected-transport tests +src/capability/capability.registry.ts # modify — bind SMTP for EMAIL when configured (precedence) +package.json # add nodemailer + @types/nodemailer +``` + +## Contract + +```ts +interface SmtpIdentity { host: string; port: number; secure: boolean; user: string; pass: string; from: string } + +class SmtpProvider implements CapabilityProvider { + readonly name = 'smtp'; + readonly channelTypes = ['EMAIL']; + readonly capabilities = { canSend: true }; + // `makeTransport` is injectable so tests pass a stub / nodemailer jsonTransport. + constructor(primary: SmtpIdentity, fallback?: SmtpIdentity, makeTransport?: (id: SmtpIdentity) => Transporter) {} + async send(req: CapabilityRequest): Promise; +} +``` + +`send()` builds the mail from `req.target` (recipient) + `req.payload`: +``` +{ from, to: req.target, subject, text, html, + inReplyTo?, references?, // threading, from payload.inReplyTo + messageId } // generated; returned as providerRef so replies can thread +``` +Primary transporter sends; on throw, if a fallback identity exists, retry once via it; still failing +→ `FAILED`. Success → `{ outcome: 'SENT', providerRef: messageId, latencyMs }`. + +**Review findings folded in:** +- **`providerRef` = nodemailer's returned `info.messageId`**, not a hand-generated id — nodemailer + stamps the real `Message-ID` it sent, which is what a reply's `In-Reply-To` will actually match. +- **Fallback only on PRE-acceptance failures** (connection refused, auth failure, timeout) — NOT on + an error raised after the SMTP server already accepted the message. Retrying a post-acceptance + failure via the fallback identity would **double-deliver**. `send()` inspects the error (nodemailer + `err.responseCode` / code) and falls back only when the server never accepted. +- **Registry precedence is registration ORDER:** `register()` does `byChannel.set(ch, provider)`, so + the LAST registration for `EMAIL` wins. Bind sandbox first (all channels), then HTTP `EmailProvider` + if its URL is set, then `SmtpProvider` **last** if SMTP env is set → SMTP > HTTP > sandbox falls out. + +## Task-by-task (TDD) + +Each: failing test → red → implement → green → commit. Tests inject a stub transporter (no network). + +**T1 — provider skeleton + config parse** +- `smtpIdentityFromEnv()` reads the env trio; returns null if incomplete. +- Test: full env → identity; missing pass → null; fallback trio → fallback identity. + +**T2 — `send()` builds the correct envelope** +- Inject a recording stub transporter. Test: `from`/`to`/`subject`/`html`/`text` map from target+payload; + `inReplyTo` → header set when present; `messageId` generated and returned as `providerRef`; outcome `SENT`. + +**T3 — failure handling + fallback** +- Stub throws on primary. Test: with a fallback identity → retries via fallback, `SENT` via fallback + transporter; without fallback → `FAILED` with `errorCode`, **never throws**. + +**T4 — registry precedence** +- `capability.registry.spec` (or extend): with SMTP env set, `forChannel('EMAIL')` returns the SMTP + provider (not sandbox/HTTP); with only `IIOS_PROVIDER_URL_EMAIL` → HTTP; with neither → sandbox. +- The registry reads env in its constructor, so each case sets env, constructs a fresh + `CapabilityProviderRegistry`, asserts, then restores env (mirror the env save/restore other specs use). + +**T5 — gate + manual real send** +- `vitest run` (all), `boundary`, `build` green. +- **Manual (ops):** point env at a real mailbox (or nodemailer **Ethereal** test SMTP for a no-mailbox + end-to-end), boot, `POST /v1/templates/send` the `welcome` seed to your own address, confirm receipt + and that From = `accounts@lynkeduppro.com`. + +## Error handling +- Incomplete SMTP env → provider not bound → EMAIL falls back to sandbox (no accidental silent prod send). +- Transport failure → `FAILED` on the command (+ delivery attempt), never thrown. +- Fallback used → `providerRef` notes the fallback identity for audit. + +## Risks / prerequisites +- 🔴 **Rotate `Qwerty@a2` BEFORE enabling.** This is the switch that turns queued sends into real + emails to real addresses — a forgeable IIOS token now reaches customer inboxes under your brand. +- **Workspace App Password, not the account password** (2FA accounts reject the raw password over SMTP). +- **Deliverability:** SPF + DKIM + DMARC on `lynkeduppro.com` or mail lands in spam. Ops task, before real customers. +- **Sending limits:** Google Workspace SMTP ≈ 2000/day. Fine — instant welcome/receipt is low volume; the + drip goes via Mailchimp, not SMTP. +- **Test safety:** never point CI/test env at a real mailbox; tests use an injected stub, the manual step + uses Ethereal or a throwaway inbox. + +## Out of scope (separate plans) +Attachments over SMTP (extends `EmailPayload` + this provider); the inbox mirror / INTERNAL delivery +(no SMTP dependency). From 74cca2d53444916296c8692e340b1c44a80184a3 Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 13:31:04 +0530 Subject: [PATCH 10/14] feat(smtp): SMTP egress provider (nodemailer) for the EMAIL channel MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Makes external email actually leave the building (was sandbox-only). SmtpProvider implements CapabilityProvider; env-activated (accounts@ primary, ceo@ fallback); transporter injected for tests. - send() maps target+payload -> {from,to,subject,html,text,inReplyTo,references}; providerRef = nodemailer's real Message-ID (so replies thread via In-Reply-To). - Fallback ONLY on pre-acceptance failures (connect/auth/timeout) — a post-acceptance error is terminal, so a message the server already took can't be double-delivered. - Never throws — transport failure -> FAILED, per the adapter doctrine. - Registry precedence via registration order: SMTP > HTTP relay > sandbox for EMAIL. Verified: 13 unit tests (config/envelope/fallback/precedence) + a REAL SMTP round-trip against nodemailer Ethereal (SENT, genuine Message-ID). Full suite 281/281, boundary+build clean. Co-Authored-By: Claude Opus 4.8 --- packages/iios-service/package.json | 2 + .../capability/capability.registry.spec.ts | 34 ++++ .../src/capability/capability.registry.ts | 7 + .../src/capability/smtp.provider.spec.ts | 90 +++++++++++ .../src/capability/smtp.provider.ts | 145 ++++++++++++++++++ pnpm-lock.yaml | 19 +++ 6 files changed, 297 insertions(+) create mode 100644 packages/iios-service/src/capability/capability.registry.spec.ts create mode 100644 packages/iios-service/src/capability/smtp.provider.spec.ts create mode 100644 packages/iios-service/src/capability/smtp.provider.ts diff --git a/packages/iios-service/package.json b/packages/iios-service/package.json index 1113aec..2d11fc5 100644 --- a/packages/iios-service/package.json +++ b/packages/iios-service/package.json @@ -28,6 +28,7 @@ "ioredis": "^5.11.1", "jsonwebtoken": "^9.0.3", "jwks-rsa": "^4.1.0", + "nodemailer": "^9.0.3", "prisma": "^6.2.1", "reflect-metadata": "^0.2.2", "rxjs": "^7.8.2", @@ -41,6 +42,7 @@ "@types/express": "^5.0.6", "@types/jsonwebtoken": "^9.0.10", "@types/node": "^26.0.1", + "@types/nodemailer": "^8.0.1", "@types/web-push": "^3.6.4", "socket.io-client": "^4.8.3", "typescript": "^5.7.3" diff --git a/packages/iios-service/src/capability/capability.registry.spec.ts b/packages/iios-service/src/capability/capability.registry.spec.ts new file mode 100644 index 0000000..7f5f1f8 --- /dev/null +++ b/packages/iios-service/src/capability/capability.registry.spec.ts @@ -0,0 +1,34 @@ +import { describe, it, expect, afterEach } from 'vitest'; +import { CapabilityProviderRegistry } from './capability.registry'; + +// The registry reads env in its constructor, so each case sets env → constructs a fresh registry → +// asserts → restores env. +const SMTP_KEYS = ['IIOS_SMTP_HOST', 'IIOS_SMTP_USER', 'IIOS_SMTP_PASS', 'IIOS_PROVIDER_URL_EMAIL']; +const saved: Record = {}; +function set(env: Record) { + for (const k of SMTP_KEYS) { saved[k] = process.env[k]; delete process.env[k]; } + for (const [k, v] of Object.entries(env)) if (v != null) process.env[k] = v; +} +afterEach(() => { for (const k of SMTP_KEYS) { if (saved[k] == null) delete process.env[k]; else process.env[k] = saved[k]; } }); + +describe('CapabilityProviderRegistry — EMAIL precedence (SMTP > HTTP > sandbox)', () => { + it('binds SMTP for EMAIL when the SMTP env trio is set', () => { + set({ IIOS_SMTP_HOST: 'smtp.test', IIOS_SMTP_USER: 'accounts@x', IIOS_SMTP_PASS: 'p' }); + expect(new CapabilityProviderRegistry().forChannel('EMAIL').name).toBe('smtp'); + }); + + it('binds the HTTP EmailProvider when only IIOS_PROVIDER_URL_EMAIL is set', () => { + set({ IIOS_PROVIDER_URL_EMAIL: 'https://relay.test/send' }); + expect(new CapabilityProviderRegistry().forChannel('EMAIL').name).toBe('email-http'); + }); + + it('SMTP wins over the HTTP relay when both are set', () => { + set({ IIOS_SMTP_HOST: 'smtp.test', IIOS_SMTP_USER: 'accounts@x', IIOS_SMTP_PASS: 'p', IIOS_PROVIDER_URL_EMAIL: 'https://relay.test/send' }); + expect(new CapabilityProviderRegistry().forChannel('EMAIL').name).toBe('smtp'); + }); + + it('falls back to the sandbox when neither is configured', () => { + set({}); + expect(new CapabilityProviderRegistry().forChannel('EMAIL').name).toBe('sandbox'); + }); +}); diff --git a/packages/iios-service/src/capability/capability.registry.ts b/packages/iios-service/src/capability/capability.registry.ts index c055fc0..fa5cd98 100644 --- a/packages/iios-service/src/capability/capability.registry.ts +++ b/packages/iios-service/src/capability/capability.registry.ts @@ -3,6 +3,7 @@ import type { CapabilityProvider } from '@insignia/iios-contracts'; import { SandboxProvider } from './sandbox.provider'; import { HttpProvider } from './http.provider'; import { EmailProvider } from './email.provider'; +import { SmtpProvider, smtpFallbackFromEnv, smtpIdentityFromEnv } from './smtp.provider'; const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'SMS', 'WHATSAPP', 'PORTAL']; @@ -11,6 +12,9 @@ const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'SMS', 'WHATSAPP', 'PORTAL']; * default; if `IIOS_PROVIDER_URL_` is set, a real HttpProvider * overrides the sandbox for that channel (the "flip the binding" swap). Unknown * channels fail closed — no silent egress path. + * + * Precedence is registration ORDER (register() does Map.set → last wins). For EMAIL: + * sandbox → HTTP EmailProvider (if URL set) → SMTP (if SMTP env set), so SMTP > HTTP > sandbox. */ @Injectable() export class CapabilityProviderRegistry { @@ -24,6 +28,9 @@ export class CapabilityProviderRegistry { // EMAIL gets an email-shaped envelope provider; other channels use the generic HTTP one. this.register(ch === 'EMAIL' ? new EmailProvider(url) : new HttpProvider(ch, url)); } + // Real SMTP for EMAIL wins over the HTTP relay when configured (registered last). + const smtp = smtpIdentityFromEnv(); + if (smtp) this.register(new SmtpProvider(smtp, smtpFallbackFromEnv() ?? undefined)); } register(provider: CapabilityProvider): void { diff --git a/packages/iios-service/src/capability/smtp.provider.spec.ts b/packages/iios-service/src/capability/smtp.provider.spec.ts new file mode 100644 index 0000000..eb99c47 --- /dev/null +++ b/packages/iios-service/src/capability/smtp.provider.spec.ts @@ -0,0 +1,90 @@ +import { describe, it, expect } from 'vitest'; +import { SmtpProvider, smtpIdentityFromEnv, smtpFallbackFromEnv, type MailTransport, type SmtpIdentity } from './smtp.provider'; +import type { CapabilityRequest } from '@insignia/iios-contracts'; + +const ID: SmtpIdentity = { host: 'smtp.test', port: 587, secure: false, user: 'accounts@lynkeduppro.com', pass: 'p', from: 'accounts@lynkeduppro.com' }; +const FB: SmtpIdentity = { ...ID, user: 'ceo@lynkeduppro.com', from: 'Justin ' }; + +const req = (payload: Record): CapabilityRequest => ({ + capability: 'channel.send', channelType: 'EMAIL', target: 'dana@acme.com', payload, idempotencyKey: 'k1', +}); + +/** A recording transport; optionally throws a given error on send. */ +function stub(opts: { throwErr?: unknown; messageId?: string } = {}) { + const calls: Array<{ id: SmtpIdentity; mail: Parameters[0] }> = []; + const make = (id: SmtpIdentity): MailTransport => ({ + async sendMail(mail) { + calls.push({ id, mail }); + if (opts.throwErr) throw opts.throwErr; + return { messageId: opts.messageId ?? '' }; + }, + }); + return { make, calls }; +} + +describe('smtpIdentityFromEnv', () => { + const base = { IIOS_SMTP_HOST: 'smtp.test', IIOS_SMTP_USER: 'accounts@x', IIOS_SMTP_PASS: 'p' }; + + it('builds the identity from a complete env trio (defaults port 587, secure false)', () => { + const id = smtpIdentityFromEnv({ ...base } as NodeJS.ProcessEnv); + expect(id).toMatchObject({ host: 'smtp.test', port: 587, secure: false, user: 'accounts@x', from: 'accounts@x' }); + }); + + it('returns null when the trio is incomplete', () => { + expect(smtpIdentityFromEnv({ IIOS_SMTP_HOST: 'smtp.test', IIOS_SMTP_USER: 'a@x' } as NodeJS.ProcessEnv)).toBeNull(); + }); + + it('reads the fallback identity, reusing the primary host', () => { + const fb = smtpFallbackFromEnv({ ...base, IIOS_SMTP_FALLBACK_USER: 'ceo@x', IIOS_SMTP_FALLBACK_PASS: 'q', IIOS_SMTP_FALLBACK_FROM: 'CEO ' } as NodeJS.ProcessEnv); + expect(fb).toMatchObject({ host: 'smtp.test', user: 'ceo@x', from: 'CEO ' }); + }); + + it('returns null fallback when not configured', () => { + expect(smtpFallbackFromEnv({ ...base } as NodeJS.ProcessEnv)).toBeNull(); + }); +}); + +describe('SmtpProvider.send — envelope', () => { + it('maps target + payload into the mail and returns the messageId as providerRef', async () => { + const t = stub({ messageId: '' }); + const res = await new SmtpProvider(ID, undefined, t.make).send(req({ subject: 'Hi Dana', html: '

x

', text: 'x' })); + expect(res).toMatchObject({ outcome: 'SENT', providerRef: '' }); + expect(t.calls[0].mail).toMatchObject({ from: 'accounts@lynkeduppro.com', to: 'dana@acme.com', subject: 'Hi Dana', html: '

x

', text: 'x' }); + }); + + it('sets In-Reply-To + References headers for a reply', async () => { + const t = stub(); + await new SmtpProvider(ID, undefined, t.make).send(req({ subject: 're', inReplyTo: '' })); + expect(t.calls[0].mail).toMatchObject({ inReplyTo: '', references: '' }); + }); +}); + +describe('SmtpProvider.send — failure + fallback', () => { + it('retries via the fallback identity on a pre-acceptance failure (SENT via fallback)', async () => { + // primary throws ECONNREFUSED (server never accepted) → fallback used. + const primaryThrows = { make: (id: SmtpIdentity): MailTransport => ({ + async sendMail(mail) { + if (id.user === ID.user) throw Object.assign(new Error('refused'), { code: 'ECONNREFUSED' }); + return { messageId: '' }; + }, + }) }; + const res = await new SmtpProvider(ID, FB, primaryThrows.make).send(req({ subject: 'x' })); + expect(res.outcome).toBe('SENT'); + expect(res.providerRef).toBe('fallback:'); + }); + + it('does NOT retry a post-acceptance failure (avoids double delivery) → FAILED', async () => { + // responseCode present = the server already spoke; retrying could double-send. + const t = stub({ throwErr: Object.assign(new Error('rejected after data'), { responseCode: 550 }) }); + const res = await new SmtpProvider(ID, FB, t.make).send(req({ subject: 'x' })); + expect(res.outcome).toBe('FAILED'); + expect(t.calls).toHaveLength(1); // primary only — no fallback attempt + }); + + it('with no fallback, a failure is FAILED and never throws', async () => { + const t = stub({ throwErr: Object.assign(new Error('boom'), { code: 'ETIMEDOUT' }) }); + const res = await new SmtpProvider(ID, undefined, t.make).send(req({ subject: 'x' })); + expect(res.outcome).toBe('FAILED'); + expect(res.errorCode).toBe('ETIMEDOUT'); + }); +}); diff --git a/packages/iios-service/src/capability/smtp.provider.ts b/packages/iios-service/src/capability/smtp.provider.ts new file mode 100644 index 0000000..675a5c6 --- /dev/null +++ b/packages/iios-service/src/capability/smtp.provider.ts @@ -0,0 +1,145 @@ +import { randomUUID } from 'node:crypto'; +import { createTransport } from 'nodemailer'; +import type { CapabilityProvider, CapabilityRequest, ProviderResult } from '@insignia/iios-contracts'; + +/** One SMTP sending identity (a mailbox + how to reach its server). */ +export interface SmtpIdentity { + host: string; + port: number; + secure: boolean; + user: string; + pass: string; + from: string; +} + +/** The subset of a mail transport this provider needs — lets tests inject a stub (no live server). */ +export interface MailTransport { + sendMail(mail: { + from: string; + to: string; + subject?: string; + text?: string; + html?: string; + inReplyTo?: string; + references?: string; + }): Promise<{ messageId: string; accepted?: unknown[] }>; +} + +const bool = (v: string | undefined): boolean => v === 'true' || v === '1'; + +/** Build the primary SMTP identity from env, or null if the required trio is incomplete. */ +export function smtpIdentityFromEnv(env: NodeJS.ProcessEnv = process.env): SmtpIdentity | null { + return identityFrom(env, ''); +} + +/** Build the optional fallback identity (accounts@ → ceo@), or null if not configured. */ +export function smtpFallbackFromEnv(env: NodeJS.ProcessEnv = process.env): SmtpIdentity | null { + const fb = identityFrom(env, 'FALLBACK_'); + if (fb) return fb; + // Fallback may reuse the primary host/port and only override the mailbox identity. + const host = env.IIOS_SMTP_HOST; + const user = env.IIOS_SMTP_FALLBACK_USER; + const pass = env.IIOS_SMTP_FALLBACK_PASS; + if (!host || !user || !pass) return null; + return { + host, + port: Number(env.IIOS_SMTP_PORT ?? 587), + secure: bool(env.IIOS_SMTP_SECURE), + user, + pass, + from: env.IIOS_SMTP_FALLBACK_FROM ?? user, + }; +} + +function identityFrom(env: NodeJS.ProcessEnv, prefix: string): SmtpIdentity | null { + const host = env[`IIOS_SMTP_${prefix}HOST`] ?? (prefix ? undefined : env.IIOS_SMTP_HOST); + const user = env[`IIOS_SMTP_${prefix}USER`]; + const pass = env[`IIOS_SMTP_${prefix}PASS`]; + if (!host || !user || !pass) return null; + return { + host, + port: Number(env[`IIOS_SMTP_${prefix}PORT`] ?? env.IIOS_SMTP_PORT ?? 587), + secure: bool(env[`IIOS_SMTP_${prefix}SECURE`] ?? env.IIOS_SMTP_SECURE), + user, + pass, + from: env[`IIOS_SMTP_${prefix}FROM`] ?? user, + }; +} + +interface EmailPayload { + subject?: string; + text?: string; + html?: string; + inReplyTo?: string; +} + +/** + * SMTP egress provider (nodemailer). Bound for EMAIL when the SMTP env trio is set (else the sandbox + * stays). A transport failure surfaces as FAILED, never thrown. On a PRE-acceptance failure it retries + * once via the fallback identity (accounts@ → ceo@); a post-acceptance failure is NOT retried, so a + * message the server already accepted can't be double-delivered. + */ +export class SmtpProvider implements CapabilityProvider { + readonly name = 'smtp'; + readonly channelTypes = ['EMAIL']; + readonly capabilities = { canSend: true }; + + private readonly makeTransport: (id: SmtpIdentity) => MailTransport; + + constructor( + private readonly primary: SmtpIdentity, + private readonly fallback?: SmtpIdentity, + makeTransport?: (id: SmtpIdentity) => MailTransport, + ) { + this.makeTransport = makeTransport ?? defaultTransport; + } + + async send(req: CapabilityRequest): Promise { + const started = Date.now(); + const p = (req.payload ?? {}) as EmailPayload; + + const attempt = async (id: SmtpIdentity): Promise<{ messageId: string }> => + this.makeTransport(id).sendMail({ + from: id.from, + to: req.target, + subject: p.subject ?? '(no subject)', + text: p.text, + html: p.html, + ...(p.inReplyTo ? { inReplyTo: p.inReplyTo, references: p.inReplyTo } : {}), + }); + + try { + const info = await attempt(this.primary); + return { providerRef: info.messageId, outcome: 'SENT', latencyMs: Date.now() - started }; + } catch (err) { + // Retry via the fallback identity ONLY if the primary never got the message accepted. + if (this.fallback && isPreAcceptanceFailure(err)) { + try { + const info = await attempt(this.fallback); + return { providerRef: `fallback:${info.messageId}`, outcome: 'SENT', latencyMs: Date.now() - started }; + } catch (err2) { + return { providerRef: `smtp-error-${randomUUID().slice(0, 8)}`, outcome: 'FAILED', errorCode: codeOf(err2), latencyMs: Date.now() - started }; + } + } + return { providerRef: `smtp-error-${randomUUID().slice(0, 8)}`, outcome: 'FAILED', errorCode: codeOf(err), latencyMs: Date.now() - started }; + } + } +} + +/** True for connect/auth/timeout errors (server never accepted); false once the server responded 2xx. */ +function isPreAcceptanceFailure(err: unknown): boolean { + const e = err as { code?: string; responseCode?: number }; + const preCodes = ['ECONNECTION', 'ETIMEDOUT', 'ECONNREFUSED', 'EDNS', 'EAUTH', 'ESOCKET', 'EENVELOPE']; + if (e.code && preCodes.includes(e.code)) return true; + // A responseCode present means the server spoke — treat 5xx after acceptance as terminal (no retry). + return e.responseCode == null && e.code == null; +} + +function codeOf(err: unknown): string { + const e = err as { code?: string; message?: string }; + return (e.code ?? e.message ?? 'SMTP_ERROR').slice(0, 60); +} + +function defaultTransport(id: SmtpIdentity): MailTransport { + return createTransport({ host: id.host, port: id.port, secure: id.secure, auth: { user: id.user, pass: id.pass } }) as unknown as MailTransport; +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 32beda2..bab7336 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -376,6 +376,9 @@ importers: jwks-rsa: specifier: ^4.1.0 version: 4.1.0 + nodemailer: + specifier: ^9.0.3 + version: 9.0.3 prisma: specifier: ^6.2.1 version: 6.19.3(typescript@5.9.3) @@ -410,6 +413,9 @@ importers: '@types/node': specifier: ^26.0.1 version: 26.0.1 + '@types/nodemailer': + specifier: ^8.0.1 + version: 8.0.1 '@types/web-push': specifier: ^3.6.4 version: 3.6.4 @@ -1520,6 +1526,9 @@ packages: '@types/node@26.0.1': resolution: {integrity: sha512-fc3KiUoBt6kie0N9bIW3E47vZsuaMf0PM2AaUpLCLT0s/LvX1nxAim6Fc049cNxODPpGm6qRAuUOB86SkRuPQw==} + '@types/nodemailer@8.0.1': + resolution: {integrity: sha512-PxpaInm8V1JQDd4j0ds5HfvWQk8JupS1C0Picb96QJsrrRDjBH+DlK7L4ZdNSqNULhiZRQHc40nLVShaGxXAMw==} + '@types/qs@6.15.1': resolution: {integrity: sha512-GZHUBZR9hckSUhrxmp1nG6NwdpM9fCunJwyThLW1X3AyHgd9IlHb6VANpQQqDr2o/qQp6McZ3y/IA2rVzKzSbw==} @@ -2545,6 +2554,10 @@ packages: resolution: {integrity: sha512-J6l92tKHX6w8Jy5nO1Vuc01NoIiRGi/d6qBKVxh+IQ8Cr3b6HbVNfKiF8ZpFKufTwpwxMmce2W3iQZ861ZRyTg==} engines: {node: '>=18'} + nodemailer@9.0.3: + resolution: {integrity: sha512-n+YP+NKwR5zRWa60k3GiQ6Q3B4KXCoAw40dAKeCtYn020iNN74aWK2liXIC3ZEATeGql7we3tE3t8QwhY0eskw==} + engines: {node: '>=6.0.0'} + notepack.io@3.0.1: resolution: {integrity: sha512-TKC/8zH5pXIAMVQio2TvVDTtPRX+DJPHDqjRbxogtFiByHyzKmy96RA0JtCQJ+WouyyL4A10xomQzgbUT+1jCg==} @@ -4195,6 +4208,10 @@ snapshots: dependencies: undici-types: 8.3.0 + '@types/nodemailer@8.0.1': + dependencies: + '@types/node': 26.0.1 + '@types/qs@6.15.1': {} '@types/range-parser@1.2.7': {} @@ -5317,6 +5334,8 @@ snapshots: node-releases@2.0.50: {} + nodemailer@9.0.3: {} + notepack.io@3.0.1: {} nypm@0.6.8: From bba5fae0617f748aaa64fd8b11e049d7cc73df1f Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 13:49:19 +0530 Subject: [PATCH 11/14] docs(mail): inbox mirror + INTERNAL delivery plan Co-Authored-By: Claude Opus 4.8 --- docs/inbox-mirror-internal-delivery-plan.md | 113 ++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100644 docs/inbox-mirror-internal-delivery-plan.md diff --git a/docs/inbox-mirror-internal-delivery-plan.md b/docs/inbox-mirror-internal-delivery-plan.md new file mode 100644 index 0000000..fb9d1fc --- /dev/null +++ b/docs/inbox-mirror-internal-delivery-plan.md @@ -0,0 +1,113 @@ +# Inbox Mirror + INTERNAL Delivery — Implementation Plan + +**Owner:** Maaz · **Repo:** `iios` (`packages/iios-service`) · **Purpose:** every message the app +sends appears in the customer's in-app inbox; and users can send app-to-app "mail" with no SMTP. + +## Goal + +Two capabilities on one mechanism: +1. **Mirror** — when an external EMAIL is sent, also record it as an in-app interaction so the + customer sees a copy in their CRM inbox. Vivek: *"जो भी communication…उसकी एक copy inbox में चाहिए ही चाहिए."* +2. **INTERNAL delivery** — a user sends a mail-style message (subject + body) to another user with + **no SMTP**; it lands only in the recipient's in-app inbox. Vivek: *"app-to-app…without smtp."* + +Both reduce to the same primitive: **render a template → create an `Interaction(kind=EMAIL)` with +subject + HTML + TEXT parts on a thread.** External additionally does the SMTP send (already built). + +## Architectural guardrail (carried from the earlier inbox work) + +This is the **mail-style inbox (a projection over `Interaction`s)** — NOT the `InboxItem` work-surface. +- An email/message becomes an `Interaction(kind=EMAIL)` on a thread. The inbox view lists interactions. +- An `InboxItem` is created ONLY when the projector decides action is needed (NEEDS_REPLY/MENTION) — + that's the existing projector, unchanged. **We do not write InboxItems here.** Mixing them is the + KG-15 "inbox fatigue" failure. + +## What already exists (reuse, do NOT rebuild) + +- `IngestService.ingest(req, idempotencyKey)` — the generic create-an-interaction entry: resolves + source handle → actor → channel → thread, writes `Interaction` (kind from `req.kind`) + parts + + outbox event, idempotent per (scope, idempotencyKey). Inbound email already uses it to make + `EMAIL` interactions with HTML/TEXT parts — **the exact model for the outbound mirror.** +- `TemplateService.render()` (exported) → `{subject, html, text}`. +- `TemplatedSender.sendTemplated()` → SMTP egress (built). +- `IiosMessagePartKind` has `HTML` + `TEXT`; `IiosInteractionKind` has `EMAIL`. + +## Design (locked) + +- **New `MailService`** (new `src/mail/` module) orchestrates `TemplateService` + `IngestService` + + `TemplatedSender` + `ActorResolver`. Templates/outbound stay unaware of each other. + - `postInternal(...)` — render → `ingest()` an `EMAIL` interaction on a per-email thread. No SMTP. + - `sendExternalWithMirror(...)` — render → `TemplatedSender.sendTemplated()` (SMTP) → **and** mirror + via `ingest()` **iff the recipient is a registered user** (timing rule below). +- **Visibility (resolved review finding):** `ingest()` creates the interaction + thread but adds NO + participants, and `listThreads` shows only threads where the caller is a participant. So after each + ingest the MailService `ensureParticipant`s **both** the sender's actor and the recipient's actor + (`ActorResolver.resolveActor` → `ensureParticipant`). Without this the mirror is invisible. +- **`ingest()` returns `threadId`** — used directly to add the two participants. +- **Rendered content → parts:** part 0 `HTML` (bodyHtml), part 1 `TEXT` (bodyText); `subject` → the + thread subject (email threads share a subject). Attachments are the separate attachments plan. +- **Idempotency:** the ingest idempotencyKey = the send's key (e.g. `mirror:`), so a + retried send never doubles the inbox copy. +- **Reply/threading:** `parentInteractionId` for in-thread replies (already modeled); a mirrored + email's `inReplyTo` maps to the parent interaction. + +## The timing rule (locked, from the meeting) + +**Mirror only AFTER the recipient is registered.** The welcome/receipt go out *before* registration — +there is no in-app inbox to mirror into yet. So `sendExternalWithMirror` mirrors only when the target +resolves to a registered actor; pre-registration sends are email-only. Vivek: *"just time app pe +register kar liya, uske baad se jitna communication…uske inbox mein chahiye."* + +## Thread model (DECIDED: one thread per email) + +**Each send is its own thread / inbox entry; a reply threads onto it.** Matches email semantics and +pairs with the reply (`parentInteractionId`) feature. Implementation: the ingest `externalThreadId` +is **derived from the send's idempotency key**, so a retried send reuses the same thread (no dupe) +while distinct emails get distinct threads. A reply posts onto the parent's thread. + +## Files + +``` +src/mail/mail.service.ts # new — postInternal, sendExternalWithMirror +src/mail/mail.service.spec.ts # new — DB-backed +src/mail/mail.module.ts # new — imports TemplateModule + AdaptersModule + interactions +src/mail/mail.controller.ts # new? — OR extend template.controller with a `deliverInternal` route +``` +(Whether INTERNAL gets its own HTTP route or rides the template controller is a small call made at build time.) + +## Task-by-task (TDD) — pending the thread-model decision + +**T1 — `renderToParts()` helper**: `{subject,html,text}` → `IngestInteractionRequest.parts` + +thread subject. Test: HTML+TEXT parts produced; empty parts omitted. + +**T2 — `postInternal()`**: render an INTERNAL template → `ingest()` an `EMAIL` interaction on the +thread between sender + recipient (thread model per the decision). Test: interaction created with +kind EMAIL + parts; idempotent per key; lands on the recipient's thread. + +**T3 — `sendExternalWithMirror()`**: render → `sendTemplated` (SMTP/sandbox) → mirror `ingest()` +**only if** the recipient resolves to a registered actor. Test: registered → one outbound command + +one mirror interaction; unregistered → outbound only, no mirror; idempotent (replay → no dupes). + +**T4 — controller/module wiring + HTTP verify** (route for INTERNAL send; mirror invoked from the +external send path). Boot + drive over HTTP against the sandbox. + +**T5 — gate**: full suite + boundary + build; manual: send external → confirm a mirror interaction +appears on the recipient's thread. + +## Out of scope (follow-ons) +- **Frontend mail-inbox view** — surfacing `EMAIL` interactions as a mail-style inbox in the CRM + (the current CRM inbox is the InboxItem work-surface; the mail view is separate UI). +- **Attachments** (separate plan). **Stripe webhook** (be-crm) — the trigger. + +## Risks +- **Don't write InboxItems here** (KG-15). Interactions only; the projector owns InboxItems. +- **Idempotency must cover BOTH** the SMTP send and the mirror ingest, or a retried webhook doubles + the inbox copy. Same key threaded through both. +- **Unregistered recipients:** resolving "is this a registered user?" must be cheap and correct, or a + pre-registration send could either error or wrongly mirror into a non-existent inbox. +- **Review finding — recipient participation:** `ingest()` resolves and attaches the *source* actor. + For the interaction to appear in the *recipient's* inbox, the **recipient must be a thread + participant.** T2/T3 must ensure this — either by making the thread's participant set include the + recipient at create time, or an explicit `ensureParticipant` after ingest. A mirror the recipient + isn't a participant of is invisible — silent failure. Cover it with an assertion in the tests + ("recipient can list the thread / the interaction shows in their inbox query"). From 9b075f46f9fd0e845e399453171d97757d283b2c Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 13:54:32 +0530 Subject: [PATCH 12/14] feat(mail): inbox mirror + INTERNAL (app-to-app) delivery MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MailService renders a template then deposits it as an EMAIL interaction on a per-email thread (thread model: one thread per email), reusing IngestService. Because ingest adds no participants and a thread is only visible to its participants, it ensureParticipant()s BOTH sender and recipient — so the mirror is actually visible. - postInternal: app-to-app mail, no SMTP → recipient's in-app inbox. - sendExternalWithMirror: SMTP send (TemplatedSender) + mirror an interaction ONLY for a registered recipient (pre-registration sends are email-only — no inbox exists yet). Idempotent across both the send and the mirror. - Writes Interactions, NEVER InboxItems (the projector owns those — KG-15). - POST /v1/mail/internal, POST /v1/mail/send. MailModule in AppModule. Verified over HTTP: internal mail → recipient SEES the thread in their inbox; external send → command + mirror; no-source/no-auth 400. 7 unit tests. Co-Authored-By: Claude Opus 4.8 --- packages/iios-service/src/app.module.ts | 2 + .../iios-service/src/mail/mail.controller.ts | 55 +++++++++ packages/iios-service/src/mail/mail.dto.ts | 30 +++++ packages/iios-service/src/mail/mail.module.ts | 18 +++ .../src/mail/mail.service.spec.ts | 106 ++++++++++++++++++ .../iios-service/src/mail/mail.service.ts | 105 +++++++++++++++++ 6 files changed, 316 insertions(+) create mode 100644 packages/iios-service/src/mail/mail.controller.ts create mode 100644 packages/iios-service/src/mail/mail.dto.ts create mode 100644 packages/iios-service/src/mail/mail.module.ts create mode 100644 packages/iios-service/src/mail/mail.service.spec.ts create mode 100644 packages/iios-service/src/mail/mail.service.ts diff --git a/packages/iios-service/src/app.module.ts b/packages/iios-service/src/app.module.ts index a1345a2..1afdae9 100644 --- a/packages/iios-service/src/app.module.ts +++ b/packages/iios-service/src/app.module.ts @@ -11,6 +11,7 @@ import { ThreadsModule } from './threads/threads.module'; import { MessageModule } from './messaging/message.module'; import { InboxModule } from './inbox/inbox.module'; import { TemplateModule } from './templates/template.module'; +import { MailModule } from './mail/mail.module'; import { MediaModule } from './media/media.module'; import { NotificationModule } from './notifications/notification.module'; import { SupportModule } from './support/support.module'; @@ -39,6 +40,7 @@ import { DevController } from './dev/dev.controller'; MessageModule, InboxModule, TemplateModule, + MailModule, MediaModule, NotificationModule, SupportModule, diff --git a/packages/iios-service/src/mail/mail.controller.ts b/packages/iios-service/src/mail/mail.controller.ts new file mode 100644 index 0000000..968b1ff --- /dev/null +++ b/packages/iios-service/src/mail/mail.controller.ts @@ -0,0 +1,55 @@ +import { BadRequestException, Body, Controller, Headers, Post } from '@nestjs/common'; +import { SessionVerifier } from '../platform/session.verifier'; +import type { MessagePrincipal } from '../identity/actor.resolver'; +import { MailService } from './mail.service'; +import { MailInternalDto, MailSendDto } from './mail.dto'; +import type { TemplateSource } from '../templates/template.model'; + +@Controller('v1/mail') +export class MailController { + constructor( + private readonly mail: MailService, + private readonly session: SessionVerifier, + ) {} + + /** App-to-app mail — renders a template and posts it into the recipient's in-app inbox (no SMTP). */ + @Post('internal') + async internal(@Body() body: MailInternalDto, @Headers('authorization') authorization?: string) { + const principal = this.principal(authorization); + return this.mail.postInternal(principal, { + source: this.source(body), + recipientUserId: body.recipientUserId, + vars: body.vars ?? {}, + ...(body.locale ? { locale: body.locale } : {}), + idempotencyKey: body.idempotencyKey, + }); + } + + /** External email (SMTP) + an in-app mirror when a registered recipient is named. */ + @Post('send') + async send(@Body() body: MailSendDto, @Headers('authorization') authorization?: string) { + const principal = this.principal(authorization); + return this.mail.sendExternalWithMirror(principal, { + source: this.source(body), + target: body.target, + vars: body.vars ?? {}, + ...(body.locale ? { locale: body.locale } : {}), + idempotencyKey: body.idempotencyKey, + ...(body.purpose ? { purpose: body.purpose } : {}), + ...(body.mirrorToUserId ? { mirrorToUserId: body.mirrorToUserId } : {}), + }); + } + + private source(body: { key?: string; version?: number; inline?: { subject?: string; html?: string; text?: string; variables?: string[] } }): TemplateSource { + if (body.inline && body.key) throw new BadRequestException('provide either "key" or "inline", not both'); + if (body.inline) return { inline: body.inline }; + if (body.key) return body.version != null ? { key: body.key, version: body.version } : { key: body.key }; + throw new BadRequestException('one of "key" or "inline" is required'); + } + + private principal(authorization?: string): MessagePrincipal { + const token = (authorization ?? '').replace(/^Bearer\s+/i, ''); + if (!token) throw new BadRequestException('Authorization bearer token is required'); + return this.session.verify(token); + } +} diff --git a/packages/iios-service/src/mail/mail.dto.ts b/packages/iios-service/src/mail/mail.dto.ts new file mode 100644 index 0000000..92844aa --- /dev/null +++ b/packages/iios-service/src/mail/mail.dto.ts @@ -0,0 +1,30 @@ +import { Type } from 'class-transformer'; +import { IsInt, IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator'; +import { InlineTemplateDto } from '../templates/template.dto'; + +/** POST /v1/mail/internal — app-to-app mail (no SMTP). Provide EITHER `key` OR `inline`. */ +export class MailInternalDto { + @IsOptional() @IsString() @IsNotEmpty() key?: string; + @IsOptional() @IsInt() version?: number; + @IsOptional() @ValidateNested() @Type(() => InlineTemplateDto) inline?: InlineTemplateDto; + + @IsString() @IsNotEmpty() recipientUserId!: string; + @IsOptional() @IsObject() vars?: Record; + @IsOptional() @IsString() locale?: string; + @IsString() @IsNotEmpty() idempotencyKey!: string; +} + +/** POST /v1/mail/send — external email via SMTP + optional in-app mirror for a registered recipient. */ +export class MailSendDto { + @IsOptional() @IsString() @IsNotEmpty() key?: string; + @IsOptional() @IsInt() version?: number; + @IsOptional() @ValidateNested() @Type(() => InlineTemplateDto) inline?: InlineTemplateDto; + + @IsString() @IsNotEmpty() target!: string; + @IsOptional() @IsObject() vars?: Record; + @IsOptional() @IsString() locale?: string; + @IsString() @IsNotEmpty() idempotencyKey!: string; + @IsOptional() @IsString() purpose?: string; + /** The registered recipient to mirror to; omit for a pre-registration send (email only). */ + @IsOptional() @IsString() mirrorToUserId?: string; +} diff --git a/packages/iios-service/src/mail/mail.module.ts b/packages/iios-service/src/mail/mail.module.ts new file mode 100644 index 0000000..3ff835d --- /dev/null +++ b/packages/iios-service/src/mail/mail.module.ts @@ -0,0 +1,18 @@ +import { Module } from '@nestjs/common'; +import { TemplateModule } from '../templates/template.module'; +import { InteractionsModule } from '../interactions/interactions.module'; +import { MailService } from './mail.service'; +import { MailController } from './mail.controller'; + +/** + * Mail orchestration: render a template (TemplateModule) → deliver it either app-to-app (an EMAIL + * interaction via IngestService) or externally (TemplatedSender/SMTP) with an in-app mirror. + * SessionVerifier + ActorResolver are global. + */ +@Module({ + imports: [TemplateModule, InteractionsModule], + controllers: [MailController], + providers: [MailService], + exports: [MailService], +}) +export class MailModule {} diff --git a/packages/iios-service/src/mail/mail.service.spec.ts b/packages/iios-service/src/mail/mail.service.spec.ts new file mode 100644 index 0000000..ea1dc08 --- /dev/null +++ b/packages/iios-service/src/mail/mail.service.spec.ts @@ -0,0 +1,106 @@ +import { describe, it, expect, beforeAll, afterAll, beforeEach } from 'vitest'; +import { PrismaClient } from '@prisma/client'; +import { makeFakePorts } from '@insignia/iios-testkit'; +import { resetDb } from '../test-utils/reset-db'; +import { ActorResolver, type MessagePrincipal } from '../identity/actor.resolver'; +import { IngestService } from '../interactions/ingest.service'; +import { MessageService } from '../messaging/message.service'; +import { OutboundService } from '../adapters/outbound.service'; +import { CapabilityBroker } from '../capability/capability.broker'; +import { CapabilityProviderRegistry } from '../capability/capability.registry'; +import { IdempotencyService } from '../idempotency/idempotency.service'; +import { TemplateRepository } from '../templates/template.repository'; +import { TemplateService } from '../templates/template.service'; +import { TemplatedSender } from '../templates/templated-sender'; +import { MailService, contentToParts } from './mail.service'; +import type { PrismaService } from '../prisma/prisma.service'; + +const url = process.env.DATABASE_URL ?? 'postgresql://iios:iios@localhost:5434/iios_test?schema=public'; +const prisma = new PrismaClient({ datasources: { db: { url } } }); +const asService = prisma as unknown as PrismaService; +const actors = new ActorResolver(asService); + +function mail(): MailService { + const templates = new TemplateService(new TemplateRepository(asService)); + const ingest = new IngestService(asService, makeFakePorts(), actors); + const outbound = new OutboundService(asService, new CapabilityBroker(makeFakePorts(), new CapabilityProviderRegistry()), new IdempotencyService(asService)); + const sender = new TemplatedSender(templates, outbound, makeFakePorts()); + return new MailService(templates, ingest, sender, actors); +} +const messages = () => new MessageService(asService, makeFakePorts(), actors); + +const alice: MessagePrincipal = { userId: 'alice', orgId: 'org_demo', appId: 'portal-demo', displayName: 'Alice' }; +const bob: MessagePrincipal = { userId: 'bob', orgId: 'org_demo', appId: 'portal-demo', displayName: 'Bob' }; +const inline = { inline: { subject: 'Welcome Dana', html: '

Hi Dana

', text: 'Hi Dana', variables: [] } }; + +beforeAll(async () => { await prisma.$connect(); }); +afterAll(async () => { await prisma.$disconnect(); }); +beforeEach(async () => { await resetDb(prisma); }); + +describe('contentToParts (pure)', () => { + it('produces HTML + TEXT parts and the subject', () => { + expect(contentToParts({ subject: 'S', html: '

h

', text: 't' })).toEqual({ subject: 'S', parts: [{ kind: 'HTML', bodyText: '

h

' }, { kind: 'TEXT', bodyText: 't' }] }); + }); + it('never yields zero parts (empty text fallback)', () => { + expect(contentToParts({ subject: 'S' }).parts).toEqual([{ kind: 'TEXT', bodyText: '' }]); + }); +}); + +describe('MailService.postInternal', () => { + it('creates an EMAIL interaction and makes the thread visible to BOTH sender and recipient', async () => { + const res = await mail().postInternal(alice, { source: inline, recipientUserId: 'bob', idempotencyKey: 'int:1' }); + + const interaction = await prisma.iiosInteraction.findUniqueOrThrow({ where: { id: res.interactionId }, include: { parts: true } }); + expect(interaction.kind).toBe('EMAIL'); + expect(interaction.parts.map((p) => p.kind).sort()).toEqual(['HTML', 'TEXT']); + + const thread = await prisma.iiosThread.findUniqueOrThrow({ where: { id: res.threadId } }); + expect(thread.subject).toBe('Welcome Dana'); + + // The load-bearing assertion: the RECIPIENT can see the thread in their inbox. + const bobThreads = await messages().listThreads(bob); + expect(bobThreads.map((t) => t.threadId)).toContain(res.threadId); + // ...and so can the sender. + const aliceThreads = await messages().listThreads(alice); + expect(aliceThreads.map((t) => t.threadId)).toContain(res.threadId); + }); + + it('is idempotent per key — a replay reuses the same thread, no duplicate interaction', async () => { + const a = await mail().postInternal(alice, { source: inline, recipientUserId: 'bob', idempotencyKey: 'int:dup' }); + const b = await mail().postInternal(alice, { source: inline, recipientUserId: 'bob', idempotencyKey: 'int:dup' }); + expect(b.threadId).toBe(a.threadId); + expect(await prisma.iiosInteraction.count({ where: { threadId: a.threadId } })).toBe(1); + }); +}); + +describe('MailService.sendExternalWithMirror', () => { + const ext = { source: inline, target: 'dana@acme.com', idempotencyKey: 'recv:1' } as const; + + it('sends via the outbound pipeline AND mirrors into a registered recipient inbox', async () => { + const res = await mail().sendExternalWithMirror(alice, { ...ext, mirrorToUserId: 'bob' }); + // outbound command exists (SENT via sandbox in tests) + const cmd = await prisma.iiosOutboundCommand.findUniqueOrThrow({ where: { id: res.commandId } }); + expect(cmd.channelType).toBe('EMAIL'); + expect(cmd.target).toBe('dana@acme.com'); + // mirror interaction visible to the recipient + expect(res.mirror).toBeDefined(); + const bobThreads = await messages().listThreads(bob); + expect(bobThreads.map((t) => t.threadId)).toContain(res.mirror!.threadId); + }); + + it('does NOT mirror when there is no registered recipient (pre-registration send)', async () => { + const res = await mail().sendExternalWithMirror(alice, { ...ext, idempotencyKey: 'recv:2' }); // no mirrorToUserId + expect(res.mirror).toBeUndefined(); + // an outbound command was created, but no mirror interaction + expect(await prisma.iiosOutboundCommand.count({ where: { id: res.commandId } })).toBe(1); + expect(await prisma.iiosInteraction.count()).toBe(0); + }); + + it('is idempotent — a replay yields one command and one mirror', async () => { + const a = await mail().sendExternalWithMirror(alice, { ...ext, idempotencyKey: 'recv:dup', mirrorToUserId: 'bob' }); + const b = await mail().sendExternalWithMirror(alice, { ...ext, idempotencyKey: 'recv:dup', mirrorToUserId: 'bob' }); + expect(b.commandId).toBe(a.commandId); + expect(await prisma.iiosOutboundCommand.count({ where: { idempotencyKey: 'recv:dup' } })).toBe(1); + expect(await prisma.iiosInteraction.count({ where: { threadId: a.mirror!.threadId } })).toBe(1); + }); +}); diff --git a/packages/iios-service/src/mail/mail.service.ts b/packages/iios-service/src/mail/mail.service.ts new file mode 100644 index 0000000..7d696f5 --- /dev/null +++ b/packages/iios-service/src/mail/mail.service.ts @@ -0,0 +1,105 @@ +import { Injectable } from '@nestjs/common'; +import { IngestService } from '../interactions/ingest.service'; +import { ActorResolver, type MessagePrincipal } from '../identity/actor.resolver'; +import { TemplateService } from '../templates/template.service'; +import { TemplatedSender } from '../templates/templated-sender'; +import type { RenderedContent } from '../templates/template.renderer'; +import type { TemplateSource } from '../templates/template.model'; + +export interface MailPart { kind: 'HTML' | 'TEXT'; bodyText: string } + +/** Turn rendered content into ingest parts (HTML + TEXT) + the thread subject. Pure. */ +export function contentToParts(content: RenderedContent): { subject?: string; parts: MailPart[] } { + const parts: MailPart[] = []; + if (content.html != null) parts.push({ kind: 'HTML', bodyText: content.html }); + if (content.text != null) parts.push({ kind: 'TEXT', bodyText: content.text }); + // A message must carry at least one part; fall back to an empty text part rather than fail ingest. + if (parts.length === 0) parts.push({ kind: 'TEXT', bodyText: '' }); + return { ...(content.subject != null ? { subject: content.subject } : {}), parts }; +} + +export interface PostInternalInput { + source: TemplateSource; + recipientUserId: string; + vars?: Record; + locale?: string; + idempotencyKey: string; +} + +export interface SendExternalInput { + source: TemplateSource; + target: string; // email address (external) + vars?: Record; + locale?: string; + idempotencyKey: string; + purpose?: string; + /** The registered recipient to mirror to; omit for a pre-registration send (email only, no mirror). */ + mirrorToUserId?: string; +} + +/** + * Mail orchestration: render a template, then deliver it. + * - INTERNAL (app-to-app): create an EMAIL interaction on a per-email thread — no SMTP. + * - EXTERNAL: send via SMTP (TemplatedSender) AND mirror a copy into the recipient's in-app inbox, + * but only when they are a registered user (pre-registration sends have no inbox yet). + * + * ingest() writes the interaction + thread but adds no participants, and a thread is only visible to + * its participants — so after each ingest we add BOTH the sender and the recipient as participants. + */ +@Injectable() +export class MailService { + constructor( + private readonly templates: TemplateService, + private readonly ingest: IngestService, + private readonly sender: TemplatedSender, + private readonly actors: ActorResolver, + ) {} + + async postInternal(principal: MessagePrincipal, input: PostInternalInput): Promise<{ threadId: string; interactionId: string }> { + const { content } = await this.templates.render(input.source, input.vars ?? {}, { channel: 'INTERNAL', ...(input.locale ? { locale: input.locale } : {}) }); + return this.deposit(principal, input.recipientUserId, content, input.idempotencyKey, 'PORTAL'); + } + + async sendExternalWithMirror(principal: MessagePrincipal, input: SendExternalInput): Promise<{ commandId: string; mirror?: { threadId: string; interactionId: string } }> { + const command = await this.sender.sendTemplated({ + source: input.source, channel: 'EMAIL', target: input.target, vars: input.vars ?? {}, + ...(input.locale ? { locale: input.locale } : {}), idempotencyKey: input.idempotencyKey, ...(input.purpose ? { purpose: input.purpose } : {}), + }); + + // Mirror only for a registered recipient (timing rule: no inbox exists pre-registration). + if (!input.mirrorToUserId) return { commandId: command.id }; + + const { content } = await this.templates.render(input.source, input.vars ?? {}, { channel: 'EMAIL', ...(input.locale ? { locale: input.locale } : {}) }); + const mirror = await this.deposit(principal, input.mirrorToUserId, content, `mirror:${input.idempotencyKey}`, 'EMAIL'); + return { commandId: command.id, mirror }; + } + + /** Ingest the rendered content as an EMAIL interaction on a per-email thread, visible to both parties. */ + private async deposit(principal: MessagePrincipal, recipientUserId: string, content: RenderedContent, key: string, channelType: string): Promise<{ threadId: string; interactionId: string }> { + const { subject, parts } = contentToParts(content); + const res = await this.ingest.ingest( + { + scope: { orgId: principal.orgId, appId: principal.appId, ...(principal.tenantId ? { tenantId: principal.tenantId } : {}) }, + channel: { type: channelType, externalChannelId: channelType.toLowerCase() }, + source: { handleKind: 'PORTAL_USER', externalId: principal.userId, ...(principal.displayName ? { displayName: principal.displayName } : {}) }, + kind: 'EMAIL', + thread: { externalThreadId: key, ...(subject ? { subject } : {}) }, + parts, + occurredAt: new Date().toISOString(), + providerEventId: key, + }, + key, + ); + + // Make the thread visible to both the sender and the recipient (ingest adds no participants). + const scope = await this.actors.resolveScope(principal); + const senderActor = await this.actors.resolveActor(scope.id, principal); + const recipientActor = await this.actors.resolveActor(scope.id, { + userId: recipientUserId, appId: principal.appId, orgId: principal.orgId, ...(principal.tenantId ? { tenantId: principal.tenantId } : {}), + }); + await this.actors.ensureParticipant(res.threadId, senderActor.id); + await this.actors.ensureParticipant(res.threadId, recipientActor.id); + + return { threadId: res.threadId, interactionId: res.interactionId }; + } +} From d28c873d4069cf249097d96a499d64ed80813c40 Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 14:14:20 +0530 Subject: [PATCH 13/14] feat(mail): email attachments over SMTP MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Email can now carry attachments (e.g. an invoice PDF). The kernel already stored attachments as message parts + the media StoragePort holds the bytes; the gap was the email envelope. - EMAIL payload carries attachment REFS ({filename, contentRef, mimeType}), not bytes — the ledger + T8 PII redaction stay small; bytes are fetched at send time. - SmtpProvider takes an AttachmentResolver; resolves each ref via storage and attaches (nodemailer). FAILS CLOSED if a declared attachment can't be resolved (or no resolver is wired) — never send a receipt/invoice missing its file; a FAILED command retries. - CapabilityProviderRegistry injects the resolver from STORAGE_PORT (@Optional); MediaModule exports STORAGE_PORT, CapabilityModule imports MediaModule. No cycle. - TemplatedSender + MailService + the /v1/mail/send DTO pass attachments through. Verified: 6 new unit tests (attach, fail-closed x2, plain-unaffected, resolver, pass-through) + a REAL Ethereal SMTP send WITH a PDF attachment (SENT). Full suite 294/294, boundary + build clean. Co-Authored-By: Claude Opus 4.8 --- docs/email-attachments-plan.md | 67 +++++++++++++++++++ .../src/capability/capability.module.ts | 2 + .../src/capability/capability.registry.ts | 15 +++-- .../src/capability/smtp.provider.spec.ts | 47 ++++++++++++- .../src/capability/smtp.provider.ts | 40 +++++++++++ .../iios-service/src/mail/mail.controller.ts | 1 + packages/iios-service/src/mail/mail.dto.ts | 10 ++- .../iios-service/src/mail/mail.service.ts | 3 + .../iios-service/src/media/media.module.ts | 2 + .../src/templates/templated-sender.spec.ts | 10 +++ .../src/templates/templated-sender.ts | 12 +++- 11 files changed, 199 insertions(+), 10 deletions(-) create mode 100644 docs/email-attachments-plan.md diff --git a/docs/email-attachments-plan.md b/docs/email-attachments-plan.md new file mode 100644 index 0000000..855d304 --- /dev/null +++ b/docs/email-attachments-plan.md @@ -0,0 +1,67 @@ +# Email Attachments — Implementation Plan + +**Owner:** Maaz · **Repo:** `iios` (`packages/iios-service`) · Extends the SMTP provider + mail plumbing. + +## Goal + +Let an external email carry attachments (e.g. an invoice PDF). The kernel already STORES attachments +as message parts (`contentRef` + mime + size); the media `StoragePort` holds the bytes. The one gap is +the **email envelope** — `SmtpProvider` (and the payload) don't carry attachments. Close that. + +## Design (locked) + +- **Payload carries REFS, not bytes:** the EMAIL payload gains + `attachments?: [{ filename, contentRef, mimeType? }]`. Refs keep the outbound-command ledger small + (and keep T8's PII redaction cheap) — bytes are fetched at send time. +- **Resolver seam:** `AttachmentResolver = (contentRef) => Promise<{ filename?; content: Buffer; contentType? } | null>`. + `SmtpProvider` takes an optional resolver; on send it resolves each ref and attaches + (nodemailer `attachments: [{ filename, content, contentType }]`). +- **Fail closed on a missing attachment:** if a declared attachment can't be resolved, the send is + `FAILED` (so it retries) — NOT sent without it. A receipt/invoice missing its file is worse than a + retry. (No resolver wired at all + attachments present → also FAILED, same reasoning.) +- **Wiring:** `MediaModule` exports `STORAGE_PORT`; `CapabilityModule` imports `MediaModule`; + `CapabilityProviderRegistry` `@Optional() @Inject(STORAGE_PORT)` → builds the resolver from + `storage.get(contentRef)` → passes it to `SmtpProvider`. `@Optional` so contexts without storage + still boot (attachments simply can't resolve → FAILED if any are declared). +- **Scope:** SMTP path only. The HTTP relay `EmailProvider` attachment support is a separate follow-up + (it would base64 the bytes into the relay POST). INTERNAL/mirror attachments already work via message + parts and are not this plan. + +## Files + +``` +src/capability/smtp.provider.ts # attachments in EmailPayload + resolve+attach in send() +src/capability/smtp.provider.spec.ts # attach resolved bytes; missing → FAILED +src/capability/capability.registry.ts # inject STORAGE_PORT → resolver → SmtpProvider +src/capability/capability.module.ts # import MediaModule +src/media/media.module.ts # export STORAGE_PORT +src/templates/templated-sender.ts # accept + pass `attachments` +src/mail/mail.service.ts # accept + pass `attachments` (external send) +``` + +## Tasks (TDD) + +**T1 — SmtpProvider attaches / fails closed** +- Inject a stub resolver. Tests: two refs → nodemailer `attachments` has both (filename + content + + contentType); a ref the resolver returns `null` for → outcome `FAILED`, nothing sent; no attachments + in payload → unchanged (plain send still SENT). + +**T2 — registry wires the resolver from STORAGE_PORT** +- `MediaModule` exports `STORAGE_PORT`; `CapabilityModule` imports `MediaModule`; registry injects it + `@Optional`. Test: with a fake storage bound, `forChannel('EMAIL')` SMTP resolves an attachment; + without storage, the registry still constructs (attachments would FAIL, but boot is fine). + +**T3 — pass-through: TemplatedSender + MailService** +- `sendTemplated`/`sendExternalWithMirror` accept `attachments` and place them in the EMAIL payload. + Tests: the outbound command's payload carries the attachment refs. + +**T4 — gate + real send** +- Full suite + boundary + build. Manual: a real Ethereal send with a small attachment → SENT, and the + Ethereal message shows the attachment. + +## Risks +- **Fail-closed is deliberate** — don't silently send an invoice email without the invoice. +- **Payload holds refs, not bytes** — so the ledger and T8 redaction stay small; the resolver reads + bytes only at send time. +- **`@Optional` storage** — a context without `STORAGE_PORT` boots fine but can't send attachments; + that's correct (fail-closed), not a silent drop. diff --git a/packages/iios-service/src/capability/capability.module.ts b/packages/iios-service/src/capability/capability.module.ts index 2809916..6eac20c 100644 --- a/packages/iios-service/src/capability/capability.module.ts +++ b/packages/iios-service/src/capability/capability.module.ts @@ -1,4 +1,5 @@ import { Module } from '@nestjs/common'; +import { MediaModule } from '../media/media.module'; import { CapabilityProviderRegistry } from './capability.registry'; import { CapabilityBroker } from './capability.broker'; @@ -8,6 +9,7 @@ import { CapabilityBroker } from './capability.broker'; * provider. PLATFORM_PORTS (for the opa gate) comes from the @Global PlatformModule. */ @Module({ + imports: [MediaModule], providers: [CapabilityProviderRegistry, CapabilityBroker], exports: [CapabilityBroker, CapabilityProviderRegistry], }) diff --git a/packages/iios-service/src/capability/capability.registry.ts b/packages/iios-service/src/capability/capability.registry.ts index fa5cd98..4728e7a 100644 --- a/packages/iios-service/src/capability/capability.registry.ts +++ b/packages/iios-service/src/capability/capability.registry.ts @@ -1,9 +1,10 @@ -import { Injectable, NotFoundException } from '@nestjs/common'; +import { Inject, Injectable, NotFoundException, Optional } from '@nestjs/common'; import type { CapabilityProvider } from '@insignia/iios-contracts'; import { SandboxProvider } from './sandbox.provider'; import { HttpProvider } from './http.provider'; import { EmailProvider } from './email.provider'; -import { SmtpProvider, smtpFallbackFromEnv, smtpIdentityFromEnv } from './smtp.provider'; +import { SmtpProvider, smtpFallbackFromEnv, smtpIdentityFromEnv, storageResolver } from './smtp.provider'; +import { STORAGE_PORT, type StoragePort } from '../media/storage.port'; const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'SMS', 'WHATSAPP', 'PORTAL']; @@ -20,7 +21,7 @@ const DEFAULT_CHANNELS = ['WEBHOOK', 'EMAIL', 'SMS', 'WHATSAPP', 'PORTAL']; export class CapabilityProviderRegistry { private readonly byChannel = new Map(); - constructor() { + constructor(@Optional() @Inject(STORAGE_PORT) private readonly storage?: StoragePort) { for (const ch of DEFAULT_CHANNELS) this.register(new SandboxProvider([ch])); for (const ch of DEFAULT_CHANNELS) { const url = process.env[`IIOS_PROVIDER_URL_${ch}`]; @@ -28,9 +29,13 @@ export class CapabilityProviderRegistry { // EMAIL gets an email-shaped envelope provider; other channels use the generic HTTP one. this.register(ch === 'EMAIL' ? new EmailProvider(url) : new HttpProvider(ch, url)); } - // Real SMTP for EMAIL wins over the HTTP relay when configured (registered last). + // Real SMTP for EMAIL wins over the HTTP relay when configured (registered last). Attachments + // resolve through the media StoragePort when one is bound (else attachments FAIL closed). const smtp = smtpIdentityFromEnv(); - if (smtp) this.register(new SmtpProvider(smtp, smtpFallbackFromEnv() ?? undefined)); + if (smtp) { + const resolver = this.storage ? storageResolver(this.storage) : undefined; + this.register(new SmtpProvider(smtp, smtpFallbackFromEnv() ?? undefined, undefined, resolver)); + } } register(provider: CapabilityProvider): void { diff --git a/packages/iios-service/src/capability/smtp.provider.spec.ts b/packages/iios-service/src/capability/smtp.provider.spec.ts index eb99c47..4aed5b6 100644 --- a/packages/iios-service/src/capability/smtp.provider.spec.ts +++ b/packages/iios-service/src/capability/smtp.provider.spec.ts @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest'; -import { SmtpProvider, smtpIdentityFromEnv, smtpFallbackFromEnv, type MailTransport, type SmtpIdentity } from './smtp.provider'; +import { SmtpProvider, smtpIdentityFromEnv, smtpFallbackFromEnv, storageResolver, type MailTransport, type SmtpIdentity } from './smtp.provider'; import type { CapabilityRequest } from '@insignia/iios-contracts'; const ID: SmtpIdentity = { host: 'smtp.test', port: 587, secure: false, user: 'accounts@lynkeduppro.com', pass: 'p', from: 'accounts@lynkeduppro.com' }; @@ -88,3 +88,48 @@ describe('SmtpProvider.send — failure + fallback', () => { expect(res.errorCode).toBe('ETIMEDOUT'); }); }); + +describe('SmtpProvider.send — attachments', () => { + const resolver = (map: Record) => + async (ref: string) => map[ref] ?? null; + + it('resolves attachment refs to bytes and attaches them', async () => { + const t = stub(); + const res = await new SmtpProvider(ID, undefined, t.make, resolver({ 'obj/1': { content: Buffer.from('PDFDATA'), contentType: 'application/pdf' } })) + .send(req({ subject: 'Invoice', attachments: [{ filename: 'invoice.pdf', contentRef: 'obj/1', mimeType: 'application/pdf' }] })); + expect(res.outcome).toBe('SENT'); + expect(t.calls[0].mail.attachments).toEqual([{ filename: 'invoice.pdf', content: Buffer.from('PDFDATA'), contentType: 'application/pdf' }]); + }); + + it('FAILS closed when a declared attachment cannot be resolved (never sends without it)', async () => { + const t = stub(); + const res = await new SmtpProvider(ID, undefined, t.make, resolver({})) + .send(req({ subject: 'Invoice', attachments: [{ contentRef: 'missing' }] })); + expect(res.outcome).toBe('FAILED'); + expect(res.errorCode).toBe('ATTACHMENT_UNRESOLVED'); + expect(t.calls).toHaveLength(0); // nothing sent + }); + + it('FAILS when attachments are requested but no resolver is wired', async () => { + const t = stub(); + const res = await new SmtpProvider(ID, undefined, t.make) // no resolver + .send(req({ subject: 'x', attachments: [{ contentRef: 'obj/1' }] })); + expect(res).toMatchObject({ outcome: 'FAILED', errorCode: 'NO_ATTACHMENT_RESOLVER' }); + }); + + it('a plain send with no attachments is unaffected', async () => { + const t = stub(); + const res = await new SmtpProvider(ID, undefined, t.make, resolver({})).send(req({ subject: 'x', text: 'y' })); + expect(res.outcome).toBe('SENT'); + expect(t.calls[0].mail.attachments).toBeUndefined(); + }); +}); + +describe('storageResolver (media StoragePort → AttachmentResolver)', () => { + it('maps storage bytes into an attachment; missing ref → null', async () => { + const storage = { get: async (k: string) => (k === 'obj/1' ? { data: Buffer.from('X'), mime: 'application/pdf' } : null) }; + const r = storageResolver(storage); + expect(await r('obj/1')).toEqual({ content: Buffer.from('X'), contentType: 'application/pdf' }); + expect(await r('nope')).toBeNull(); + }); +}); diff --git a/packages/iios-service/src/capability/smtp.provider.ts b/packages/iios-service/src/capability/smtp.provider.ts index 675a5c6..2485b3a 100644 --- a/packages/iios-service/src/capability/smtp.provider.ts +++ b/packages/iios-service/src/capability/smtp.provider.ts @@ -12,6 +12,23 @@ export interface SmtpIdentity { from: string; } +export interface MailAttachment { + filename: string; + content: Buffer; + contentType?: string; +} + +/** Fetch an attachment's bytes by its opaque contentRef (backed by the media StoragePort). */ +export type AttachmentResolver = (contentRef: string) => Promise<{ filename?: string; content: Buffer; contentType?: string } | null>; + +/** Build an AttachmentResolver over the media StoragePort (contentRef = the storage object key). */ +export function storageResolver(storage: { get(key: string): Promise<{ data: Buffer; mime: string } | null> }): AttachmentResolver { + return async (contentRef) => { + const o = await storage.get(contentRef); + return o ? { content: o.data, contentType: o.mime } : null; + }; +} + /** The subset of a mail transport this provider needs — lets tests inject a stub (no live server). */ export interface MailTransport { sendMail(mail: { @@ -22,6 +39,7 @@ export interface MailTransport { html?: string; inReplyTo?: string; references?: string; + attachments?: MailAttachment[]; }): Promise<{ messageId: string; accepted?: unknown[] }>; } @@ -71,6 +89,8 @@ interface EmailPayload { text?: string; html?: string; inReplyTo?: string; + /** Attachment REFS (not bytes) — resolved to bytes at send time via the injected resolver. */ + attachments?: Array<{ filename?: string; contentRef: string; mimeType?: string }>; } /** @@ -90,6 +110,7 @@ export class SmtpProvider implements CapabilityProvider { private readonly primary: SmtpIdentity, private readonly fallback?: SmtpIdentity, makeTransport?: (id: SmtpIdentity) => MailTransport, + private readonly resolveAttachment?: AttachmentResolver, ) { this.makeTransport = makeTransport ?? defaultTransport; } @@ -98,6 +119,20 @@ export class SmtpProvider implements CapabilityProvider { const started = Date.now(); const p = (req.payload ?? {}) as EmailPayload; + // Resolve attachment bytes up front. Fail CLOSED — never send an invoice/receipt email missing + // its file; a FAILED command retries instead. Resolved once so a fallback retry doesn't re-fetch. + let attachments: MailAttachment[] | undefined; + if (p.attachments && p.attachments.length > 0) { + if (!this.resolveAttachment) return this.failed('NO_ATTACHMENT_RESOLVER', started); + const out: MailAttachment[] = []; + for (const a of p.attachments) { + const r = await this.resolveAttachment(a.contentRef).catch(() => null); + if (!r) return this.failed('ATTACHMENT_UNRESOLVED', started); + out.push({ filename: a.filename ?? r.filename ?? 'attachment', content: r.content, ...(a.mimeType ?? r.contentType ? { contentType: a.mimeType ?? r.contentType } : {}) }); + } + attachments = out; + } + const attempt = async (id: SmtpIdentity): Promise<{ messageId: string }> => this.makeTransport(id).sendMail({ from: id.from, @@ -106,6 +141,7 @@ export class SmtpProvider implements CapabilityProvider { text: p.text, html: p.html, ...(p.inReplyTo ? { inReplyTo: p.inReplyTo, references: p.inReplyTo } : {}), + ...(attachments ? { attachments } : {}), }); try { @@ -124,6 +160,10 @@ export class SmtpProvider implements CapabilityProvider { return { providerRef: `smtp-error-${randomUUID().slice(0, 8)}`, outcome: 'FAILED', errorCode: codeOf(err), latencyMs: Date.now() - started }; } } + + private failed(errorCode: string, started: number): ProviderResult { + return { providerRef: `smtp-error-${randomUUID().slice(0, 8)}`, outcome: 'FAILED', errorCode, latencyMs: Date.now() - started }; + } } /** True for connect/auth/timeout errors (server never accepted); false once the server responded 2xx. */ diff --git a/packages/iios-service/src/mail/mail.controller.ts b/packages/iios-service/src/mail/mail.controller.ts index 968b1ff..8cc2b1c 100644 --- a/packages/iios-service/src/mail/mail.controller.ts +++ b/packages/iios-service/src/mail/mail.controller.ts @@ -36,6 +36,7 @@ export class MailController { ...(body.locale ? { locale: body.locale } : {}), idempotencyKey: body.idempotencyKey, ...(body.purpose ? { purpose: body.purpose } : {}), + ...(body.attachments && body.attachments.length > 0 ? { attachments: body.attachments } : {}), ...(body.mirrorToUserId ? { mirrorToUserId: body.mirrorToUserId } : {}), }); } diff --git a/packages/iios-service/src/mail/mail.dto.ts b/packages/iios-service/src/mail/mail.dto.ts index 92844aa..0847751 100644 --- a/packages/iios-service/src/mail/mail.dto.ts +++ b/packages/iios-service/src/mail/mail.dto.ts @@ -1,7 +1,14 @@ import { Type } from 'class-transformer'; -import { IsInt, IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator'; +import { IsArray, IsInt, IsNotEmpty, IsObject, IsOptional, IsString, ValidateNested } from 'class-validator'; import { InlineTemplateDto } from '../templates/template.dto'; +/** An email attachment reference — bytes live in the media store under `contentRef`. */ +export class AttachmentDto { + @IsOptional() @IsString() filename?: string; + @IsString() @IsNotEmpty() contentRef!: string; + @IsOptional() @IsString() mimeType?: string; +} + /** POST /v1/mail/internal — app-to-app mail (no SMTP). Provide EITHER `key` OR `inline`. */ export class MailInternalDto { @IsOptional() @IsString() @IsNotEmpty() key?: string; @@ -25,6 +32,7 @@ export class MailSendDto { @IsOptional() @IsString() locale?: string; @IsString() @IsNotEmpty() idempotencyKey!: string; @IsOptional() @IsString() purpose?: string; + @IsOptional() @IsArray() @ValidateNested({ each: true }) @Type(() => AttachmentDto) attachments?: AttachmentDto[]; /** The registered recipient to mirror to; omit for a pre-registration send (email only). */ @IsOptional() @IsString() mirrorToUserId?: string; } diff --git a/packages/iios-service/src/mail/mail.service.ts b/packages/iios-service/src/mail/mail.service.ts index 7d696f5..f0ed3a3 100644 --- a/packages/iios-service/src/mail/mail.service.ts +++ b/packages/iios-service/src/mail/mail.service.ts @@ -33,6 +33,8 @@ export interface SendExternalInput { locale?: string; idempotencyKey: string; purpose?: string; + /** Attachment refs (bytes resolved at send time). */ + attachments?: Array<{ filename?: string; contentRef: string; mimeType?: string }>; /** The registered recipient to mirror to; omit for a pre-registration send (email only, no mirror). */ mirrorToUserId?: string; } @@ -64,6 +66,7 @@ export class MailService { const command = await this.sender.sendTemplated({ source: input.source, channel: 'EMAIL', target: input.target, vars: input.vars ?? {}, ...(input.locale ? { locale: input.locale } : {}), idempotencyKey: input.idempotencyKey, ...(input.purpose ? { purpose: input.purpose } : {}), + ...(input.attachments && input.attachments.length > 0 ? { attachments: input.attachments } : {}), }); // Mirror only for a registered recipient (timing rule: no inbox exists pre-registration). diff --git a/packages/iios-service/src/media/media.module.ts b/packages/iios-service/src/media/media.module.ts index 7f1df2a..a9c00ff 100644 --- a/packages/iios-service/src/media/media.module.ts +++ b/packages/iios-service/src/media/media.module.ts @@ -11,5 +11,7 @@ import { STORAGE_PORT } from './storage.port'; controllers: [MediaController], // Dev binds local disk; prod swaps STORAGE_PORT to an S3/Supabase adapter. providers: [MediaService, { provide: STORAGE_PORT, useClass: LocalDiskStorage }], + // Exported so the capability layer can resolve email attachment bytes at send time. + exports: [STORAGE_PORT], }) export class MediaModule {} diff --git a/packages/iios-service/src/templates/templated-sender.spec.ts b/packages/iios-service/src/templates/templated-sender.spec.ts index b1f52e2..fc2d6b5 100644 --- a/packages/iios-service/src/templates/templated-sender.spec.ts +++ b/packages/iios-service/src/templates/templated-sender.spec.ts @@ -67,6 +67,16 @@ describe('TemplatedSender.sendTemplated', () => { expect(cmd.renderedHash).toMatch(/^[a-f0-9]{64}$/); }); + it('carries attachment refs into the EMAIL command payload', async () => { + await seedReceipt(); + const cmd = await sender().sendTemplated({ + source: { key: 'payment.receipt' }, channel: 'EMAIL', target: 'dana@acme.com', + vars: { firstName: 'Dana', amount: '$1' }, idempotencyKey: 'att:1', + attachments: [{ filename: 'invoice.pdf', contentRef: 'obj/1', mimeType: 'application/pdf' }], + }); + expect((cmd.payload as { attachments: unknown[] }).attachments).toEqual([{ filename: 'invoice.pdf', contentRef: 'obj/1', mimeType: 'application/pdf' }]); + }); + it('shapes an SMS send as text only', async () => { await prisma.iiosMessageTemplate.create({ data: { key: 'payment.receipt', channel: 'SMS', locale: 'en', version: 1, bodyText: 'Paid {{amount}}', variables: ['amount'] }, diff --git a/packages/iios-service/src/templates/templated-sender.ts b/packages/iios-service/src/templates/templated-sender.ts index b7a96b6..3d8acaf 100644 --- a/packages/iios-service/src/templates/templated-sender.ts +++ b/packages/iios-service/src/templates/templated-sender.ts @@ -10,6 +10,9 @@ import { isInlineSource, type TemplateSource } from './template.model'; /** External-egress channels only. INTERNAL (in-app, no SMTP) delivery is the messaging module's job. */ export type ExternalChannel = 'EMAIL' | 'SMS'; +/** An email attachment REF (bytes resolved by the provider at send time). */ +export interface AttachmentRef { filename?: string; contentRef: string; mimeType?: string } + export interface SendTemplatedInput { source: TemplateSource; channel: ExternalChannel; @@ -19,6 +22,7 @@ export interface SendTemplatedInput { locale?: string; idempotencyKey: string; // REQUIRED — e.g. "receipt:" purpose?: string; + attachments?: AttachmentRef[]; // EMAIL only } /** @@ -48,7 +52,7 @@ export class TemplatedSender { return this.outbound.send( input.channel, input.target, - this.toPayload(input.channel, content), + this.toPayload(input.channel, content, input.attachments), input.idempotencyKey, input.scopeId, input.purpose, @@ -57,8 +61,10 @@ export class TemplatedSender { } /** Shape rendered content into the channel's egress envelope (EmailProvider reads subject/text/html). */ - private toPayload(channel: ExternalChannel, content: RenderedContent): Record { - if (channel === 'EMAIL') return { subject: content.subject, html: content.html, text: content.text }; + private toPayload(channel: ExternalChannel, content: RenderedContent, attachments?: AttachmentRef[]): Record { + if (channel === 'EMAIL') { + return { subject: content.subject, html: content.html, text: content.text, ...(attachments && attachments.length > 0 ? { attachments } : {}) }; + } return { text: content.text ?? content.subject ?? '' }; // SMS: text only } } From 0c6650f3c6a06510e83e017129b7598e2cba040d Mon Sep 17 00:00:00 2001 From: maaz519 Date: Sat, 18 Jul 2026 14:26:53 +0530 Subject: [PATCH 14/14] feat(media): S3/MinIO storage adapter (StoragePort) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drop-in S3-compatible backend for object storage (media + email attachments) — the swap the StoragePort seam was designed for. MinIO/self-hosted/R2/Supabase all work via endpoint + path-style. - S3Storage implements StoragePort (put/get/remove); sha256 computed locally on put (matches LocalDiskStorage); a missing object reads back as null, not an error. - Env-driven binding in MediaModule: IIOS_S3_BUCKET + keys set → S3Storage, else LocalDiskStorage. forcePathStyle defaults true (MinIO); endpoint omitted → AWS. - S3 client is injectable so tests run with no live server. 6 unit tests (config parse, put size/sha, get round-trip, NoSuchKey→null, remove). Full suite 300/300, boundary + build clean. Co-Authored-By: Claude Opus 4.8 --- packages/iios-service/.env.example | 9 + packages/iios-service/package.json | 1 + .../iios-service/src/media/media.module.ts | 18 +- .../iios-service/src/media/s3.storage.spec.ts | 66 ++++ packages/iios-service/src/media/s3.storage.ts | 87 +++++ pnpm-lock.yaml | 302 ++++++++++++++++++ 6 files changed, 479 insertions(+), 4 deletions(-) create mode 100644 packages/iios-service/src/media/s3.storage.spec.ts create mode 100644 packages/iios-service/src/media/s3.storage.ts diff --git a/packages/iios-service/.env.example b/packages/iios-service/.env.example index 9053606..04e6e61 100644 --- a/packages/iios-service/.env.example +++ b/packages/iios-service/.env.example @@ -59,3 +59,12 @@ IIOS_ATTESTATION_AUDIENCE=iios-core # When '1', every guarded request MUST carry a valid X-Context-Attestation (else 403). # Leave OFF until callers (AppShell/be-crm) forward attestations. Verified-if-present regardless. IIOS_REQUIRE_ATTESTATION=0 + +# ─── Object storage (media + email attachments) ─────────────────── +# Unset → local disk (MEDIA_DIR). Set these → S3-compatible (AWS S3 / MinIO / R2 / Supabase). +# IIOS_S3_ENDPOINT=https://minio.your-server:9000 # omit for AWS S3 +# IIOS_S3_BUCKET=iios-media +# IIOS_S3_ACCESS_KEY=... +# IIOS_S3_SECRET_KEY=... +# IIOS_S3_REGION=us-east-1 # any value for MinIO +# IIOS_S3_FORCE_PATH_STYLE=true # true for MinIO/self-hosted diff --git a/packages/iios-service/package.json b/packages/iios-service/package.json index 2d11fc5..07b68b7 100644 --- a/packages/iios-service/package.json +++ b/packages/iios-service/package.json @@ -12,6 +12,7 @@ "prisma:studio": "prisma studio" }, "dependencies": { + "@aws-sdk/client-s3": "^3.1090.0", "@insignia/iios-adapter-sdk": "workspace:*", "@insignia/iios-contracts": "workspace:*", "@nestjs/common": "^11.1.27", diff --git a/packages/iios-service/src/media/media.module.ts b/packages/iios-service/src/media/media.module.ts index a9c00ff..2350ef7 100644 --- a/packages/iios-service/src/media/media.module.ts +++ b/packages/iios-service/src/media/media.module.ts @@ -1,16 +1,26 @@ -import { Module } from '@nestjs/common'; +import { Module, Logger } from '@nestjs/common'; import { PlatformModule } from '../platform/platform.module'; import { IdentityModule } from '../identity/identity.module'; import { MediaController } from './media.controller'; import { MediaService } from './media.service'; import { LocalDiskStorage } from './local-disk.storage'; -import { STORAGE_PORT } from './storage.port'; +import { S3Storage, s3ConfigFromEnv } from './s3.storage'; +import { STORAGE_PORT, type StoragePort } from './storage.port'; + +/** S3/MinIO when its env is set (IIOS_S3_BUCKET + keys), else local disk. Env-driven swap. */ +function makeStorage(): StoragePort { + const cfg = s3ConfigFromEnv(); + if (cfg) { + new Logger('MediaStorage').log(`using S3 storage (bucket "${cfg.bucket}"${cfg.endpoint ? ` @ ${cfg.endpoint}` : ''})`); + return new S3Storage(cfg); + } + return new LocalDiskStorage(); +} @Module({ imports: [PlatformModule, IdentityModule], controllers: [MediaController], - // Dev binds local disk; prod swaps STORAGE_PORT to an S3/Supabase adapter. - providers: [MediaService, { provide: STORAGE_PORT, useClass: LocalDiskStorage }], + providers: [MediaService, { provide: STORAGE_PORT, useFactory: makeStorage }], // Exported so the capability layer can resolve email attachment bytes at send time. exports: [STORAGE_PORT], }) diff --git a/packages/iios-service/src/media/s3.storage.spec.ts b/packages/iios-service/src/media/s3.storage.spec.ts new file mode 100644 index 0000000..abadaf1 --- /dev/null +++ b/packages/iios-service/src/media/s3.storage.spec.ts @@ -0,0 +1,66 @@ +import { describe, it, expect } from 'vitest'; +import { DeleteObjectCommand, GetObjectCommand, PutObjectCommand } from '@aws-sdk/client-s3'; +import { S3Storage, s3ConfigFromEnv, type S3Config, type S3Like } from './s3.storage'; + +const CFG: S3Config = { endpoint: 'https://minio.test', region: 'us-east-1', bucket: 'iios', accessKeyId: 'k', secretAccessKey: 's', forcePathStyle: true }; + +/** A recording stub S3 client; GetObject returns whatever `store[key]` holds (or a NoSuchKey error). */ +function stub(store: Record = {}) { + const calls: unknown[] = []; + const client: S3Like = { + async send(command: unknown) { + calls.push(command); + if (command instanceof PutObjectCommand) { store[command.input.Key!] = { body: command.input.Body as Buffer, mime: command.input.ContentType ?? '' }; return {}; } + if (command instanceof DeleteObjectCommand) { delete store[command.input.Key!]; return {}; } + if (command instanceof GetObjectCommand) { + const hit = store[command.input.Key!]; + if (!hit) throw Object.assign(new Error('missing'), { name: 'NoSuchKey' }); + return { Body: { transformToByteArray: async () => new Uint8Array(hit.body) }, ContentType: hit.mime }; + } + return {}; + }, + }; + return { client, calls, store }; +} + +describe('s3ConfigFromEnv', () => { + it('builds config from env (path-style default true for MinIO)', () => { + const cfg = s3ConfigFromEnv({ IIOS_S3_ENDPOINT: 'https://minio.x', IIOS_S3_BUCKET: 'b', IIOS_S3_ACCESS_KEY: 'k', IIOS_S3_SECRET_KEY: 's' } as NodeJS.ProcessEnv); + expect(cfg).toMatchObject({ endpoint: 'https://minio.x', bucket: 'b', region: 'us-east-1', forcePathStyle: true }); + }); + it('returns null when bucket/keys are missing', () => { + expect(s3ConfigFromEnv({ IIOS_S3_ENDPOINT: 'https://minio.x' } as NodeJS.ProcessEnv)).toBeNull(); + }); +}); + +describe('S3Storage (StoragePort over S3/MinIO)', () => { + it('put stores the object and returns size + sha256', async () => { + const s = stub(); + const store = new S3Storage(CFG, s.client); + const res = await store.put('scope/obj1', Buffer.from('hello'), 'text/plain'); + expect(res.sizeBytes).toBe(5); + expect(res.checksumSha256).toMatch(/^[a-f0-9]{64}$/); + const put = s.calls[0] as PutObjectCommand; + expect(put.input).toMatchObject({ Bucket: 'iios', Key: 'scope/obj1', ContentType: 'text/plain' }); + }); + + it('get round-trips the bytes + mime', async () => { + const s = stub(); + const store = new S3Storage(CFG, s.client); + await store.put('scope/obj2', Buffer.from('PDFDATA'), 'application/pdf'); + const got = await store.get('scope/obj2'); + expect(got).toEqual({ data: Buffer.from('PDFDATA'), mime: 'application/pdf', sizeBytes: 7 }); + }); + + it('get returns null for a missing key (NoSuchKey → null, not throw)', async () => { + const store = new S3Storage(CFG, stub().client); + expect(await store.get('nope')).toBeNull(); + }); + + it('remove issues a DeleteObject', async () => { + const s = stub({ 'k': { body: Buffer.from('x'), mime: 't' } }); + await new S3Storage(CFG, s.client).remove('k'); + expect(s.calls.some((c) => c instanceof DeleteObjectCommand)).toBe(true); + expect(s.store['k']).toBeUndefined(); + }); +}); diff --git a/packages/iios-service/src/media/s3.storage.ts b/packages/iios-service/src/media/s3.storage.ts new file mode 100644 index 0000000..55667e6 --- /dev/null +++ b/packages/iios-service/src/media/s3.storage.ts @@ -0,0 +1,87 @@ +import { createHash } from 'node:crypto'; +import { Injectable } from '@nestjs/common'; +import { DeleteObjectCommand, GetObjectCommand, PutObjectCommand, S3Client } from '@aws-sdk/client-s3'; +import type { StoragePort } from './storage.port'; + +export interface S3Config { + endpoint?: string; // MinIO/self-hosted URL; omit for AWS + region: string; + bucket: string; + accessKeyId: string; + secretAccessKey: string; + forcePathStyle: boolean; // true for MinIO +} + +/** The one method this adapter uses — lets tests inject a stub client (no live S3/MinIO). */ +export interface S3Like { + send(command: unknown): Promise; +} + +/** Build an S3Config from env, or null if the required bits are missing (→ fall back to disk). */ +export function s3ConfigFromEnv(env: NodeJS.ProcessEnv = process.env): S3Config | null { + const bucket = env.IIOS_S3_BUCKET; + const accessKeyId = env.IIOS_S3_ACCESS_KEY; + const secretAccessKey = env.IIOS_S3_SECRET_KEY; + if (!bucket || !accessKeyId || !secretAccessKey) return null; + return { + ...(env.IIOS_S3_ENDPOINT ? { endpoint: env.IIOS_S3_ENDPOINT } : {}), + region: env.IIOS_S3_REGION ?? 'us-east-1', + bucket, + accessKeyId, + secretAccessKey, + // MinIO/self-hosted needs path-style; default true unless explicitly disabled for AWS. + forcePathStyle: env.IIOS_S3_FORCE_PATH_STYLE !== 'false', + }; +} + +/** + * S3-compatible object storage (AWS S3, MinIO, R2, Supabase Storage). A drop-in for LocalDiskStorage: + * same StoragePort contract. sha256 is computed locally on put (S3 doesn't return it), matching the + * disk adapter. A missing object reads back as null (not an error). + */ +@Injectable() +export class S3Storage implements StoragePort { + private readonly client: S3Like; + private readonly bucket: string; + + constructor(config: S3Config, client?: S3Like) { + this.bucket = config.bucket; + this.client = client ?? new S3Client({ + region: config.region, + ...(config.endpoint ? { endpoint: config.endpoint } : {}), + forcePathStyle: config.forcePathStyle, + credentials: { accessKeyId: config.accessKeyId, secretAccessKey: config.secretAccessKey }, + }); + } + + async put(objectKey: string, data: Buffer, mime: string): Promise<{ sizeBytes: number; checksumSha256: string }> { + const checksumSha256 = createHash('sha256').update(data).digest('hex'); + await this.client.send(new PutObjectCommand({ Bucket: this.bucket, Key: objectKey, Body: data, ContentType: mime })); + return { sizeBytes: data.length, checksumSha256 }; + } + + async get(objectKey: string): Promise<{ data: Buffer; mime: string; sizeBytes: number } | null> { + try { + const res = (await this.client.send(new GetObjectCommand({ Bucket: this.bucket, Key: objectKey }))) as { + Body?: { transformToByteArray(): Promise }; + ContentType?: string; + }; + if (!res.Body) return null; + const bytes = await res.Body.transformToByteArray(); + const data = Buffer.from(bytes); + return { data, mime: res.ContentType ?? 'application/octet-stream', sizeBytes: data.length }; + } catch (err) { + if (isNotFound(err)) return null; + throw err; + } + } + + async remove(objectKey: string): Promise { + await this.client.send(new DeleteObjectCommand({ Bucket: this.bucket, Key: objectKey })); + } +} + +function isNotFound(err: unknown): boolean { + const e = err as { name?: string; $metadata?: { httpStatusCode?: number } }; + return e.name === 'NoSuchKey' || e.name === 'NotFound' || e.$metadata?.httpStatusCode === 404; +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index bab7336..251f3b0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -328,6 +328,9 @@ importers: packages/iios-service: dependencies: + '@aws-sdk/client-s3': + specifier: ^3.1090.0 + version: 3.1090.0 '@insignia/iios-adapter-sdk': specifier: workspace:* version: link:../iios-adapter-sdk @@ -487,6 +490,78 @@ packages: resolution: {integrity: sha512-/PZmyAlb2NGWPikRRuiWLdfHQd8Wrx6lX4HqvTcaDhlU43M3T0ud4PH2T3QDp7BzHYY92xtD8iPxX2asg67G1A==} engines: {node: ^18.19.1 || ^20.11.1 || >=22.0.0, npm: ^6.11.0 || ^7.5.6 || >=8.0.0, yarn: '>= 1.13.0'} + '@aws-sdk/checksums@3.1000.18': + resolution: {integrity: sha512-IImkbEyXdV6/uaF5r6Wkk+8718mQw1ll83j0a4a30R3JM/rHVFdWAiT4jtJpFjJiIwM/oJ6SxIxr0z2TaQUGqw==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/client-s3@3.1090.0': + resolution: {integrity: sha512-R6GX9cd1jljwzZ8xFmgAI/hHCuX1MobIKBdsymv7WL9SENvO9Vgz9KOR6avTnu0Ao+w1LmxnTe+jqmZXEn7Q/Q==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/core@3.975.3': + resolution: {integrity: sha512-7ur3kCKuvPLqlsZ2XlvnNBVQ7KkpSu6Y6dOTwSPHLrFpTEfZM8isLBJc4cgv96WB7GifeVM436mpycwxBd2vEA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-env@3.972.59': + resolution: {integrity: sha512-Ny5e4Mfh3QPmiAc0AiUe+cbTXDlxkU3Rc+EpWOfyWeWEy6yp7Fa1KmfNeCc+1a8by9zQ9gtohmiQUkMPScF3ng==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-http@3.972.61': + resolution: {integrity: sha512-8jAjgStl5Ytq4+HF3X/9f+EmRinaRbGRRtQGktlPfBRVx73H+R1y48vIeXerQtYGFaUqkEp3fT6jP854rVO2yQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-ini@3.973.4': + resolution: {integrity: sha512-e6ZvVsj90aRALf1kHP+J4iqC1496ZpVgqI/+u0LJ5HL7q7ATauGy4gdDvRCP13L1pN/fMiZLah162PGIYkbUVQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-login@3.972.66': + resolution: {integrity: sha512-g2fsqm87r/nKthLZ0VkkDBElkGg0PvSa8d97HQ6EilMbJTZ6hxa8FxkSZyJfgPfFdZn0TTmkOffQmTSUcAHIng==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-node@3.972.70': + resolution: {integrity: sha512-3xzvkGdykBunxqh8WudmUpSyLWvIhfI6aBQo1b5rb3mDO5mNLadK+0hiI0qBQBMVynJbfLO+Ajy9dztMwy9O8w==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-process@3.972.59': + resolution: {integrity: sha512-DlZF2/MhLlatDdlrIy3CUCpfdbLrKx+3SMjVo+WyHnPpwzkc/M3vwAHw4OVJf7DMvO+4vfRqSCMc/E9I1auN0g==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-sso@3.973.3': + resolution: {integrity: sha512-hmdDHoy2G5Es2e8IgelNMYUuSQI6uCIAKZMJ2u2PdKDhxvbk1uWD/g4+R7R5c/tJfKEB1+KjjWiaoCr/S+ZTiQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-web-identity@3.972.65': + resolution: {integrity: sha512-gHQb/Kt0chjk/JQDa/GJDqmAvEuVn8n7z10wK2h0LFM9TUDRkohgOO4aEF+s2sBLM0br7Cl5W6P7phgjrrJvLQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/middleware-sdk-s3@3.972.64': + resolution: {integrity: sha512-RBi43anhDBUv+HCfxCOXwGOE7GmT4n7ChV04Mwr22RhXTNcamW/iWnJlOotDPCZSrJ4dEvhZSiWWQMwLX+ZhFA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/nested-clients@3.997.33': + resolution: {integrity: sha512-dVZOroI/r3/ENvqNGgjMPul+jjlz9GddfVusgTXlVjfZj5isibOxecLkGQbRPp8XOuX+RAfjXLFgPkD1JS5xrw==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/signature-v4-multi-region@3.996.41': + resolution: {integrity: sha512-QMUytg+FQMGouc8gHS00KoYih3+N6cqmVI/pQGOIo7Nr7OpQaiXjSYOuL+vsPZ1tymY4LAQ8MYcHJmws5LRxng==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/token-providers@3.1088.0': + resolution: {integrity: sha512-4ObatWt2qpJg5FBk4LOOKrTQYzaqeewAtdO3r9ZO8lH9YqLtpTzLyIdy0mJ+nVdfYOnqISkKNfmzP22bNDhwyw==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/types@3.974.2': + resolution: {integrity: sha512-3W6IUtSxFbH6X7Wb7DzGCV5QiFQsd0g8bOfntpmDxQlzBoKWUMBu/JPQR0DwkE+Hpnxd6db1tXbOwdeHddG6cA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/xml-builder@3.972.36': + resolution: {integrity: sha512-RdGmS1GLrtaTOLE1ElSluMldNrpk9Emq6uYs8SS8iHlu5xTAmM9rRkM91o48+rIRryBtyO9t+uLYCoMG6jVMVA==} + engines: {node: '>=20.0.0'} + + '@aws/lambda-invoke-store@0.3.0': + resolution: {integrity: sha512-sl4Bm6yiMNYrZKkqqDFWN0UfnWhlS8ivKxrYl+6t0gCLrqr8y3B2IqZZbFRkfaVVp7C/baApyh71P+LeE1A2sQ==} + engines: {node: '>=18.0.0'} + '@babel/code-frame@7.29.7': resolution: {integrity: sha512-Aup7aUOfpbAUg2ROOJN6Iw5f9DMBlzu0mIkm/malLQFN/YQgO48wCj0Kxa3sEHJvPVFg7siR+qRInwXd2qhQKw==} engines: {node: '>=6.9.0'} @@ -1450,6 +1525,30 @@ packages: cpu: [x64] os: [win32] + '@smithy/core@3.29.5': + resolution: {integrity: sha512-i0dk2t5B+CwV/dcJdUHILYkOQF5lof8f44dFCfDWToGCxjT9YQ+CgHqTAvJxzc3+zqQwm2QtVoJ5IqiNar/CnQ==} + engines: {node: '>=18.0.0'} + + '@smithy/credential-provider-imds@4.4.10': + resolution: {integrity: sha512-MJenAe4OKRZUo1LdYYFDCsSHxaHvInIU/z52GsheO9vl1/VSySVCr0zkyKD6TFiGkSUaWGxvKZ/70OvgUZR5HQ==} + engines: {node: '>=18.0.0'} + + '@smithy/fetch-http-handler@5.6.7': + resolution: {integrity: sha512-3zpg8yqqyXzoK2TsRDdkqVOj2RDBFfLXwCczOZ5c7TWB4eiaebfSCsbMjDPYB3PJ9ihV62QaeadZ+wLadZtNGA==} + engines: {node: '>=18.0.0'} + + '@smithy/node-http-handler@4.9.7': + resolution: {integrity: sha512-wCU8HCLjAtAVqxxe0j2xff9LcEPw3yjBbg5IdQDIYFnxnPxbxcSLc7rgex7kqm9L/WYOnJEgaWQlfDkZleozMA==} + engines: {node: '>=18.0.0'} + + '@smithy/signature-v4@5.6.6': + resolution: {integrity: sha512-efP6DN3UTFrzIsGO42/xcabv8jU7+9nwEdphFUH7yL0k010ERyAWaO41KFQIDLcFZLZ8xzIQr4wplFxNzslSGQ==} + engines: {node: '>=18.0.0'} + + '@smithy/types@4.16.1': + resolution: {integrity: sha512-0JFs3V2y2M9tKW5na/qxe69Zv+uxLMO7QBbhxF/FHu/Gp2NFZAAL9tWl9PU02xxo07pb3G9FTyjNc6D5uZrJIg==} + engines: {node: '>=18.0.0'} + '@socket.io/component-emitter@3.1.2': resolution: {integrity: sha512-9BCxFwvbGg/RsZK9tjXd8s4UcwR0MWeFQ1XEKIQVVvAGJyINdrqKMcTRyLoK8Rse1GjzLV9cwjWV1olXRWEXVA==} @@ -1766,6 +1865,9 @@ packages: resolution: {integrity: sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==} engines: {node: '>=18'} + bowser@2.14.1: + resolution: {integrity: sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==} + brace-expansion@1.1.15: resolution: {integrity: sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==} @@ -3362,6 +3464,171 @@ snapshots: transitivePeerDependencies: - chokidar + '@aws-sdk/checksums@3.1000.18': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/client-s3@3.1090.0': + dependencies: + '@aws-sdk/checksums': 3.1000.18 + '@aws-sdk/core': 3.975.3 + '@aws-sdk/credential-provider-node': 3.972.70 + '@aws-sdk/middleware-sdk-s3': 3.972.64 + '@aws-sdk/signature-v4-multi-region': 3.996.41 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/fetch-http-handler': 5.6.7 + '@smithy/node-http-handler': 4.9.7 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/core@3.975.3': + dependencies: + '@aws-sdk/types': 3.974.2 + '@aws-sdk/xml-builder': 3.972.36 + '@aws/lambda-invoke-store': 0.3.0 + '@smithy/core': 3.29.5 + '@smithy/signature-v4': 5.6.6 + '@smithy/types': 4.16.1 + bowser: 2.14.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-env@3.972.59': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-http@3.972.61': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/fetch-http-handler': 5.6.7 + '@smithy/node-http-handler': 4.9.7 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-ini@3.973.4': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/credential-provider-env': 3.972.59 + '@aws-sdk/credential-provider-http': 3.972.61 + '@aws-sdk/credential-provider-login': 3.972.66 + '@aws-sdk/credential-provider-process': 3.972.59 + '@aws-sdk/credential-provider-sso': 3.973.3 + '@aws-sdk/credential-provider-web-identity': 3.972.65 + '@aws-sdk/nested-clients': 3.997.33 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/credential-provider-imds': 4.4.10 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-login@3.972.66': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/nested-clients': 3.997.33 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-node@3.972.70': + dependencies: + '@aws-sdk/credential-provider-env': 3.972.59 + '@aws-sdk/credential-provider-http': 3.972.61 + '@aws-sdk/credential-provider-ini': 3.973.4 + '@aws-sdk/credential-provider-process': 3.972.59 + '@aws-sdk/credential-provider-sso': 3.973.3 + '@aws-sdk/credential-provider-web-identity': 3.972.65 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/credential-provider-imds': 4.4.10 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-process@3.972.59': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-sso@3.973.3': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/nested-clients': 3.997.33 + '@aws-sdk/token-providers': 3.1088.0 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-web-identity@3.972.65': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/nested-clients': 3.997.33 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/middleware-sdk-s3@3.972.64': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/signature-v4-multi-region': 3.996.41 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/nested-clients@3.997.33': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/signature-v4-multi-region': 3.996.41 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/fetch-http-handler': 5.6.7 + '@smithy/node-http-handler': 4.9.7 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/signature-v4-multi-region@3.996.41': + dependencies: + '@aws-sdk/types': 3.974.2 + '@smithy/signature-v4': 5.6.6 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/token-providers@3.1088.0': + dependencies: + '@aws-sdk/core': 3.975.3 + '@aws-sdk/nested-clients': 3.997.33 + '@aws-sdk/types': 3.974.2 + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/types@3.974.2': + dependencies: + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws-sdk/xml-builder@3.972.36': + dependencies: + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@aws/lambda-invoke-store@0.3.0': {} + '@babel/code-frame@7.29.7': dependencies: '@babel/helper-validator-identifier': 7.29.7 @@ -4105,6 +4372,39 @@ snapshots: '@rollup/rollup-win32-x64-msvc@4.62.2': optional: true + '@smithy/core@3.29.5': + dependencies: + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@smithy/credential-provider-imds@4.4.10': + dependencies: + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@smithy/fetch-http-handler@5.6.7': + dependencies: + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@smithy/node-http-handler@4.9.7': + dependencies: + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@smithy/signature-v4@5.6.6': + dependencies: + '@smithy/core': 3.29.5 + '@smithy/types': 4.16.1 + tslib: 2.8.1 + + '@smithy/types@4.16.1': + dependencies: + tslib: 2.8.1 + '@socket.io/component-emitter@3.1.2': {} '@socket.io/redis-adapter@8.3.0(socket.io-adapter@2.5.8)': @@ -4492,6 +4792,8 @@ snapshots: transitivePeerDependencies: - supports-color + bowser@2.14.1: {} + brace-expansion@1.1.15: dependencies: balanced-match: 1.0.2