iios/docs/email-template-module-plan.md

# Email / Message Template Module — Implementation Plan

**Owner:** Maaz · **Repo:** `iios` (`packages/iios-service`) · **Target:** first templates sending by Friday go-live.

## Goal

A reusable template module so **any** message the system sends — welcome, payment receipt,
onboarding reminders, drip — is produced from a stored (or ad-hoc) template with variables filled
in, then handed to the **existing** outbound pipeline. One render path, one send path, provenance
recorded for every send.

From the July 16 meeting: *"template का एक पूरा module बनाना है… template में value भरोगे, और वो
outbound क्यू में डाल दोगे।"*

## What already exists (the substrate — do NOT rebuild)

- `OutboundService.send(channelType, target, payload, idempotencyKey?, scopeId?, purpose?)` —
  idempotency + per-target/per-tenant rate limits + delivery ledger (`IiosOutboundCommand`).
- `CapabilityBroker` — policy gate + obligations + provider selection for egress.
- `EMAIL` is a registered channel; `EmailProvider` exists (**HTTP**, not SMTP — see Out of Scope).
- `EMAIL` is a first-class `IiosInteractionKind`; `IiosMessagePartKind` has `HTML`/`TEXT`.
- `IiosActorKind` includes `SERVICE`/`BOT` (system sender is first-class).
- `InboxModule` uses `OnModuleInit` — copy that pattern for the template seeder.

The module adds **content/rendering** in front of this. It never talks to a provider directly.

## Design decisions (locked)

| # | Decision | Rationale |
|---|---|---|
| 1 | **Channel-generic**: one template per `(key, channel)`; channels `EMAIL` / `SMS` / `INTERNAL` | Vivek wants the same confirmation on email *and* SMS; INTERNAL = app-to-app, no SMTP |
| 2 | **Seed-from-files, DB-is-truth**: templates are files in the repo, seeded into the DB on boot if absent | Copy is version-controlled + code-reviewed; a later admin UI can edit the DB with no deploy; Friday needs no UI |
| 3 | **Handlebars** rendering | Auto-escapes HTML (customer names go into email → XSS risk), logic-less, no code execution, one dep |
| 4 | **Global default + scope override**: `scopeId` nullable — `NULL` = platform default, set = tenant override; resolve scoped-first-else-global | Seeds cleanly at boot (IIOS scopes are created lazily, so a boot seeder has no scope to seed into); leaves room for white-label |
| 5 | **Provenance on `IiosOutboundCommand`** (4 columns), not a new `template_snapshot` table | Rendered content is already in `payload`; only provenance is missing. Honours the SOT's intent at 4 columns |
| 6 | **`TemplateSource` = stored `{key,version?}` OR `{inline:{subject,html,text}}`** | Marketing hands over finished HTML (*"Maaz, HTML भेज सकते हैं"*); inline still renders vars + records provenance |
| 7 | **Integration pattern B**: pure `render()` + thin `sendTemplated()` composer | Single entry point for callers; provenance guaranteed by construction; `OutboundService` stays content-agnostic |
| 8 | **Caller = a service** (be-crm's system token); **recipient = a `target` address, never a login** | System mail has no user on the sending side; the recipient may have no account yet |

## Caller & auth model

Every send is authenticated as the **calling app/service** (e.g. be-crm via its `APP_SECRETS`
entry), verified by `SessionVerifier` like every other IIOS endpoint. The recipient is a plain
`target` string — **not** an IIOS principal and **not** required to be logged in or registered.
Sending a welcome email to an anonymous payer is the normal case: the app is the sender, the
address is data. `scopeId` for the send is derived from the caller's principal (`org/app/tenant`).

## Data model

### New table — `IiosMessageTemplate`

```prisma
enum IiosTemplateChannel {
  EMAIL
  SMS
  INTERNAL
}

/// The template SOURCE. DB is runtime truth; platform defaults are seeded from repo files on boot.
/// Versions are immutable: a change writes a new (higher) version, never edits in place.
model IiosMessageTemplate {
  id        String              @id @default(cuid())
  /// NULL = platform default (seeded). Set = a tenant scope's override of the same key.
  scopeId   String?
  scope     IiosScope?          @relation(fields: [scopeId], references: [id], onDelete: Cascade)
  key       String              // "welcome", "payment.receipt", "onboarding.reminder"
  channel   IiosTemplateChannel
  locale    String              @default("en")
  version   Int                 @default(1)
  subject   String?             // EMAIL only
  bodyHtml  String?             // EMAIL / INTERNAL
  bodyText  String?             // SMS, and EMAIL plaintext fallback
  /// Declared variable names — render throws if a declared var is missing (fail loud).
  variables Json?
  active    Boolean             @default(true)
  createdAt DateTime            @default(now())
  updatedAt DateTime            @updatedAt

  @@unique([scopeId, key, channel, locale, version])
  @@index([key, channel, locale, active])
}
```

`IiosScope` needs the back-relation `messageTemplates IiosMessageTemplate[]`.

**Resolution:** `resolve(key, channel, locale, scopeId?)` returns the highest-`version` `active`
row, preferring `scopeId = <caller scope>` and falling back to `scopeId IS NULL`. No match → 404.

### Provenance columns on `IiosOutboundCommand`

```prisma
  templateKey     String?
  templateVersion Int?
  templateLocale  String?
  renderedHash    String?   // sha256 of the rendered content — replay/audit
```

All nullable: non-templated sends (if any) leave them null.

**How provenance is written (review finding):** `OutboundService.send` **creates the command row
itself** (both the RATE_LIMITED and PENDING paths), so a caller cannot stamp provenance after the
fact without a racy update. Therefore `send()` gains one optional trailing arg —
`provenance?: { templateKey; templateVersion; templateLocale; renderedHash }` — written into the
same `create()` in both paths. `OutboundService` stays content-agnostic: it does not render or
resolve templates, it only persists four opaque strings it is handed. This is what makes
decision 7's "provenance guaranteed by construction" true. **This is a change to an existing file**
(`adapters/outbound.service.ts`) and its spec — call it out in the PR.

### Migrations (hand-written, non-destructive)

1. `add_message_template` — the enum + table + `IiosScope` back-relation. **Plus a partial unique
   index for global rows (review finding):** the `@@unique([scopeId, key, channel, locale, version])`
   does **not** prevent duplicate *platform-default* rows, because Postgres treats `NULL` scopeId as
   distinct (`NULL != NULL`) — the same footgun handled in the inbox idempotency migration. Add:
   `CREATE UNIQUE INDEX "IiosMessageTemplate_global_key" ON "IiosMessageTemplate"("key","channel","locale","version") WHERE "scopeId" IS NULL;`
   so a double-seed or race cannot create two defaults for the same key.
2. `add_outbound_template_provenance` — the 4 nullable columns on `IiosOutboundCommand`.

## Module structure

```
packages/iios-service/src/templates/
  template.channel.ts        // IiosTemplateChannel re-export/helpers if needed
  template.model.ts          // TemplateSource, RenderedContent, CreateSendInput types
  template.repository.ts     // resolve(): scoped ?? global, highest active version
  template.renderer.ts       // Handlebars compile+render; escaping; missing-var throw
  template.service.ts        // render(source, vars) — pure; resolve + renderer
  templated-sender.ts        // sendTemplated(): render -> OutboundService.send -> stamp provenance
  template.seeder.ts         // OnModuleInit: seed file defaults into DB if (key,channel,locale,version) absent
  template.controller.ts     // POST /v1/templates/send, POST /v1/templates/preview
  template.dto.ts            // SendTemplateDto, PreviewTemplateDto (class-validator)
  template.module.ts
  seeds/
    welcome.email.ts         // { key, channel, locale, version, subject, html, text, variables }
    payment-receipt.email.ts
    payment-receipt.sms.ts
    onboarding-reminder.email.ts
  templates.spec.ts          // TDD, real Postgres (localhost:5434), mirrors inbox.spec.ts
```

**Scope boundary:** `render()` is channel-generic (it can render an `INTERNAL` template to
`{subject,html,text}`). `sendTemplated()` covers **external** channels (`EMAIL`/`SMS`) via
`OutboundService`. `INTERNAL` *delivery* (render → create an in-app `Interaction`, no SMTP) reuses
`render()` but is wired by the messaging/mirror spec — this module never imports the messaging layer.

## Contract

```ts
type TemplateSource =
  | { key: string; version?: number }
  | { inline: { subject?: string; html?: string; text?: string } };

interface RenderedContent { subject?: string; html?: string; text?: string }

// template.renderer.ts — PURE (no I/O): given a template's raw strings + vars, produce content.
// template.service.ts render() — resolves the template from the DB (I/O), then calls the pure renderer.
// For an inline source there is no DB row, so declared-variable validation is skipped — inline
// content is the caller's responsibility; only stored templates enforce their declared `variables`.
render(source: TemplateSource, vars: Record<string, unknown>, opts: { channel: IiosTemplateChannel; locale?: string; scopeId?: string }): Promise<RenderedContent>

// templated-sender.ts — external egress
sendTemplated(input: {
  source: TemplateSource;
  channel: 'EMAIL' | 'SMS';
  target: string;                 // email address / phone
  vars: Record<string, unknown>;
  scopeId?: string;
  idempotencyKey: string;         // REQUIRED — e.g. "receipt:<stripe_session_id>"
  purpose?: string;
}): Promise<IiosOutboundCommand>
```

**HTTP** (`SessionVerifier`-auth'd; `scopeId` from the caller's principal):
- `POST /v1/templates/send` → `sendTemplated`. OPA action `iios.template.send`;
  inline source additionally gated on `iios.template.send.inline` (arbitrary HTML to customers).
- `POST /v1/templates/preview` → `render` only, returns `RenderedContent`. No send. For marketing/QA.

## Task-by-task (TDD)

Each task: write the failing test → run it (confirm red) → implement → run (green) → commit.

**T1 — Renderer (`template.renderer.ts`)**
- Tests: substitutes `{{firstName}}`; **escapes `<script>` in a name**; `{{#if}}`/`{{#each}}`;
  a declared-but-missing variable throws; renders `bodyHtml` and `bodyText` independently.
- Impl: Handlebars, `noEscape:false`; validate declared `variables` present.

**T2 — Migrations + repository (`template.repository.ts`)**
- Migration 1 (table+enum). Regenerate client.
- Tests: `resolve` returns highest active version; **scoped overrides global**; unknown key → NotFound;
  inactive versions ignored.

**T3 — `render()` service tying resolve+renderer, incl. inline source**
- Tests: stored `{key}` resolves+renders; `{inline}` renders without a DB row; wrong channel → 404.

**T4 — Provenance: `OutboundService.send` param + migration + `sendTemplated()`**
- Migration 2 (4 columns).
- Modify `OutboundService.send` to accept the optional `provenance` arg and write it into the command
  `create()` on both the RATE_LIMITED and PENDING paths. Extend `outbound.service.spec.ts`: a send
  with provenance persists all four fields; a send without leaves them null (no regression).
- Then `sendTemplated()`. Tests: composes render→`OutboundService.send`; **stamps
  `templateKey/version/locale/renderedHash`** on the command; **idempotent per key** (replay → one
  `IiosOutboundCommand`); inline → `templateKey` null, `renderedHash` set.

**T5 — Seeder (`template.seeder.ts`, `seeds/*`)**
- Tests: boot seeds the file defaults as `scopeId NULL`; **re-seed is idempotent** (no dupes);
  a bumped file version inserts a new row, leaves the old.

**T6 — Controller + DTOs**
- Tests (HTTP, boot against sandbox provider so no real mail): `POST /send` renders+queues;
  unknown key → 404; bad body → 400; missing auth → 400/401; `POST /preview` returns content, sends nothing.

**T7 — Whole-suite gate**
- `vitest run` (all packages), `npm run boundary`, `npm run build` all green.
- Manual: boot locally, `POST /v1/templates/send` with the `welcome` seed via sandbox, inspect the
  `IiosOutboundCommand` row for payload + provenance.

**T8 — PII redaction of outbound commands (fast-follow; lever #2)**
- Extend `RetentionService` so its sweep also redacts aged `IiosOutboundCommand` rows: raw `target`
  and rendered `payload` → redacted, while `templateKey/version/renderedHash` + `scopeId` are kept
  for audit. Reuse the existing redact-in-place pattern (currently applied to `iiosMessagePart`).
- Tests: a command past its window has `target`/`payload` redacted but provenance intact; a command
  under compliance hold is skipped; audit row `retention.redacted` written.
- Independent of T1–T7 — can land immediately after the module without blocking Friday.

## Error handling

- Unknown key/channel/locale → `NotFoundException` (404).
- Declared variable missing → throw (400) — never send a half-rendered receipt.
- Transport failure → surfaced as `FAILED` on the command by `OutboundService`, never thrown to caller.
- Idempotent replay (same key) → returns the existing command; never double-sends.

## Out of scope (separate specs — this module unblocks them)

1. **`SmtpProvider`** (nodemailer, `accounts@`/`ceo@lynkeduppro.com`) in IIOS's capability registry.
   Today's `EmailProvider` is HTTP; SMTP is a new provider flipped on by env. Without it, sends land
   in the sandbox.
2. **`POST /webhooks/stripe` in be-crm** — verify Stripe signature → mint a **service token** →
   call `POST /v1/templates/send` (welcome + receipt), keyed on the Stripe object id. This is the
   "backend" the frontend-only payment site lacks.
3. **Inbox mirror** (post-registration): render an email → also create an `Interaction(kind=EMAIL)`
   so it appears in the customer's in-app inbox. `INTERNAL` delivery lives here. Mirror only *after*
   registration (no inbox exists before).

## PII minimization

Sending email means IIOS must *touch* the recipient's address (you can't mail without it) and the
rendered body holds their name — so `IiosOutboundCommand.target`/`payload` become PII. You cannot
avoid IIOS touching it; you avoid it **accumulating** and being **cheaply reachable**. Three levers,
by impact:

1. **Rotate the `Qwerty@a2` signing secret — precondition for prod, highest impact, not code.**
   The stored PII is only dangerous because any leaked token can be brute-forced back to the secret,
   letting an attacker forge a token and read the outbound table. A strong random secret leaves the
   PII in place but **unreachable by forgery** — 90% of the risk closed by one env change. **Gate:
   do not point the real SMTP provider at production until this is rotated.**

2. **Redact the raw address + body after delivery (fast-follow task — see T8).**
   IIOS keeps `templateKey/version/renderedHash` + the opaque `scopeId` for audit, but the raw
   `target` and rendered `payload` are redacted-in-place once past their retention window. The
   `RetentionService` already does exactly this for interaction message parts (`bodyText →
   '[redacted]'`); it just doesn't cover `IiosOutboundCommand` yet. Reuse the same sweep.

3. **Tokenized recipient (future, when MDM ships).** The clean model is: callers pass a `userId`,
   IIOS resolves the address from MDM *at send time* and never persists it. Not available today
   (MDM isn't deployed; `externalId` is a UUID with no email). **Design accommodation now:** keep
   `target: string` for Friday, but treat it as an opaque "where to send" so it can later become
   `{ userId }` resolved via MDM without changing the `sendTemplated` contract shape.

**Rejected shortcut:** having be-crm send SMTP directly so IIOS never sees the address. It "avoids"
the PII but breaks the inbox mirror and the single-send-path (IIOS would have no record to copy into
the inbox) — trading a solvable security problem for a broken feature.

## Other risks

- **Friday, no test env:** first real sends go to paying customers from an unproven path. Insist on a
  sandbox target / 100%-off test coupon before wiring the real SMTP provider (compounds with lever #1
  above — don't send real mail from prod until the secret is rotated *and* a safe test target exists).