Refactor code structure for improved readability and maintainability
This commit is contained in:
+59
-1
@@ -30,7 +30,65 @@ These edits *generate new pixels* from a prompt: **Restore, Colorize, Replace Sk
|
||||
Prompt**. Pick a backend with the `AI_EDIT_PROVIDER` env var (default `auto`). The server route
|
||||
(`/api/ai/edit`) proxies to it so no key ever reaches the browser.
|
||||
|
||||
`auto` chooses the first configured of: **local → huggingface → gemini**.
|
||||
`auto` chooses the first configured of: **runpod → local → huggingface → gemini**.
|
||||
|
||||
### Option 0 — RunPod serverless GPU endpoints (each model is its own endpoint) 🚀
|
||||
|
||||
This is the production backend for the models in `model-api-spec.docx`. Each model runs as a
|
||||
separate RunPod endpoint; the app proxies to them **server-side** (`/api/ai/edit`,
|
||||
`/api/ai/classify`) so `RUNPOD_API_KEY` and the URLs are **never** exposed to the browser.
|
||||
|
||||
```env
|
||||
AI_EDIT_PROVIDER=runpod
|
||||
RUNPOD_API_KEY=rpa_xxxxxxxx
|
||||
RUNPOD_SD_IMG2IMG_URL=https://api.runpod.ai/v2/<id>/runsync # #10
|
||||
RUNPOD_SD_INPAINT_URL=https://api.runpod.ai/v2/<id>/runsync # #9
|
||||
RUNPOD_UPSCALE_URL=https://api.runpod.ai/v2/<id>/runsync # #7
|
||||
RUNPOD_COLORIZE_URL=https://api.runpod.ai/v2/<id>/runsync # #8
|
||||
RUNPOD_BG_REMOVE_URL=https://api.runpod.ai/v2/<id>/runsync # #6 (optional; default is in-browser @imgly)
|
||||
RUNPOD_YOLO_URL=https://api.runpod.ai/v2/<id>/runsync # #1 (also set NEXT_PUBLIC_APG_RUNPOD_DETECT=true)
|
||||
```
|
||||
|
||||
**Which editor action hits which endpoint** (the app maps them for you):
|
||||
|
||||
| Editor action | `op.type` | RunPod endpoint | Notes |
|
||||
|---|---|---|---|
|
||||
| Restore & Enhance | `restore` | #7 Real-ESRGAN (`RUNPOD_UPSCALE_URL`) | ×4 + face-enhance |
|
||||
| Upscale | `upscale` | #7 Real-ESRGAN | ×2 / ×4 (`op.factor`) |
|
||||
| Colorize | `colorize` | #8 DDColor (`RUNPOD_COLORIZE_URL`) | |
|
||||
| Apply Prompt | `prompt` | #10 SD 3.5 img2img (`RUNPOD_SD_IMG2IMG_URL`) | whole-image, no mask |
|
||||
| Replace Sky | `replace-sky` | #9 SD 3.5 inpaint **if a mask is sent**, else #10 img2img (low strength) | see masking below |
|
||||
| Magic Eraser | `magic-eraser` | #9 SD 3.5 inpaint (`RUNPOD_SD_INPAINT_URL`) | **mask required** |
|
||||
| Generative Fill | `generative-fill` | #9 SD 3.5 inpaint | **mask + prompt required** |
|
||||
|
||||
**Endpoint I/O contract:** the app POSTs `{ input: { image, mask?, prompt?, strength?, guidance_scale?,
|
||||
num_inference_steps?, seed?, … } }` and reads the image back from `output` (any of `image`,
|
||||
`image_png`, `images[0]`). Deploy your handlers to accept that `input` shape (or adjust
|
||||
`apps/web/src/lib/runpod/endpoints.ts`). `/runsync` is preferred; `/run` + `/status/{id}` polling is
|
||||
supported as a fallback but must finish inside Vercel's 60 s function limit.
|
||||
|
||||
**Masking (SD 3.5 #9):** masked ops send a `maskBase64` PNG **the same pixel size as the image**
|
||||
(white = regenerate, black = keep). The client rasterizes the editor's `ImageData` mask to match the
|
||||
downscaled image automatically (`apps/web/src/lib/ai/imageEncode.ts` → `maskToBase64`). **Magic
|
||||
Eraser / Generative Fill therefore need a selection** — until a brush-mask UI is added to the editor
|
||||
they return a clear "needs a mask/selection" error; **Replace Sky** works today by degrading to a
|
||||
low-strength img2img (#10) when no mask is present (add sky-segmentation or a drawn mask to get true
|
||||
masked #9).
|
||||
|
||||
**Detection (#1):** set `NEXT_PUBLIC_APG_RUNPOD_DETECT=true` to route object detection to the YOLO
|
||||
construction-material classifier via `/api/ai/classify`; it falls back to in-browser COCO-SSD if the
|
||||
endpoint is unreachable. Verify the box format in `endpoints.ts` (`normalizeBox`) against your model —
|
||||
it assumes ultralytics `xyxy` pixel coordinates.
|
||||
|
||||
**Already local (no RunPod needed):** screenshot detection (#4), EXIF geolocation (#15), and the tag
|
||||
rename lookup (#5) run in the app/browser — see `lib/classify.ts`, `lib/media.ts`. Background removal
|
||||
(#6) also runs free in-browser by default.
|
||||
|
||||
**Not yet wired (typed client + env only):** camera tilt (#2), voice-to-text (#3), audio denoise
|
||||
(#12), and video frame-rate/resolution (#13/#14) have `rp*` functions in `endpoints.ts` and env vars,
|
||||
but **no route or UI** — they have no consuming feature yet. #13/#14 in particular need an out-of-band
|
||||
pipeline (presigned upload → async job → webhook), because per-frame video can't fit Vercel's 4.5 MB
|
||||
body / 60 s limits or the in-browser video export path. Add these when the corresponding feature is scheduled.
|
||||
|
||||
### Option 1 — Your own local Stable Diffusion (free + private, recommended) 🏆
|
||||
|
||||
|
||||
+19
-7
@@ -60,12 +60,21 @@ install the whole workspace). Do not commit `.env.local` (it's gitignored).
|
||||
**Server-only (optional — do NOT prefix with `NEXT_PUBLIC`):**
|
||||
| Name | Value |
|
||||
|---|---|
|
||||
| `GEMINI_API_KEY` | a Gemini key *(only for the AI generative-edit button)* |
|
||||
| `AI_EDIT_PROVIDER` | `runpod` *(or `local` / `huggingface` / `gemini` / `auto`)* |
|
||||
| `RUNPOD_API_KEY` | your RunPod API key *(shared by every RunPod endpoint)* |
|
||||
| `RUNPOD_SD_IMG2IMG_URL` | #10 prompt endpoint URL *(…/runsync)* |
|
||||
| `RUNPOD_SD_INPAINT_URL` | #9 masked sky/eraser/fill endpoint URL |
|
||||
| `RUNPOD_UPSCALE_URL` | #7 restore/upscale endpoint URL |
|
||||
| `RUNPOD_COLORIZE_URL` | #8 colorize endpoint URL |
|
||||
| `RUNPOD_YOLO_URL` | #1 detection endpoint URL *(+ `NEXT_PUBLIC_APG_RUNPOD_DETECT=true`)* |
|
||||
| `GEMINI_API_KEY` | a Gemini key *(alternative generative-edit backend)* |
|
||||
| `GEMINI_IMAGE_MODEL` | `gemini-2.5-flash-image` *(optional)* |
|
||||
|
||||
Without the Supabase vars the app still runs, but on browser-local storage only (data won't sync
|
||||
across devices). Without `GEMINI_API_KEY` the generative-edit route returns 503 while all the free
|
||||
in-browser AI keeps working.
|
||||
The RunPod URLs and `RUNPOD_API_KEY` are read **only** in the `/api/ai/*` route handlers (server
|
||||
side) — never send them to the browser (no `NEXT_PUBLIC_` prefix). Full endpoint map + the op →
|
||||
endpoint table is in [`docs/AI-SETUP.md`](./AI-SETUP.md). Without the Supabase vars the app still
|
||||
runs on browser-local storage; without any AI-edit provider the generative-edit route returns 503
|
||||
while all the free in-browser AI (detection, faces, OCR, background removal) keeps working.
|
||||
|
||||
Optional theming/feature flags (`NEXT_PUBLIC_APG_*`) are documented in [`docs/ENV.md`](./ENV.md).
|
||||
|
||||
@@ -90,9 +99,12 @@ they're intentionally not stored.
|
||||
|
||||
## Gotchas / limits (free tier)
|
||||
|
||||
- **Gemini AI edit + Vercel body limit:** Vercel Hobby caps request bodies at ~4.5 MB. The AI-edit
|
||||
route is capped accordingly and large images are downscaled client-side first; very large images
|
||||
may still be rejected. The free in-browser AI is unaffected.
|
||||
- **AI edit + Vercel body limit:** Vercel Hobby caps request bodies at ~4.5 MB. The AI-edit route is
|
||||
capped accordingly and large images are downscaled client-side first (≤1280 px). For masked SD 3.5
|
||||
ops the **image + mask share one body**, so they're budgeted together; a mask PNG is mostly flat
|
||||
black/white and compresses small, so this holds. Very large images may still be rejected. Prefer
|
||||
RunPod `/runsync` endpoints so a job returns within the 60 s function limit. The free in-browser AI
|
||||
is unaffected.
|
||||
- **Single-user demo:** the Supabase adapter does a full-state sync (no auth). For multi-user, add
|
||||
Supabase Auth and scope rows per user before going to production (the RLS policies are open for the
|
||||
demo — tighten them with `auth.uid()`).
|
||||
|
||||
Reference in New Issue
Block a user