commit d1308bf149b68758801ec2db937a7ab780a46908 Author: kaushik Date: Sat Jul 18 01:34:52 2026 +0530 first commit diff --git a/.claude/memory/MEMORY.md b/.claude/memory/MEMORY.md new file mode 100644 index 0000000..dd19696 --- /dev/null +++ b/.claude/memory/MEMORY.md @@ -0,0 +1,18 @@ +# Project-local AI memory (`.claude/memory/`) + +This folder is the **project-local memory for AI assistants**, committed with the code (not in any +machine's global `~/.claude`). Any developer or Claude/AI account that opens this package should read +these files for full context. Keep them updated as work proceeds. + +- [status.md](status.md) — current build status + session log (what's done, what's next). +- [requirements.md](requirements.md) — the COMPLETE requirements backlog (every request, tracked). +- Human-facing docs live in [`/docs`](../../docs/) (ROADMAP / SETUP / ARCHITECTURE) and [`/CLAUDE.md`](../../CLAUDE.md). + +## Hard rules (do not violate) +- **Never `git commit` / `git push`** — the user does that. Never commit secrets. +- API keys live only in `apps/web/.env.local` (gitignored), read **server-side** only. +- Every feature is **optional via `features` flags**. End goal: **backend (Supabase), no localStorage reliance** + (localStorage is the dev fallback only until Supabase creds are provided). +- Prefer **free** tech; if no free option, build our own. If a cloud key (e.g. Gemini) is missing/invalid, + fall back to a **local model** (e.g. `@imgly/background-removal` for remove-bg). +- UI must be **meaningful for a WEB gallery** (not iOS) — e.g. no "App Store"; "Albums" not "Projects". diff --git a/.claude/memory/requirements.md b/.claude/memory/requirements.md new file mode 100644 index 0000000..8be8f00 --- /dev/null +++ b/.claude/memory/requirements.md @@ -0,0 +1,46 @@ +# Requirements backlog (complete) + +Status: ✅ done · 🚧 in progress · ⬜ planned · 🔑 needs key/creds. Cross-ref `/docs/ROADMAP.md`. + +## Done (Phases 1–3) +- ✅ macOS-Photos UI: sidebar, toolbar, Library (Years/Months/All), Collections, Map (map/satellite/grid), + People, Recently Deleted (locked), lightbox, light/dark/**semi-dark**, responsive to 280px, a11y, nonce CSP. +- ✅ Albums (CRUD/nested/smart), source auto-classify, recycle bin, duplicates, multi-select, import (file+drag), + export, **EXIF** (GPS/date/camera) on import. +- ✅ Object detection (TF.js, free) → tags/search/find-similar/**Objects browser**/object focus. +- ✅ Annotations (rect/oval/line/arrow/**double-arrow measurement**/text/freehand + colors) in editor + lightbox. +- ✅ Info panel (full metadata + **mini-map**). Themes menu + `accentColor`/`borderRadius` props. +- ✅ Gemini generative editor (Remove BG/Restore/Colorize/Replace Sky/prompt) — 🔑 needs billed key (free=429). + +## Now / next (in user's priority order) +1. 🚧 **Custom camera** (#1): getUserMedia preview; capture photo; record video; switch front/back; grid; + **live annotation + measurement**; geolocation on capture (permission); auto object-detect after capture. +2. ⬜ **Supabase backend + storage** (#last, 🔑): Postgres data + Supabase Storage. **No localStorage reliance** — + backend is the source of truth. Ask user for creds (URL, anon key, service-role key, bucket) with steps. +3. ⬜ Face clustering → People; OCR (tesseract.js) → document search; semantic "show me beach photos" + (transformers.js / Hugging Face CLIP) — all free, in-browser. +4. ⬜ Video editor + annotations; macOS-parity player polish. + +## New requests (this session — must add) +- ⬜ **Editor: straighten / tilt** (angle slider) — rotate by arbitrary degrees. +- ⬜ **Crop: free-form AND fixed-ratio** — when a ratio (e.g. 1:1 square) is selected, dragging/stretching the + crop box keeps that aspect ratio; also free crop. +- ⬜ **Import sync of existing metadata**: if an imported/mobile image already has tags (IPTC/XMP keywords) + + location + details, read and apply them (exifr supports XMP/IPTC). Keep object detection + object albums. +- 🚧 **Web-meaningful UI** (deep pass): "All Projects" → **"All Albums"** (this is a web gallery, not iOS); + remove "Open App Store" (meaningless on web); review sidebar tabs, **corner radius**, touch-ups so everything + is meaningful for a web gallery and matches macOS more closely. +- ⬜ **Search dropdown defaults**: show quick filters **Recently Viewed**, **Recently Edited** (+ Recently Shared) + when the search field is focused. +- ⬜ **Info → map click opens full Map** centered on that photo's location. +- ⬜ **Map parity w/ macOS**: locations + images + tags; latest image shown for a tag; **click a tag → open + that tag's images grouped by date** (date-wise). Tag = album-like view, date-grouped. +- ⬜ **Local-model fallback**: if Gemini key missing/invalid, use a local model. Concretely wire + `@imgly/background-removal` (free, in-browser) for Remove Background so it always works without a key. +- ⬜ **Everything free**; build our own if no free option. +- ✅ **Memory in project `.claude/` folder** (this folder) — done. +- ⬜ More **advanced features** (open-ended; user will add more later). + +## Notes +- getUserMedia needs HTTPS or localhost (dev OK). Permissions-Policy already allows camera/mic/geolocation=self. +- When Supabase is wired, switch the demo adapter from localStorage → Supabase and drop localStorage reliance. diff --git a/.claude/memory/status.md b/.claude/memory/status.md new file mode 100644 index 0000000..ba5afbf --- /dev/null +++ b/.claude/memory/status.md @@ -0,0 +1,309 @@ +# Build status & session log + +## Run +`pnpm install && pnpm dev` → http://localhost:3000 (landing → Open Gallery → /gallery). `pnpm build` to ship. + +## Current state (2026-06-29) +Phases 1–3 complete & verified (see [requirements.md](requirements.md)). `pnpm build` passes; /gallery ~170 KB +(TF.js + exifr lazy). Zero console errors. Default persistence = localStorage adapter (until Supabase wired). + +## Session log +- S1: monorepo + SDK + demo; full macOS Photos UI; albums/smart-albums/recycle/import/editor/lightbox; light+dark; + responsive; nonce CSP; 19-finding review fixed. +- S2: free in-browser object detection (TF.js COCO-SSD) + Objects browser; Gemini generative editor (server route, + key server-side; 429 on free tier); fixed ORB video (local /sample-clip.mp4). +- S3: map popups; annotations incl. measurement double-arrow; EXIF; Info panel + mini-map; semi-dark + theming props; + docs/ + CLAUDE.md. +- S4: `.claude/memory/` created; web-meaningful UI (All Projects→**All Albums**, removed App Store; AlbumsOverview); + **custom camera (#1) DONE** (`components/Camera.tsx`: getUserMedia preview, photo capture, video record, + front/back switch, rule-of-thirds grid, live annotation+measurement, geolocation on capture, auto object-detect; + store `cameraOpen`/openCamera/closeCamera; toolbar 📷 button). Verified modal UI (no camera device in test env; + works with a real webcam). `pnpm build` passes (/gallery 172KB). + STILL OPEN: editor straighten/ratio-crop, search defaults (Recently Viewed/Edited), import IPTC/XMP tags, + local remove-bg fallback (@imgly), info-map→Map, tag→date-wise, face clustering, OCR, semantic search, video annotations. +- S5: **#2 Supabase backend DONE** — `apps/web/src/lib/adapters/supabaseAdapter.ts` (Postgres JSONB tables + + Storage blobs via putBlob); store gained `importFiles`/`uploadBlob` (uploads on import + camera capture); + TopToolbar/LibraryView/Camera route through them; GalleryClient uses Supabase when `NEXT_PUBLIC_SUPABASE_*` set + (else localStorage). CSP (middleware) now allows `https://*.supabase.co` (connect+img+media). Bucket `media` made + public. Creds in `apps/web/.env.local` (gitignored). Adapter degrades gracefully if tables missing (probe→warn→seed). + **USER MUST run `docs/supabase-setup.sql` once (SQL Editor) + reload** to activate persistence. `pnpm build` passes + (/gallery 234KB — supabase-js bundled; TODO lazy-load to trim). Verified app boots clean on Supabase adapter (seed + fallback) — only the expected gallery_media 404 probe until SQL is run. + NOTE: service_role key is in .env.local but the client adapter uses the anon key; service key only for admin/setup. +- S6: VERIFIED Supabase end-to-end (user ran SQL): app loads 45 rows from Postgres, 0 console errors; a camera + capture persisted + round-trips. FIXES: camera annotations moved to the REVIEW stage (annotate the captured still + → saved as edits.annotations) + Undo/Clear; geolocation prefetched on camera open (10s, high-accuracy) + cached; + device info attached to capture exif (Make 'Web Camera', Model=track label, Resolution). Auto-albums: addMedia now + auto-creates+fills "Camera" (source camera) and "Screenshots" (source screenshot) user albums. Recycle: init purges + items trashed > 30 days (deletePermanently → syncs to Supabase); restore/permanent-delete already sync via save. + NEW FEATURES: editor **Straighten** slider (EditState.straighten + editTransformCss cover-scale); **info mini-map + click → focusMap → opens full Map centered** (store mapFocus); **tag chip click → setTagFocus → library filtered by + tag, date-grouped** (store tagFocus, libraryScale='days'); object/tag chips close overlays. VERIFIED tag→date-wise live. + `pnpm build` passes. + DEFERRED (next): interactive fixed-ratio/free crop (bake on save), search dropdown defaults (Recently Viewed/Edited), + import IPTC/XMP keywords, local remove-bg fallback (@imgly), face clustering/OCR/semantic search, video annotations, + lazy-load supabase-js (trim /gallery bundle), storage blob deletion on permanent-delete, reverse-geocode place names, + optional pg_cron for server-side 30-day purge. +- S7: **Interactive crop DONE & verified** — `components/editor/CropBox.tsx` (drag/resize, 4 corner handles, masks, + thirds grid, aspect lock so square stays square), ratio presets (Free/Square/4:3/3:4/16:9/9:16/Original) in the + Crop tab; `lib/bake.ts` flattens crop+flip+filter+vignette+rotation/straighten to a JPEG on Done and re-maps + annotations; `PhotoEditor.save()` bakes when `hasGeometry(edits)` and uploads via `uploadBlob` (Supabase Storage). + FIXED Supabase storage 400: upsert/overwrite violated RLS → adapter now uploads to a UNIQUE path per call + (`${id}-${rand}.ext`, plain INSERT). VERIFIED: cropped a portrait screenshot to 16:9, baked + uploaded to bucket + (object `eMdEj_Ow32-…jpg`), displays cropped; 0 console errors; `pnpm build` passes. + STILL OPEN (next, in order): search dropdown defaults (Recently Viewed/Edited), import IPTC/XMP keywords, + local remove-bg fallback (@imgly), face clustering / OCR / semantic search, video annotations, lazy-load supabase-js. +- S8: **search dropdown defaults DONE** (store recentlyViewed[] + searchPreset 'viewed|edited|added'; openLightbox + records views; setSearchPreset; selectors handle presets; TopToolbar Recents dropdown on focus). FIX: 'edited' + preset keys off `editedAt||edits` (baked crops clear `edits` but set editedAt) — VERIFIED shows both crops + selfie. + **IPTC/XMP import DONE** (readExif parses iptc+xmp → Keywords/dc:subject → tags). **Lazy-load supabase-js DONE** + (GalleryClient dynamic-imports the adapter; /gallery 234KB→176KB). All typecheck + `pnpm build` pass; verified live. + STILL OPEN (next): local remove-bg fallback (@imgly; needs wasm-unsafe-eval CSP + connect-src for model host), + face clustering → People, OCR → documents, semantic search (transformers.js/HF), video annotations. +- S9: **Local Remove Background DONE & verified** — `@imgly/background-removal` (lazy import) in + `createDemoAIProvider.generativeEdit`: op 'remove-background' runs the in-browser WASM model (no Gemini); + other ops still hit /api/ai/edit. CSP: added `'wasm-unsafe-eval'` to script-src + `blob: data:` and + `https://staticimgly.com` to connect-src (the ML worker fetches model/wasm from blob: URLs). VERIFIED: + removed background from the user's selfie capture → clean transparent PNG, 0 errors, saved+uploaded to Supabase. + `pnpm build` passes (/gallery 177KB; @imgly lazy). protobufjs build script declined in pnpm-workspace.yaml. + STILL OPEN (next, model-heavy): face clustering → People (face-api.js), OCR → Documents (tesseract.js), + semantic search (transformers.js/CLIP), video annotations. Each needs a model CDN host in connect-src. +- S10: **Face clustering → People DONE & verified** — `apps/web/src/lib/ai/faceProvider.ts` (`@vladmandic/face-api`, + lazy import; tinyFaceDetector+faceLandmark68+faceRecognition; models from `cdn.jsdelivr.net/npm/@vladmandic/face-api@1.7.15/model`, + added to connect-src). Wired as `detectFaces` in createDemoAIProvider. SDK: `lib/cluster.ts` (greedy euclidean + clustering, threshold 0.55, biggest-face-first); store `rebuildPeople()` (reuses existing person id/name by + media overlap so names survive re-clustering) + `personFocus`/`setPersonFocus`; selectors filter by personFocus; + AIAnalyzer now runs detectObjects+detectFaces (only what's missing per item) and calls rebuildPeople() after the + pass (and once on load if faces exist but People empty); PeopleView click → setPersonFocus (filters library + + shows count); LibraryView banner shows "Person: …". BUG FIXED: `hooks/useViewMedia.ts` omitted personFocus from + the state passed to mediaForView → filter was a no-op; added it (+dep). VERIFIED: face-api detected the selfie's + face (1 face, 128-D emb), People shows 1 person w/ circular cover + "1 photo", click → grid filtered to that 1 + photo. Persisted to Supabase (gallery_people + media.personIds). `pnpm build` passes (/gallery 178KB; face-api lazy, + benign "Critical dependency: require" webpack warning only). Picsum seed photos / screenshots have no detectable + frontal faces → only the real selfie clusters, as expected. + STILL OPEN (next): OCR → Documents (tesseract.js), semantic search (transformers.js/CLIP), video annotations. +- S11: **OCR → Documents + searchable text DONE & verified** (planned via an `ultracode` Workflow that fanned out + 5 parallel readers to map integration points, then an Opus synthesizer). `apps/web/src/lib/ai/ocrProvider.ts` + (NEW, tesseract.js@5.1.1, lazy import, single memoized worker; worker/core/lang paths ALL pinned to + cdn.jsdelivr.net so nothing falls back to the non-allow-listed tessdata.projectnaptha.com; `meaningfulText()` + confidence filter: per-word conf>=70 + >=4 words, else mean-conf>=72 + >=6 word-shaped tokens → '' for photos). + Wired as `ocr` in createDemoAIProvider. SDK: `types.ts` MediaItem.ocrText + SmartRule 'hasText' + ViewId + `sys:${string}`; `lib/text.ts` (NEW) sanitizeOcrText (control-char strip via RegExp-from-string, ws-collapse, + 10k cap); `smartAlbums.ts` hasText eval + `sys:documents` album; `selectors.ts` searchMedia haystack += ocrText; + `AIAnalyzer.tsx` 3rd OCR leg (own `ocrText===undefined` gate, inner .catch so a slow/failed OCR can't abort + objects+faces, patches even '' so it isn't reprocessed); Sidebar + CollectionsView 'Documents' + 'document' icon. + CSP: cdn.jsdelivr.net added to script-src (connect-src already had it). CAUGHT VIA GROUND-TRUTH: first pass put + ALL 41 Picsum photos in Documents because tesseract hallucinates low-conf gibberish (meanConf 21-29) on + textureless photos → added the confidence filter; re-verified Picsum→'' , imported a text invoice → 213 chars of + accurate text, Documents shows exactly it, search "oceanfront villa" (words only inside the image) finds it. + Persisted to Supabase. `pnpm build` passes (/gallery 179KB; tesseract lazy). NOTE: ~189 benign console WARNINGS + ("kernel … already registered for backend webgl") appear when face-api's bundled tfjs + coco-ssd's tfjs coexist + (triggered by analyzing a newly-imported image) — warnings only, 0 errors. A test invoice now lives in the demo + library as a live Documents example (uploaded to Supabase; user can delete). + ADVERSARIAL REVIEW (ultracode Workflow, 4 dims × find→refute, 15 agents): correctness + security CLEAN (0); + 6 false alarms correctly refuted (worker leak, rebuildPeople N+1 writes, sequential model load, CSP-nonce-for- + blob-workers, etc.). FIXED 3 confirmed: (1) exported pure utils clusterFaces/FaceCluster/sanitizeOcrText/ + OCR_TEXT_MAX_CHARS from index.ts (SDK-pattern consistency); (2) pinned tesseract.js EXACT 5.1.1 in package.json + (was ^5.1.1) so the hardcoded worker CDN version can't drift; (3) sanitizeOcrText truncates on a word boundary + at the 10k cap (was mid-word → search false-negative). DOCUMENTED (not fixed — minor/pre-existing, not regressions): + (a) OCR backfill gate `ocrText===undefined` won't re-run if a host disables then re-enables OCR (same pattern as + faces; fine while OCR stays on); (b) persist() debounce means a mid-analysis crash loses recomputable derived + metadata (pre-existing, affects all analysis; source media safe, re-derived next load). + STILL OPEN (next): semantic search (transformers.js/CLIP), video annotations. +- S12: **BIG new backlog from user (13 items)** — kicked off. WAVE 1 DONE & verified: + (1) Selection is now a TOGGLE — plain click selects, clicking the sole selected tile deselects + (PhotoTile.onClick); (2) the "..." More menu shows **Select All** then flips to **Unselect All** once all + current-view items are selected (TopToolbar.openMoreMenu; removed the stray clearSelection-on-open); (3) editor + now has **Save** (overwrite) + **Save as Copy** (uploads under a fresh id via new store action + `duplicateWithEdits`, then opens the album picker) + **Cancel** with an "unsaved changes" confirm (isDirty check; + Escape routes through it too via cancelRef); (4) **Copy to Album** (was "Add to Album") + **Move to Album** + (new `moveToAlbumPicker`, album-view only) + Remove from Album. VERIFIED in browser: toggle true→false, + Select All→46 selected→"Unselect All", editor bar = Cancel/Revert/Save as Copy/Save, context menu = Copy to Album. + `pnpm build` passes (/gallery 180KB). + PLANNING (ultracode background Workflow, 7 architects): file-level plans for the remaining big features saved at + tasks/wck4qn659.output (sharing+links, password-lock, map-pins, grid-mosaic, glass-design, semantic-search, + video-annotations) — use as the blueprint for the next waves. + REMAINING WAVES (in this order): People rename + pet detection; Objects browser verify/expand; Map thumbnail pins + + click->location date-wise + screenshot pins; Map "Grid" mosaic (hero + verticals + auto-slide); Sharing/Shared + Albums/Activity + download links; Recently Deleted password lock + Settings; macOS glass/dark/radius design polish; + semantic search (CLIP); video annotations; then README + memory. + Design ref captured from user's macOS screenshots: glass dropdowns/sidebar (backdrop-blur + translucent dark + fill + hairline border + caret), thumbnail map pins w/ count badge, irregular mosaic Grid. +- S13: **WAVE 2 DONE & verified** — People rename + pet detection + Objects browser. + PETS: `constants.PET_LABELS` (dog/cat/bird/horse/rabbit); `store.rebuildPeople` now also builds pet groups from + objectLabels with STABLE ids `pet: