Show HN: AI search for every photo and every frame of video on macOS

SCM — Screen Memories

Deep AI search for every photo and every frame of video in any folder on macOS. Local-first — no accounts, no cloud, no uploads. Inference runs on your Mac.

Mode Finds

Files Whole photos/videos by meaning — vision rank with filename and phrase boosts

Scenes Moments inside video — search a shot, jump to its timecode

OCR Text visible in images and frames, matched literally (Tesseract; eng + 35 language toggles)

Dialogue Exact spoken words in videos (Whisper), tiered exactness

LLMs (opt-in) Local chat over the dialogue, OCR, and filenames your Mac already extracted — cited answers

Requirements

macOS (packaged with electron-builder; menu-bar/tray features are macOS-only)

Bun — the project uses bun as package manager and runner

Node modules installed: bun install

First use of a model downloads its weights (~435MB for the default CLIP); after that, fully offline.

Download

The easiest install is via Homebrew (Apple Silicon, macOS 12+). The tap's cask clears the macOS quarantine flag automatically on every install and upgrade, so the app launches with no manual Gatekeeper steps:

brew tap allenv0/scm brew trust allenv0/scm brew install --cask allenv0/scm/scm

Upgrades keep the same behavior:

brew upgrade --cask allenv0/scm/scm

Prefer least privilege? Trust just the cask instead of the whole tap:

brew tap allenv0/scm brew trust --cask allenv0/scm/scm brew install --cask scm

The tap lives at allenv0/homebrew-scm.

bun run dev # build the renderer bundle, then launch the Electron app bun start # launch the Electron app without rebuilding bun run build # just rebuild the renderer bundle into dist/

Building the app package (DMG / ZIP)

bun run dist # signed if an identity is in the keychain bun run dist:unsigned # skip code-sign discovery

This runs two steps in sequence:

Output: the installers land in dist-app/ (see build.directories.output in package.json) — look for SCM-0.2.4.dmg and SCM-0.2.4.zip.

Files

Typing starts an instant filename-keyword pre-pass, then the vision model takes over: results are scored by cosine similarity against image embeddings, with gated phrase and filename boosts, an honesty floor calibrated per model, and a near-duplicate diversity filter. Every tile carries a "why it matched" badge (Visual match / Filename match / …) and a hover tooltip with the per-component score breakdown. CJK queries search as overlapping bigrams ("台北車站" also matches 台北, 車站).

Every scene segment across all videos is scored, so a hit lands on the exact shot: tiles show the scene poster with a timecode badge, and opening the video jumps straight to that moment. A noise gate returns "no scene match" instead of flooding the grid with gibberish, and each video contributes at most 3 scenes.

Matches the fraction of query tokens literally visible in each image's OCR text — the filename is ignored and no vision model is involved, so it works even while the AI engine is warming up or offline. Matched words are boxed in amber on tiles and in the lightbox.

Exact literal retrieval over Whisper transcripts — no embeddings, no thresholds, works with the AI engine down. Results come in three tiers: Exact line (contiguous phrase in one utterance), Exact words (all words in one utterance or an ≤8s window), and Words spoken (all words in the same video). Matching words are highlighted in a speech snippet; opening a result seeks straight to the line.

Opt-in — nothing downloads or runs until enabled in Settings → LLMs Chat. A llama.cpp sidecar bound to loopback answers your question from evidence the app already extracted — dialogue lines, OCR text, and filename keyword hits — with numbered citations you can click, streamed token-by-token with a live tok/s readout. Leading /screenshots, /videos, /email narrow the corpus; Stop keeps the partial answer; empty evidence short-circuits before the model ever runs.

Built-in browse tabs: All, Videos, plus Screenshots and Email — the latter two toggleable in Settings → Smart Tabs. Selecting Videos auto-enables Scenes mode.

Save any query as a tab: the pin pill under the search bar saves the current prompt with its mode (Files/Scenes/OCR/Dialogue) — up to 20 tabs, renameable, each restored exactly as saved.

Five semantic views behind remappable shortcuts (⌘1–⌘5 by default), plus ⌘I import / AI Insights, ⌘, for Settings.

Search stays in the selected tab: pick Screenshots, Email, Videos, or any saved tab and results are filtered to it — scope first, then search. In LLMs chat the same idea is explicit: leading /screenshots, /videos, /email narrow the corpus before the model ever runs.

Email tab

Surfaces photos whose visible OCR text contains an email address — an overlapping view (a photo keeps its category too). Detection is OCR-tolerant: it reassembles addresses Tesseract fractures across word boxes, and handles comma-for-dot noise ("gmail,com"), split TLDs ("gmail. com"), bracketed obfuscation ("allen [at] gmail [dot] com"), and dictated addresses ("allen at gmail dot com"). Tiles show a contact strip; expand it to copy or compose.

Screenshot classification is rename-proof. Four signals, in priority order: a manual override (right-click any tile) → filename vocabulary (30+ localized OS screenshot names in 20+ languages) → a PNG/JPEG metadata probe (reads "screenshot" from PNG text chunks / EXIF UserComment, so a renamed Bildschirmfoto still classifies) → source-folder hint. Everything else lands in Projects.

Four switchable models via ONNX Runtime; the active one is chosen per library:

Switching models re-embeds the whole library: the flip lands instantly with the tail filled in the background, and search falls back to filename keywords until it completes. Per-model text-mean centering de-biases text embeddings so similarity scores stay honest across models.

ffmpeg scans each video for shot boundaries and builds a segment plan sampled to the density you pick in Settings → Video Search — each preset shows its measured time and disk cost before you commit:

Each segment embeds its midpoint frame and keeps a poster; shot plans are cached per file (path + size + mtime + config fingerprint), so re-imports skip detection entirely.

Dialogue transcription: Whisper tiny.en (~150MB, default) or base.en (~300MB) — switching re-transcribes every video. Whole videos embed three frames (20/50/80%) averaged; GIFs embed an average of middle frames.

Tesseract runs in its own worker, separate from the vision model. English is always on; 35 more languages are toggleable in Settings → Photo Search (default: Simplified + Traditional Chinese, Japanese, Korean). Each language pack downloads once (~2.4–5MB; ~17MB for the default set), then everything is offline. Word boxes are stored with the text so matches highlight in place; CJK text is joined without spaces and email fragments fractured across word boxes are reassembled.

Import via ⌘I, drag-and-drop, or watched folders — importing a folder starts watching it (live fs.watch plus a re-sync at every launch). Problem files retry up to 3 times, then sit out watch-syncs until they change.

Rename-proof dedupe: every file is content-hashed (SHA-256) before copy, and lying extensions are normalized by MIME sniffing.

Named embedding versions (Settings → Library): point-in-time snapshots of the entire searchable state — the index, every model's embedding bins, scene and transcript sidecars — with restore (auto-backup first) and a Fresh Start danger zone. Cap: 10.

Everything lives under ~/Library/Application Support/scm (MEMORIES_DATA_DIR overrides it): the index JSON, Float32 embedding bins per model, scene and transcript sidecars, thumbnails, and posters.

Privacy by construction

The renderer is a sandboxed app:// bundle — contextIsolation, OS sandbox, and a CSP pinned to 'self' (+ Google Fonts CDN for display type, with a monospace fallback when offline).

Only main-process workers ever download, once per thing: vision weights (Hugging Face), OCR language packs (Tesseract CDN), Whisper weights, and — only if you opt in — the llama.cpp sidecar and GGUF chat models (GitHub + Hugging Face), sha256-verified at download time.

Media is copied into the app-managed library and streamed from disk. No telemetry, no accounts, no uploads.

Settings & polish

A macOS-style settings sheet with ten panels (Library, Appearance, Grid, Smart Tabs, Photo Search, Video Search, LLMs Chat, Global Shortcut, Keyboard, Menu Bar). Around the core: light/dark/system theme, a CRT screen effect for the lightbox, 6 alternate app icons, menu-bar-only mode, a recordable global shortcut, a first-run onboarding tour, background-work trays (scenes / transcripts / OCR) with a global pause, and a status bar with version and indexed-video counts.