返回资讯中心
外部精选
软件工程
#开发者实时流

Show HN: Umless, open-source on-device dictation that writes what you meant

Local, open-source voice dictation that types what you meant, not just what you said. Hold a key, talk the way you actually talk ("um, can you look at my auth middleware, no wait, the session handler…"), let go, and cle…

Hacker News 最新jatinbishla2026/11/10 · 7 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Local, open-source voice dictation that types what you meant, not just what you said. Hold a key, talk the way you actually talk ("um, can you look at my auth middleware, no wait, the session handler…"), let go, and clean text appears at your cursor.

Requirements: macOS 13 or later on Apple Silicon, Windows 10 or 11 (x64), or Linux (X11 or Wayland). All three tiers run on every OS, and the tray app types into any app with a hold key (Fn on macOS, Ctrl + Win / Ctrl + Super elsewhere). Reading on-screen text is macOS-only for now, and the Windows and Linux desktop app is newer and less tested (details). First install builds from source and downloads one tier. On an M5 MacBook Air, a fresh-clone build took 71 s and 94 s on an idle machine and 635 s on a heavily loaded one; a Mac without Rust also downloads the toolchain and crates first. Lite's models downloaded in 57 s and Standard's (1.77 GB) in 240 s; your connection decides this part.

Rust · Parakeet-TDT / Whisper via sherpa-onnx · Qwen via llama.cpp (Metal) · a LoRA fine-tuned 0.8B cleanup model (Hugging Face) · macOS Accessibility.

¹ measured with the base 0.8B cleanup model; the fine-tune's prompt is 77 tokens instead of 575 for the same dictation.

Methodology and the full history are in Benchmarks and docs/benchmarks.md; the design is in ARCHITECTURE.md.

Umless runs on macOS, Windows and Linux. Each needs 1–6 GB free (by tier). The macOS app is the most complete; on Windows and Linux everything except reading the screen works, see docs/cross-platform.md.

Needs macOS 13 or later and an admin account. On a brand-new Mac, run xcode-select --install first: git clone needs Apple's command line tools.

git clone https://github.com/jatinbishla/Umless.git && cd Umless ./install.sh

Windows

Windows 10 or 11 (x64). In PowerShell:

git clone https://github.com/jatinbishla/Umless.git; cd Umless powershell -ExecutionPolicy Bypass -File install.ps1

The installer offers to install what's missing with winget (Rust, the Visual Studio C++ build tools, cmake, LLVM), builds Umless, downloads the tier you pick, installs umless.exe to %LOCALAPPDATA%\Programs\Umless with a Start menu entry, and starts it in the tray. Hold Ctrl + Win anywhere, speak, let go; Ctrl + Win + Alt is command mode. UMLESS_HOLD_KEY=right-ctrl (or right-alt, ctrl+alt) picks another hold key. No permissions to grant; Windows won't let it type into apps running as administrator.

X11 or Wayland; tested on Ubuntu 24.04 in CI. apt, dnf and pacman are supported:

git clone https://github.com/jatinbishla/Umless.git && cd Umless ./install.sh sudo usermod -aG input $USER # once, then log out and back in: lets Umless see the hold key

The installer installs the build tools and libraries (GTK 3, WebKitGTK 4.1, ALSA) and the typing tools (wl-clipboard + wtype/ydotool on Wayland, xclip + xdotool on X11), builds Umless, downloads your tier, and installs ~/.local/bin/umless with an app-menu entry. Hold Ctrl + Super, speak, let go; + Alt is command mode. Wayland blocks global keyboard hooks, so Umless reads the keyboard devices directly (hence the input group); on X11 it falls back to XRecord without it. On GNOME Wayland, typing uses ydotool (start ydotoold), and per-app styles fall back to Clean because GNOME doesn't tell apps which window is focused. A server without a desktop can build only the CLI: cargo build --release -p umless-cli --no-default-features. GPU offload for the LLM: add --features vulkan (or cuda, rocm) to the build.

The installer builds Umless (installing Rust and cmake first if you don't have them), suggests a tier for your machine, downloads exactly the models that tier needs and, on macOS, adds a local signing certificate to your login keychain (so updates keep their permissions), installs /Applications/Umless.app and opens it. Nothing else to clone or download. The app reads its models from this folder's models/, so keep the folder where it is (or set UMLESS_MODELS before installing). To skip the questions, pass the tier: ./install.sh lite, ./install.sh standard, ./install.sh max, or ./install.sh all.

./target/release/umless models install max # or: lite, standard, all ./target/release/umless models list

Use

Dictate into any app (macOS): hold Fn, speak, let go, and the cleaned text is typed at your cursor.

install.sh already set up the app. After pulling updates or building by hand, rebuild it with:

scripts/make-signing-cert.sh # once: a local signing certificate, so updates keep their permissions scripts/make-app.sh standard # builds Umless.app and installs it in /Applications

Open Umless from Applications or Spotlight. It lives in the menu bar (no Dock icon). A pill at the bottom of the screen shows "Listening" while you hold Fn, with a sound at the start and when the text is typed. The menu has Edit Dictionary… (names and terms spelled your way, applied as soon as you save) and Launch at Login.

The first time, allow Umless under System Settings → Privacy & Security, then quit and reopen it:

Also set System Settings → Keyboard → "Press 🌐 key to" → Do Nothing, so Fn doesn't open the emoji picker too. Pressing another key while holding Fn (Fn+←, Fn+F5) cancels, so Fn's shortcuts keep working. Fn + Control is command mode: say what to write instead of the text itself. With nothing selected, Umless writes at the cursor using what's on screen ("reply saying thanks, I'll send it Thursday" in a Gmail thread). With text selected, it rewrites the selection ("make this more polite", "turn this into bullet points").

It reads the screen, locally: while you speak, Umless reads the text around your cursor through Accessibility. It uses it to spell names the way the thread does (you say "pria", it types Priya), to match the conversation's tone (a jokey Teams chat gets casual text, not formal), and as command mode's context. That text never leaves your Mac and is never saved.

It learns from your fixes: if you correct a name Umless typed, it adds your spelling to the dictionary. Turn either off in Settings. It writes for where you are: an AI prompt in Claude, ChatGPT, Codex, Cursor or a terminal; polished sentences in Mail, Gmail, Outlook and Teams; your own casual voice in Slack, WhatsApp and Messages; plain cleanup everywhere else. Open Umless from the menu bar to see your history, edit your dictionary or fix one style. Your clipboard is restored after each dictation. Models stay loaded; the mic is on only while Fn is held. Log: ~/Library/Logs/Umless.log.

To remove Umless completely (app, permission entries, history, dictionary, settings, log and signing certificate), run scripts/uninstall.sh. Add --keep-data to keep your history and dictionary.

From a terminal, ./target/release/umless app does the same. Grant the permissions to the terminal app instead.

Try it in the terminal (speak, then press Enter):

./target/release/umless listen --tier standard

--tier works on every command. Other commands (umless is ./target/release/umless; to type just umless, run ln -s "$PWD/target/release/umless" /usr/local/bin/umless):

Names, products and jargon get spelled your way on every tier. Put one term per line in a text file and pass it with --dictionary my-words.txt (or set UMLESS_DICTIONARY), or list a few inline with --hotwords "SubFlo,Jatin,lien waivers":

my-words.txt

Jatin Bishla SubFlo lien waivers

The speech model usually hears a rare name right and only spells it wrong ("Subflow", "firman", "Zirodhaa"), so Umless respells near misses that sound the same, before and after the cleanup LLM. When a word keeps coming out as something else, list what you hear after =: Gaurav = Gorov, Barraf. Ordinary words that merely look similar ("former", "subway") are left alone. --hotword-boost 1.5 additionally biases the Parakeet speech model (Lite, Standard) toward the terms; above ~2 it starts inventing them.

Benchmark clips: ./scripts/make-bench-clips.sh (macOS) synthesises 11 clips with references, and ./scripts/record-passages.sh records you reading 6 passages, so umless bench --clips recordings tests your real voice.

Tuning flags: --threads N, --step / --context (streaming), --asr <model dir> (override the tier's model).

How is this different from Wispr Flow and macOS's built-in dictation? macOS dictation transcribes what you said, fillers and false starts included, and doesn't adapt to the app. Wispr Flow cleans up and adapts, but it processes your audio in the cloud and its full version is a paid subscription. Umless does the cleanup and per-app writing locally, with open models you can inspect, swap or fine-tune, and it publishes its benchmarks.

Does anything leave my Mac? No. Models are downloaded once (from Hugging Face and the sherpa-onnx GitHub releases); after that, audio, transcripts and the screen text Umless reads stay on your machine. It has no account, no analytics and no network calls while you dictate. History and your dictionary are plain files under ~/Library/Application Support/Umless.

Which tier should I pick? Standard for most people: 3.6% word error rate, 760 ms typical key-up → text, 25 European languages. Lite if disk or memory is tight (678 MB, English only). Max if accuracy matters more than speed or you need Hindi or another non-European language (1.8% word error rate, 1250 ms typical). ./install.sh suggests one for your machine, and you can add others any time.

Why build from source? Is there a DMG? Not yet. A signed, notarised DMG needs an Apple Developer ID, which is on the roadmap along with Homebrew. Until then install.sh builds it and signs it with a certificate it creates in your login keychain, so macOS keeps the app's permissions across updates.

mic ─► ring buffer ─► 16 kHz ─► streaming ASR ───────────► rules ─► cleanup LLM ─► guard ─► typed at cursor (cpal, RT-safe) (resample) (decodes while you talk) (fillers, (style from (rejects (clipboard + ⌘V,

  • Silero VAD self- the app + invented clipboard restored) corrections,screen text) names/numbers) digits, "?")
  1. Streaming speech recognition, so key-up is cheap. Parakeet is an offline model, so Umless runs it in a LocalAgreement-2 loop: every second it re-decodes the uncommitted window (plus 1.5 s of already-committed audio as left context, snapped to a word start) and commits the words two consecutive decodes agree on, ignoring filler flips like "uh"↔"a". At key-up only the short uncommitted tail is left to decode, so latency doesn't grow with how long you spoke (a 62 s take finishes in 85 ms of ASR on Lite). Whisper has no reliable word timestamps, so Max decodes VAD-cut phrases instead, capped at 4 s so the phrase still open at key-up stays small (8 s → 4 s took Max's p50 from 2,911 ms to 1,250 ms, WER 1.5% → 1.8%).

  2. A safety net for lost words. A decode that starts mid-word can make Parakeet drop seconds of speech for good once the window slides past it. A per-session Silero VAD marks speech as it arrives; at key-up, any stretch VAD heard as speech that no committed word covers is decoded again and spliced back in by timestamp, skipping words already committed and hesitation-only re-decodes. It costs nothing on normal takes (VAD runs while you talk).

  3. Deterministic rules before any model. Fillers and stammers, explicit self-corrections ("Tuesday, no wait, Wednesday", "at ten. Actually, make it eleven" → the replaced word must be the same kind: day, number, determiner…), spoken numbers to digits ($18,750, 15%, 2:30, +91, while "two blocking bugs" stays words), question marks small models drop, misheard work phrases ("let's sing" → sync, "deploy field" → failed). These run on every tier and are what's typed when a model's output is rejected.

  4. A cleanup LLM that writes for the app you're in. The focused app (and, in browsers, the site in the window title) picks the style: a concise, structured prompt in Claude, ChatGPT, Cursor, Codex or a terminal; formal in Mail, Gmail, Outlook, Teams; casual in Slack, WhatsApp, Messages; minimal clean elsewhere. The

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →