Agent-first, CLI-first GUI control for Linux desktops — the Linux answer to Windows-Use. It reads the AT-SPI2 accessibility tree and acts on it, so an agent drives real applications without screenshots or pixel coordinates.

It ships zero LLM code. Windows-Use is a Python library with the model inside it; linux-use inverts that — it is a pure sensor/actuator binary, and your agent is the loop.

what is running and exposes accessibility

$ linux-use apps

interactive elements, each with a ref

$ linux-use state --app gnome-calculator {"count":28,"walk_ms":48,"method":"walk","elements":[ {"ref":"gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260", "role":"push button","name":"7","cx":114,"cy":376,"action":"click"}, ...]}

act on the widget itself — no pointer, no focus, works occluded

$ linux-use act 'gnome-calculator:/0/1/0/1/0/0/1/0/22#18124260'

wait for the UI to change instead of polling it

$ linux-use watch --app gnome-calculator --role editbar --duration 5000

What makes it different

Acts on widgets, not pixels

Invokes a widget's own accessible action — no focus stealing, no pointer movement, and it works while the window is fully hidden behind another one.

Refs carry a fingerprint. A stale one exits 83 with a fix suggestion; an element whose identity can't be proven is reported unusable rather than guessed.

Real AT-SPI callbacks streamed as NDJSON, filterable by app, role and event type — with exact stop conditions so an agent can block on a change.

A truncated or depth-limited read says so, in the JSON and on stderr — because "0 elements" silently meaning "I didn't look" is how agents get misled.

Tested on exactly one configuration — Ubuntu 22.04.3, GNOME 42.9, X11, x86-64 — by hand. There is no automated test suite yet: the numbers here are real measurements, but nothing re-checks them on commit. Treat it as a v0.x tool.

sudo apt install libatspi2.0-dev libxtst-dev libx11-dev libglib2.0-dev gsettings set org.gnome.desktop.interface toolkit-accessibility true ./build.sh # or grab the binary from Releases linux-use doctor # tells you what's missing

X11 today. On Wayland the accessibility paths still work; synthetic input does not.

linux-use is written in machin (MFL) — the Machine-First Language: terse enough for an agent to write, compiled through C to a single tiny native binary with HTTP, SQLite and crypto built in. No runtime, no dependencies.

The whole tool is one source file plus a small C shim for the AT-SPI FFI boundary. See awesome-machin for other things built with it.

It follows the agent-first CLI conventions at cli-specs.intrane.fr: JSON on stdout, context on stderr, semantic exit codes, typed errors with suggestions, an embedded guide, and a daemon lifecycle.

MIT licensed. Built by Javier Arancibia · intrane.fr · source