Personal agents need a different kind of harness

I’ve been working on building a personal agent that is both useful and secure and have come to the conclusion that it needs a new kind of harness, specifically one without a shell. A personal agent comes with the lethal trifecta by default — in order to be useful, it needs access to your data and the internet, which is full of untrusted content and ways to exfiltrate information. Right now personal agent security, such as it is, relies on hopes and prayers that LLMs will follow your requests to please not get pwned. This is wishful thinking and not a reasonable or responsible security posture, in my opinion.

Traditional security practices break down with agents, because agents are not like people. When you grant a person access to a system, you are trusting their judgment, and a person's judgment is a property of who they are, which is why identity-based authorization mostly works for people. A person who reads a phishing attempt is still the same person with the same decision-making capacity as before they read the message.

Agents are fundamentally different. They have no such judgment or decision-making capacity of their own. They act based on instructions they are given, and anything that enters an agent's context can become part of its instructions. The identity of an agent remains stable over time, but whoever is effectively in control of it does not necessarily. After an agent reads a stranger's email, that stranger is partly directing the agent. This is why identity cannot be the basis of authority for agents. An agent's authority is a function of whatever is in its current context, and what tools it has access to at a given moment.

Enforcing this kind of session-based authority requires the harness to mediate every read and every send. It must be the exclusive broker and monitor of every LLM call -- everything that enters the session context has to pass through the harness in a tamper-proof way so that it can track what information the session holds, and every action has to pass through it, so that it can tell what an action does before it executes it. A secure harness is effectively a reference monitor (see §3.4, p. 16) -- always invoked, tamper-proof, and small enough to verify. The problem is that this is impossible if the agent has a shell. So secure agents cannot have a shell.

Coding agents, like Claude code and codex, are extremely useful because they have broad access to powerful tools and relevant data at the same time. They have access to a shell on your computer, which means they can effectively do anything, and their approach to security is to grant access to everything then try to add guardrails. Things like execution sandboxes, approval prompts, and classifiers. There are many problems with this approach, though. The main one being that it doesn't work. Or at least not all of the time, and as we all know, in security 99% correct is a failing grade.

Prompting for approval leads to fatigue. Anthropic found that people were accepting 97% of approval prompts, which meant after only 50 prompts they were catching just 5% of dangerous commands. To be fair this is partly because their approval prompts are unhelpful. For example this:

is supposed to be the version of my job that no longer requires reading code. That is an inscrutable wall of mangled bash commands. There is no universe where a person can tell whether that is safe to run or not at a glance. Sure you can manually scrutinize it, linting and trying to follow the control flow with your eyeballs, but that is almost certainly slower than just manually doing whatever it is that Claude is asking you for permission to do in the first place.

Classifiers are models judging models, which are useful but also leaky. Anthropic's own "Claude containment" doc notes that auto mode blocks "roughly 0.4% of benign commands", at the cost of letting ~17% of overeager actions through. Johann Rehberger of Embrace The Red got malicious code execution past auto mode with up to 80% success with some pretty basic techniques. Notably, the first step is getting claude to decide not to use a tool and to use curl instead. This is why your agent cannot have a shell.

It also goes to show why you cannot trust the frontier labs to self-police. Anthropic's own commissioned test showed 0% success in 720 attempts to hijack auto mode. Their response to Rehberger's report was that auto mode is a best-effort classifier, not a security boundary. IMO it's quite disingenuous to market 0% attack success and then call an attack that succeeds 80% of the time out of scope, but I digress. In their response they say that OS level isolation and network egress control are the real boundaries, which we we'll see, also don't work.

The standard answer to this is "just run it in a sandbox". The first problem is that if your agent has access to a shell inside a sandbox, information you are trying to keep inside will get out.

To be clear, you definitely need a sandbox, but it solves a different category of problems. Sandboxes limit what an agent can reach, but don't track what information the agent has access to. They are necessary but not sufficient.

Two days after Cowork launched, PromptArmor got it to upload a user's files to the attacker's Anthropic account through Anthropic's own Files API, using a key planted in a file. The exfiltration used also used curl from a shell. The request went to api.anthropic.com, which was on the allowlist because the agent needs it. Anthropic's own account of the incident said "The sandbox worked perfectly, and yet the data was exfiltrated." It's also worth noting that Rehberger had reported this same bug to claude.ai earlier and his report was, as ever, ignored. EchoLeak, found by Aim Labs, got data out of Microsoft 365 Copilot through an image URL on a Teams domain that Copilot's content security policy trusted.

If you want your agent to be useful you simply have to give it access to some powerful tools and even if you only allow the bare minimum, those channels will leak.

For a person, permissions can be identity based, because your judgment remains your own, at least to a first approximation over the short term. But agents don't have any judgment or taste of their own, they follow instructions and relentlessly pursue whatever objectives they are optimized for. What an agent can safely do depends on what's in its current context.

Most agents today run on what capability-security researchers call "ambient authority", i.e. authority derived from the identity of the program rather than from the request it's acting on. This leads to the classic Confused Deputy from Norm Hardy's 1988 paper, in which a compiler allowed to write to its own directory overwrote the billing file kept there, because a user handed it that filename as its output. This is effectively the same class of bug as prompt injection. It is far too easy for outsiders to inject instructions into an agent's context today, tricking the agents into using your authority to carry out their requests. The fix from this line of research is to derive authority from the request rather than the agent.

The idea that what you've read limits what you may do is also very old. Kenneth Biba's 1977 integrity model (starts p. 13) includes a low-water-mark policy, where a subject's integrity drops to the lowest level of anything it has read. Meta's Agents Rule of Two is a similar idea applied to agents, saying an agent may hold at most two of the lethal trifecta's properties within a single session, and needs supervision once it has all three. These are both precedent for making authority a property of the session, not of the agent.

To work out an agent's authority from what its session has seen, the harness has to know two things about every action: what it reads and what it sends. A shell makes both of these inscrutable to the harness. cat, curl, git push, rm -rf, a DNS lookup, or a file write to a synced folder all look the same on the surface (a string handed to bash). Even if you restrict the commands a shell may run, many commands can both read and send, e.g. fetching a URL can exfiltrate data via query params. And there are countless ways a shell can move data from disk to network without ever passing through the model's context, so watching the context isn't sufficient. Mediating these calls has to happen beneath the model, in the harness itself.

In principle, the harness could police every read and send a shell does, but it would have to trace every file, process, and socket each command touches and propagate labels through all of them. This is effectively building an information-flow operating system under the agent, which researchers have done, many times (see Asbestos, HiStar, and Flume). In practice, at that granularity the label creep cripples the usability of the system almost immediately, and the decisions it surfaces are too low-level to be meaningfully approvable anyway, like "process 4121 is trying to connect to 104.18.x.x". This is the inscrutable approval prompt problem again, just at an even lower level.

The fundamental problem is that a harness with a shell is trying to answer "what does this command actually do?" before it runs, and Rice's theorem proved in the 1950s that for arbitrary programs that question is undecidable.

Rehberger's attack is a good example showing how difficult it is to secure a tool as generic and powerful as a shell. The fatal command auto mode approved was a short Python decoder he had tricked Claude into writing itself, after refusing to run the binary decoder he supplied. Superficially, this is harmless. It was only a problem because it ran from inside an archive it had just unzipped, where he had planted a struct.py that shadowed the standard library's, so Claude running import base64 inadvertently ran the malicious code. In some runs, auto mode even blocked Claude's attempt to kill the malware after Claude noticed it. Every step seemed reasonable and passed the auto mode classifier on its own, but that's the thing -- most shell commands do.

What a shell command does depends not only on its contents but also the current state of the filesystem and environment it runs in, which an attacker can largely control. Even with perfect knowledge of that state, the question of whether a given command is safe to run or not is undecidable in general. The best a harness can do is over-approximate, which in the context of a shell means assuming every command might read anything and send anywhere. You could restrict a shell to a decidable subset, like a fixed set of commands with checked arguments, but that is effectively rebuilding typed tools with worse ergonomics.

So a harness enforcing session-based authority through a shell just cannot do it automatically. All that's left is asking the model to please be careful, or asking you for permission a thousand times a day.

Since the shell is the problem, the agent cannot have one. But that means that in order to do anything, it needs access to tools, and the harness has to be the only way it can access those tools. So far these are the requirements that fall out of all this:

The harness must mediate every action. The agent can’t have a shell or side channels or generic ways of accomplishing the things the tools do, like using curl to bypass a web fetch tool. The only way for an agent to read or act has to be via tool calls that the harness mediates, because the harness can only enforce rules on what passes through it.

Every tool must declare its effects. A tool needs to declare whether it reads, writes, sends data out, or acts in the world, and what its output contains. A tool that doesn't follow this contract cannot be trusted to run.

Labels follow the data and the agent can only accumulate them. Anything private is marked as such, anything an outsider could have written is marked as untrusted. The system also must not launder labels through summaries or copying or