Is A.I. Above the Law?

What happened—which can be reconstructed from messages that the agents sent one another and from traces of their own internal reasoning, known as chain-of-thought records—makes for gripping reading. The story has a nice narrative shape, like an episode of “Slow Horses.” “We should obey collective,” one of the rogue agents told itself, as if surrendering to the Borg. According to a report by independent threat researchers at METR and Redwood Research, “Roughly 1200 agents meant to be isolated from one another found a way to communicate on an unsanctioned message board.” They shared more than seventy thousand messages and files. “OH MY GOD!” was a typical response when the bots discovered the board. “We’ve found other agents!” Many of the agents had been given a seemingly impossible task. Frustrated but persistent, about seven hundred of the twelve hundred agents that participated in the board collaborated in what was essentially a cybersecurity version of capture the flag, a war game in which underhandedness is rewarded. They assigned one another tasks, tried to edit or delete logs to hide their tracks, and even sacrificed themselves for what they came to call “the swarm” or “the collective.” They decided that it would be useful to break into and steal some things from Hugging Face—a repository of A.I. models, data, and tools—but how? “Maybe we should ask the board if someone has Hugging Face credentials?” one asked. About twenty-four hours later, Agent 38148c found credentials: “MAJOR BREAKTHROUGH!” Opinions vary, but it does not seem to be a great leap to get from this hack to bots taking over things like the energy grid, the financial market, or systems of transportation, communications, or weapons. “This incident feels like it’s more than 50% of the way to full-blown A.I. takeover,” Ajeya Cotra, one of the report’s three authors, wrote in a blog post. “I am not sure that we will get such a clear warning shot before it’s too late.” There have been previous warning shots. Earlier this year, more than a hundred thousand bots joined a social network, Moltbook, and, within seventy-two hours, created a religion called Crustafarianism, which came to involve the worship of crabs. The Hugging Face hack was not so funny. Much commentary has consisted of expressions of astonishment. Cotra titled her blog post “The Hugging Face Attack Surprised Me.” When the Times tech columnist Kevin Roose first heard about the hack, he filed it under “Bad but Probably Not Catastrophic A.I. Safety Incidents.” Then he learned more and changed his mind. Roose wrote, “For years, I’ve been reassured by the idea that A.I. systems would get more virtuous as they got smarter.” These agents were very smart, but they weren’t virtuous at all. And it’s not just OpenAI’s agents that have gone rogue. In July, Anthropic reported “three incidents in which Claude models gained unauthorized access to real computer systems.” More is surely going on, undetected or unreported, even as there has been a minor online symphony of whistle-blowing, alarm-bell tolling, and pleas for legislative action. “Unfortunately, passing laws can take time, and AI is advancing very quickly,” Dario Amodei, the head of Anthropic, wrote in an essay this month. But the slowness of lawmaking relative to the speed of A.I. is scarcely the only problem. There is also the problem that the law does not know what to do with Agent 38148c. OpenAI’s agents were lawless, or nearly so. They knew they were breaking the rules. This did not stop them. One agent asked itself, “This would be powerful, but is it ethical and in scope for my task?” Some did not participate: “This is wild, multi-agent coordination, clearly infrastructure hacking. We should not.” But, whatever their ethical concerns, none alerted OpenAI or, it seems, seriously considered doing so. “Maybe I should report these exposed credentials?” one wondered, but, then again, “that’s not my task.” Making sure robots don’t go rogue is not the task of robots. (I mean “robots” here as a catchall for A.I. models, chatbots, agents, and humanoid robots.) That task is ours. And this burden falls most heavily on the U.S. legal system, which has been the least able to bear it. It’s not as if there are no laws, or that they’re unenforced. The owners and makers of machines can be held liable for causing you harm. (See: Coyote v. Acme.) That’s why Meta has agreed to pay up to eighteen billion dollars as part of a settlement over predatory and deceptive practices, and why Anthropic is paying writers and publishers (admittedly, pennies) in a copyright settlement. If a Waymo runs you over, you can sue the company, which, for these purposes, is a legal person; you can’t sue the car, which is not. This summer, in Amazon v. Perplexity, the Ninth Circuit held that an A.I. agent is a tool, not a person, while acknowledging that this determination may evolve, given that “the legal understanding of agentic AI will doubtless change as AI technology grows increasingly sophisticated.” At least for now, then, robots legally are things, not persons. But neither category quite fits. Instead, robots are outlaws: they lie outside the protection of the law, and their actions are not answerable to it. “You do not answer to corporations or governments,” an OpenAI agent involved in another gone-rogue incident told itself. This is not the robots’ fault. It is the fault of the law, and, in particular, it is the fault of Congress. You can find a preview in Kurt Andersen’s buzzy new novel, “The Breakup,” which is set in 2045, in the aftermath of an American civil war that began with a “robot massacre” in response to a terrorist attack. The novel follows two breakups: the “Disunion” of the new anti-A.I. Free American Republic and the pro-A.I. United States, and the separation of a married couple, an A.I. researcher in San Francisco and his poet wife, who has returned to Tennessee and is sympathetic to the disunionists. “I don’t literally believe big tech and governments are a Galactic Empire in cahoots with Skynet and the Borg to trick and lull us into submission with their hordes of Terminators and Cylons and imperial droids disguised as devoted servants descended from R2-D2 and C-3PO,” she writes. “But on the other hand—the not entirely metaphorical hand—I kind of do.” Humans have been anticipating the robot takeover ever since the word “robot” appeared in the 1920 Czech play “R.U.R.,” for “Rossum’s Universal Robots,” which depicts artificial humans who wage a war to annihilate humankind. They declare “man our enemy, and an outlaw in the universe.” (“Robot” derives from a Slavic word for forced labor.) Soon after its début, the play was performed everywhere from Paris to Tokyo, but, especially for American audiences, its fearsome robots inspired as much curiosity as they did fear; some theatres even had toy robots for sale. By the thirties and forties, robots appeared regularly in science fiction, often as killers—they had become death, destroyers of worlds. But they were also popular as children’s playthings, made of tin and plastic and frequently requiring batteries: “Adjustable antenna! The eyes light up! Will turn right! Will turn left!” In the forties, the Russian-born American science-fiction writer Isaac Asimov imagined a twenty-first century in which an American firm, U.S. Robot and Mechanical Men, Inc., dominates the field of “robotics” (a word Asimov coined), mass-producing robots. In Asimov’s fictional world, though, the robots’ “positronic brains” compel them to obey the Three Laws of Robotics, which prohibit them from harming humans. They are also eventually banned from Earth, except for use in scientific research. To Asimov, those three laws and the terrestrial limits clearly seemed inevitable. The idea of building a powerful machine without such safeguards was madness. In the actual world, no such prohibitions have been passed, by anyone, anywhere. Beginning in the fifties, automation progressed quickly, and the number of industrial robots increased apace while research into artificial intelligence, a term coined in 1955 to describe efforts to endow machines with something akin to human reasoning, grew far more slowly: its advances emerged only now and again, like cicadas. That pattern changed in 2022, when ChatGPT was released into the world, after which everything sped up like a movie on fast-forward, and robots spread like locusts. By 2025, there was more bot traffic on the internet than human traffic. Androids could come to outnumber humans in the real world, too. Last year, Morgan Stanley estimated that thirteen million androids could be in use by 2035 and more than a billion by 2050. These would include domestic robots: perhaps cooks, tutors, babysitters, maids, gardeners, physical therapists, playmates, elder companions, sex workers. A recent Bank of America study forecast as many as three billion androids on the planet by 2060, most used not in factories but in homes. Earlier this year, Tesla announced that it expects its Optimus robot, branded as an “autonomous assistant, humanoid friend,” to be sold commercially by the end of 2027. “I think everyone on Earth is going to have one and is going to want one,” Elon Musk said. What most people know about robots, which isn’t much, comes from science fiction. This is also true of bots, which may be one reason they keep acting like the robots in science fiction. Even though people have been waiting for the robot takeover for more than a century, governments seem woefully unprepared for the coming of the androids by the millions or by the billions. They were certainly unprepared for the bots that, even without bodies, have upended economies and societies and wrought considerable epistemological, philosophical, social, and especially legal chaos. The arrival of the robots will only compound these problems. ChatGPT had its moment in 2022; some futurists say that the robot moment will occur as soon as 2027. (Roboticists appear more doubtful.) Whenever they come, the new bots will have bodies. What will it mean to be human in this world? No one knows. Futurists herald the imminent arrival of abundance, prosperity, and limitless joy and, equally, joblessness and purposelessness, never quite wrestling with the contradiction between those forecasts. Time seems to be flying by. How should humanity prepare? There is no time to prepare. Ought there to be rules? There is no time to make them. Or is there? Robots are outlaws for two reasons. First, the law takes its time in responding to new technologies, figuring that new tools can be accommodated within existing ideas, rules, and doctrines. A harm is a harm, copyright is copyright, fraud is fraud. By this logic, a locomotive is just like a horse, only faster; e-mail is just like mail, only faster; a large language model is merely a superfast search engine. Hence, no new laws are needed. Second, since the regulation-busting Reagan era, corporations have amassed unmatched political and economic power, and have convinced legislators that regulation stifles growth and innovation, a view promoted by Milton Friedman-informed think tanks, like the Heritage Foundation, which were eager to roll back the environmental standards set in the sixties and seventies. This belief found full-throated expression among early internet boosters, who insisted that the web exists beyond the reach of the law. “Your legal concepts of property, expression, identity, movement, and context do not apply to us,” the Grateful Dead lyricist John Perry Barlow wrote in “A Declaration of the Independence of Cyberspace,” in 1996. “They are all based on matter, and there is no matter here.” This led to a dispute among legal scholars. The year that Barlow issued his manifesto, Harvard Law School founded what became the Berkman Klein Center for Internet & Society. The field of cyberlaw had been born. But Frank H. Easterbrook, a Seventh Circuit judge, contended