Achieving AI-style reasoning without an LLM

In the last post we introduced Minotaur, an AI that reasons with no language model and no GPU. This post looks at why language models think the way people do, at what was tried before them and where each attempt stopped, and at how Minotaur’s brain does the same kinds of reasoning differently: not by imitating how people write their thinking down, but by working on the structure of the system it’s reasoning about.

In the last post we introduced Minotaur, an AI architecture built on ELC that works out its answers by measuring rather than predicting. We showed it checking real regulations against records of how they were applied: finding which organisation broke a rule, by how much and where, and telling a broken rule from a badly recorded measurement.

Checking rules against records is one kind of thinking. Today’s language models are admired for something broader: they can deduce, explain, diagnose, weigh a hypothesis, imagine what would have happened if things had been different, and change their minds when told something new. Can a machine with no language model, and no GPU, do all of that?

A large language model is trained to predict the next word of a piece of text. It learns this from an enormous amount of writing, from books, papers, textbooks, forums and code, by doing it trillions of times on thousands of GPUs.

To predict the next word of a proof, you have to follow the proof. To predict the next line of a doctor’s notes, you have to follow the diagnosis. Much of the writing in the world is people thinking on paper, so a model that gets very good at continuing it ends up very good at imitating the thinking. That’s why simply asking a model to “think step by step” makes its answers better.

So a language model mirrors the way people think. It inherits our strengths: explaining, drawing analogies, using common sense, taking a question in ordinary words. Researchers have also found that it inherits our weaknesses. Like people, language models find a logical argument easier to accept when its conclusion sounds believable, and harder when it doesn’t. Their answers can shift with how a question is framed or the order things are put in. And like ours, its explanations are often written after the fact, not a record of how the answer was reached.

The model never sees the world, only our words about it.

It does mean that if you want reasoning that can be checked, replayed and certified, imitating the way people write their thinking down may not be the best place to start. Exact, checkable reasoning was what most approaches to AI before language models were trying for.

People tried to build machines that reason for some sixty years before language models. Each attempt worked up to a point and then stopped, and where they stopped explains why we ended up with language models.

In 1956 Allen Newell, Cliff Shaw and Herbert Simon built the Logic Theorist, a program that proved theorems from Whitehead and Russell’s Principia Mathematica. Their next program, the General Problem Solver, tried to do the same for any problem that could be written as a starting point, a goal and a set of moves. The idea was that reasoning is search: try the possible steps until you reach the goal.

It worked on puzzles. On real problems the number of possible steps grows so fast that no computer could search them. In 1973 the Lighthill report to the British government named this the combinatorial explosion, and funding in Britain was cut sharply.

The answer in the 1970s was the expert system, which gave the machine knowledge instead of search: hundreds or thousands of hand-written rules of the form if this, then that, collected from human experts. MYCIN, built at Stanford, recommended antibiotics for serious infections, and in a 1979 evaluation its advice compared well with that of specialists. XCON, at Digital Equipment Corporation, configured computer orders and saved the company a great deal of money.

They stopped for two reasons. Every rule had to be got out of an expert’s head and written down by hand, which came to be called the knowledge acquisition bottleneck. And they were brittle: just outside what the rules covered, the system had nothing sensible to say. By the end of the 1980s the market for them had collapsed.

Logic programming aimed to have the machine reason exactly, in a proper logic. That was the bet behind Prolog, created in Marseille in 1972, and behind Japan’s Fifth Generation Computer Systems project, which ran from 1982 to 1992 and was built around logic programming.

Logic gives exact answers, and its descendants, the solvers that check whether a set of conditions can all be true at once, are now used every day to verify computer chips. But logic only reasons about what someone has written down as logic. The world doesn’t arrive in that form, and the Fifth Generation project ended without the thinking machines it had promised.

In 1984 Douglas Lenat started Cyc, an attempt to write down, by hand, the common sense that expert systems lacked: that water runs downhill, that people have mothers, that a thing can’t be in two places at once. Over the following decades its team entered millions of such facts.

It showed how large the common sense people take for granted is, and that writing it down by hand never comes to an end.

From the late 1980s, led above all by Judea Pearl, reasoning under uncertainty got a proper mathematical footing. A Bayesian network says which things affect which and how strongly, and lets a machine work out what an observation implies. Pearl went on to make precise the difference between seeing something and doing it: seeing a wet pavement suggests it rained, but hosing the pavement yourself tells you nothing about the weather. In the same years, Raymond Reiter, Johan de Kleer and Brian Williams turned diagnosis into a precise method for finding which component of a system has failed.

These are some of the best ideas in the field, and Minotaur reproduces them. But someone has to hand them the model first: the causal graph, or a description of every component. Given only a column of numbers, they can’t begin.

From 2012, neural networks trained on large amounts of data, known as deep learning, began to beat everything else at recognising images and speech. They need nothing handed to them except examples.

What they learn is a pattern, with no structure to reason on. A network that has learned how a production line behaves can predict its output. It can’t tell you that one of its inputs is a safety cut-off, or answer the question “what if that cut-off were stuck open?”, because there’s no cut-off anywhere inside it to be stuck.

Language models, built on the transformer design published in 2017, reached the public in 2022, and from 2024 began to be trained to reason step by step before answering. Language models won because they’re the one approach that needs nothing handed to it. You don’t write a model of your problem for it; you describe the problem in plain words, and it reads them.

The price is the one we discussed in the last post. The reasoning runs through hundreds of billions of weights on data-centre GPUs, and the answer comes with no certificate of how it was reached.

Each way of getting a machine to reason has had a price:

Minotaur sits in the middle. It can be given a description or learn one from data, puts it into typed roles it can reason on, and reasons exactly enough to certify each answer, on one ordinary processor core.

The ideas behind ELC are set out on the MINOS page, and the last post gave the short version. Every system, whatever it’s about, is read in three roles:

E and C never connect directly. A drive doesn’t say what is allowed, and a limit doesn’t say which drive it answers. The relation between them always has to be supplied, and supplying it is what the system’s logic is.

Each element of a system is read as a share. Of what arrives at it, some passes and some is held back, and the two always add up to the whole. At one end everything passes, at the other everything is held, and in between there is a balance point.

This is a minimal representation of a system; MINOS stands for the minimal ordinal representation of systems. A whole boiler can be read as a single element: something drives it, something constrains it, and its logic relates the two. When more detail is needed, the same reading is applied to its parts. The boiler becomes a pump, a fan, a burner and a gas valve, each an ELC element with its own drive, constraint and logic, connected so that what one element passes on becomes the next one’s drive. The description gets finer, but it never leaves the ELC format: it’s the same three roles at every scale, just more finely divided and more interconnected.

On this representation we built Minotaur’s brain.

A brain that reasons in general turned out to need two kinds of faculty, best kept apart, a little like the two hemispheres of a human brain.

The ELC side holds the system being reasoned about, as a network of ELC elements. It has exactly one operation, settling, described below. It can also learn a system it has never been told about, from records of what that system did. By design, it contains no counting, no exact arithmetic and no matching of text. Its results are measured: the network came to rest, or it didn’t, to within a stated tolerance.

The exact side is a calculator. It reads a description of a system, lists every case when every case has to be checked, does exact arithmetic and checks results. Its results are exact: true in every case, or here is a case where it isn’t.

Nothing passes between the two sides except through a named, one-way crossing:

Every crossing is recorded, along with the certificate it carries. When an answer rests on several steps, it carries the weakest certificate among them.

That record is also the brain’s chain of thought. When a language model shows its reasoning, it writes a new piece of text. When Minotaur shows its reasoning, it shows the crossings it actually made, in order.

Picture the network as a set of nodes, each holding a share between “all passed” and “all held”, joined along the lines ELC allows: drives into logic, constraints into logic, logic on to logic and to constraints. Never drives straight to constraints, or back.

To ask a question, you say what each node is doing:

The rest follow their laws. Then the network settles, a little like water finding its level:

If the stress falls to almost nothing and every fact reads back as it was held, the world closes: there’s a way for things to be that fits everything given. If no setting of the holes will close it, the stress says where the contradiction sits.

Every kind of reasoning is the same settle, asked differently. What changes is only which nodes are held, which are cut and which are left free.

The ELC side has no special code for any of these kinds of reasoning. Where an answer needs every case checked, as deduction does, the exact side lists the cases and the ELC side settles each one. So deduction gets an exact certificate, because no case is left out.

All of the systems below were written in a simple format that lists a system’s drives, constraints and laws. The first two were written by us for development. The others were written by people who never saw Minotaur’s code, and we come to how they were tested afterwards.

Start with a house’s lighting. The supply comes in through a main breaker and splits into a hall circuit and a kitchen circuit, each with its own fuse, switch and bulb, and the kitchen circuit also feeds a socket and a kettle. All the switches are on. The kitchen lamp is lit and the hall lamp is dark. What has failed?

The exact side supposes, one at a time, that each part has failed, starting with the smallest possibilities. For each supposition, the ELC side settles the network and reports whether the world closes. Supposing nothing has failed, it doesn’t close, and the