The Paradox of Contemporary AI: Engineering Success and Institutional Failure

OpenAI released GPT-3 in June of 2020, though not to the general public. Those who had access to it were variously impressed or stunned: Is this it? Is AI really coming at long last? Though I was not in the direct access circle, I had indirect access and was deeply impressed, using that access in an interview with Hollis Robbins that I published in 3QD on July 20, 2020: An Electric Conversation with Hollis Robbins on the Black Sonnet Tradition, Progress, and AI, with Guest Appearances by Marcus Christian and GPT-3. That August I issued a working paper: GPT-3: Waterloo or Rubicon? Here be Dragons. I stuck this at the top of the first page:

But I fear the community that has created it may, like other communities have done before – machine translation in the mid-1960s, symbolic computing in the mid-1980s – triumphantly walk over the edge of a cliff and find itself standing proudly in mid-air.

This is not necessary and certainly not inevitable.

As far as I can tell, the AI industry is now well out over the edge of that cliff. While I see prospects for a brilliant and productive long-term future, I fear that the next decade or two will be chaotic.

Here’s how ChatGPT put it to me in a recent dialog:

The engineering success is real. LLMs [large language models] and related systems have given us access to a new conceptual continent. They work, and at extraordinary scale. But the institutional failure lies in the monoculture: too much intellectual, financial, and training-path dependence on one family of architectures and one style of thought about intelligence. The result is that we are building out the utility before we have adequately explored the space of possible successor technologies or developed the conceptual tools needed to understand what these systems are revealing about language, cognition, and cultural structure.

The paradox we are facing, then, is that the unexpected and extravagant success of the transformer technology has resulted in a furious maelstrom of entrepreneurial, investment, and commercial activity that may inhibit the further development of that technology.

The story is a complicated one, but I can convey its basic shape with two analogies. The European discovery, exploration, and settling of North America can give a rough sense of where we are and how far we have to go in the development and use of AI technology. The structure of knowledge and skills involved in whaling, seamanship vs. whale hunting, can give a rough sense of the institutional failing.

When Columbus set sail he was expecting to reach India. When he struck land in the Bahamas in 1492 he was a bit south of an entirely different continent, North America. Roughly three centuries later much of the eastern part of the continent had been settled by Europeans and their descendants. Late in 1805 Lewis and Clark reached the Pacific Ocean near the mouth of Columbia River in Oregon. It took a century after that before European Americans had more or less settled the entire continent.

Let’s say that OpenAI’s release of GPT-3 corresponds to Columbus first setting sail in August of 1492 while the release of ChatGPT in late November of 2023 corresponds to Columbus landing in October of 1492. Within a relatively short time tens of millions of people had hooked up to ChatGPT. By the end of the year artificial intelligence was no longer little more than the dream of relatively small group of engineers and scientists. It had moved from the world of science fiction into what is colloquially known as the real world. Just how real it was and is, that became the subject fierce debate, and remains so. The debate has become widespread and unavoidable.

Within roughly a year and a half, as Claude noted in an article I recently published (“The Shock and the Narrowing”), “capital, talent, and institutional attention all funneled toward a single paradigm,” a computing monoculture based the kind of technology that powered the GPTs. The idea was to scale it up: more compute (memory and processing), bigger models (more parameters), and more data. Then build-out the infrastructure and ship products. Claude continues:

Google, which had invented the transformer architecture in 2017, was caught flat-footed and scrambled. Meta pivoted its AI strategy around LLMs. Microsoft integrated OpenAI’s models into its core products. A hundred startups raised money to build on top of the new foundation models. The venture capital flowing into AI, measured as a share of total U.S. deal value, went from 23% in 2023 to nearly two-thirds in the first half of 2025.

The infrastructure investment that followed is staggering by any historical standard. The four largest hyperscalers — Amazon, Google, Microsoft, and Meta — are expected to spend more than $350 billion on capital expenditures in 2025 alone, most of it AI-related.

To return to our crude analogy, this monoculture believes that all we need to do to reach the Pacific Ocean is to build more of the same. We have all the fundamental ideas we need.

One reason for this is that most of the scientists and engineers currently involved in machine learning and AI are relatively young. For them, the triumph of AlexNet in image recognition in 2012 is the setting-foot-on-a-new-land event. AlexNet’s triumph was achieved through scaling up enabled by using graphics processing units (GPUs). If that’s your starting point, then it is easy to see why you would believe that scaling up would take LLMs across the continent to the Pacific Ocean and that it’s going to happen fairly soon.

My time horizon is much longer and encompasses a wider range of technologies. I did research on Old School computational semantics in the 1970s and saw the so-called AI Winters of the 1980s. Moreover my teacher, David G. Hays, was a first generation researcher in computational linguistics in the 1950s and saw that field defunded in the mid-1960s for promising more than it delivered.

Thus I have a different view. I’m hardly alone. And some within the industry are beginning to realize that large language models like those powering ChatGPT, Claude, Gemini, Grok, and others may be limited technologies. As Claude put it in that article:

Gary Marcus has been making this argument for years, often dismissed as a contrarian. But the intellectual tide has shifted. Ilya Sutskever — the man who arguably did more than anyone else to build the scaling paradigm that produced GPT-3 and GPT-4 — left OpenAI in May 2024 and founded Safe Superintelligence, a company explicitly premised on the idea that the current path does not lead to genuine intelligence. Fei-Fei Li’s World Labs, which raised $1 billion, is pursuing spatial intelligence and world models — the ability to understand and predict the physical world, which LLMs trained on text cannot do. And Yann LeCun, in a January 2026 interview with MIT Technology Review given from his Paris apartment shortly after leaving Meta, was direct: “The breakthroughs are not going to come from scaling up LLMs. The most exciting work on world models is coming from academia, not the big industrial labs.”

That is to say, LLMs will not take us to the Pacific Ocean, much less support our efforts to settle the continent from coast to coast.

They believe, as do I, that we are going to need some fundamentally new ideas to make it across the continent. A number of ideas are already visible, but are not yet fully developed. The problem is that so much attention is focused on, and so many resources are being devoted to, the transformer-based LLM paradigm that it is not at all clear how and whether or not these other architectures can be brought to maturity. The problem does not lie entirely with industry. We must also consider the educational “pipeline,” undergraduate and graduate, that funnels people to the industry.

The title of a 3QD article I published in December 2023 states the analogy: “Aye Aye, Cap’n! Investing in AI is like buying shares in a whaling voyage captained by a man who knows all about ships and little about whales.” The skills required to navigate and handle a ship are real, and they are necessary to any whaling voyage. But they are different from the skills needed to find, successfully pursue, and ultimately to kill whales and extract the oil and ambergris. While one can imagine, as an abstract possibility, that such a voyage might have happened, as far as I know, no real whaling expedition of that nature ever took place. It’s too absurd.

Contemporary AI technology, however, is a very different beast, one where a distinction of that kind can be made without, however, a deficiency of knowledge about whale behavior being recognized as such. Contemporary chatbots can generate very sophisticated texts, but you don’t need to know much of anything about language and linguistics, or cognition, much less rhetoric, discourse, structure or literary criticism, in order to create the bots. When a Gary Marcus points to the limitations of large language models, he does so because he has studied language and cognition. He has a deeper understanding of the human mind that do most of the experts who have given us the current AI revolution.

“But, wait,” you might be thinking, “don’t the engineers who design automobiles, for example, don’t they understand how cars work? Do they not grasp the thermodynamics of exploding gas inside a small chamber? Do they not grok how the reciprocating motion of the pistons can be mechanically transformed into rotary motion and how the transmission and drivetrain transfer that motion to the axels? Then there’s the friction between the tires and the road, and between the air and the car’s exterior, not to mention the steering mechanism, the brake mechanism, the windows, the trunk, the air-conditioning, and so forth. Don’t they have to understand these things in order to design and build cars?” Yes, they do. But language technology is different from traditional electro-chemical-mechanical technology, very different.

Just why that is so is a difficult and tricky business, something well beyond the scope of this essay. But I can make some observations. Last month I published “Chess and Language As Paradigmatic Cases for Artificial Intelligence.” My point was simply that they present very different computational problems, but I didn’t say much about the computational techniques used in each case. For roughly the first three to four decades, from the 1950s into the 1990s, the computational techniques were analogous to those involved in automobile engineering in the sense the people who designed and wrote the programs did so on the basis of some understanding of how chess and language worked. To construct a chess program you need to know, not only the rules of chess, but chess strategy and tactics: How do you evaluate a board position? How do you plan a sequence of moves? Similarly, to construct a program to translate a text from one language to another, or to answer questions, you need have base your program on some theory of language and even cognition. If you want to build a program that tells small stories, a standard task for the last quarter of a century – something that David Ferrucci worked on and then went on to lead IBM’s Watson project – you need to have some theory of story structure, something AI researchers picked up from folklorists and narratologists.

This older style (which I previously referred to as Old School) is generally known as symbolic computing because it is about computing with symbols, such as language strings, logical propositions, or ordinary arithmetic, with its Arabic style numerals and limited set of operators (addition, subtraction, multiplication, division, and equivalence). More informally, and with a touch of disdain, it is known as GOFAI (Good Old-Fashioned Artificial Intelligence). The newer style is statistical in character, with artificial neural networks such as large language models being a specific kind of statistica