Out of My Other Mind – What are the defining characteristics of a conscious AI?

The Official ACM

Oct 28, 2026

4

2

Share

by Jim Waldo

In many of the discussions of AI agents exhibiting surprising or unwanted behavior, the agents are talked about as though they had intentions, beliefs, plans, and ulterior motives. All of these are associated with consciousness, and sometimes there are claims that these AIs have, in fact, exhibited at least limited consciousness. But how could you tell? This is a question that has been discussed by philosophers for far longer than there have been computers.

The most famous discussion having to do with computers showing consciousness centers on the Turing test. Proposed by Alan Turing in his 1950 article Computing Machinery and Intelligence (published in Mind, a philosophical journal), the simplified version of the test is that if you can converse with a computer (in whatever form) for a period of time and not be able to know if you are talking to a computer or a human, then you can consider the computer to be conscious. The core idea is that consciousness is discovered by behavior; if something acts like it is conscious, then it is.

By this definition, the program Eliza would have to be considered conscious. Developed in the 1960s, Eliza was a simple program designed to interact like a Rogerian therapist (also known as a person-centered therapist). Intended to show how AI was not conscious, it surprised its creators by having just the opposite effect, with many users revealing far more than they should have to a machine. The simplicity of the program, which just asked questions based on certain keywords or echoed back statements made by the user, perhaps revealed more about the trusting nature of humans than it did about the mental state of the machine.

A more principled philosophical response to the adequacy of the Turing test came in the 1980s from John Searle, whose “Chinese room” thought experiment discussed directly the notion of consciousness. In the thought experiment, queries were made into the Chinese room (in Chinese). Inside the room, a person who does not read or understand Chinese finds a reply by using a large lookup table that matches the input characters with some output. The output, also in Chinese, is then returned. Searle’s point is that while there is a conscious entity in the room, that consciousness is not being used to pass the Turing test. Consciousness, said Searle, requires some understanding of the world, not just an ability to give a certain output when fed a certain input.

While Searle was talking about a person doing the lookup, the Chinese room thought experiment does feel a lot like a large language model, where the output is determined by a lookup based on the input. The lookup is over a much larger dimension than a simple table, but it is still based on the probabilities of what word follows another and has no connection to the meaning of the words.

While the Chinese room thought experiment calls into question the authority of the Turing test, it does not do much to reveal how you would decide if something was conscious. Coming to grips with this idea leads to an even older discussion in philosophy, centered not around machines or programs, but around the more general question of how to know that anything other than ourselves is conscious. Known as “the problem of other minds,” this has been a question that dates back at least to René Descartes, generally considered to be one of the first philosophers of the enlightenment.

Descartes is best known as the author of cogito, ergo sum (I think, therefore I am). This in turn was the result of his approach in his Meditations on First Philosophy (1641), which begins with a program of radical doubt. Descartes wants to know what he can be absolutely sure of as a basis for his construction of knowledge. Descartes brings up a number of now classic problems—how do I know that I’m not dreaming, what if there is some evil demon doing all it can to confuse or mislead me?—and finally decides that the one thing on which he can rely is that he exists since he is conscious. Even to be deceived requires someone to be deceived, so his consciousness entails his existence. My own consciousness is something with which I have direct experience, and therefore I must exist.

While you have direct experience of your own consciousness, you do not have such experience of any other consciousness, at least not in your normal state of mind. Mystics may claim to experience the mind of God, psychedelics might allow experiencing the mind of the universe, but normal experience is just of your own mind. So how do you know that others are conscious?

Descartes’ solution was to first prove that God exists, and then conclude that other people have consciousness because a benevolent God would not fool us about this. This approach has been found less than convincing by later philosophers. Descartes also believed that only humans (and, in some writings, perhaps only men) had consciousness, and that anything else did not, which led him to some remarkable experiments on animals that we would find horrifying today. It also meant that he had to find some way for the conscious mind to interact with the body (his conclusion had the mind tickling the pineal gland). While these views are, to say the least, not widely held today, the problem of determining if others are conscious remains.

This is not to say that philosophers generally worry about the existence of consciousnesses other than their own. There may be some die-hard skeptics out there, but it isn’t a widely held position. If you find someone who argues this point, there is an irony in them trying to convince you since, if they are right, there is no you to convince (and, from your point of view, no them to argue). The question that does linger is not whether there are other conscious beings, but how you can go about knowing that.

One argument, first put forward by John Stuart Mill in 1865, is that you know by analogy. Other people act like you act, react to stimulus in the way you react, reason similarly to the way you reason, and are biologically constructed the way you are constructed. So, you can reasonably conclude that they are conscious in the same way that you are conscious. You can argue that dogs and cats (and other animals) also have some level of consciousness given the way they react to stimulus and the fact that they are biologically similar. Where to draw the line is an interesting question—are insects conscious, and what about plants? But not knowing where to draw the line is different from saying that you can’t know others are conscious in clear cases like other humans.

Another set of approaches, put forward by those such as Wittgenstein or Popper, say that the question is really not well formed. We have shared language and we communicate with each other, which requires that there be shared consciousness. If you aren’t conscious, we can’t really converse. So, there is no problem, other than that I might think there is a problem.

In fact, the philosophical discussion around the problem of other minds seems to have peaked in the latter part of the last century but has dropped off the philosophical radar more recently. That is, until very recently, when it has been resurrected by the question of how to tell that an AI is conscious.

Here the discussion often rests on whether the AI is behaving enough like something that we believe is conscious to enable us to say that the AI is conscious. This is a form of Mill’s argument by analogy—if the AI reacts the way we do, it must be conscious in the way we are. The analogy is deepened when we think of how the AI thinks—neural nets are supposed to be models of the way the brain functions, and an underlying assumption of neuroscience is that we will understand consciousness when we understand the way the brain works.

This is brought out nicely in a recent paper, “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness.” Written by a group of philosophers, neuroscientists, and computer scientists, the paper proposes a set of criteria that would allow you to say that an AI is conscious. The authors conclude that no AI system meets the criteria (at least as of the writing in August 2023), but also that there are “no obvious technical barriers to building AI systems” that do.

But this conclusion, found in the abstract, leads to an interesting footnote, where the authors point out that their original conclusion was that there is no technical barrier to keep an AI from being conscious, but that they made the weaker claim because “satisfying the indicators would not mean that such an AI system would definitely be conscious.”

Which brings us back to the central question: How would we know that an AI is conscious? It may be able to converse with us as though it were conscious, but part of how we conclude that other people (or animals) are conscious is how they interact with the world, which AIs currently don’t do. The analogy and behavior are limited in ways that other candidates for consciousness are not. Coming up with the criteria is difficult enough for organic life forms; finding one for AIs is more difficult still. The question is not a technical one, and it has been discussed by philosophers for hundreds of years without a clear solution. But the need for such a solution is growing with every new release of a frontier model.

Image credit: Roman de la Rose, 14th century, National Library of Wales

Jim Waldo is a Gordon McKay Professor of the Practice of Computer Science in the John A. Paulson School of Engineering and Applied Sciences at Harvard, where he teaches courses in distributed systems and privacy. He is also the chief technology officer for the School of Engineering and Applied Sciences (SEAS), and a professor of technology policy in the Harvard Kennedy School, where he is associated with the Belfer Center, the Shorenstein Center, and the Carr Center. Prior to being the CTO at SEAS, Waldo was the first CTO for Harvard University. He received his Ph.D. in philosophy.

For more than 75 years, ACM has helped computing professionals to be their most creative, connect to peers, and see what’s next. Shape the future of computing with us.