返回资讯中心
外部精选
软件工程
#开发者实时流

Why No One Has Implemented the Turing Test

In his famous October 1950 paper in Mind, “Computing Machinery and Intelligence”, Turing described an imitation game involving a “question and answer method” and proposed that success in this game provided a “criterion …

Hacker News 最新geox2026/10/16 · 2026/10/1630 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

In his famous October 1950 paper in Mind, “Computing Machinery and Intelligence”, Turing described an imitation game involving a “question and answer method” and proposed that success in this game provided a “criterion for ‘thinking’” in machines (1950, 435, 436).Footnote 1 Turing’s paper was a dramatic contrast to other articles in the journal in 1950 and the years immediately prior; the few papers that might be described as on philosophy of mind typically discussed the epistemology of sense data, selves as particulars or substances, mental acts as dispositions, psycho-physical parallelism, and souls—nothing on learning machines or arguments for or against the possibility of artificial intelligence (AI). Perhaps for this reason, some early replies to Turing’s paper did not take his proposal very seriously. Only a few months later, Leonard Pinsky whimsically proposed an alternative test for intelligence in machines. He suggested that one of Turing’s computers be sent, as a “Christmas gift”, a “subscription to MIND, retroactive to October, 1950” (Pinsky Reference Pinsky1951, 397–98). The computer, on reading Turing’s article, might realize that “it is thinking about the possibility of machines thinking” and consequently suffer “a nervous breakdown”—if the computer could be cured by the reassurance that it was “only tussling with a pseudo-problem”, this really would demonstrate that a machine thinks, Pinsky declared (1951, 398).

Sober treatments of Turing’s proposed test appeared quite quickly. On the one hand, in April 1952, Wolfe Mays claimed that Turing was a “mechanical necromancer”, in whose account “the meaning of the word ‘thinking’ has changed to such an extent that it has little in common with what we normally mean by it”; Turing’s “behaviouristic” approach failed to see that what is important is not “what” the machine does, “but how it does it”, Mays claimed (Reference Mays1952, 150–53, 158). Several early discussions pressed Mays’ first point; for example, Michael Scriven claimed in 1953 that the expression “intelligent” was “an improper adjective for the noun ‘computer’”, and so talk of intelligent computers implied “a change of meaning in one of these terms” (Reference Scriven1953, 236). On the other hand, in 1951 Donald MacKay said that there is no distinction in principle between the “observable behaviour of a human brain” and the “behaviour possible in a suitably designed artefact”, and in 1952 Henry Cohen stated that arguments against the possibility of thinking machines “lead inevitably, as Turing has shown, to a naïve solipsism” (MacKay Reference MacKay1951, 120; Cohen Reference Cohen1952, 206). Subsequently, F. H. George claimed that Mays had made “no impression whatsoever on Turing’s position”—and, echoing Turing’s sentiments in Mind, that “much dispute” over the capabilities of machines was due merely to “feelings of an emotional kind” (Reference George1956, 248; Reference George1958, 59).Footnote 2

Three-quarters of a century later, the verdict on Turing’s test is often cavalier. For example, an article in October 2025 in Nature claimed that large language models (LLMs) “killed the Turing test”: “[c]hatbots now ace the mathematician’s famous imitation game”, it proclaimed (Gibney Reference Gibney2025). The evidence cited was a 2025 study stating that two LLMs “pass the Turing test” (Jones and Bergen Reference Jones and Bergen2025).Footnote 3 Several recent studies making the same claim have prompted commentators to declare that the test is obsolete—either because we no longer need an existence proof of the possibility of AI, or (assuming these LLMs do not think) because their success in the test counterexamples Turing’s claim to have provided a criterion for thinking in machines. The vital question here, however, is whether these machines have passed Turing’s test, or merely some imitation test (Copeland Reference Copeland2000, Reference Copeland2004a).

Our first aim in the present paper is, taking Turing’s descriptions of his test, to establish definitively his conceptual design for the test. We argue that influential interpretations of his design are incompatible with Turing’s own words; these readings also conflict with the early perception and historical context of his ideas.Footnote 4 A consequence is that any claim that Turing’s test has been passed by unintelligent LLMs, and his criterion for thinking thereby rebutted, is false. Our second aim is to present the results of implementing Turing’s conceptual design for the first time (full technical specifications of our study, data, and analyses are available in Temtsin et al. Reference Temtsin, Proudfoot, Kaber and Bartneck2025). The results of our study do not support a “general consensus”, announced at the October 2025 Royal Society celebration of the Turing test’s 75th anniversary, that AI has passed the test.Footnote 5 When we tested a generative pre-trained transformer (GPT) machine, the machine’s performance was extremely poor.

The paper is in two parts. The first part (sections 2–4) aims to identify Turing’s conceptual design for his test on the basis of exegetical, historical, and experimental considerations.Footnote 6 From Turing’s descriptions, we extract informal specifications of the structure, scoring method, and essential features of his imitation game. The second part (section 5) presents (informally) our new empirical study implementing Turing’s design and the philosophical lessons to be derived from it. The paper, we argue, generates a challenge for AI experimenters: to earn the cachet of passing Turing’s test, their machine must do well in the imitation game presented here.

Discussions of the Turing test often begin with some claim concerning Turing’s philosophical commitments—roughly, either to behaviourism, some classical or non-classical version of computationalism, or (more recently) response-dependence concerning judgements of mind. Our aim is to identify Turing’s conceptual design without tying it to any specific theory of mind, in order that his blueprint may be useful to theorists, regardless of their interpretations of Turing’s philosophy. (We turn briefly to Turing’s philosophy of mind at the beginning of section 5.) For the same reason, we make no assumptions regarding the nature of the concept of intelligence, or of the property of being intelligent, for which success in Turing’s game is to suffice.

In addition, many discussions deny that Turing intended his test as an actual experiment to be performed. For example, at the Royal Society celebration, it was stated that Turing was only “kidding” in his 1950 paper, and that we should not take the paper “too seriously”.Footnote 7 It has been claimed that Turing intended his test merely as a “joke” (Minsky Reference Minsky2013), a “philosophical conversation-stopper” (Dennett Reference Dennett and Michael1985), or as a rhetorical device in order to counter popular objections to the possibility of AI (Sloman Reference Sloman, Barry Cooper and van Leeuwen2013). However, these claims neglect the fact that Turing discussed his test, not only in the Mind paper, but also in a 1952 radio symposium on machine learning, “Can Automatic Calculating Machines Be Said to Think?”, and he anticipated his computer-imitates-human game in “Intelligent Machinery”, an official 1948 report for the National Physical Laboratory (NPL) (Turing et al. Reference Turing, Braithwaite, Jefferson and Newman1952; Turing Reference Turing1948; Copeland Reference Copeland2004a). In addition, these claims overlook the historical context: in the late 1940s and early 1950s, the emergence of the world’s first electronic stored-program computers in the UK and the US had generated vigorous public and scientific debate on the possibility of AI—Turing was aware that a criterion of intelligence in machines was badly needed.Footnote 8

Some commentators claim that Turing intended his test only as a thought experiment (e.g., Mitchell Reference Mitchell2024). However, depicting Turing as solely an armchair philosopher would be almost as inaccurate as characterizing him as solely a pure mathematician. Turing was interested in turning his abstract universal computing machine into concrete machines; at the NPL, for example, Turing and his engineering team were designing his Automatic Computing Engine (Copeland Reference Copeland2004b). In addition, he was interested in actual human reactions to computers. In his NPL report, he described a “little experiment” in which a human (C) plays chess with another human or with a “paper machine”—which is to say, a human being simulating a computer by following a set of rules, using pencil and paper, the only sort of programmable computer then available (1948, 431). C, Turing said, “may find it quite difficult to tell which he is playing” (1948, 431). This experiment anticipated in important respects the imitation game he announced two years later, and Turing ended his description of the chess game with the words, “This is a rather idealized form of an experiment I have actually done” (1948, 431).

In the second paragraph of his 1950 paper, Turing introduces “a game which we call the ‘imitation game’”, as follows:

It is played with three people, a man (A), a woman (B), and an interrogator (C) who may be of either sex. The interrogator stays in a room apart from the other two. The object of the game for the interrogator is to determine which of the other two is the man and which is the woman. He knows them by labels X and Y, and at the end of the game he says either “X is A and Y is B” or “X is B and Y is A”. (1950, 433)Footnote 9

After more discussion of this man-imitates-woman game, Turing said:

We now ask the question, “What will happen when a machine takes the part of A in this game?” Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman? (1950, 434)

Later in the paper Turing suggests that “the question, ‘Can machines think?’ should be replaced by ‘Are there imaginable digital computers which would do well in the imitation game?’” (1950, 442). When a computer is a player, Turing says, the computer takes “the part of A in the imitation game, the part of B being taken by a man” (1950, 442).

A persistent and sometimes torturous debate has grown up around these remarks. In moving from a man-imitates-woman game to a new game involving a computer, is Turing proposing that, in the latter, (1) the interviewees are a computer and a woman, and the computer’s task is to imitate a woman? Or is it that (2) the interviewees are a computer and a man (i.e., male or masculine human being), and the computer’s task is to imitate a man? Or is it that (3) the interviewees are a computer and a human being (of any gender), and the computer’s task is to imitate a human being? All three interpretations can be found in the recent scholarly literature as well as in popular discussions of Turing’s test.

In an influential paper criticizing Turing’s test as “actively harmful”, Patrick Hayes and Kenneth Ford propose (1)—the computer-imitates-woman interpretation.Footnote 10 They say, “If we take [Turing] at his word, the test is rather clever”:

It has a woman and a machine each trying to convince the judge that they are a woman, and the judge’s task is still to decide which is the woman and which, therefore, is not. But this judge is not thinking about the differences between women and machines, but between women and men. The hypothesis that one of his subjects is not human is not even in his natural space of initial possibilities. This judge has exactly the same problem to solve as a judge in the original imitation game and could be expected to bring the same attitudes and skills to the problem. We will call this the gender test. (Hayes and Ford Reference Hayes, Ford and Chris1995, 972)

The conception of Turing’s test as a “gender test” is st

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →