
Report from the Vatican
In my view, Pope Leo XIV is by far the clearest thinker on AI among prominent world leaders. It was a genuine thrill to meet him yesterday.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Gary Marcus
Oct 31, 2026
197
41
42
Share
In my view, Pope Leo XIV is by far the clearest thinker on AI among prominent world leaders. It was a genuine thrill to meet him yesterday.
30 October 2026
It is understatement to say that I am a huge fan of his Encyclical on AI, the Magnifica Humanitas. It goes beyond words that I was able listen to him yesterday in person, from just a few feet away.
He gave a brilliant, short speech, which you can find here.
The Pope, delivering his remarks on AI
His central five points, which I fully endorse:
First, the future of artificial intelligence is not pre-determined. As a technology developed by human beings, its future depends on the choices we make. We therefore have the responsibility to guide, change and, when necessary, reject its applications (cf. nos. 97, 104). The decisive question, then, is not simply “What can AI do?” but rather, “What purpose should it serve?” Those who regard its development merely as a race to be won risk focusing exclusively on the finish line, forgetting those who are left behind. A reliable system that serves the human person is of greater value than one that is exceptionally powerful but whose consequences cannot be adequately foreseen or controlled.
Second, machines are tools, not persons. “So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean” (no. 99). Today, some propose granting these systems rights, or even legal personhood. Such an approach risks obscuring human responsibility and leaving those who suffer harm to confront a faceless entity.
Third, certain decisions must never be delegated. When a person is placed on a target list or entered into a border control database, there is a danger that he or she will be reduced to a mere data point. Decisions affecting human life, freedom and the use of force must always remain subject to effective, informed and responsible human oversight (cf. nos. 198-200). For this reason, artificial intelligence must be “disarmed” (no. 110), and I encourage the States meeting in Geneva in November to take concrete steps toward banning lethal autonomous weapons.
Fourth, intelligent technologies must be directed toward the common good. “Justice requires access to the benefits of innovation, including care, knowledge, tools and opportunities” (no. 161). Indeed, they must never become ends in themselves or serve merely the interests of a privileged few. Rather, their development should foster solidarity and participation at various levels of the global community, ensuring that the benefits of progress are shared equitably, especially with those most in need (cf. nos. 71–72).
Finally, I would like to emphasize a challenge, which is perhaps the most urgent. We already know many of the ethical principles that should guide the development of artificial intelligence. What we currently need are institutions capable of putting these principles into practice, particularly effective international institutions that can safeguard the global common good (cf. no. 64). This calls for shared standards, independent oversight and clear mechanisms of accountability, so that decisions concerning the development and use of AI can be examined, challenged when necessary, and accompanied by appropriate remedies for any harm caused (cf. nos. 105, 107).
As a bonus, here’s a transcription of own (improvised) remarks on technical, political, and philosophical aspects of AI, given earlier in the day, a commentary after Max Tegmark, Stuart Russell, and Valérie Pisano spoke, at a small private meeting just before our group met with the Pope:
There are so many ideas here [today] that are interesting and deserve a response.
I’ll start by saying it’s a tremendous honor to be here and to be in this illustrious audience.
For those who don’t know me, I’m first and foremost a cognitive scientist. I trained with Steven Pinker at MIT, and my dissertation was on a mix of things. It was on how humans learn about the world, how children learn language, but also about neural networks, neural networks that are the ancestors of the ones that are popular right now.
And to the extent that I’ve had something to say about the field, It’s been by comparing how human intelligence works, how natural intelligence. I also worked with animals, how natural intelligence works with how machine intelligence works. And I think that’s given me a little bit of a different perspective on how things go.
I also spent some time as an entrepreneur. I built an AI company that I sold to Uber, and so I’ve seen both the industry side and the academic side. So that’s where some of my background comes.
So I agree, for example, with Max [Tegmark] about many of his recommendations, for what the commission might put forward. And maybe the most profound thing that I agree with is with Stuart [Russell], that the current path is untenable. and that the sooner that the industry recognizes that, the better. And I agree with Valérie [Pisano], that we’re mistaking things for inevitability, that far too much of the discussion is people taking for granted that we are on the only path, rather than maybe a different path.
And I agree that we still have agency, and that we need to look at things differently. So, as a cognitive scientist, I begin to look at things differently from the very statement of what AI is. As a cognitive scientist, I look at the nature of intelligence.
We sometimes talk about the modularity of intelligence, that there are many different components that go into intelligence. So, there’s verbal intelligence, and mathematical intelligence. And, kinesthetic intelligence about how we move our bodies in the world, and so forth.
So one of the disagreements that I would mark is actually with something that Stuart said, which I think was partly true. Stuart said that we are humans, we predict, and we do, we certainly predict. There’s ample evidence from the cognitive science literatre that we make predictions, including literally of the next word.
But there’s also lots of evidence from cognitive science that we do lots of other things. And so, in addition to making predictions, we do reasoning, we have emotional judgments and so forth. The mind is, as one lovely quote [put it “not one thing, but many.”
And I think that the fundamental mistake that the field has made, is to assume that capturing one component of intelligence, prediction extraordinarily well, is enough to capture all of intelligence. And we now, though we didn’t always, have an entire industry around this, as Stuart, who wrote the most famous textbook or co-wrote the most famous textbook on AI, knows well, we used to actually have many different approaches to AI, and the fields settled on one, the large language model, which is the best prediction machine that we have. And it does that very well, but it doesn’t do some of the other things very well.
I think it’s still very poor reasoning. And the thing that’s maybe most dangerous and that, I think, captures what all of us [speakers at this session] are saying here, is [current AI] doesn’t follow instructions very well. And that might be surprising, but it turns out that the way that these prediction machines work is they follow correlations rather than causation.
I would say that there’s semantics of the world, their understanding of the meaning of the world is fairly superficial. And so they will often match the things that we hope that they will, but there’s no guarantee, and we see that in many different ways. For example, you can tell a large language model, don’t make up any references.
People have done this in a legal context, and the systems will still make up references. You can say don’t use any copyrighted materials, and the systems will continue to use copyrighted materials. So they will do the prediction very well of what a citation in an article looks like, and they will make up a citation that sounds like the kind of thing that you would cite in the legal brief that you’re doing.
So they’re really good at the prediction. They’re not good at basic instructions, like, only find something in a table of references that you already have. And so, we are stuck on this untenable path because we are building machines that are actually sort of unequally skilled.
They’re really skilled at predicting what happens next, and not so skilled at reasoning, not so skilled at sticking to facts, and not very good at sticking to instructions. And so what I like about what Yoshua {Bengio] is trying to do, though I might differ on some of the technical details, is he’s trying to build a system that is safe by design, that will follow instructions by design. And I think we are indeed in really deep trouble unless someone like him succeeds.
Maybe he succeeds, maybe someone else takes a [different] approach. But one of the things that I think is underemphasized here, but maybe not in [Valérie‘s talk] is we need different research approaches. I understand Stuart’s claim that we’re not on a great path research wise.
He’s right about that. And so I understand the urgency to want to take a policy approach, and I think we actually need both. We need policy approaches, and I’m going to give you an example in a second.
We need policy approaches, but we also still need those research approaches. And in fact, I think if we don’t have the new research approaches, like what Joshua is trying to do, for example, and I think we need many such approaches, then I think we were in deep trouble, and the policy alone will not get us there. The policy can help push us towards those approaches.
So here’s an example where I think we are currently stuck and where we should reflect pretty deeply. Not all current AI is dangerous. A lot of current AI is not dangerous.
I used a GPS navigation system to get most of the way here today [with] a little bit of hand editing for the final bit. GPS navigation systems do not hack into Australian government websites. They do not hack into places like Hugging Face and so forth.
They have a little bit of Internet access, but it’s restricted. GPS navigation systems are form of intelligence. They find the best possible route from point A to point B, using what we call a world model, a literal world model in their case.
And they’re not so dangerous. There are lots of forms of AI that are not very dangerous. Nobody has ever noted [danger from] AlphaFold, which holds proteins from the description of the nucleotides.
Nobody has ever seen it hack into a Hugging Face or anything else, right? It works with a particular problem, a particular database, in a particular way. It does not cause harm.
But we now have these set of systems that people call generically agents. Really, it’s a special form of agents. The agents that we have right now that are causing the problems or the ones that are open-ended, can kind of do anything on the internet.
If you have an agent that can only do one thing on the Internet, GPS system, actually, can look up a traffic website and give you some information. If you have these open-ended things, then you get into trouble. Well, how much trouble?
Well, the Hugging Face incident was really bad, although you can look at it and say, open AI failed in a bunch of ways that are kind of interesting. But it turns out there’s been like 10,000 incidents like that. We’ve seen the French refrigeration system was attacked.
There’s all kinds of cyber attacks in Japan now. The number of incidents is much higher, I think, like in the last month and maybe in the previous year, which was probably higher than ever before. So right now, we’re confronted with kind of like an experiment or a dress rehearsal.
The experiment is, what do we do about that? We now have an AI technology that I think we can all agree is act
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。