Why Sam Altman is against a single superintelligence

Translate

Why Sam Altman is against a single superintelligence

Cheeky Pint

Oct 29, 2026

7

1

2

Share

Sam Altman, CEO of OpenAI, joins John to break down how models uncovered critical PHP vulnerabilities, why test-time compute unlocks massive agent swarms, and OpenAI’s internal experiments proving complex math problems like Navier-Stokes. He also shares why sandboxing superhuman models is an unsolved safety dilemma, how robotics will scale future gigawatt data centers, and why businesses must adapt as personal agents replace traditional web forms and mobile apps.

00:00:00 Introduction 00:00:12 PHP week 00:02:37 Internet early days 00:07:33 Cyber Blue 00:08:54 Sandboxing 00:15:47 Math 00:23:12 Agent swarms 00:24:48 Stripe Atlas 00:25:17 Compute 00:37:53 Revenue per gigawatt 00:49:27 Token pricing is fake

We’re going off script. Just having a beer.

It’s not the first non-Guinness, but it is the first IPA. There’s a first time for everything.

First off, how did it... Why did it take so long to find these bugs? Because it sounds like there were—

You mean why did it take the 20 years, or why did it take the last few months?

No, why did it take... Exactly, why did it take a year? Maybe start by explaining the bug. Or the problem.

So the problem is that although we have known for a while that these models are getting good at finding vulnerabilities in the wild. A) They continue to get better fairly rapidly, and B) People just hadn’t really used them to go look at everything out there. There’s a lot of surface area and a lot of code on the internet. And, as people have started to do that, I think we’re going to find that we are in a world of many more serious vulnerabilities than we expect. Even though many people who have been following this for a while expect a lot.

Yeah, I just feel like a lot of the discussion ends up being around we need to fix the software. And sure, yeah, we should fix the software. But also, we maybe need to change the architectures, which is harder, but a much more durable fix.

It would seem really hard, except we have this amazing new tool to do it with. So, I kind of think that’s what’s going to happen. And you’re already seeing, like, a new generation of startups talking about building this new kind of internet or new kind of application architecture for this world where you assume that the models get smarter and smarter.

What do you think the global software upgrade looks like? For example, is a PHP codebase just a bad idea and you should rewrite a nice, clean version in Rust? Or, do we upgrade PHP? But yeah, as we... Everyone thinks that we need to kind of upgrade all hand-spun software. Just what form will that actually take? Is this huge for Infosys and all those guys? Because, it’s going to be a lot more software consulting projects.

I mean, people are talking more and more about just, “Let’s re-architect what it means to be a router, or an operating system, or an application.” And, you know, I’ve seen some of these new startups saying that they’re just going to make a whole new stack. And that would have seemed insane two years ago. You would have been like, “There’s no way. There’s too much surface. You can’t write all that.” And now maybe soon you can. I would bet that we have a completely new sort of model of how we make systems and networks.

What else reminds you of the early internet days?

The thing that most reminded me of the early internet was... There is this spirit of just unbelievable, creative unleash in people using these tools. You see it in kids particularly.

And again, maybe this is just because I remember what it was like when I first got a computer with internet and I could do all these things that adults just couldn’t even think to ask for. And watching an 8- or 10-year-old make a complete video game, with some crazy ideas they have that no adult would really have.

And expect that to just be the way that technology is going to work for the rest of their lives. In fact, getting better and better. And, just this constant “I’m going to make useful stuff for me, and show it to my friends.”

We had to suffer when I was a kid to make a piece of software.

That barely did anything. And now you can have a full, 3D video game. Unbelievable, anything you can think of in 10 minutes. That reminds me of the early internet, including the just... These people. They’re never going be mind-blown by that, because that’s always what they expect out of the world.

Any ideas they can come up with, anything they want to create, anything that they think might be useful, anything that they just need to make their lives better. The computer’s supposed to make it. There was a crazy moment at OpenAI where we had a SEV... And Codex went down.

I mean, not all the way. But, it was like a “Whoa, what if you couldn’t?”

I remember when I was a student, and I would meet the really old guys. And they would, once in a while, pull out their gigantic Xerox Alto disk. Or they would pull out their...Line-by-line written assembly. Or they would pull out their punch cards. And I felt... It is incredible to me that you all bootstrapped us to this point.

Yes, no, I agree. And keeping programs in your head, right? You know, programs running the first time after, 100s–1,000s of lines were written. No, there’s few lost arts like that.

Going back to the cyber side of things. In retrospect, was the cyber RL a mistake? Or, did we just have to go there? It’s a little like the SNL thing of... We urge you, to urge us to stop. It’s like the models are getting very good at cyber capabilities. I mean, they are getting very good, but not just totally emergently.

Look, the reason that SNL thing was so funny is because it was so true. Humor like that doesn’t stick unless there’s an element of... Clearly pointing out a ridiculous but true thing about society. Or someone. And the cyber thing, there is a point which is like, “Okay. Sure. The models are really good at cyber. But, who asked for this? Why are you doing this?”

One of my big disappointments, criticisms of our field or whatever—and I think maybe the cyber model’s getting good at cyber. Cyber is an example of this is: I don’t think we have done a great job of explaining to people why we think all of the potential downside of this technology is worth it. The caricature of this is people feel like we’re like... Fine. You say the model’s going to cure cancer, someday. Maybe it will. And if so, I guess that’s good. And you say it’s going to make stuff cheaper. And okay, maybe I already see that’s good and I get access to medical advice or legal advice. But, then you paint this picture of the future where it’s like a new industrial revolution. And maybe the last one wasn’t that good and this one sounds worse. And I will have no job, no ability to have control, or choice, or freedom of my future.

And AI is so dangerous that you’re going to have to have a crazy surveillance state so we don’t all die. This is a terrible—

One of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people, we have really stuck to that principle. And I think this is the only kind of good path. I think people... It’s not enough to tell people: “You are going to get the model to do these great things for you.”

It’s got to be: “We are going to enable you to have a better life that you define for yourself, to do useful things. You’re not going to just sit and get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future.”

Iterative deployment is this idea that you just have to get the advances out there and you can’t... Stopper them up inside the government—

I mean, not all. You still have to make them safe before you put them out there. You can’t cause havoc in the world. But, building up a capability overhang is a thing to fight against. That directionally speaking, yes, you want to get this into the world. You want to get this into people’s hands. And you want to let people use the technology to figure out all these wonderful things.

The exploits get a lot of headlines. The prevented exploits do not. How quickly are people making progress with cyber blue?

I don’t want to declare victory at all on the PHP thing, because I still don’t think we know. And I assume some bad things are going to happen. Relative to my worst fears, if I can think back 10 days ago, the first day of that went much better than I expected.

And, we heard a bunch of stories from people that were using our models to not only build patches and build cyber defense, but individual companies who were just: “I need to just do a quick scan and deployment. I’m going to take my thing down for an hour. I’m going to use this thing and I want to put it back up because my application’s offline.”

Now I feel like I really jinxed it and I shouldn’t have said that, but the first day did go a lot better than I thought. And I think people have been able to use that. The thing that I’ve heard a lot of cyber experts talk about is... They just no longer are confident that we have the right assumptions baked into our threat model. And they’re like: “Look. If everything is what we think, and hardware exploits are as hard as we think they are, then yes, we can see how this is going to work.”

But if AI is going to discover really deep novel bugs in hardware that require physics we didn’t understand, but all of a sudden means some assumption doesn’t hold. Then I think, people aren’t sure what happens.

One thing that always strikes me is that, when I talk to very knowledgeable people inside OpenAI and similar companies, there is a pessimism that sandboxing is possible, basically. And, that would not have been my mental model going in. It would’ve been that, yeah, you have to build a really tight sandbox and we haven’t hardened them historically enough compared to the current capabilities. But ultimately, you can have a maximum security prison for the model to work in and keep it inside there. It seems like you guys have a pessimism that sandboxing models is possible.

My instinct... Well, first of all, I think we can do much better than we’re doing now. And we should. That is the main thing to focus on right now is a better security model. If you imagine a theoretical AI that is a million times smarter than the smartest security researcher—would you really be willing to trust that sandbox? Especially if the model can discover new science that people don’t already understand—

You can use the models to construct the sandboxes.

I think you can... Well, so this gets... Let me come back to that. I think sandboxing works to a point. Then I think monitoring works to a further point. Where we can look at what the models are doing, not only in their chain of thought, but we can do activation monitoring and try to really understand: are the models... Are we happy that the models are not deceiving us? And then, eventually, I think you need to believe with huge confidence that the models are truly aligned. And are going to value and try faithfully to follow their actual... The intent of the people operating them. And that they’re going to value people being in control of the future, forever. And, if you had a model that you could fully trust the alignment—that you were absolutely certain of the alignment of. Then yes, I think you can have that to help you build better and better sandboxes. But, if you really let yourself project forward into the: “What does it mean to have something a million times smarter than we are?” I don’t know. I think that just gets... Strange.

Because you need to really, really trust the alignment to trust that the sandbox is staying ahead of capabilities.

Can you not give models competing incentives and give them incentives to defect from each ot