Plan R: AI Safety by ASICs

9artemium1moMaybe someone already mentioned this here, but if I remember correctly, this is exactly how they approached AI safety in Neuromancer novel with Turing police monitoring chips that should instantly melt if they detect signs of recursive-self-improvement.

So yeah, his is the canonical way for us to choose to a Cyberpunk timeline!

4Daniel Kokotajlo1moOh wow I didn't know that. Nice.

7Roko1moI mean this does sound like a future I would look forward to as opposed to imminent extermination under plan D!

3Roko1moProbably not - Plan R philosophy is to seriously limit synthetic intelligence to the absolute minimum needed to do the task.

[-]Ben Pace1d16

2

Curated. Love to see more specific plans being worked out, which is still greatly undervalued, and I like how this utilizes ASICs and separates out financial incentives from research groups and companies. Also loved Kokotajlo's comment visualizing this world.

2Roko18hthankyou!

[-]joec1mo15

1

For those who haven't seen it, LLMs burnt into ASICs can run shockingly fast. Here's a demo of an ASIC that can run Llama-3.1-8B (which came out only slightly more than two years ago! This chip did not take that long to make!) The numbers here are around 17,000 tokens/sec, compared to the B300's ~500 tokens/sec.

I'd guess that if capabilities did stagnate at the current level for ~years (whether due to a pause or some ceiling in ability), companies would begin to turn to ASIC inference for their frontier models for purely financial reasons, which could set up for this. I'm not sure what other worlds lead to a pursual of Plan R.

[-]Vladimir_Nesov1mo15

3

SRAM or hardcoded weights (with SRAM KV cache) doesn't really work for the biggest models. You need too many chips to fit a big model, many more than you need if you also use HBM for each chip (to store both weights and KV), at which point the networking gets infeasible. With B300s and such you can serve the biggest models batch-size-1 with no MTP to get 10x faster (by only reading the active-not-total params and KV for a single request each pass), but that's maybe 20x more expensive per token.

[-]Mlxa1mo9

2

I might be missing something, but why is (B) relevant, given (A)? If (A) also involves the likely destruction of the controlling entity itself (as is the case with unaligned ASI), then for a rational player there is no incentive to pursue it even given (B). And if (A) doesn't harm the controlling entitiy, and just gives it control over the world, then it automatically implies (B) as well, since no one can stop it? The analogy with nuclear weapons breaks down, because there the relevant fact was not (B), but rather that other players also got nuclear weapons, and in particular, second-strike capability / MAD.

6Roko1moDestruction is not certain.

If there are many players each racing and creating a lot of risk, the risk acts like a public bad that they each contribute to, but it is still rational at the margin for them to contribute more risk if, in the case that they win the race AND the risk is avoided they get a massive reward.

The Nash Equilibrium of this game can be risk approaching 100% for a large number of players and appropriate parameters.

Armstrong, S., Bostrom, N. & Shulman, C. (2016). "Racing to the precipice: a model of artificial intelligence development." AI & Society 31, 201–206. doi:10.1007/s00146-015-0590-y

https://nickbostrom.com/papers/racing-to-the-precipice.pdf

3noma1moAs I understand it, by having to burn the outputs of the the R&D firms into ASICS by law, you are not able to get a runaway intelligence explosion. Through controlling the ASIC, you control the model.

[-]Vaniver1mo8

2

Nuclear technology and bioweapon technology both approximate (A) and they are mostly okay because without (B), there isn't an incentive for people controlling them to push their luck on safety.

Wait, what? I thought the massive runup in nuclear weapons was primarily because of the 'unbounded claim' dynamics (first strike capability maybe means you can rule the world).

Obviously, the humans made a mistake in letting the AI company write the IOUs to them. But once there's a system for stocks, banking, money, etc, letting people write you IOUs is the default an

[-]cqb1mo7

1

Burning the weights into an ASIC would effectively make them public to anyone willing to expend with one of their purchased chips, right? They could extract the weights and then run them in their own GPUs/TPUs. One would need to encrypt the weights in the chip to avoid extraction, but that would increase the transistors per weight immensely with of the decryption circuits (or slow it down with serial decryptions). Or, one could FHE everything, but this would also come with an immense size/transistor count cost.

1Roko1moWhat GPUs? Unless that someone is an AI R&D lab, they will not have GPUs or TPUs.

1cqb1moI guess this does depend a lot on the regulation regime for access to GPUs and TPUs.

Even if we're under a regime that restricts access to GPUs/TPUs to the R&D labs I think it would be prudent treating the AI R&D labs as adversarial here, if only because the AIs under their development may be adversarial. I'm unsure of how much of an issue it is for the labs to have direct access to the weights of their competitors (though my first impression is somewhat negative), but mostly noting that it is a possibility, that the systems under their development may also have access to these weights, and that burning the weights into the ASICs alone does not prevent others from accessing them.

2Roko1moThey would be legally forbidden from taking ASICs to pieces and each one would be a physical sealed unit that would have to be accounted for. Also, the teardown and deconstruction of a ternarized LLM ASIC might be extremely hard.

[-]JennaS1mo5

3

I wonder what a red teaming of this idea looks like?

[-]Maxsimal1mo25

9

For me the biggest problem with this is it requires some crazy level of civilizational cooperation. Everyone has to agree to this regime of giving up 'high spec compute', among many other things that need to be hard and fast rules. Roko.claims this is easy because only China and the US are involved - but that's clearly not the case when everyone has to agree to this.

And despite that massive civilizational realignment...it doesn't actually solve that much. It makes it difficult for an AI to self replicate, unless it can take over and ASIC fab and reverse engineer itself...which seems a dubious claim. It does nothing to make sure the models are fully aligned on reaching production - the labs still have to be trusted to do this correctly.

What it mostly seems to do is throw sand in the gears, which, ok but doesn't really need this convoluted scheme. And try to make sure something like the a misaligned attack from a model cannot spread outside a lab. Though if one did happen in a lab but was not detected, nothing is stopping a misaligned AI from polluting a production model that then is burned to ASICs and gets to do whatever in the outside world.