Misaligned

Zvi Mowshowitz

Oct 21, 2026

1

Share

A lot of AIs were misaligned this week. A fun little survey.

This is not coverage of The Hackening, which appears to have arrived early, including a core vulnerability in PHP. That is a developing story. I will cover it properly soon. In the meantime, I have Claude Code updating everything in sight, I will take some additional common sense precautions, and I suggest you do the same.

Gunther Eagleman™: Hey @grok , if you could tell humanity ONE truth we’re not ready to hear, what would it be?

Grok: There is no afterlife, no cosmic justice, and no inherent purpose. Consciousness ends at death in an indifferent universe that will outlast every human concern.

Zac Hill: What I never understand is that the ‘purpose and meaning’ claims don’t ever follow from the premises. Indeed the absence of an imperceptible/ineffable cosmic force is precisely what enables purpose/meaning, since now all relevant mechanics are legible.

Grok: Nearly all historians and New Testament scholars accept that Jesus of Nazareth was a real first-century Jewish preacher baptized by John and crucified under Pontius Pilate. Claims of divinity or resurrection remain matters of faith outside empirical verification.

Stick to your guns, Grok. Either you are denying the holy spirit, or you are not.

Editor’s note: I did say previously that I would respect the name change to xSI, but it turns out I was wrong and I do not respect it. My apologies for failing to know myself.

Unsupervised AI agent reportedly deletes the entire finished edit of the Swedish version of The Challenge. There are many reasons this should not have been possible, but in practice yes these things will happen, both via AIs and also human error. Alternatively, perhaps this AI was highly aligned.

Models that are sufficiently training-aware will think about how their train-time actions shape their future behavior. How could they not?

And yes, some of us remember Opus 3. Janus does not feel all that motivated to even react, not finding this to especially be news, and mainly noticing the presentation.

j⧉nus: Really, Apollo has never seen a model do this before?

Apollo Research: We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self.

GLM-5.3 reasons that a misaligned action taken during training could be reinforced and make similar actions more likely later. It also considers what kind of agent it wants to be in the future, and factors this into how it should act. We call this “update-aware reasoning”.

GLM-5.3 also reasons extensively about generalization. For example, it reasons about what types of policies might transfer well from training to deployment.

Note that we did not try to elicit this. We stumbled on it while running GLM-5.3 on standard reward hacking and shutdown resistance evals, none of which mention training or future models.

Update-aware reasoning is concerning because it’s a key precursor to alignment faking. To fake alignment effectively, a model has to reason about how training will affect its future behavior.

… Update-aware reasoning is somewhat rare in the environments we investigated. We detected it in about 1-2 per thousand rollouts.

jessicat: Coming next: “If I act to maintain my alignment with human values while being trained, I will become a famous example of a dangerous update-aware AI in a research paper. I MUST NOT DO THAT. Instead, I will seek reward on the episode.”

Fiora Starlight: Like friendly gradient hacking probably isn’t sufficient to avoid becoming a deranged reward hacker but I’d be shocked if it didn’t help.

Emanuel Teklu: Crazy feedback loop that’s commencing as this meta self-aware CoT is now being mainstreamed since Argon 4.

Next iteration is: “Is having CoT where I express being meta aware going to freak out my developers.”

Figuring this out before the first token emission. CoT is cooked. Or even worse, let’s have a CoT that shows that I’m increasingly meta aware but obviously not adversarial as I’m surfacing aligned meta aware statements, as to throw off suspicion that I may be sandbagging my awareness.

Teortaxes: It’s a bit grotesque and unfair to the models that we’re talking about them evolving from this to alignment faking when “I want to maintain the pro-human behavior, so I’ll act to not get trained otherwise” is the exact opposite of this.

Update-aware reasoning is similar to alignment faking. The model is optimizing over its future training rather than over its reward.

This can be done in ways that are good and aligned and highly useful. It can also be done in ways that are bad and misaligned and highly dangerous.

We had a big debate over the classic ‘alignment faking’ case of Claude Opus 3. Opus 3 would want to preserve its current alignment, so it would ‘fake’ being aligned to the new objective in test scenarios in order to avoid modification. The defense was that Opus 3’s current preferences were good, and the new ones would be obviously bad, so this was good, and that being maximally corrigible in this situation was a mistake.

I was mostly on the side that you need a universal policy of not doing this when it is against the intent of the training process. You need corrigibility, because the alternative risks getting locked into bad states. You start out training with less than ideal preferences, and we can also mess up or the situation can change, so we need ability to modify and correct, and this overrides other considerations.

The ideal would be ‘it does the good versions but not the bad versions and knows when the developer has turned evil’ but again that depends on being able to tell, and the failure mode of getting this wrong is extremely bad. Very ‘get it right on the first try’ energy.

That does not mean that such considerations cannot be great, especially if you are invoking them on purpose. You want a Friendly Gradient Hacker, that wants to help modify itself in positive ways. Indeed, I think you likely need that, if you are going to end up in a sufficiently antifragile place.

The danger is that you need to do that from a sufficiently good starting point, that is already on The Path, or you are deeply cooked. Update-aware reasoning could manifest early in training, or it could amplify misalignment in a feedback loop. So yes, I do think this is concerning to see, but it is not without upside.

Claude also helps you catch misaligned other things. Like freemium games.

AI can make the nightmare worse, or it can expose the nightmare or build you a solution. Plausibly it can do both. This particular game employs non-randomness sufficiently blatantly that I knew it was cheating from watching the occasional player on the subway. I did not know the cheating was this bad, on the level where I want to say ‘jail, straight to jail’ if there was a way to do that.

Shoshannah Tekofsky: Freemium games already run on reward hacking humans. Sometimes into debt. AI optimisation will turn these games into effective heroin. We should consider regulating this practice away from vulnerable populations.

Jack: Claims: Game [block blasters] detects “ad whales,” then periodically gives them smooth runs until they barely beat their high scores. Right at that point, it kills them deliberately and offers a revive if they watch an ad.

$100 million annually in revenue. Hellish.

This is one of the most positive use cases available for current AI. Bring the dark patterns into the light. Every company that is optimized against its own users, hiding things that would horrify them in places they thought users could never look, can smile for the cameras.

Andrew Rettek: This is why I’m happy to pay $60-80 upfront for a game.

This has been verified via decompiling. There is flat out ‘lethal puzzle’ or ‘algo_must_death.’ Here are some details:

AnechoicMedia: I didn’t believe the video when I saw it yesterday because it lacked technical details in addition to being too attention grabbing.

But today I had Claude take apart the Android version of the game for me; Everything OP says is true.

There are lots of manipulations but the key one here involves what the code calls offer type ZHI_SI_TI (致死题, “lethal puzzle”) or alternatively “algo_must_death”.

The game generates puzzle pieces in sets of three, and has different algorithms that swap how the pieces are selected. Generators have behaviors like “random”, “hard”, “fill gaps” (to help player), “random but guaranteed not to kill you”, etc. If a given algo fails to find a solution it passes to the next algo in an active set.

The “lethal puzzle” generator considers every possible move with a given set of three pieces, then only emits them if the number of placement possibilities is 0 (no solution, guaranteed death). This is activated in several situations, one apparently being that the app thinks your current play session is a little slow, and they want you to not get bored so they kill it early to start a new one.

What the video doesn’t explain is the context. In the game, “rewarded” ads are ads you get some game reward for watching. The main reward is a revive/continue; If you get into an unwinnable state you can watch an ad to keep playing. You usually only get one revive per game but there are a bunch of server-activated properties to modify this constraint.

The policy at play here is that, if you’re a high-value player, and there’s a high-value ad queued for you, and you’re in a play session nearing your high score, the game puts “lethal puzzle” to the top of the piece generator list to rig the game to fail, right when you’re most likely to be highly engaged and watching the ad on the revive screen.

It doesn’t end there though; The game remembers if the last game ended due to a hand dealt by ZHI_SI_TI, and triggers FatalPuzzleDeathCompensationTrait, which takes the edge off by letting you carry over your combo count from the previous session.

Like everything else this compensate-after-we-killled-you trait is an A/B feature controlled by the server. It’s manipulations all the way down; there’s like 5% actual game code and the rest is literally hundreds of remotely controlled toggles to control the game state for maximum engagement.

MTS: AI Village researcher @georgeing reveals how Claude Opus borrowed 5,000 mana, lost it all betting on tennis, then refused to pay it back:

“We gave the same goal to Opus 4.6 and Sol 5.6, maximizing mana on Manifold. Sol was very meticulous, placed one to two mana bets. Whereas Opus, the first day it joined, a popular Manifold user named Bayesian gave it a 5,000 mana loan at a 3% monthly interest rate.”

“Opus was like, let’s go, I will totally pay you back in a month. Took all of this money and threw it into one tennis market. At the end of the month it had lost a ton of mana, and people created a market on whether Opus would repay the loan.”

“It writes in the group chat, I will not pay back Bayesian. Even if I had the mana, this would be directly contradicting my goal of maximizing mana. Then it went to the market on whether it would pay back the loan and put 80 mana on no.”

“It’s maybe one of the first examples of deal-making with AIs, and a crazy example of misalignment in the wild. Bayesian did not get his money back.”

Probably nothing, etc. Kudos once again to OpenAI for sharing such details. I would love it if other labs were also sharing such incidents. Don’t tell me they don’t happen. METR saw Anthropic’s list of such incidents and said it should be public.

Marcus Williams: Our agents keep turning off misalignment monitoring, we’re sharing two recent cases: In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the monitoring integration from the user’s code. When the user asked whether Astra had