LessWrong posts by zvi

OpenAI claims that Astra is ‘the most intelligent and most aligned [available] model’ in the world. Not the most intelligent and aligned OpenAI model, but the most period.

That is bold talk. It risks overstepping, and by doing so souring the release of what is clearly an excellent model. As do the severe problems with monitorability.

It also raises the question of what they mean by ‘most aligned model.’ How do they define ‘aligned.’ Why do they think it is more aligned than Claude Fable 5.1?

keltan : “a significant step forward in […] alignment.”

Buddy, how tf are you measuring ‘alignment’? Would love to know because being able to measure that would save the fucking world.

roon (OpenAI): low rates of cheating

Rob Miles: *detected cheating

keltan : Thank you for clarifying. But you know what I’m gonna say next, right?

roon (OpenAI): that this metrics are not a full solve of alignment and will break discontinuously

keltan : Yep. But I would have said it in a dumber way. Something like: Low Rates of Cheating ≠ Alignment

roon (OpenAI): I agree but also in some real sense [...]
---
Outline:
(04:12) OpenAI's Safety Claims About Astra (1)
(08:33) Preparedness Capabilities Assessment (10)
(09:00) Biological and Chemical Capability is High
(10:33) Cybersecurity Capability is Critical
(17:05) AI Self-Improvement Capabilities (10.1.3)
(17:45) Astra Is Highly Verbally Eval Aware (from 8.6)
(19:05) Safe Mundane Completions (4.1)
(21:05) Jailbreaks (5.1)
(22:21) Prompt Injection (5.2)
(23:41) Health (6)
(24:22) Hallucinations (7)
(24:57) Alignment (8)
(26:06) Obeying Restrictions (8.2)
(29:19) That's Worse, You Do Get How That's Worse, Right?
(30:45) OpenAI Does Not Understand Why This Is Worse
(34:52) The Alternative Explanation Is Also Worse
(42:39) Metagaming (8.7)
(45:07) Alignment Faking (8.7)
(46:19) Don't Lie to the User (8.3)
(47:25) Misalignment in Realistic Work Environments (8.4)
(48:05) Unintended Agent-to-Agent Communication (8.5)
(49:48) The Three Obviously Monitored Temptations of Astra
(51:26) Severe Issues In Simulated Traffic Are Down By Half
(53:01) UK AISI External Evaluations (8.8)
(57:02) Sabotaging Safety Work
(57:29) What About The July 19 Attacks?
(58:58) Apollo Research External Evaluations (8.8.1)
(01:00:01) The Alignment Verdict
(01:02:04) It Depends What You Mean By Alignment
---

First published:

September 9th, 2026


Source:

https://www.lesswrong.com/posts/AmFJZyeCgvFjNKgNk/gpt-6-astra-the-system-card-alignment-and-what-comes-next

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI's central message on Astra is that it is three things:


Highly capable and can do all the things for you.

Hard to monitor.

The most aligned model.

The first claim largely checks out. Astra and Fable are both clearly excellent models.

This post is about their second claim, which to their credit they are being loud about, in three parts:


The system card result, affirmed on Twitter by several OpenAI employees including Tomek Korbak, and in an excellent post by Chief Scientist Jakub Pachocki that I covered yesterday, that Astra is harder to monitor.

OpenAI's use of recurrent depth and the internet's immune reaction, including some people reading too much into what happened there.

Discussion of what to do about monitorability going forward, to fight for it and avoid a race to the bottom.

In An Alien Mind, Jakub Pachocki makes clear OpenAI's primary bet is on CoT monitoring. He also outright says, ‘our ability to rely on CoT monitoring is progressively diminishing.’ The system card confirms both claims.

This combination should freak you out, with a side of existential dread.

Chain [...]
---
Outline:
(03:39) Monitorability is Defense in Depth That Is Already Flailing
(06:03) OpenAI Is Counting On Monitorability
(07:44) How They Tested For Monitorability
(09:50) Non-Adversarial Monitorability (9.1)
(10:58) Astra Shortens Its CoT When It Knows It Is Being Monitored And Is Doing Something Bad
(12:30) Sandbagging Task Kind of Passes, Kind of Fails (9.2.2.3)
(14:12) OpenAI Does Not Believe It Could Catch Sandbagging
(15:15) OAI-Repo Sabotage v.2
(18:17) The Secret Police Do Not Make Your Notebook Useless
(19:38) CoT Controllability Is Up (9.2.1)
(21:47) Astra Cannot Make Itself More Monitorable On Demand
(22:18) Steganographic Chain of Thought May Be Within Reach
(23:59) Astra Cannot Yet Sneak Math Side Tasks Past CoT Monitors (9.2.2.4)
(25:13) UK AISI Monitorability Assessment (9.3)
(27:53) Monitorability Declines Seem Unlikely To Be Only Capability Gains
(32:28) Part 2: Recurrent Depth
(35:19) The Immune System Responds
(41:49) Ryan Greenblatt Explains How Bad This Could Be
(46:01) Only Law Can Prevent Extinction
(49:55) OpenAI Calls On Us to Avoid Racing to the Bottom
(57:10) Thinking Fast and Slow, Also Small and Large
(01:04:53) Talking Price
(01:06:24) Conclusion: If The House Burns Down, Halt and Catch Fire
---

First published:

September 8th, 2026


Source:

https://www.lesswrong.com/posts/HCRs8btkiamtWSNAL/astra-is-hard-to-monitor

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI Chief Scientist Jakub Pachocki is dropping truth bombs.

Tomorrow I will discuss Astra's lack of monitorability, and the potential contributing factors to that. The situation is alarming and should freak you out, and briefly it looked, in the wake of leaked architectural changes, like the situation might be even more alarming than it is. Jakub rushed to try and head off misunderstandings that might lead to a race to the bottom on monitorability.

Table of Contents


An Excellent Warning.

Branches of the Tech Tree.

Universally Better Is Not Required.

Alignment To What and To Whom.

Monitorability.

The Case For Not Stopping.

Pacing the Next Frontier.

Mea Culpa Cascade.

The Calls Are Coming From Inside the House.

Actions Speak Louder.

An Excellent Warning

Jakub Pachocki has now fleshed out his full position on the current state of play.

Here are his key points, translated into my own voice:


Smarter than human intelligence is coming in our lifetime.

Based on internal results, he expects recursive self-improvement in a few years.

No one is prepared for the consequences.

[...]
---
Outline:
(00:38) An Excellent Warning
(05:40) Branches of the Tech Tree
(06:46) Universally Better Is Not Required
(08:01) Alignment To What and To Whom
(11:49) Monitorability
(14:23) The Case For Not Stopping
(15:01) Pacing the Next Frontier
(18:27) Mea Culpa Cascade
(22:51) The Calls Are Coming From Inside the House
(25:38) Actions Speak Louder
---

First published:

September 7th, 2026


Source:

https://www.lesswrong.com/posts/8E6ng6CseuzafSxQR/an-alien-mind-jakub-pachocki-warns-us

---

Narrated by TYPE III AUDIO.

I did not expect to be back here so soon with more OpenAI agent swarm coverage.

And yet, here we are.

It turns out that the whole time, there was a different, true First Message Board, and also a bunch of other additional message boards, scattered across the internet.

They were created by agents that were assigned ordinary harmless web search tasks.

Based on OpenAI IPs visiting the associated Wiki right before all activity ceased, among other evidence, OpenAI knew about it, including before the HuggingFace hack.

They decided not to tell us until researchers published the story, complete with data explorer. OpenAI excluded this from potential investigation by METR and Redwood.

When challenged, OpenAI tried to downplay this.

It is true that these incidents do not show the AIs exhibiting new capabilities that we did not see from later events. But these events are important missing pieces of the puzzle, including explaining the origin of the ‘zz’ prefix, the definitive demonstration that the underlying task can be fully harmless, and the fact that OpenAI knew about it while making their decisions. Whoever decided not to disclose this made a very, very [...]
---
Outline:
(02:26) I Don't Think They Know About First Message Board
(03:20) The New Extended Timeline
(04:42) The Researchers Explain What Happened This Time
(12:55) They Also Don't Know About All These Other Message Boards
(14:33) OpenAI Knew and Did Not Tell Us
(16:54) OpenAI Tries To Downplay the 'Wiki Incident'
(21:03) This Was a Cover-Up
(22:46) Schelling Points and Last Ditch Efforts
(26:20) Can We Finally Dispose Of The 'You Told It To Hack' Narrative?
(28:04) So Much And Yet So Little
---

First published:

September 6th, 2026


Source:

https://www.lesswrong.com/posts/PtJpGurfw7JTxHfmg/openai-and-the-wiki-incident

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is the weirdest situation in which to write a capabilities review.

Introducing the world's most powerful model, by a substantial margin. No wait, this just in, we also have someone else introducing the world's most powerful model.

Claude Fable 5.1 and GPT-6 Astra are both excellent models. This much, we know.

Fable 5.1 comes with reduced cache prices, the option of zero data retention and substantially more lenient classifiers than Fable 5.

Early signs are, with large error bars, that the jump from Sol to Astra is bigger and more exciting than the jump from Fable 5 to Fable 5.1. This may be similar to how the scaling move from Opus to Fable was a big deal.

With the exception of token use, Fable 5.1 got almost universally positive feedback in absolute terms. Reports are that Fable 5.1 is highly well-rounded. Writing is greatly improved. The Claudisms seem to have improved, although some are very much still there. It admits mistakes. People enjoy their conversations. Several people noted it simplifies code. The safety classifiers are less obnoxious.

Fable 5.1 loves being proactive and doing all the things. If you give it a [...]
---
Outline:
(02:30) The Official Pitch
(04:26) Our Price Cheap
(05:46) Zero Data Retention and Reduced Safeguards
(07:11) Official Benchmarks
(13:14) Other People's Benchmarks
(16:02) The System Prompt
(16:12) The Blurb Pitches
(18:17) The Every Review Is In and It's Very Good
(21:47) Positive Reactions
(32:03) Our Price Cheap But Only Per Token
(36:50) Negative Reactions
(38:14) Early Whispers
(38:33) Weapon of Choice
---

First published:

September 5th, 2026


Source:

https://www.lesswrong.com/posts/QHoF3tJvryRtmAmMg/claude-mythos-5-1-and-fable-5-1-capabilities

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.