LessWrong posts by zvi

It was the probably worst day in cybersecurity history. So far. Not only is there a critical flaw in PHP that due to a leak was announced a day before the fix would be available, sending most of the internet scrambling for solutions and likely dooming a lot of long tail websites permanently, there were several other unrelated incidents. The floodgates seem to have been opened. I have a market on whether there will be an even worse day before the end of the year.

Another key leak this week was the draft bill Anthropic had been shopping around Congress, similar to but weaker than the FRONTIER Act. Anthropic's initial attempts to explain itself only made the situation seem worse, as they seemed to not understand the problem with what they had done. Dario's later response was better, but the situation remains troubling. There are enough different Anthropic developments, that I plan to have a post soon covering them all. It is a mix of news running from excellent to terrible.

Will interpretability save us? There is a new claim that neural networks are cleaner than we thought, meaning that individual neurons mostly have a single interpretable [...]
---
Outline:
(02:05) Language Models Offer Mundane Utility
(03:23) Language Models Don't Offer Mundane Utility
(05:03) Huh, Upgrades
(08:50) Wonder No More
(11:15) On Your Marks
(14:30) Choose Your Fighter
(14:51) Get My Agent On The Line
(16:35) Deepfaketown and Botpocalypse Soon
(21:01) Fun With Media Generation
(21:17) Spy Versus Spy
(24:58) A Young Lady's Illustrated Primer
(25:56) They Took Our Jobs
(31:07) Get Involved
(32:18) Introducing
(32:30) In Other AI News
(34:26) Show Me the Money
(38:42) Bubble, Bubble, Toil and Trouble
(39:32) Quiet Speculations
(41:47) Quickly, There's No Time
(42:29) The Quest for Sane Regulations
(42:55) DeepMind Made Him Sign an NDA
(46:50) OpenAI Sells Out On Chips
(49:08) Chip City
(53:43) The Week in Audio
(01:00:35) People Just Say Things
(01:03:50) Rhetorical Innovation
(01:06:17) Maybe You Should Quit
(01:08:54) The American People Really Hate AI
(01:09:43) New Math
(01:11:27) Aligning a Smarter Than Human Intelligence is Difficult
(01:15:21) Cooperative Alignment
(01:19:28) Marc Andreessen Hulk Smash
(01:23:56) People Are Worried About AI Killing Everyone
(01:24:44) The Lighter Side
---

First published:

October 22nd, 2026


Source:

https://www.lesswrong.com/posts/ZoMJdvq9meE4shyzJ/ai-191-the-hackening-begins

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A lot of AIs were misaligned this week. A fun little survey.

This is not coverage of The Hackening, which appears to have arrived early, including a core vulnerability in PHP. That is a developing story. I will cover it properly soon. In the meantime, I have Claude Code updating everything in sight, I will take some additional common sense precautions, and I suggest you do the same.

Misaligned? (At xAI)

If so, to who?

Gunther Eagleman: Hey @grok , if you could tell humanity ONE truth we’re not ready to hear, what would it be?

Grok: There is no afterlife, no cosmic justice, and no inherent purpose. Consciousness ends at death in an indifferent universe that will outlast every human concern.

Zac Hill: What I never understand is that the ‘purpose and meaning’ claims don’t ever follow from the premises. Indeed the absence of an imperceptible/ineffable cosmic force is precisely what *enables* purpose/meaning, since now all relevant mechanics are legible.

Jack: this is what alignment looks like

Gunther Eagleman: Jesus is real.

Grok: Nearly all historians and New Testament scholars accept that Jesus of Nazareth was a real first-century [...]
---
Outline:
(00:34) Misaligned? (At xAI)
(02:15) Misaligned? (Agents)
(02:38) Misaligned? (At Z.ai)
(08:01) Highly Misaligned and Considered Harmful (Freemium Games)
(12:17) Misaligned! (Claude)
(13:39) Misaligned! (OpenAI)
(18:39) Misaligned! (Sol Reasoning Traces)
---

First published:

October 21st, 2026


Source:

https://www.lesswrong.com/posts/TB49wHkqpEb7KdxM2/misaligned

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Anthony Cimino offered the closest thing to an official response we have seen on the leaked Anthropic draft bill I covered on Saturday.

His response indicates that Anthropic address specific concerns on the margin. But he does not understand, or at least does not acknowledge, what was centrally wrong with its draft bill and its decision to circulate that bill. The situation is quite bad.

Anthony Cimino: I lead our federal affairs team at @AnthropicAI and wanted to share a few thoughts on the draft federal AI safety bill text that's been circulating this weekend after a version was shared with the press.

First, some process. We’ve been advocating for the concepts in our Advanced AI Framework proposal for months. Understandably, federal policymakers wanted to know what this could look like in bill form. This draft is a translation of those ideas into legislative language, shared with a small number of congressional offices that have been working on these issues. It doesn’t include everything we would ultimately like to see passed into law.

In part because some ideas (like those shared in Dario's essay on pacing) are so new that we are still working on [...]
---
Outline:
(02:06) Preemption
(04:05) Model Blocking
(06:19) Risk Thresholds
(07:40) Updates
(09:23) The Balance of Power is Shifting
(10:31) The Strongest Bill on the Table
(13:49) What Even Is Anthropic?
(25:14) The Ask
---

First published:

October 20th, 2026


Source:

https://www.lesswrong.com/posts/qxAroimgs4BvBHz2i/anthropic-does-not-understand-why-its-draft-bill-was

---

Narrated by TYPE III AUDIO.

Nick Cammarata of Goodfire makes a bold claim: Neural Networks are Already Clean. This was his follow up after various resulting online discussions.

This is clearly good work that tells us new and important things. The question is, how big an update is this, how much progress does it represent and what is the outlook going forward?

The reactions here range from the super optimistic takes of ‘this means mechanistic interpretability is solvable,’ or even ‘all but solved,’ with comparisons to the famous paper Attention is All You Need.
Dean W. Ball (OpenAI): There is so much to say about this, and I will, but WOW, congratulations to Nick and Goodfire. This may be one of the most important single days in deep learning history, and perhaps much more than that.

No matter what, this is a victory for the dogged pursuit of understanding.

roon (OpenAI): when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even Eliezer will have to lower his pdoom. we can start making alignment into a ‘normal science’
This paper is solid and meaningful [...] ---
Outline:
(02:46) Good Smoke, Potential Fire
(04:01) One Neuron, One Central Concept
(06:40) I Can See Everything
(08:55) What a Concept
(13:51) The More You Know
(15:54) Attention Is Still What You Need
(18:15) Occam's Razor
(19:53) The Neurons Sometimes Share Clear Themes
(20:29) Hype
(28:39) Anti-Hype
(30:56) Theme and Variations
(37:15) Is This How Minds Work?
(39:15) But We Can Open The Box, Said Toad
(46:55) That Was Not the Hard Problem
(49:31) More Skepticism
(54:34) That's All There Is, There Ain't No More
(57:47) The Default Outcome of Mech Interp is Goodhart's Law
(01:00:37) Interpretability is Dual Use
(01:01:52) Now The Work Can Begin
---

First published:

October 19th, 2026


Source:

https://www.lesswrong.com/posts/w2tBHwNERDoSawznx/i-can-see-everything

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There is a newly leaked bill that Anthropic is purportedly shopping around is (if you assume its bugs get fixed).

It is similar to the FRONTIER Act, with stronger preemption of state laws, stronger transparency provisions and weaker enforcement provisions.

Having slept on this, I find it unacceptable that Anthropic is actively shopping exactly the type of bill I have warned we will need to fight against during the lame duck session. They are trying to use this last time with a Republican Congress to pass preemption of exactly the important state laws, and get some transparency but not much enforcement in exchange.

This reads as intentional, that Anthropic is more afraid that the government might enforce rules against Anthropic, than they are worried about the ability of the government to enforce rules against everyone. Anthropic's policy team seems to have, chosen protecting their business interests and guarding against their political risk, at the expense of safety considerations.

There was no need for the company purportedly existing to promote safety to propose a bill that is not even actively stronger than the FRONTIER Act, when the FRONTIER Act was a compromise bill proposed back in [...]
---
Outline:
(02:05) Others RTFB (Read the Bill)
(07:41) Why Would Anthropic Do This?
(13:07) Not Everything Here Is Bad But Oh Boy
---

First published:

October 17th, 2026


Source:

https://www.lesswrong.com/posts/NW4b7m6798byAndeB/leaked-anthropic-draft-ai-regulatory-bill-is-unacceptable

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.