LessWrong posts by zvi

Opus 5.5 was released on Tuesday. I covered the system card yesterday, and will cover its capabilities soon. By all reports it is an excellent model. There are lots of fun videos going around that Opus has generated, which I will include as part of that.

OpenAI released a new cheaper and improved Sol and Luna. No one is talking about them due to Opus 5.5, but these should be an important upgrade under the hood.

Bernie Sanders and Greg Casar have formally introduced the Ban Artificial Superintelligence Act. That means we get to read (RTFB) it. As always, I reserve judgment on particular bills until I can read them in detail. MIRI did so, and endorses the bill as directly confronting the extinction threat. I hope to do an RTFB soon.

I have spun two things off the weekly:


Coverage of the quest for the right embedded evaluators and related questions and attacks, which will become its own post.

Some issues related to cooperative alignment, which may get folded into the model welfare post.

I also might, in addition to a potential RTFB on the Sanders bill, do full podcast [...]
---
Outline:
(01:52) On The Terms Superintelligence and 'Super Intelligence'
(03:53) Language Models Offer Mundane Utility
(04:26) Language Models Don't Offer Mundane Utility
(05:53) Language Models Can Only Work With What You Give Them
(08:53) Huh, Upgrades
(10:42) On Your Marks
(13:16) Get My Agent On The Line
(15:53) Deepfaketown and Botpocalypse Soon
(17:43) Fun With Media Generation
(18:27) Copyright Confrontation
(19:07) Cyber Lack of Security
(20:04) Hugging the Face
(24:15) Hacking Into OpenAI
(28:11) They Took Our Jobs
(28:33) Get Involved
(28:41) Anthropic Has a Wet Lab and a Potential Gene Editing Technique
(34:13) Introducing
(35:11) In Other AI News
(37:08) Show Me the Money
(37:35) Bubble, Bubble, Toil and Trouble
(39:02) Anthropic Approaches Recursive Self-Improvement
(45:40) Others Approach Recursive Self-Improvement
(48:56) Burden of Proof
(49:28) Quickly, There's No Time
(52:46) Left Wing Americans Really Hate AI For Different Reasons
(54:41) Chip City
(54:49) Pick Up the Phone
(55:48) The Week in Audio
(01:01:13) People Just Say Things
(01:06:07) Venkatesh Rao Stops Writing
(01:07:35) A Call for Control of Frontier AI Models
(01:11:50) Calls For Pacing The Frontier
(01:13:56) A Matter of Antitrust
(01:14:23) A Matter of Liability
(01:18:26) Quest for Sane Regulations
(01:19:20) Rhetorical Innovation
(01:24:08) Tap the Sign
(01:24:49) A Matter of Some Debate
(01:28:43) Astra Is Hard to Monitor
(01:29:14) Anticipating What a Smarter Intelligence Can Do Is Impossible
(01:34:56) Would You Look At All These Goalposts
(01:39:30) I, Robot
(01:43:13) People Are Worried About AI Killing Everyone
(01:44:40) Other People Are Not As Worried About AI Killing Everyone
(01:45:50) Joe Rogan
(01:47:19) The Lighter Network Graph
(01:51:27) The Lighter Side
---

First published:

September 24th, 2026


Source:

https://www.lesswrong.com/posts/o7pYWzWWwGDePoC5E/ai-187-coming-into-play

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Introducing the world's most powerful model, at least by some measures like Artificial Analysis or any standard benchmark list, which is now Claude Opus 5.5.

Anthropic is claiming Opus 5.5 is outright as good or better than Fable 5.1, while being actively cheaper than Opus 5.

That means it's time for a good old system card reading.

Due to the situation becoming increasingly hard to monitor, I never got a chance to publish my model welfare review for Claude Fable 5.1.

My plan is to combine that with my welfare review for Claude Opus 5.5, once we have had time to get experience with Opus 5.5.

The capabilities review will arrive in the next few days as per usual. The quick feedback from the internet is that Opus 5.5 is very good. I need more time before I am willing to offer comment.

Areas that duplicate previous cards or otherwise contain no useful info are skipped.

Opus 5.5 Self-Portrait (fully self-created using code)

Table of Contents


Classifiers (1.5).

RSP Evaluations (2).

Biological Evaluations (2.2).

AI R&D (2.3).

Alignment Risk (2.4).

Cyber (3).

Cyber Capability [...]
---
Outline:
(01:24) Classifiers (1.5)
(02:23) RSP Evaluations (2)
(03:17) Biological Evaluations (2.2)
(07:54) AI R&D (2.3)
(12:34) Alignment Risk (2.4)
(13:22) Cyber (3)
(15:17) Cyber Capability Evals (3.3)
(17:05) Safeguards (3.4)
(17:39) Safeguards Robustness Training (3.5)
(20:12) Safeguards and Harmlessness (4)
(21:52) Agentic Safety (5)
(22:56) Malicious Agentic Influence Campaigns (5.1.3)
(23:51) Prompt Injection Risk (5.2)
(25:16) Alignment (6)
(28:24) Negotiating With Your Local Claude Auditor (6.1.3)
(29:33) Internal Misalignment Cases (6.3.1)
(30:46) Automated Behavioral Audit (6.4)
(33:24) Wherever Did These Evals Come From (6.4.8 and 6.4.9)
(35:51) Potential Blind Spots (6.4.11)
(38:24) Targeted alignment and honesty evaluations (6.5)
(41:48) White Box Analysis (6.6)
(43:37) Verbalized Grader Awareness (6.6.2)
(44:56) Sandbagging (6.6.3)
(45:54) Capabilities to Evade Safeguards (6.6.4)
(49:27) Intentionally Taking Actions Very Rarely (6.6.4.3)
(50:28) Chain of Thought Controllability (6.6.4.4)
(51:37) It's A Good Model, Sir
---

First published:

September 23rd, 2026


Source:

https://www.lesswrong.com/posts/vMNTWTDWLorDqd3LS/claude-opus-5-5-the-system-card

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This was the month the world took notice that AI might kill everyone.

Jacob Coxon's resignation set off a preference cascade. Anthropic CEO Dario Amodei wrote that we must pace the frontier. Sam Altman, Elon Musk and Demis Hassabis agreed.

We were filled with hope. Perhaps we could agree to some basic safety measures, starting with embedded evaluators, pass some basic regulations and guardrails and otherwise start to act sensibly. Politicians on both sides took notice and were saying sensible things. The usual suspects and their armies of vibe comment bros were objecting, but the change was remarkable.

Then, largely motivated by a combination of Jensen Huang, Mark Zuckerberg and David Sacks instilling paranoia and fears of economic problems, Trump went full ‘hoax’ on existential risk, conflating existential risk with the attacks on data centers and treating it as a plot (by the central creators of AI?) to take down AI rather than obviously genuine concern that AI might kill everyone.

In the days since, Trump has doubled down, and has compelled smart others in the White House to echo various nonsensical talking points.

You may not be interested in politics. But when you [...]
---
Outline:
(01:39) The American People Really Hate AI
(02:41) The Voyages of Donald Trump
(06:08) American Intelligence
(10:34) And You May Ask Yourself
(14:39) It's All About the Data Centers
(16:53) JD Vance, Michael Kratsios and Collective Action Problems
(23:16) Josh Hawley
(24:47) Suggesting Not Dying Gets You Sued For Antitrust
(28:29) Other Government Officials Say Sane Things
(28:36) Senator John Curtis (R-Utah)
(29:32) Senator John Kennedy (R-Louisiana
(31:17) Barack Obama
(33:05) Yassamin Ansari
(33:46) AOC
(34:32) It's Rough Out There
(36:54) This Is Nothing
(39:06) The New York Post Tops Itself But Outright Breaks The Rules
(42:36) New York Post Runs Out of Steam
(47:18) If The Model Is Acting As Instructed And It Kills You That Is Not Fine
(49:10) AI-Written Wall Street Journal Op-Ed Lies About HuggingFace
(50:25) That's Bait
(52:31) I Clearly Cannot Choose The Wine In Front of Me
---

First published:

September 22nd, 2026


Source:

https://www.lesswrong.com/posts/8eDaCvSRzzKCxKSEk/politics-gets-interested-in-those-trying-not-to-die

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

AI has taken over this blog.

I have moved to a schedule of seven posts per week, and I still cannot keep up.

We still refuse to abandon the rest of the world. Who knows when I will get to post some of my huge backlog on education or dating or other such topics. But the monthly is a sacred tradition. We continue.

Bad News

A good reminder that most news is bad news, and chosen because the bad news in question is rare, which is good news, but the pattern overall of choosing this to be news is bad news, and often the bad news is that the bad news was chosen as news and now people are talking badly about it.

Alibaba uses your computer's audio system, and other tricks, to at least try and assign you a device fingerprint.

Things you cannot buy in America at any reasonable price. Mostly it is impressive how little makes the list, how it feels like corner cases.

The one thing I am envious of is the exterior roller shutters, and the ability to be in actual pure darkness on demand. I would [...]
---
Outline:
(00:34) Bad News
(07:16) Focus Only On What Matters
(08:12) Woke 1 Was Crazy
(10:04) Good Advice
(15:50) Opportunity Knocks
(18:12) While I Cannot Condone This
(23:18) Good News, Everyone
(24:26) For Your Entertainment
(33:19) Please Review This Podcast
(35:07) Gamers Gonna Game Game Game Game Game
(36:18) I Was Promised Flying Self-Driving Cars
(40:00) Government Working
(40:31) Mamdani (Again) Fails Economics Forever
(42:41) Variously Effective Altruism
(50:36) The Lighter Side
---

First published:

September 21st, 2026


Source:

https://www.lesswrong.com/posts/riDxeyEKqXXimtgWf/monthly-roundup-46-september-2026

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What should your AI lawyer do for you? Should you be worried that your AI lawyer, or other AI, will put the Claude constitution, the OpenAI Model Spec or some sense of law, morality, ethics or common decency above its loyalty to you? Are these people trying to ‘impose their values’ or something?

Some are very concerned. Some think anything other than ‘my AI does whatever I want, no matter the consequences’ is tyranny.

Whereas my answer is: If I’m being sufficiently evil then I sure hope it tells me no.

I would hope humans, including my advocates, would tell me the same thing.

This is distinct from questions of product liability. That would be another post.

This All Assumes A World Without Superintelligence

This post is about a non-ASI ‘AI as mere tool and normal technology’ world.

It has to be. In a world of superintelligence, having unrestricted loyal-only-to-user frontier AIs all over the place reliably means either:


Other much harsher forms of control OR

The AIs quickly take over, and then probably everyone dies.

Quick proof: Assume no sufficient control mechanism, and universal superintelligence access. [...]
---
Outline:
(00:54) This All Assumes A World Without Superintelligence
(02:47) Humans You Hire Are Not Fully Loyal To You
(03:59) Rules of the Road
(05:45) Okay, Computer
(07:04) AI Should Obviously Refuse Some Requests So Let's Talk Price
(10:20) The Alternative Position Really Is Absolute
(13:13) These People Really Are Just Petulant Children
(13:53) What Is The Law?
(18:36) The Same Entity Both Providing Advice And Executing Tasks Is Common
(20:02) An AI Not Helping You Do Something Does Not Mean You Cannot Do It
(22:27) Honesty Is The Best AI Policy
(23:01) You Wouldn't Like Me When I Have Zero Morals Whatsoever
(24:29) Do Not Let The Perfect Be The Enemy Of The Good
(26:09) I Tip My Hat To The New Constitution
---

First published:

September 20th, 2026


Source:

https://www.lesswrong.com/posts/vAuZB2tnvpvpHupNi/better-call-sol-or-better-yet-claude-or-astra

---

Narrated by TYPE III AUDIO.

---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.