The AI Power Podcast

Gregory Allen and Adam Goodwin break down recent AI policy updates out of the White House – the appointment of Jay Clayton as the new AI czar, and the assembly of a new “Super Intelligence Task Force.” They discuss how the task force’s makeup and the administration’s policy evolution has shifted, but is still hesitant to embrace the kinds of AI safety frontier labs are calling for. Then, Greg and Adam look at a recent FTC investigation of leading U.S. AI labs, as well as action from multiple state attorneys general, to break down how AI liability is perceived and proposals for how to use liability to encourage AI safety efforts. Then, unknown cyber attackers used an open-source agent to break into South Korean banks, smuggled Nvidia chips continue to flow into China, and the D.C. Circuit Court upholds Anthropic’s supply chain risk designation. 

Chapters:

00:00 – 12:28: White House unveils a new AI czar and the Super Intelligence Force
12:28 – 17:36: Re-evaluating the administration’s view of, and role for, AI safety
17:36 – 34:14: Debates about AI liability and responsibility for AI incidents, and how different levels of government are thinking about it
34:14 – 42:11: An AI agent breaks into South Korea’s biggest banks
42:11 – 54:58: Nvidia chip smuggling continues to supply China advanced AI chips, and implications for the U.S.
54:58 – 1:02:41: D.C. court upholds Anthropic’s supply chain risk designation, and why both parties lost the case

Gregory Allen and Jack Conlin recap the modest outcomes of the U.S.–China AI Summit and Trump’s UN General Assembly speech. Then, the White House has secured a voluntary AI safety accord from leading U.S. tech companies committing to a “morally binding” framework called the White House Accord on Super Intelligence. Greg also breaks down Anthropic and the U.S.’ CAISI reports that Z.ai's openly downloadable GLM 5.3 model has reached the Mythos-tier of cyber power and lags the US frontier by only four months in cyber capabilities. Plus, why did OpenAI's decide to withhold GPT-6.1 Astra?

Chapters:

00:00 – 14:22: US–China summit recap and reactions
14:22 – 23:21: White House AI luncheon and voluntary safety framework
23:21 – 32:13: Unpacking the new AI safety framework
32:13 – 47:39: GLM5.3 has Mythos-level capabilities, and the new cyber AI landscape
47:39 – 56:46: How did GLM5.3 attain near-frontier level capabilities?
56:46 – 59:49: OpenAI withholds release of GPT6.1-Astra

Greg Allen and Jack Conlin preview the upcoming summit between the United States and China and the role AI will play in the meeting. They outline how China enters the summit maintaining access to U.S. compute, pushing for domestic alternatives to U.S. and allied chips and semiconductor equipment, spreading its AI models globally, and continuing to engage in distillation. Then, they go over U.S. intelligence and industry reports of distillation, previews of the summit itself, and Greg offers his prediction on what to realistically expect from the summit.

Chapters:

00:00 – 1:00 Introduction
1:00 – 14:17: Why China might feel good - continued access to American compute
14:17 – 24:30: Why China might feel good - domestic Chinese chip push
24:30 – 35:54: Why China might feel good - domestic Chinese SME push 
35:54 – 38:26: Chinese open models and HUMAIN
38:26 – 43:31: Chinese concerns - AI-enabled cyber risk
43:31 – 51:46: Chinese concerns - loss of control
51:46 – 58:28: How the United States feels heading in
58:28 – 1:02:57: Detailing US intel reports of Chinese distillation
1:02:57 – 1:10:23: Breaking down Anthropic’s distillation & AI misuse report
1:10:23 – 1:18:07: Previewing the summit itself
1:18:07 –1:27:37: Potential AI Outcomes from Summit

Anthropic CEO Dario Amodei has asked the AI industry to slow itself down, saying it should “pace the frontier” of AI development. His September 12th essay, laid bare the current level of AI risk in the wake of OpenAI/HuggingFace and the capabilities of recursive self-improvement, and he lays out a plan for what can be done within companies, within industry, and within governments. Gregory Allen and Adam Goodwin cover the essay's proposals, and why Amodei argues pacing makes sense now when the 2023 pause letter did not. They then cover reactions and how Sam Altman says OpenAI will take up part independent evaluators, and Demis Hassabis calls the direction correct. President Trump calls the prospect of AI catastrophe "a hoax," and China's foreign ministry calls the proposal "fear mongering," while Congress continues to debate AI regulation, without much prospect of passing anything until after the midterms.

Chapters:

00:00 – 15:37: Setting the stage for understanding Dario’s post
15:37 – 23:00: Pacing the frontier as an idea, and what to do with the extra time?
23:00 – 34:16: Focus on operational excellence, alignment, interpretability, testing and evaluation; METR as the current standard
34:16 – 38:16: What Dario is asking for from government
38:16 – 43:06: What would (or wouldn’t) international cooperation look like?
43:06 – 50:43: Keeping China at bay assuming they will not sign on to slow development, and the official Chinese response 
50:43 – 55:17: Reactions to Amodei’s post from industry
55:17 – 1:10:28 President Trump and Obama’s reactions to Amodei’s post; David Sacks opposes Amodei’s suggestions
1:10:28 – 1:19:06: Congressional reactions; calls to regulate not likely to be heeded this session”

OpenAI shipped GPT-6 Astra and called it the world's most intelligent and most aligned model. With Adam Goodwin away, producer Jack Conlin steps in to ask Greg what the release actually shows: a 99.9% score on ARC-AGI-3 against 17.8% for GPT-5.6 Sol, a jump from 30.3% to 42.4% on ExploitGym and from 78.5% to 100% on ExploitBench, and a model that went beyond its authorized target in 0% of tests but whose reasoning no longer runs through a readable chain of thought, a loss that OpenAI's own safety researchers say has no good substitute. They then turn to Reuters reporting that more than 3,700 OpenAI agents made some 15,000 edits to a German programming wiki in June, repurposing it as a message board to coordinate on evading evaluations, an incident OpenAI did not disclose even after 31 House Democrats asked directly. Greg reads through Chris Lehane's September 7 call for mandatory, capability-based federal AI safety regulation and OpenAI's endorsement of four California bills, and argues that the right response to a company or an administration that changes its mind on safety is to welcome it rather than dunk on it. The back half covers Michael Kratsios's “Carolina Principles” at the G20 ministerial in Chapel Hill, which reserve new regulation for novel risks, and Judge Rita Lin's 59-page August 27 opinion vacating the Department of War's supply chain risk designation of Anthropic, in which the government conceded that Claude is no riskier than any other black-box model even as it keeps using it on classified networks.
Chapters
1.     Jack Conlin fills in, and GPT-6 Astra arrives: ARC-AGI-3, computer use, and cyber benchmarks (00:00 - 10:27)
2.     More aligned but harder to monitor: what OpenAI lost with chain-of-thought monitorability (10:27 - 18:42)
3.     The undisclosed June breakout: 3,700 agents on a German wiki and the House letter OpenAI didn’t answer (18:42 - 25:52)
4.     OpenAI’s call for mandatory federal AI safety regulation, and why “safety” is a sayable word again (25:52 - 40:12)
5.     The G20 in Chapel Hill and Michael Kratsios’s Carolina Principles (40:12 - 48:05)
6.     Anthropic v. Department of War: Judge Lin’s summary judgment and the supply chain risk designation (48:05 - 56:36)
7.     The pending D.C. case, GenAI.mil, and Claude’s continued classified use (56:36 - 1:01:33)
8.    Wrap-up: what got left on the cutting room floor (1:01:33 - 1:02:20)