Latent Space: The AI Engineer Podcast

• Scaling Test Time Compute to Multi-Agent Civilizations — Noam Brown, OpenAI

  Solving Poker and Diplomacy, Debating RL+Reasoning with Ilya, what's *wrong* with the System 1/2 analogy, and where Test-Time Compute hits a wall Timestamps 00:00 Intro – Diplomacy, Cicero & World Championship 02:00 Reverse Centaur: How AI Improved Noam’s Human Play 05:00 Turing Test Failures in Chat: Hallucinations & Steerability 07:30 Reasoning Models & Fast vs. Slow Thinking Paradigm 11:00 System 1 vs. System 2 in Visual Tasks (GeoGuessr, Tic-Tac-Toe) 14:00 The Deep Research Existence Proof for Unverifiable Domains 17:30 Harnesses, Tool Use, and Fragility in AI Agents 21:00 The Case Against Over-Reliance on Scaffolds and Routers 24:00 Reinforcement Fine-Tuning and Long-Term Model Adaptability 28:00 Ilya’s Bet on Reasoning and the O-Series Breakthrough 34:00 Noam’s Dev Stack: Codex, Windsurf & AGI Moments 38:00 Building Better AI Developers: Memory, Reuse, and PR Reviews 41:00 Multi-Agent Intelligence and the “AI Civilization” Hypothesis 44:30 Implicit World Models and Theory of Mind Through Scaling 48:00 Why Self-Play Breaks Down Beyond Go and Chess 54:00 Designing Better Benchmarks for Fuzzy Tasks 57:30 The Real Limits of Test-Time Compute: Cost vs. Time 1:00:30 Data Efficiency Gaps Between Humans and LLMs 1:03:00 Training Pipeline: Pretraining, Midtraining, Posttraining 1:05:00 Games as Research Proving Grounds: Poker, MTG, Stratego 1:10:00 Closing Thoughts – Five-Year View and Open Research Directions Chapters 00:00:00 Intro & Guest Welcome 00:00:33 Diplomacy AI & Cicero Insights 00:03:49 AI Safety, Language Models, and Steerability 00:05:23 O Series Models: Progress and Benchmarks 00:08:53 Reasoning Paradigm: Thinking Fast and Slow in AI 00:14:02 Design Questions: Harnesses, Tools, and Test Time Compute 00:20:32 Reinforcement Fine-tuning & Model Specialization 00:21:52 The Rise of Reasoning Models at OpenAI 00:29:33 Data Efficiency in Machine Learning 00:33:21 Coding & AI: Codex, Workflows, and Developer Experience 00:41:38 Multi-Agent AI: Collaboration, Competition, and Civilization 00:45:14 Poker, Diplomacy & Exploitative vs. Optimal AI Strategy 00:52:11 World Models, Multi-Agent Learning, and Self-Play 00:58:50 Generative Media: Image & Video Models 01:00:44 Robotics: Humanoids, Iteration Speed, and Embodiment 01:04:25 Rapid Fire: Research Practices, Benchmarks, and AI Progress 01:14:19 Games, Imperfect Information, and AI Research Directions

  --------  
• The Shape of Compute (Chris Lattner of Modular)

  Chris Lattner of Modular (https://modular.com) joined us (again!) to talk about how they are breaking the CUDA monopoly, what it took to match NVIDIA performance with AMD, and how they are building a company of "elite nerds". X: https://x.com/latentspacepod Substack: https://latent.space 00:00:00 Introductions 00:00:12 Overview of Modular and the Shape of Compute 00:02:27 Modular’s R&D Phase 00:06:55 From CPU Optimization to GPU Support 00:11:14 MAX: Modular’s Inference Framework 00:12:52 Mojo Programming Language 00:18:25 MAX Architecture: From Mojo to Cluster-Scale Inference 00:29:16 Open Source Contributions and Community Involvement 00:32:25 Modular's Differentiation from VLLM and SGLang 00:41:37 Modular’s Business Model and Monetization Strategy 00:53:17 DeepSeek’s Impact and Low-Level GPU Programming 01:00:00 Inference Time Compute and Reasoning Models 01:02:31 Personal Reflections on Leading Modular 01:08:27 Daily Routine and Time Management as a Founder 01:13:24 Using AI Coding Tools and Staying Current with Research 01:14:47 Personal Projects and Work-Life Balance 01:17:05 Hiring, Open Source, and Community Engagement

  --------  
• The Utility of Interpretability — Emmanuel Amiesen

  Emmanuel Amiesen is lead author of “Circuit Tracing: Revealing Computational Graphs in Language Models” (https://transformer-circuits.pub/2025/attribution-graphs/methods.html ), which is part of a duo of MechInterp papers that Anthropic published in March (alongside https://transformer-circuits.pub/2025/attribution-graphs/biology.html ). We recorded the initial conversation a month ago, but then held off publishing until the open source tooling for the graph generation discussed in this work was released last week: https://www.anthropic.com/research/open-source-circuit-tracing This is a 2 part episode - an intro covering the open source release, then a deeper dive into the paper — with guest host Vibhu Sapra (https://x.com/vibhuuuus ) and Mochi the MechInterp Pomsky (https://x.com/mochipomsky ). Thanks to Vibhu for making this episode happen! While the original blogpost contained some fantastic guided visualizations (which we discuss at the end of this pod!), with the notebook and Neuronpedia visualization (https://www.neuronpedia.org/gemma-2-2b/graph ) released this week, you can now explore on your own with Neuronpedia, as we show you in the video version of this pod. Chapters 00:00 Intro & Guest Introductions 01:00 Anthropic's Circuit Tracing Release 06:11 Exploring Circuit Tracing Tools & Demos 13:01 Model Behaviors and User Experiments 17:02 Behind the Research: Team and Community 24:19 Main Episode Start: Mech Interp Backgrounds 25:56 Getting Into Mech Interp Research 31:52 History and Foundations of Mech Interp 37:05 Core Concepts: Superposition & Features 39:54 Applications & Interventions in Models 45:59 Challenges & Open Questions in Interpretability 57:15 Understanding Model Mechanisms: Circuits & Reasoning 01:04:24 Model Planning, Reasoning, and Attribution Graphs 01:30:52 Faithfulness, Deception, and Parallel Circuits 01:40:16 Publishing Risks, Open Research, and Visualization 01:49:33 Barriers, Vision, and Call to Action

  --------  
• [AIEWF Preview] Containing Agent Chaos — Solomon Hykes

  Solomon most famously created Docker and now runs Dagger… which has something special to share with you on Thursday. Catch Dagger at: - Tuesday: Dagger’s workshop https://www.ai.engineer/schedule#ship-agents-that-ship-a-hands-on-workshop-for-swe-agent-builders - Wednesday: Dagger’s talk: https://www.ai.engineer/schedule#how-to-trust-an-agent-with-software-delivery - Thursday: Solomon’s Keynote https://www.ai.engineer/schedule#containing-agent-chaos Chapters 00:00 Introduction & Guest Background 00:29 What is Dagger? Post-Development Automation 01:08 Dagger’s Community & Platform Engineers 02:32 AI Agents and Developer Workflows 03:40 Environment Isolation & The Power of Containers 06:28 The Need for Standards in Agent Environments 07:25 Design Constraints & Challenges for Dev Environments 11:26 Limitations of Current Tools & Agent-Native UX 14:11 Modularity, Customization, and the Lego Analogy 16:24 Convergence of CICD and Agentic Systems 17:41 Ephemeral Apps, Resource Constraints, and Local Execution 21:01 Adoption, Ecosystem, and the Role of Open Source 23:30 Dagger’s Modular Approach & Integration Philosophy 25:38 Looking Ahead: Workshops, Keynotes, and the Future of Agentic Infrastructure

  --------  
• [AIEWF Preview] CloudChef: Your Robot Chef - Michellin-Star food at $12/hr (w/ Kitchen tour!)

  One of the new tracks at next week’s AI Engineer conference in SF is a new focus on LLMs + Robotics, ft. household names like Waymo and Physical Intelligence. However there are many other companies applying LLMs and VLMs in the real world! CloudChef, the first industrial-scale kitchen robotics company with one-shot demonstration learning and an incredibly simple business model, will be serving tasty treats all day with Zippy (https://www.cloudchef.co/zippy ) their AI Chef platform. This is a lightning pod with CEO Nikhil Abraham to preview what Zippy is capable of! https://www.cloudchef.co/platform See a real chef comparison: https://www.youtube.com/watch?v=INDhZ7LwSeo&t=64s See it in the AI Engineer Expo at SF next week: https://ai.engineer Chapters 00:00 Welcome and Introductions 00:58 What is Cloud Chef? 01:36 How the Robots Work: Culinary Intelligence 05:57 Commercial Applications and Early Success 07:02 The Software-First Approach 10:09 Business Model and Pricing 13:10 Demonstration Learning: Training the Robots 16:03 Call to Action and Engineering Opportunities 18:45 Final Thoughts and Technical Details

  --------  

More Business podcasts

Trending Business podcasts

About Latent Space: The AI Engineer Podcast