Engineering Enablement by DX

Are engineering teams measuring the impact of AI, or just how much it's being used?
In this episode of Engineering Enablement, host Brian Houck talks with researcher André Meyer about his DevEx Metrics Compass, why traditional productivity metrics often miss what matters, and how to measure the value AI actually delivers.
They also revisit André's Today Was a Good Day study to explore what makes developers' work rewarding, how AI is changing that experience, and practical ways to balance focused work, collaboration, and growing code review demands.

Where to find André Meyer-Baron:
• LinkedIn: https://www.linkedin.com/in/andrenmeyer 
• Website: https://andre-meyer.ch and www.devexcompass.com

Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(02:42) Why AI dashboards fail to capture actual value
(05:08) What the DevEx Metrics Compass is and why it was created 
(06:58) Why measuring collaboration matters
(09:47) Determining the right metrics for your organization 
(12:03) Distinguishing measurement gaps from gaps in public reporting
(14:35) Why some metrics are harder to act on at scale
(18:11) Why customer and business value are difficult to measure
(20:03) How to know when it's time to retire a metric
(24:20) Hierarchies of metrics 
(27:31) How to measure AI's impact beyond activity metrics
(30:20) André's Good Day study: What makes a good workday for developers 
(31:56) What observing developers revealed about context switching
(34:28) What makes developers' work rewarding in the age of AI
(36:27) The tradeoffs of delegating coding to AI
(38:01) Balancing collaboration with focused work
(42:11) Two ways to improve developers' workdays

Referenced:
• CAFE(S): Your Agent Is Only as Good as Its Context
• Too Many DevEx Metrics, Too Little Guidance
• Today was a Good Day: The Daily Life of Software Developers
• DevEx Metrics Compass 
• Gail Murphy | LinkedIn
• Thomas Fritz | LinkedIn
• Patrick Meyer | LinkedIn
• The SPACE Framework
• From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI

In this episode of Engineering Enablement, Brian Houck talks with Andrew Swerdlow, VP of Engineering at Roblox, about how the company is building trust in AI agents to review, test, and deploy code autonomously. Andrew explains how Roblox turns past code review feedback into rules for agents and why engineering is shifting toward specifications and intent. They explore what that means for the skills engineers need, how Roblox measures AI’s impact on productivity, and how it manages costs. They also discuss the guardrails needed to let agents safely ship code with minimal human intervention.

Where to find Andrew Swerdlow:
• LinkedIn: https://www.linkedin.com/in/andrewswerdlow/ 
• X:  https://x.com/andrewswerdlow 
• Website: https://andrewswerdlow.net

Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(02:20) Building trust in AI across the software development lifecycle
(04:44) When human oversight helps and when it gets in the way
(06:50) What AI agents need to know about your organization
(08:45) How Roblox turns engineering knowledge into rules for AI agents 
(12:40) How AI code review compares to human code review
(14:59) Capturing engineering knowledge beyond documentation
(18:14) How AI changes what engineers need to understand
(20:17) The shift to spec-driven development
(23:24) How AI changes how we evaluate engineers
(26:33) Rethinking engineering interviews in the age of AI
(28:23) How Roblox tracks AI-generated code
(35:56) Model routing and choosing the right AI tools
(39:50) Measuring AI's impact on engineering productivity
(44:44) Optimizing AI costs at scale
(48:12) Builders at Roblox and the future of software engineering
(50:19) Building trust in autonomous code review
(53:11) The cultural shift toward autonomous engineering

Referenced:
• CAFE(S): Your Agent Is Only as Good as Its Context
• Roblox
• Tech Leadership by Andrew Swerdlow
• Margaret-Anne Storey
• Malte Ubl - Vercel | LinkedIn
• Grafana Labs
• Kushan Mehta | LinkedIn 
• Diff Risk Score: AI-driven risk-aware software development - Engineering at Meta

As AI agents become more capable, context quality is increasingly the limiting factor in their performance. In this episode of Engineering Enablement, DX researcher Brian Houck by his CAFE(S) is joined by co-authors Max Kanat-Alexander (Capital One), Eirini Kalliamvakou (DX), and Margaret-Anne Storey (University of Victoria) to discuss CAFE(S), a new framework for improving context quality.
The panelists introduce the five dimensions of CAFE(S): Clarity, Actionability, Fidelity, Efficiency, and Security. They walk through common context failures, how poor context accrues technical and cognitive debt, what it costs in developer productivity and operating spend, how AI is changing the skills engineers need, and why context review could become as important as code review.

Where to find Margaret-Anne Storey:
• LinkedIn: https://www.linkedin.com/in/margaret-anne-storey 
• X:  https://x.com/margaretstorey 
• Website: https://margaretstorey.com
Where to find Max Kanat-Alexander: 
• LinkedIn: https://www.linkedin.com/in/mkanat
• X: https://x.com/mkanat
• Website: https://www.codesimplicity.com
Where to find Eirini Kalliamvakou: 
• LinkedIn: https://www.linkedin.com/in/eirini-kalliamvakou-1016865 
• X: https://x.com/irina_kAl
Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(01:18) Meet the CAFE(S) authors
(03:13) Introducing CAFE(S): Why context quality matters
(06:32) Common context failure modes
(08:30) The five dimensions of high-quality context
(12:33) When model failures are actually context failures
(18:24) Why the panel prioritizes clarity in CAFE(S)
(22:34) How context quality relates to technical, cognitive, and intent debt
(25:35) How human interventions reveal context failures
(28:11) How context quality relates to AI fluency
(30:28) How AI is changing what makes a good engineer
(33:48) The case for context review as a discipline 
(38:44) Wrap-up

Referenced:
• CAFE(S): Your Agent Is Only as Good as Its Context
• Nicole Forsgren
• The Productivity-Experience Paradox - Annie Vella
• Startup Studio
• From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI
• The AI-native developer
• CodeScene

Meta researcher Moritz Beller joins Brian Houck to discuss how Meta measures developer productivity. Moritz explains diff authoring time and how Meta uses it to evaluate tools and guide engineering decisions. They explore how AI is changing engineers’ work and revealing gaps in traditional metrics. They also revisit Moritz’s “Mind the Gap” study and discuss the promise and risks of AI agents for testing.

Where to find Moritz Beller:
• LinkedIn: https://www.linkedin.com/in/inventitech 
• X: https://x.com/Inventitech 
• GitHub: https://github.com/Inventitech 
• Website: https://inventitech.com

Where to find Brian Houck: 
• LinkedIn: ⁠⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠⁠

In this episode, we cover:
(00:00) Intro
(02:10) Moritz’s role at Meta
(04:01) Measuring diff authoring time
(08:20) Measuring A/B experiments
(12:11) What diff authoring time reveals
(14:45) Planning and leadership reporting
(18:36) Why Meta measures teams, not individuals
(22:33) Where developer time goes with AI
(26:06) AI’s impact on junior and senior developers
(27:18) Capturing invisible work with AI
(29:32) The “Mind the Gap” study
(33:40) Revisiting “Mind the Gap” in 2026
(38:32) AI agents for software testing
(41:49) What remains hard to measure

Referenced:
• State of AI Impact in Engineering Q2 Report 2026
• GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis
• From Technical Debt to Cognitive and Intent Debt 
• Mind the Gap: On the Relationship Between Automatically Measured and Self-Reported Productivity
• The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study
• When, how, and why developers (do not) test in their IDEs | Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering
• DORA, SPACE, and DevEx: Which framework should you use
• A Tale of Two Cities: Software Developers Working from Home During the COVID-19 Pandemic

In this episode of Engineering Enablement, Robert Lucero, Chief Architect at Okta, joins host Brian Houck to discuss how agentic AI is changing identity and access management. They explore how organizations should define agent identities, why authentication and authorization remain foundational, and how sandboxing, fine-grained permissions, and just-in-time access can enable agents to operate autonomously without creating unacceptable risk.
Robert shares what Okta has learned from driving AI adoption among security-minded engineers, what makes a repository ready for AI agents, and why strong testing, CI, and review processes matter even more as AI generates more code. They also discuss AI’s role in software validation, the need for human judgment, and whether AI will ultimately give the advantage to security teams or attackers.

Where to find Robert Lucero:
• LinkedIn: https://www.linkedin.com/in/rlucero

Where to find Brian Houck: 
• LinkedIn: ⁠⁠https://www.linkedin.com/in/brianhouck⁠⁠

In this episode, we cover:
(00:00) Intro
(02:16) Meet Robert Lucero
(02:56) Why identity becomes critical as AI agents access more systems
(06:07) Governing automated access for non-deterministic agents
(08:01) Defining and managing agent identities
(10:32) Authentication and authorization for AI agents
(13:33) Determining how much autonomy to give AI agents
(19:44) Identity systems as the control plane for AI agents
(22:40) Driving AI adoption among security-minded engineers
(25:48) Why AI’s impact on engineering extends beyond coding
(28:44) The three components of AI readiness for repositories
(30:55) AI’s role in software testing and validation
(34:26) Whether AI gives defenders or attackers the advantage
(36:01) Where to find the best food in the country

Referenced:
• State of AI Impact in Engineering Q2 Report 2026
• Okta
• The Hugging Face incident and the road ahead | OpenAI
• GitHub Copilot · Your AI pair programmer
• Claude Code by Anthropic | AI Coding Agent, Terminal, IDE
• Eight Myths on Software Engineering and GenAI | Queue
• Adriana Corona