Complete Beginner's Course on AI Evaluations: Step by Step (2025) | Aman Khan
Today, I want to share a new episode with Aman Khan.The best way to learn about AI evaluations is to watch 2 PMs build them live from scratch. In our new episode, Aman and I walk through creating evals for an AI customer support agent — from labeling a golden dataset to aligning LLM judges. This is the complete beginners AI eval course you've been waiting for.Aman and I talked about:(00:00) What are AI evals and how to get good at them (02:52) The 4 types of AI evaluations everyone should know(06:08) Live demo: Building evals for a customer support agent (10:29) Using Anthropic's console to generate great prompts (15:13) Creating the evaluation criteria(17:40) Adding human labels to the golden dataset (31:05) Scaling evals with LLM-judge prompts (38:21) How to align LLM judges with human judgmentGet the takeaways: https://creatoreconomy.so/p/complete-beginner-course-on-ai-evaluations-aman-khanWhere to find Aman:X: https://www.linkedin.com/in/amanberkeley/Website: https://arize.com/📌 Subscribe to this channel – more interviews coming soon!
--------
51:47
--------
51:47
Cursor Tutorial for Beginners from Cursor's Head of AI Education (2025) | Lee Robinson
Today, I want to share a new episode with Lee Robinson.Lee is the VP of Developer Experience at Cursor and a legend in his field. While most people are still vibe coding, Lee showed me how to build apps properly using planning, test-driven development, AI agent workflows, and more. This is a must-watch tutorial for beginners who want to learn how to build with AI the right way.Lee and I talked about:(00:00) How the way we build software is fundamentally changing(03:49) Live demo: Building a music tracking app with Cursor(5:17) The critical setup phase most people skip (tests, git, linting)(16:03) Foreground vs background agents and when to use each(27:34) How to create cursor rules that compound your workflows(37:31) Why experienced engineers shouldn't fear AI tools(42:19) Essential advice for PMs who want to learn technical skills(44:48) How Cursor builds products without any PM(47:18) The future of generalist engineers in an AI worldGet the takeaways: https://creatoreconomy.so/p/cursor-tutorial-for-beginners-from-cursor-head-of-ai-education-lee-robinsonWhere to find Lee:X: https://x.com/leerobWebsite: https://leerob.com/📌 Subscribe to this channel – more tutorials coming soon!
--------
50:21
--------
50:21
How to Build an AI Co-Pilot to Help You Ace PM Interviews | Ben Erez (ex-Meta)
Today, I want to share a new episode with Ben Erez.Ben is a former Meta PM who has helped 100s of people ace their PM interviews and land their dream jobs in tech. I got him to show me exactly how to ace product sense interviews and build an AI-copilot that his students say feels like having a "calibrated interviewer that's available 24/7."Ben and I talked about:(00:00) The current state of the PM job market(02:07) The exact structure of the product sense interview(08:58) How to build your AI interview co-pilot step-by-step(12:36) Live demo: AI co-pilot walking through a Meta interview question(13:33) Why Ben designed the AI to be the candidate, not the interviewer(25:23) How to prioritize the right user segments and problems to solve(32:13) The biggest mistakes candidates make when challenged by interviewersThis episode covers how to build a simple AI co-pilot from scratch, but you should also check out Ben’s advanced interview AI co-pilot: https://www.benerez.com/copilotGet the takeaways: https://creatoreconomy.so/p/how-to-build-an-ai-co-pilot-to-ace-your-pm-interviews-ben-erezWhere to find Ben:LinkedIn: https://www.linkedin.com/in/benerez/Website: www.benarez.com📌 Subscribe to this channel – more interviews coming soon!
--------
36:43
--------
36:43
How to Get Your Team to Start Prototyping with AI | Jeremy Epling (CPO Vanta)
Today, I want to share a new episode with Jeremy Epling.Jeremy is the CPO of Vanta and former VP of Product at Github. He showed me the exact template that he used to get his entire team to prototype with AI, his favorite AI workflows for PM work, and why now might be the best time to be an IC PM. We also did a deep dive on how Vanta ran AI evaluations for their new AI agent.Jeremy and I talked about:(00:00) Why prototypes beat PRDs (01:43) The exact template that got his entire PM team prototyping(05:14) From prototype to product: live demo of Vanta’s AI agent(08:56) How to run prototype-first product reviews(12:47) Top AI workflows that save Jeremy hours each week(19:08) Is now the best time to be an IC PM?(25:03) Advice for big tech PMs facing industry changes(30:35) Separating the hype from craft for AI products(36:16) Deep dive on AI evaluations for AI agentsWhere to find Jeremy:LinkedIn: https://www.linkedin.com/in/jeremy-epling-j40/ Website: https://vanta.com Get the takeaways: https://creatoreconomy.so/p/how-to-get-your-team-to-start-prototyping-with-ai-jeremy-vanta📌 Subscribe to this channel – more interviews coming soon!
--------
43:11
--------
43:11
Full Tutorial: Use AI to Create Beautiful Designs (No Generic Slop) | Meng To
Meng is the founder of Aura, an AI prototyping tool for designers. He has spent 100s of hours using AI to create beautiful designs that don’t look like generic AI slop. Watch as he shares his exact process in this episode — from prompting to layouts to animations, and more without having to code.Timestamps:(00:00) Why most AI websites look generic and how to fix it(01:19) Comparing prompts across V0, Lovable, and Aura(06:42) Using references and templates instead of blank prompts(08:54) Building a great website prompt from scratch(16:51) Why AI should augment, not replace your creativity(22:43) Adding testimonial sections and scrolling animations(29:40) How to add Spline 3D backgrounds step-by-step(36:12) The 90/10 rule: When to use AI vs manual tweaksGet the takeaways: https://creatoreconomy.so/p/use-ai-to-create-beautiful-designs-meng-toWhere to find Meng:X: https://x.com/MengToWebsite: https://aura.build/📌 Subscribe to this channel – more interviews coming soon!
New Zealand