Beyond TOPS: A Holistic Framework for Edge AI Metrics
Beyond raw computational power lies the true measure of AI system effectiveness. Austin Lyons, founder of ChipStrat and analyst at Creative Strategies, challenges us to rethink how we evaluate Edge AI technologies in this thought-provoking talk on metrics that truly matter.For too long, the industry has obsessed over Trillion Operations Per Second (TOPs) as the gold standard measurement. Lyons expertly deconstructs this limited view, introducing us to a more nuanced framework that considers what users actually experience. As generative AI moves to edge devices, shouldn't we care more about tokens per second—how quickly systems respond to our prompts—than abstract computational capabilities?But speed alone doesn't tell the whole story. What happens when your lightning-fast AI assistant drains your battery in an hour? Lyons presents "tokens per second per watt" as a crucial metric for practical, everyday AI use. He also introduces the concept of "vibes"—those harder-to-quantify qualities like perceived intelligence and personality that make or break user adoption, drawing a compelling parallel to why people choose Apple products despite comparable technical specs from competitors.The most valuable insight comes from Lyons' call for cross-functional collaboration in AI system design. When hardware engineers, software developers, designers, and product managers work in isolation, optimizing for their preferred metrics, the end result often disappoints users. By approaching AI development holistically, teams can make informed trade-offs that deliver better overall experiences—sometimes with less powerful but more efficient models.Ready to transform how you think about AI performance? Subscribe to Austin's newsletter at chipstrat.com where he regularly shares insights on the evolving intersection of semiconductors, AI, and product strategy.Send us a textSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org
--------
12:14
--------
12:14
How AI Zip Is Shrinking Models for a Device-First Future
What if we could put powerful AI anywhere—from underwater cameras monitoring fish to the phone in your pocket? That's the vision driving AI Zip, a company building ultra-efficient AI models that can run on virtually any device.During this fascinating conversation, we explore how AI Zip is pioneering a different path in artificial intelligence by focusing on extreme compression and efficiency. While most companies pursue ever-larger models requiring massive cloud infrastructure, AI Zip is shrinking intelligence to fit where it's needed most—at the edge where 99% of data originates.The numbers are staggering: edge devices collectively possess about 100 times more computing power than all cloud resources combined, yet 95% of AI workloads run in the cloud. This disconnect represents an enormous untapped opportunity that AI Zip is addressing through innovations in model compression and deployment.We dive into real-world applications, including an award-winning smart fish farming solution developed with SoftBank that uses underwater computer vision to optimize feeding and dramatically reduce waste. This practical example shows how specialized AI can deliver enormous value when deployed directly at the data source.Perhaps most thought-provoking is the efficiency gap between artificial and natural intelligence. While our most advanced AI systems require thousands of watts of power, the human brain operates on just 20 watts—about the same as a smartphone. Similarly, a jumping spider can navigate complex 3D environments with millions of neurons, while autonomous vehicles need billions of parameters. Closing this three-orders-of-magnitude efficiency gap represents an exciting frontier for AI research.The future of AI won't just be about bigger models—it will be hierarchical, with specialized intelligence at every level. Subscribe now to hear more conversations with pioneers who are reimagining what's possible at the intersection of AI and edge computing.Send us a textSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org
--------
29:41
--------
29:41
How Innatera is Revolutionizing Low-Power AI with Neuromorphic Chips
What happens when we redesign computing hardware to work more like the human brain? The results are transformative for edge AI.Sumit Kumar from Inatera takes us inside the world of neuromorphic computing – a revolutionary approach that's bringing brain-like intelligence directly to sensors. Born from research at Delft University of Technology, Inatera is tackling one of the most significant challenges in modern technology: how to perform complex AI tasks on battery-powered devices without draining power.The key lies in spiking neural networks that are fundamentally different from conventional AI approaches. These event-driven networks operate with computational dynamics that mimic brain function, resulting in models 100 times smaller than traditional AI while consuming just a fraction of the power. For applications like video doorbells, acoustic scene classification, and wearable healthcare, this means continuous monitoring with millisecond latency at just a few milliwatts – outperforming traditional microcontrollers by at least 10x.Beyond current applications, neuromorphic computing opens entirely new possibilities. The technology excels not just with conventional vision but with radar sensors and other modalities, particularly in privacy-sensitive situations. Robotics represents another frontier, where neuromorphic systems can enhance environmental perception, process complex sensor fusion, and enable low-latency control. Through academic partnerships and industry collaboration via the Edge AI Foundation, Inatera is helping build the ecosystem that will make neuromorphic computing as ubiquitous as neural networks are today. The future of edge AI may indeed be neuromorphic.Send us a textSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org
--------
16:28
--------
16:28
How Edge Matrix is Transforming Video Monitoring Through AI
Smart cities face an impossible challenge - monitoring countless security cameras 24/7 with human operators alone. Edge Matrix is solving this problem with innovative AI technology that transforms how urban environments approach security and monitoring.Founded in 2019 as a Cloudian spin-off, Edge Matrix has developed sophisticated video AI solutions designed specifically for smart city applications. Their systems can continuously monitor multiple camera feeds, instantly detect anomalies, and alert security personnel when issues arise. What makes their approach unique is the comprehensive nature of their offering - they don't just provide software, but build the rugged hardware needed for outdoor deployment in challenging environments like roadways and public spaces.The team at Edge Matrix has engineered remarkably resilient systems built around NVIDIA's Jetson platform, incorporating features like supercapacitor backups and secondary control systems that can perform complete power cycles remotely when needed. This ensures their equipment maintains continuous operation without manual intervention. Their newest product, Edge AI Station, bundles ready-made AI applications for common use cases like traffic analysis, people counting, and fire detection, making advanced AI monitoring accessible to organizations without extensive technical resources.Edge Matrix represents the future of urban security - intelligent, always-vigilant systems that transform ordinary cameras into powerful monitoring tools. As cities continue growing smarter, solutions like these will become essential infrastructure for maintaining safety and security at scale. Want to see how AI is revolutionizing urban monitoring? Visit edgematrix.com to learn more about their innovative approach to smart city technology.Send us a textSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org
--------
28:32
--------
28:32
From Stanford Labs to Edge AI Pioneers: FemtoSense's Journey
The quest to create artificial intelligence as efficient as the human brain is one of computing's most fascinating challenges. While today's AI systems consume megawatts of power in massive data centers, your brain accomplishes far more complex tasks on roughly 20 watts—about the same as a dim light bulb. This efficiency gap is what FemtoSense is determined to close.In this illuminating conversation with Sam from FemtoSense, we dive into the journey of this Stanford spin-off that's revolutionizing edge AI with neuromorphic-inspired computing. The company's very name speaks to their mission—"femto" references the femtojoule, the incredibly tiny amount of energy required for a single neuron in your brain to communicate with another. Their goal? Create AI systems that approach this remarkable biological efficiency.What makes FemtoSense's approach unique is how they've evolved from pure academic research to commercial viability. Rather than pursuing neuromorphic computing in its purest form, they've distilled its key efficiency principles—sparse computing and spatial locality—into manufacturable, reliable systems. This pragmatic approach is already bearing fruit in consumer wearables like AI-powered hearing aids that maintain 24-hour battery life while delivering sophisticated audio processing, and in smart home devices that offer on-device intelligence without cloud dependence. For battery-powered devices, their technology extends runtime dramatically; for plugged-in devices, it slashes costs by reducing silicon footprint.As Sam puts it, "Efficiency is the new currency." The principles that make FemtoSense's technology possible aren't tied to any specific market or modality—they represent fundamental improvements in how computing can be done. With global operations spanning the US, Asia, and Europe, and active participation in industry groups like the Edge AI Foundation, FemtoSense isn't just building more efficient chips; they're helping shape a future where AI can be everywhere without consuming the planet's resources.Curious about the future of efficient AI? Join us at upcoming Edge AI Foundation events and discover how companies like FemtoSense are making AI that's not just smarter, but fundamentally more sustainable.Send us a textSupport the showLearn more about the EDGE AI FOUNDATION - edgeaifoundation.org
Discover the cutting-edge world of energy-efficient machine learning, edge AI, hardware accelerators, software algorithms, and real-world use cases with this podcast feed from all things in the world's largest EDGE AI community. These are shows like EDGE AI Talks, EDGE AI Blueprints as well as EDGE AI FOUNDATION event talks on a range of research, product and business topics. Join us to stay informed and inspired!
New Zealand