OpenObservability Talks

• Shopify’s Journey to Planet-Scale Observability - OpenObservability Talks S5E09

  Shopify operates at massive scale, running thousands of services and processing billions of events per second. To tackle the challenges of observability at this scale, they built Observe—an in-house observability stack that makes use of open-source tools and specifications. In fact, they replaced an older vendors-based system, in an awe-inspiring migration project. But why build their own stack? Which open source tools did they use? How did they shape the user experience to their needs?Joining us to unpack Shopify’s journey is Elijah McPherson, an engineering leader with deep expertise in observability and distributed systems. Elijah led the complete rebuild of Shopify’s observability stack and now also oversees jobs, caching, search, and ClickHouse infrastructure. Tune in to hear firsthand insights from one of the most innovative purpose-built observability implementations in production today!The episode was live-streamed on 11 February 2025 and the video is available at https://www.youtube.com/watch?v=rBfTjlXKJW0OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube.We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat.⁠⁠https://www.youtube.com/@openobservabilitytalks⁠  https://www.twitch.tv/openobservability⁠Show Notes:00:46 - Episode and guest intro03:43 - Why rebuild the observability stack in house 05:47 - Cost and vendor lock-in07:09 - Tailoring observability for the organizational processes10:27 - How to build a team to build in-house observability 13:37 - The importance of product sense in internal platforms18:05 - The functionality of Shopify’s observability platform 25:15 - The Open Source stack used at Shopify observability 29:50 - Extending open source Grafana to Shopify’s needs36:23 - Adopting open standards 42:26 - observability into business health45:16 - how to run a migration project for a live production platform53:15 - final tips and best practices 56:41 - which organizations should develop in-house observabilityResources: Episode: Scaling Platform Engineering: Shopify’s Blueprint: https://medium.com/p/f18e97140681  Shopify Observe - lectures: https://www.linkedin.com/posts/elijahmcpherson_observe-activity-7258195493657223168-mOGS/ Socials:Twitter:⁠ https://twitter.com/OpenObserv⁠YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠Dotan Horovits============Twitter:@horovitsLinkedIn:www.linkedin.com/in/horovitsMastodon: @horovits@fosstodonBlueSky: @horovits.bsky.socialElijah McPherson===============Twitter: https://twitter.com/ElijahMcPhersonLinkedIn: https://www.linkedin.com/in/elijahmcpherson/

  --------  

  1:00:24
• Open Source AI: Perspectives from the OSI - OpenObservability Talks S5E08

  We all know pretty well what open source means and what AI means. But what does open source AI mean? Is there even such a thing? Join us for an intriguing episode as we host Stefano Maffulli, Executive Director of the Open Source Initiative (OSI). The OSI has been the steward of the Open Source Definition for over two decades, and has recently launched its first draft of the Open Source AI Definition—OSAID 1.0—following intense community discussions, and amid corporate and governmental policy making and opinion shaping efforts. In this engaging fireside chat, host Dotan Horovits and Stefano delve into the evolving role of open source in the age of AI. Stefano shares insights into how open source principles are being applied to artificial intelligence, the challenges of defining openness in this rapidly advancing field, and the impact of OSI’s work in shaping the future of AI innovation. Whether you're a developer, data scientist, FOSS advocate, or simply curious about the intersection of open source and AI, this is an episode you won’t want to miss! Stefano Maffulli is a leader in the global Open Source community and has experience leading global projects across organisations, partners, and countries. Stefano is a skilled writer and speaker on community building and an active Open Source contributor. The episode was live-streamed on 21 January 2025, and the video is available at https://youtube.com/live/DIcv2YbFC6c You can read the recap blog here: https://medium.com/p/5ba89e11c26d OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. ⁠⁠https://www.youtube.com/@openobservabilitytalks⁠   https://www.twitch.tv/openobservability⁠ Show Notes: 00:44 - Episode and guest intro 02:35 - What’s the OSI’s role today 07:33 - Trend of open source projects turning to the dark side 17:38 - AI challenges the open source definition 20:39 - How is open source different in AI than in software? 25:41 - The broken social contract of data 34:17 - White paper by the OSI and Open Future 40:23 - Open Source AI Definition v1.0, industry feedback and roadmap 49:55 - The EU AI Act and legislative work for open source AI 55:31 - What’s next from the OSI Resources: Open Source Initiative (OSI): https://opensource.org/ Open Future and OSI - white paper on open source AI: https://opensource.org/data-governance-open-source-ai Mozilla and Eleuther - white paper on open source AI: https://foundation.mozilla.org/en/research/library/towards-best-practices-for-open-datasets-for-llm-training/ Dark Side of Open Source: The Community Strikes Back: https://medium.com/p/400a1a5c679e Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: https://twitter.com/horovits LinkedIn: https://www.linkedin.com/in/horovits/ Mastodon: https://fosstodon.org/@horovits BlueSky: https://bsky.app/profile/horovits.bsky.social Stefano Maffulli ============= Linkedin: https://www.linkedin.com/in/maffulli Mastodon: @[email protected] Bluesky: @maffulli.net Threads: smaffulli

  --------  

  1:01:06
• End-of-Year Observability Retrospective with Charity Majors - OpenObservability Talks S5E07

  In this special year-end episode of OpenObservability Talks, we are thrilled to host Charity Majors, co-founder and CTO of Honeycomb, for an insightful conversation on the state of observability. Charity and our host Horovits recently delivered keynotes at Open Source Observability Day, which sparked fascinating discussions on the evolution of open observability and its impact on the broader industry. Together, they run a 2024 yearly postmortem on the key insights and trends, exploring what the observability community and industry have accomplished this year. Looking ahead, they also discuss what’s on the horizon for observability in 2025 and beyond. Charity Majors pioneered the concept of modern Observability, drawing on her years of experience building and managing massive distributed systems at Parse (acquired by Facebook), Facebook, and Linden Lab building Second Life. She is the co-author of Observability Engineering and Database Reliability Engineering (O'Reilly).  Join us for this fireside chat as we wrap up the year with the influential voices in observability. The episode was live-streamed on 9 December 2024 and the video is available at https://www.youtube.com/watch?v=D7ssNKAmYMs You can read the recap post at https://medium.com/p/94f80fff77e8/ OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. ⁠⁠https://www.youtube.com/@openobservabilitytalks⁠   https://www.twitch.tv/openobservability⁠ Show Notes: 00:00 - intro 01:51 - major observability trends of 2024 05:14 - OpenTelemetry trends 07:50 - Observability 2.0 14:45 - AI for DevOps and Observability 27:02 - Platform engineering 36:37 - observability query and data analytics 43:40 - observability for business insights 46:53 - how to start observability in Greenfield projects 50:15 - additional use cases for observability 54:11 - controlling cost of observability 58:47 - outro Resources: Practitioner's guide to wide events: https://jeremymorrell.dev/blog/a-practitioners-guide-to-wide-events/ Charity Major's blog on Observability 2.0: https://www.honeycomb.io/blog/time-to-version-observability-signs-point-to-yes Observability Is A Data Analytics Problem: https://insideainews.com/2022/04/07/observability-is-a-data-analytics-problem/ Platform as a Product survey by the CNCF: https://www.linkedin.com/feed/update/urn:li:share:7267977952242397185/ SaaS observability: https://medium.com/p/b2db276305b2 Expensive Metrics: Why Your Monitoring Data and Bill Get Out Of Hand: https://medium.com/p/e5724619e3f1 Sampling best practices: https://logz.io/learn/sampling-in-distributed-tracing-guide/ Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: @horovits LinkedIn: www.linkedin.com/in/horovits Mastodon: @horovits@fosstodon BlueSky: @horovits.bsky.social Charity Majors ============ Twitter: https://x.com/mipsytipsy LinkedIn: https://www.linkedin.com/in/charity-majors Mastodon: @[email protected] BlueSky: https://bsky.app/profile/mipsytipsy.bsky.social

  --------  

  1:01:41
• CNCF Ambassadors Share the Best of KubeCon 2024 - OpenObservability Talks S5E06

  Catch up on everything you missed at KubeCon North America 2024! Join us for a special recap that brings you closer to the action. This is a special episode in collaboration with the Cloud Native Computing Foundation (CNCF), the foundation behind KubeCon+CloudNativeCon and the cloud-native projects. Dotan Horovits, our host and a CNCF Ambassador, will be joined by an all-star panel of cloud-native experts—CNCF Ambassadors Viktor Farcic and Max Körbächer—each bringing their unique insights and takeaways from the conference. Together, they unpack the major project announcements and key themes from this year’s event: the standout talks, co-located events, maintainer meetings and those memorable hallway conversations. Get insights from the experts who know the cloud-native space inside out. Viktor Farcic is a lead rapscallion at Upbound and a published author. He is a host of the YouTube channel DevOps Toolkit and a co-host of DevOps Paradox. Max is Co-Founder at Liquid Reply. He is Co-Chair of the CNCF Environmental Sustainability Technical Advisory Group and served 3 years at the Kubernetes release team. He runs the Munich Kubernetes Meetup as well as the Munich and Ukraine Kubernetes Community Days. Dotan Horovits is a DevOps specialist with special focus on observability solutions and related open source projects such as OpenTelemetry, Jaeger, Prometheus and OpenSearch. He runs the OpenObservability Talks podcast, now in its 5th year. Don't miss this expert-led KubeCon recap, in collaboration with the Cloud Native Computing Foundation’s official channel! The episode was live-streamed on 19 November 2024 in collaboration with the Cloud Native Computing Foundation, and the video is available at https://www.youtube.com/watch?v=1TrPev5IzB8 You can read the recap post: https://medium.com/@horovits/1362959030c1 OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. ⁠⁠https://www.youtube.com/@openobservabilitytalks⁠   https://www.twitch.tv/openobservability⁠ Show Notes: 00:00 - episode and speaker intro 02:45 - KubeCon Salt Lake City stats and trends 05:26 - The cloud-native stack is maturing up 08:12 - KubeCon’s role in the cloud-native space 11:23 - Platform Engineering trend 14:07 - Open specifications and Kubernetes API 18:44 - Flatcar joins the CNCF with container focused OS 24:54 - wasmCloud moves to CNCF incubation and WASMCon highlights 31:49 - CNCF Ambassador program and recent Community Awards 35:24 - KubeCon event plan and expansion, and local KCDs 43:34 - Environmental Sustainability TAG 47:46 - Dapr and cert-manager reached CNCF graduation 51:11 - Cloud Native Reference Architectures 54:39 - observability updates for Jaeger, Prometheus and more 58:53 - episode outro Resources: CNCF community awards: https://www.cncf.io/announcements/2024/11/14/cloud-native-computing-foundation-announces-the-2024-community-awards-winners/ Dapr graduation: https://www.cncf.io/announcements/2024/11/12/cloud-native-computing-foundation-announces-dapr-graduation/ wasmCloud moves to incubation: https://www.cncf.io/blog/2024/11/12/cncf-welcomes-wasmcloud-to-the-cncf-incubator/ More on wasmCloud: https://medium.com/p/02a5025c6115 OpenTelemetry expands into CI/CD observability https://www.linkedin.com/feed/update/urn:li:share:7259200802689273856 Jaeger v2 unveiled https://medium.com/p/be612dbee774 Prometheus 3.0 unveiled https://medium.com/p/1c5edca32c87 Flatcar joins the CNCF https://www.linkedin.com/feed/update/urn:li:share:7257278073824288768/ OpenCost matured into incubation https://www.linkedin.com/feed/update/urn:li:share:7257826394179522562 New Cloud Native Reference Architecture hub: https://architecture.cncf.io/ CNCF upcoming events: https://www.cncf.io/events/ Kubernetes Community Days events around the world https://www.cncf.io/kcds/ Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: @horovits LinkedIn: www.linkedin.com/in/horovits Mastodon: @horovits@fosstodon BlueSky: @horovits.bsky.social Viktor Farcic =========== Twitter: https://twitter.com/vfarcic LinkedIn: https://www.linkedin.com/in/viktorfarcic BlueSky: https://bsky.app/profile/vfarcic.bsky.social Max Körbächer ============= Twitter: https://twitter.com/mkoerbi LinkedIn: https://www.linkedin.com/in/maxkoerbaecher BlueSky: https://bsky.app/profile/mkoerbi.bsky.social Mastodon: https://fosstodon.org/@[email protected]

  --------  

  59:54
• Jaeger V2 Unveiled: Powered by OpenTelemetry - OpenObservability Talks S5E05

  In this episode of OpenObservability Talks, Dotan Horovits sits down with Yuri Shkuro, the creator of Jaeger, to unveil the highly anticipated Jaeger V2. This major release introduces a new architecture with deep OpenTelemetry integration, which promises more flexibility, performance, extensibility and ease of use. Join us as Yuri shares insider details on the challenges, innovations, and roadmap for Jaeger V2 towards a more efficient and scalable distributed tracing solution. Yuri is a software engineer who works on distributed tracing, observability, reliability, and performance problems, currently working at Meta; author of the book "Mastering Distributed Tracing"; creator of Jaeger, an open source distributed tracing platform originally developed at Uber; co-founder of the OpenTracing and OpenTelemetry CNCF projects; member of the W3C Distributed Tracing Working Group. The episode was live-streamed on 14 October 2024, and the video is available at www.youtube.com/watch?v=lICivVwm-F8. Check out the recap blog at: https://medium.com/p/be612dbee774/ OpenObservability Talks episodes are released monthly, on the last Thursday of each month and are available for listening on your favorite podcast app and on YouTube. We live-stream the episodes on Twitch and YouTube Live - tune in to see us live, and chime in with your comments and questions on the live chat. ⁠⁠https://www.youtube.com/@openobservabilitytalks⁠   https://www.twitch.tv/openobservability⁠ Show Notes: 00:00 - Intro 00:45 - Open Source Observability Day 01:46 - Episode and guest intro 04:37 - Jaeger v1.x highlights 09:04 - Jaeger scope evolution from instrumentation to backend 13:36 - Jaeger v2 - why now? 20:26 - New architecture for V2 - learnings for SW engineering 26:53 - Jaeger persistence layer, and do we need tracing-specialized database? 35:35 - extending OpenTelemetry to manage storage for Jaeger 38:57 - RC1 is out, when is GA expected and what's expected? 43:24 - Breaking changes and migration path from v1 to v2 48:31 - What's expected for Jaeger UI 51:24 - New contributors joining through mentorship programs 54:47 - Observability at Meta/Facebook: machine learning, correlation, OpenTelemetry 1:01:04 - Outro Resources: https://www.jaegertracing.io/docs/next-release-v2/getting-started/⁠  https://medium.com/jaegertracing/towards-jaeger-v2-moar-opentelemetry-2f8239bee48e https://horovits.medium.com/observability-into-your-finops-taking-distributed-tracing-beyond-monitoring-48a51e32e78a https://www.youtube.com/watch?v=35aInRLbTQo&list=PLd57eY2edRXz4djMETYTm-2p8WGTdoX3D https://www.youtube.com/watch?v=1l0HKUDoX4Q&list=PLd57eY2edRXz4djMETYTm-2p8WGTdoX3D https://research.facebook.com/publications/positional-paper-schema-first-application-telemetry/⁠ ⁠ https://research.facebook.com/publications/scuba-diving-into-data-at-facebook/⁠ ⁠ https://osoday.com/⁠ Socials: Twitter:⁠ https://twitter.com/OpenObserv⁠ YouTube: ⁠https://www.youtube.com/@openobservabilitytalks⁠ Dotan Horovits ============ Twitter: @horovits LinkedIn: www.linkedin.com/in/horovits Mastodon: @horovits@fosstodon BlueSky: @horovits.bsky.social Yuri Shkuro ========== Twitter: https://twitter.com/YuriShkuro LinkedIn: https://www.linkedin.com/in/yurishkuro/

  --------  

  1:03:01

More Technology podcasts

Trending Technology podcasts

About OpenObservability Talks