The State & Future of Apache Kafka (with Anatoly Zelenin)
I’m joined this week by one of the authors of Apache Kafka In Action, to take a look at the state of Kafka, event systems & stream-processing technology. It’s an approach (and a whole market) that’s had at least a decade to mature, so how has it done? What does Kafka offer to developers and businesses, and which parts do they actually care about? What have streaming data systems promised and what have they actually delivered? What’s still left to build?–Apache Kafka in Action: https://www.manning.com/books/apache-kafka-in-actionPat Helland, Data on the Inside vs Data on the Outside: https://queue.acm.org/detail.cfm?id=3415014Out of the Tar Pit: https://curtclifton.net/papers/MoseleyMarks06a.pdfMartin Kleppmann, Turning the Database Inside-Out: https://martin.kleppmann.com/2015/11/05/database-inside-out-at-oredev.htmlData Mesh by Zhamak Dehghani: https://www.amazon.co.uk/Data-Mesh-Delivering-Data-Driven-Value/dp/1492092398Quix Streams: https://github.com/quixio/quix-streamsXTDB: https://xtdb.com/Support Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinAnatoly’s Website: https://zelenin.de/Kris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Kris on Twitter: https://twitter.com/krisajenkins
--------
1:12:22
DataFusion - The Database Building Toolkit (with Andrew Lamb)
Building a database is a serious undertaking. There are just so many parts that you have to implement before you even get to a decent prototype, and so many hours of work before you could begin working on the ideas that would make your database unique. Apache DataFusion is a project that hopes to change all that, but building an extensible, composable toolkit of database pieces, which could let you build a viable database extremely quickly, and then innovate from that starting point. And even if you’re not building a database, it’s a fascinating project to explain how databases are built.Joining me to explain it all is Andrew Lamb, one of DataFusion’s core contributors, and he’s going to take us through the whole stack, how it’s built and how you could use it. Along the way we cover everything from who’s building interesting new databases and how you manage a large, open-source Rust project.–DataFusion Homepage: https://datafusion.apache.org/DataFusion on Github: https://github.com/apache/datafusionDataFusion Architecture (with diagrams!): https://youtu.be/NVKujPxwSBA?si=tw9ACxlbdpBuVsnv&t=1045Datalog: https://docs.racket-lang.org/datalog/Tokio: https://tokio.rs/Andrew’s Homepage: http://andrew.nerdnetworks.org/Andrew’s Blog Post about Tokio: https://thenewstack.io/using-rustlangs-async-tokio-runtime-for-cpu-bound-tasks/Velox: https://velox-lib.io/Arroyo: https://www.arroyo.dev/Synnada: https://www.synnada.ai/LanceDB: https://lancedb.com/SDF+DBT: https://docs.sdf.com/integrations/dbt/integratingSupport Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Jupyter’s become an incredibly popular programming and data science tool, but how does it actually work? How have they built an interactive language execution engine? And if we understand the architecture, what else could it be used for?Joining me to look inside the Jupyter toolbox are Afshin Darian and Sylvain Corlay, two of Jupyters long-standing contributors and project-steerers. They’ve going to take us on a journey that starts with today’s userbase, goes through the execution protocol and ends with a look at what Jupyter will be in the future - an ambitious framework for interactive, collaborative applications and more.–Support Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinJupyter Homepage: https://jupyter.org/Jupyter Xeus: https://github.com/jupyter-xeus/xeusJupyter AI: https://github.com/jupyterlab/jupyter-aiJupyter CAD: https://github.com/jupytercad/JupyterCADJupyter GIS: https://github.com/geojupyter/jupytergis/Jupyter GIS Announcement: https://blog.jupyter.org/real-time-collaboration-and-collaborative-editing-for-gis-workflows-with-jupyter-and-qgis-d25dbe2832a6QGIS: https://qgis.org/ZeroMQ: https://zeromq.org/Sylvain on LinkedIn: https://www.linkedin.com/in/sylvaincorlayDarian on LinkedIn: https://www.linkedin.com/in/afshindarianKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
--------
1:29:11
Nix, The Build-Everything Language (with Julian Arni)
Ever since we invented makefiles, the programming world has been wrestling with the problem of building software stacks reliably. This week we’re going to look at one of the most ambitious solutions available - Nix. Nix tries to do everything from invoking your compiler to installing your language, and even providing your operating system. But how does it work in theory, and how well does it work in practice?Joining me to discuss is Julian Arni, a Nix-enthusiast and creator of a build/test/deploy service called Garnix.Nix has been one of my go-to tools for years - I hope it’ll find its way into your stack.–Nix Overview: https://nixos.org/explore/Nix Tutorial: https://nix.dev/tutorials/first-steps/Nix Flakes: https://nixos.wiki/wiki/FlakesThe Nix Package List: https://search.nixos.org/packagesGarnix.IO: https://garnix.io/Julian’s NixCon Talk, Call by Hash: https://www.youtube.com/watch?v=fU9ogB9hZZASupport Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
--------
1:20:36
Graphite: Image Editing as a Syntax Tree (with Keavon Chambers & Dennis Kobert)
Graphite is a new image editor with an interesting architecture - it’s a classic UI-driven app, an image-manipulation language, and a library of programmable graphics primitives that any Rust coder could use, extend or add to. The result is something that you can use like Photoshop or Inkscape, or make use of in batch pipelines, a bit like ImageMagick.Joining me to discuss it are Keavon Chambers & Dennis Kobert, who are hammering away on building a project that’s potentially as demanding as Photoshop, but with a more ambitious architecture. How can they hope to compete? Perhaps in the short term by doing what regular image And is the future of image editing modular?–Graphite Homepage: https://graphite.rs/Graphite Web Version: https://editor.graphite.rs/Graphite on Github: https://github.com/GraphiteEditor/GraphiteSigned Distance Fields: https://jasmcole.com/2019/10/03/signed-distance-fields/Support Developer Voices on Patreon: https://patreon.com/DeveloperVoicesSupport Developer Voices on YouTube: https://www.youtube.com/@developervoices/joinKris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.socialKris on Mastodon: http://mastodon.social/@krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Deep-dive discussions with the smartest developers we know, explaining what they're working on, how they're trying to move the industry forward, and what we can learn from them.You might find the solution to your next architectural headache, pick up a new programming language, or just hear some good war stories from the frontline of technology.Join your host Kris Jenkins as we try to figure out what tomorrow's computing will look like the best way we know how - by listening directly to the developers' voices.
New Zealand