Developer Voices

This week we're back on systems programming with Hare. A C-like language for the ages. We talk to its creator, Drew DeVault, about what he thinks we can learn from the past 50 years of programming, and how we can build that hindsight into a new language that will last for the next 100. In among all that long-term ambition we talk cover everything from error handling, typed unions and linear types, to metaprogramming and Drew's microkernel operating system. It's called Ares, and it is, of course, built in Hare.--Drew's Homepage: https://drewdevault.com/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/ A summary of Hare’s features: https://harelang.org/tutorials/introduction/Hare Community Resources: https://harelang.org/community/SXMO Mobile: https://sxmo.org/QBE Compiler Backend: https://c9x.me/compile/users.htmlAres OS Source Code: https://sr.ht/~sircmpwn/helios/OSDev Wiki: https://wiki.osdev.org/Expanded_Main_PageThe Ares System [pdf]: https://mirror.drewdevault.com/ares.pdf --  #programming #podcast #harelang #qbe #microkernel

A few months ago, Michael Drogalis quit his job and decided launch 4 viable startup business ideas in 4 months, publically documenting every step of the journey. Over here at Developer Voices it seemed fun, inspired, and just crazy enough to work.We had him on the podcast a few months back just as that journey was beginning, and since he launched his first startup things have changed,. The reception has been better than he expected and the plan has been updated to go all-in on idea number one. But why? What's changed? What happened between brainstorming 4 ideas and launching #1 into the world? How is he figuring out what problems to solve, and how is he coping with the workload of being a solopreneur with a business idea and only himself to rely on?It's definitely time for an update, and to see what we can learn from a fellow geek who wants to start a business, but most of all wants to build technology that people find useful and valuable. Let's hope he succeeds...--ShadowTraffic: https://shadowtraffic.io/Michael’s Previous Appearance: https://youtu.be/jqS2TbxssQEFollow Michael’s journey: https://michaeldrogalis.substack.com/Michael on LinkedIn: https://www.linkedin.com/in/michael-drogalis-01029924/Michael on Twitter: https://twitter.com/MichaelDrogalisKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/--#podcast #softwareprogramming #programming #startup #technology #kafka #postgres #shadowtraffic #entrepreneur

Integration is probably the last, hardest, and least well thought-out part of any large software project. So anything that makes the data-streaming job easier is worth knowing about. So this week we turn our attention to Apache Flink, a flexible system for grabbing, transforming and shipping data between systems using Java, Python or good ol’ SQL. So this week Robert Metzger—Apache Flink expert and PMC member—joins us to explain what problems Flink solves and how it solves them reliably. We cover the range from simple use cases to realtime aggregations & joins to its high availability strategy.If you’re working on systems that include more than one database, then you’re definitely going to face the kinds of problems that Flink tackles.--Apache Flink: https://flink.apache.org/Robert on Twitter: https://twitter.com/rmetzger_Robert on LinkedIn: https://www.linkedin.com/in/metzgerrobert/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/–#software #programming #podcast #flink #apacheflink #dataintegration

Zig is a programming language that’s attempting to become “the new C” - the language of choice for low-level systems programming and embedded hardware. Going into that space not only puts it in competition with C and C++, but also other newcomers like Rust and Go. So what makes Zig special?Joining us to discuss it is Loris Cro from the Zig Foundation. We talk through Zig’s reasons to exist, its language design features, which parts of the C ecosystem it's tackling, and how the Zig Foundation is set up for the long-term health of the language.–Loris’ website: https://kristoff.it/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Zig homepage: https://ziglang.org/The “learn zig” guide: https://ziglearn.org/Learn Zig with Ziglings: https://ziglings.org/Find the Zig community: https://github.com/ziglang/zig/wiki/CommunityRust’s cargo-zigbuild: https://github.com/rust-cross/cargo-zigbuildUsing zig as a better linker: https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-clang.html"The Economics of Programming Languages" by Evan Czaplicki (Strange Loop 2023) - https://youtu.be/XZ3w_jec1v8–#programming #programminglanguages #software #zig #llvm #rust #go

Would you ever take on a rewrite of one of the largest and most popular Apache projects? And if so, what would you keep the same and what would you change?This week we’re talking to Christina Lin, who’s part of Redpanda, a company that’s rewriting parts of the Apache Kafka ecosystem in C++, with the aim of getting performance gains that aren’t feasible in Java. It seems like a huge mountain to climb, and a fascinating journey to be on, so let’s ask why and how they’ve taken on this challenge…Christina on Twitter: https://twitter.com/Christina_wmKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Redpanda: https://redpanda.com/Redpanda University: https://university.redpanda.com/Seestar framework: https://seastar.io/Apache Flink: https://flink.apache.org/#redpanda #kafka #apachekafka #streaming #python

This week we’re looking at Debezium - an open source project that taps into a huge number of databases and lets you stream data to other systems in real time. It’s a huge project that covers a wide range of uses: Some people use it to replicate from Oracle to MySQL, others to do smart cache invalidation, and others to build a bridge from an existing relational database to the event-sourcing world. If you’re working on a system that has more than one kind of database, it may be an essential tool. But what exactly does it do, and how does it do it?Joining us for a deep dive is Debezium expert and former project lead, Gunnar Morling. He takes us through all things Debezium, from who uses Debezium and why; which systems you can connect to and what data you get out from it; and how a project of this scope is developed.Gunnar on Twitter: https://twitter.com/gunnarmorlingGunnar on LinkedIn: https://www.linkedin.com/in/gunnar-morling-2b44b7229/Gunnar’s blog: https://www.morling.dev/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Gunnar’s talk on the new incremental snapshot mechanism: https://events.bizzabo.com/468544/agenda/session/1136877Getting into Real Time data: https://youtu.be/ftAVFxa5AwIStripe’s talk at Flink Forward: https://www.slideshare.net/FlinkForward/squirreling-away-640-billion-how-stripe-leverages-flink-for-change-data-captureFrancesco’s “When JDBC Goes Wrong” talk: https://www.confluent.io/events/kafka-summit-london-2022/jdbc-source-connector-what-could-go-wrong/#debezium #microservicesarchitecture #cdc #database #postgresql #oracle #mysql #eventsourcing #kafka #jdbc #realtime

Ever read a bad README? We all have, and most of the time, we’ve just moved right along. A programmer that can’t communicate their ideas will find no-one uses their software. And that’s true even outside of the open-source world. The best software doesn’t win - the best software _that people can understand_ wins. So how do we get better at communicating our code? What do we talk about when we talk about software?Joining to discuss that question is a data-streaming expert and skilled communicator, Francesco Tisiot. Unusually, this episode is recorded on location, as we met up in the hallway of a recent tech conference.Francesco on Twitter: https://twitter.com/FTisiotFrancesco on LinkedIn: https://www.linkedin.com/in/francescotisiot/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/#podcast #podcasts #devrel #opensource #software #presentations

As interesting and useful as LLMs (Large Language Models) are proving, they have a severe limitation: they only know about the information they were trained on. If you train it on a snapshot of the internet from 2023, it’ll think it’s 2023 forever. So what do you do if you want to teach it some new information, but don’t want to burn a million AWS credits to get there?In exploring that answer, we dive deep into the world of semantic search, augmented LLMs, and exactly how vector databases bridge that gap from the old dog to the new tricks. Along the way we’ll go from an easy trick to teach ChatGPT some new information by hand, all the way down to how vector databases store documents by their meaning, and how they efficiently search through those meanings to give custom, relevant answers to your questions.--Zain on Twitter: https://twitter.com/zainhasan6Zain on LinkedIn: https://www.linkedin.com/in/zainhasKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/HNSW Paper: https://arxiv.org/abs/1603.09320ImageBind - One Embedding Space To Bind Them All (pdf): https://openaccess.thecvf.com/content/CVPR2023/papers/Girdhar_ImageBind_One_Embedding_Space_To_Bind_Them_All_CVPR_2023_paper.pdfWeaviate: https://weaviate.io/Source: https://github.com/weaviate/weaviateExamples: https://github.com/weaviate/weaviate-examplesCommunity Links: https://forum.weaviate.io/ and https://weaviate.io/slack--#vectordb #vectordatabase #semanticsearch #openai #chatgpt #weaviate #knn

Real-time data is gradually becoming a standard requirement in systems design. Our customers are beginning to demand it, our colleagues in other departments are starting to expect it. Whether you’re letting people book a taxi, recommending their next binge-watch, or delivering business reports to management, faster data is just obviously better. Or is it?Does real-time data matter everywhere, or does it just have sweet spots in some sectors and some use-cases? Is it a cost-benefit question - is the idea great in theory, but still too hard to adopt in practice? Would everyone be streaming their data live if streaming their data live was easier? If the future of data is, “now, not later,” then what’s holding that future back?In this week’s Developer Voices we talk to Thomas Camp of Ably, and chew through the use-cases, software stacks, and education needed to speed up the way we process data. We consider everything from the front-end to the back, from user experience to business needs, and from greenfield projects to incrementally adapting existing systems.If you’re wondering what all the batch vs. streaming fuss is about, or you want to know how you can drag the industry there sooner, we have some answers. It’ll only take an hour. 😉--Thomas on LinkedIn: https://www.linkedin.com/in/thomascamp333/Ably: https://ably.com/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

How far would you go to get the kind of database you want? How deep into the stack would you dive to re-architect a system for the kind of performance, reliability and scale you believe in? Today's guest has decided to go all in, as he’s tackling the database problem from the fsync up. In this week’s Developer Voices we talk to Joran Dirk Greef, whose ambitions—combined with the lacklustre performance of his project's payment system—have led him to build a new database called TigerBeetle, that tackles some meaty problems. They’re attempting to build a database that can be durable in the face of fsync-corner cases, highly available in the face of all kinds of hidden network problems, and performant enough to outpace existing financial systems. And on top of all those goals, they’re doing it with an interesting new language you may not have heard of - Zig.What makes him want to take on this big a challenge? What problems keep him awake at night? And what is he doing to turn all that ambition into an achievable launch strategy? Listen on and find out…–TigerBeetle on Twitter: https://twitter.com/TigerBeetleDBTigerBeetle on YouTube: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGgKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Joran’s QCon ‘23 Talk: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGgViewstamped Replication Revisited (paper): https://pmg.csail.mit.edu/papers/vr-revisited.pdfGithub Test Cases for Journal recovery code: https://github.com/tigerbeetle/tigerbeetle/blob/b4dd441502894cbe9d48cb90ff0bc6a12c378591/src/vsr/journal.zig#L1181-L1213MySQL transactions per second vs fsyncs per second: https://sirupsen.com/napkin/problem-10-mysql-transactions-per-second

Ever wanted to start a tech business? Michael Drogalis has done it successfully in the past, and now he’s trying again (and again and…) as he makes a very public attempt to start 4 new tech businesses in the next 4 quarters.He’d sound completely mad, except he’s got form: His last Kafka-based company got bought out by a tech giant, giving him enough of a safety net to try something new. And for his new approach, he’s doing the exact opposite of ‘stealth mode’. He’s publishing every step of his 4-by-4 challenge, wins and losses, for all to see. It's entrepreneurship for the Reality TV era. 😁In an unusually vulnerable episode of Developer Voices, we talk about everything from solo software development and marketing strategies, to the inner struggle of starting out on your own, with no guarantees that the world will care. And this time with everyone watching. -- Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Follow Michael’s journey: https://michaeldrogalis.substack.com/Michael on LinkedIn: https://www.linkedin.com/in/michael-drogalis-01029924/Michael on Twitter: https://twitter.com/MichaelDrogalisSteal Like An Artist by Austin Kleon: https://austinkleon.com/steal/

Ask any software developer about their favourite stack, and they'll probably have strong opinions. (They might even have a snappy acronym for it, like LAMP or JAM or...) But if you ask any business about their stack, things aren’t so clear. They'll probably tell you the key components, and then add a laundry list of extra parts that got glued on later. The reality is, assembling large systems gets complicated. And with more and more options coming out every year, that potential complexity is only growing. These days, a good software developer must also be part researcher, and part explorer. So this week we're going to survey the landscape, and head out on the hunt for workable architectures. And joining us to do it is Ben Gamble. He's worked in fields as diverse as video games, enterprise software and AR headsets, and these days works for an Everything As A Service company helping people find and assemble the missing pieces their system needs. Who better to guide us through the software landscape? If you've ever worried that your system is too complicated, or is missing important parts, or you're just yearning for an architecture that feels more cohesive and less like a box of parts, come exploring with us.--Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Ben on Twitter: https://twitter.com/BenGamble7The Schemaverse: https://schemaverse.com/Nerdsniping: https://xkcd.com/356/Tremor: https://www.tremor.rs/Anna McDonald’s talk on Completion Criteria: https://www.confluent.io/events/kafka-summit-london-2023/pragmatic-patterns-and-pitfalls-for-event-streaming-in-brownfield/Temporal.IO: https://temporal.io/

In modern systems, the amount of data keeps getting larger, and the time available keeps getting shorter. So it's almost inevitable that we're augmenting our general-purpose databases with dedicated analytics databases.This week we dive into the world of OLAP with a thorough look at Clickhouse, a high-performance, columnar database designed to "query billions of rows in microseconds."Alasdair Brown joins us to discuss what Clickhouse is, how it performs queries so quickly, and where it fits into a wider system. We talk about its origins as a Google Analytics-like, and how it's grown into one of the most popular OLAP databases around.There's a lot of ground to cover, and a lot of questions to ask, all in the service of faster answers...--Alasdair's Blog: alasdairb.comAlasdair on Threads: https://www.threads.net/@sdairsabAlasdair on LinkedIn https://www.linkedin.com/in/alasdair-brownKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Clickhouse: https://clickhouse.com/Tinybird: https://www.tinybird.co/Birdhouse in your Soul: https://youtu.be/vn_or9gEB6g

Neil Buesing is a seasoned Apache Kafka® user, and a respected voice from the Kafka community who specialises in helping companies make the best use of Kafka. And that makes him the ideal person to ask the $64,000 question: What problems can Kafka actually solve for me? Because Kafka's definitely interesting, and it can be fun, but to earn a place in the toolbox it has to make life easier. In answering that question, Neil covers a tonne of ground, from queuing and quasi-databases, transitioning from batch to real-time, and solving general software integration headaches. If you have data problems, big or small, join us to figure out if Kafka is the answer. -- Kinetic Edge: https://www.kineticedge.ioNeil on Twitter: https://twitter.com/nbuesingNeil on LinkedIn: https://www.linkedin.com/in/nbuesing/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Building an Enterprise Eventing Framework (Conference Talk): https://www.confluent.io/en-gb/kafka-summit-san-francisco-2019/building-an-enterprise-eventing-framework/Synchronous Kafka (Conference Talk): https://www.confluent.io/resources/kafka-summit-2020/synchronous-commands-over-apache-kafka/Dev Local: https://github.com/kineticedge/dev-localKIP-714 - Client metrics and observability: https://cwiki.apache.org/confluence/display/KAFKA/KIP-714%3A+Client+metrics+and+observability

How do distributed systems work? If you’ve got a database spread over three servers, how do they elect a leader? How does that change when we spread those machines out across data centers, situated around the globe? Do we even need to understand how it works, or can we relegate those problems to an off the shelf tool like Zookeeper?Joining me this week is Distributed Systems Doctor—Benjamin Bengfort—for a deep dive into consensus algorithms. We start off by discussing how much of “the clustering problem” is your problem, and how much can be handled by a library. We go through many of the constraints and tradeoffs that you need to understand either way. And we eventually reach Benjamin’s surprising message - maybe the time is ripe to roll your own. Should we be writing our own bespoke Raft implementations? And if so, how hard would that be? What guidance can he offer us? Somewhere in the recording of this episode, I decided I want to sit down and try to implement a leader election protocol. Maybe you will too. And if not, you’ll at least have a better appreciation for what it takes. Distributed systems used to be rocket science, but they’re becoming deployment as usual. This episode should help us all to keep up!--KubeCon talk on the FCD bug: https://kccncna2022.sched.com/event/182N9/lessons-learned-from-etcd-the-data-inconsistency-issues-marek-siarkowicz-google-benjamin-wang-vmwareThe Raft paper by Diego Ongaro and John Ousterhout: https://raft.github.io/raft.pdfThe EPaxos Algorithm: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdfLevelDB: https://github.com/google/leveldbBenjamin on Twitter: https://twitter.com/bbengfortBenjamin on LinkedIn: https://www.linkedin.com/in/bbengfortBenjamin on GitHub: https://github.com/bbengfortRotational Labs: https://rotational.io (check out the blog!)Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

How do you get started as a programmer? And how do experienced programmers help them as they start their journey?This week's guest is a developer-turned-teacher, James Q. Quick. A former coder and developer advocate, he's s been been working on tutorials, courses and bootcamps to teach aspiring developers how to get started with JavaScript. We talk about why people get into programming as a new career, what they need to succeed, and what James thinks is the best indicator of success.We also discuss what's new and exciting in JavaScript, James' favourite up & coming libraries, and dive into the JavaScript vs. TypeScript debate.James’s YouTube channel: https://www.youtube.com/c/jamesqquickJames’s website: https://www.jamesqquick.com/Astro website builder: https://astro.build/James’ Astro course: https://astrocourse.dev/Svelte framework: https://svelte.dev/Vite build tool: https://vitejs.dev/James on Twitter: https://twitter.com/jamesqquickKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Sooner or later, every programmer will have to cross the gap between their programming language and their database. It feels like it should be easy, but in practice it’s always a much wider chasm than it seems, and every tool that bridges that gaps comes with its own strengths, weaknesses and opinions.This week we take a look at a relatively new library for database access—jOOQ—by chatting with its author, Lukas Eder. This episode takes in the simple questions like syntax, the thornier ones like supporting multiple databases, and the deeply philosophical ones like how we even think about data and data-processing.If you’re a Java (or JVM) programmer, there’s a new tool to learn here, and even if you're not there’s food for thought and ideas to borrow for the next time you need to SELECT…jOOQ: https://www.jooq.org/YesQL: https://github.com/krisajenkins/yesqlDatomic: https://www.datomic.com/XTDB: https://www.xtdb.com/The Elm Architecture: https://guide.elm-lang.org/architecture/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

Whether you’re trying to ace the coding interview, sharpen your programming skills or just have some fun learning new things, the world of competitive coding has something to offer you. Some people join with dreams of hitting the podium, and plenty of others are just competing to be their better selves.Either way, Mathis Hammel is a veteran of the competitive coding scene and he’s going to give us a view into that world, tell a few war stories and share some tips how you can play better, faster and stronger…ICPC: https://icpc.global/Advent of Code: https://adventofcode.com/Advent of Code, Day 18, 2022: https://adventofcode.com/2022/day/18Clash of Code: https://www.codingame.com/multiplayer/clashofcodeCodeForces: https://codeforces.com/CodeWars: https://www.codewars.com/HackerRank: https://www.hackerrank.com/Mathis on Twitter: https://twitter.com/mathishammelKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

“Software development has not caught up with the internet age.” So says this week’s guest, Rúnar Bjarnason. But what does that mean? What would a programming language for the internet age look like?Rúnar’s answer is Unison. A language that completely rethinks the way distributing computing can work, from the source code up. Borrowing some key ideas from git, it challenges the way we think about code-sharing, compilation, versioning and more. --Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Rúnar on Twitter: https://twitter.com/runaroramaRúnar’s book, Function Programming in Scala: https://amzn.to/46I9jewUnison website: https://unison-lang.orgComplete and Easy Bidirectional Typechecking for Higher-Rank Polymorphism (pdf): https://www.cl.cam.ac.uk/~nk480/bidir.pdfDo Be Do Be Do (pdf): https://arxiv.org/pdf/1611.09259.pdfRúnar’s Øredev conference talk: https://youtu.be/EgIVzOobD48Cloud icons created by Freepik - Flaticon: https://www.flaticon.com/free-icons/cloudComputer icons created by xnimrodx - Flaticon: https://www.flaticon.com/free-icons/computer

Have you ever been overwhelmed by the number of databases on offer? This week we welcome database expert Ben Stopford as a guide to help us map the database landscape and make sense of it all!Join us as we embark on a journey through the history of databases, tracing the path from Edgar Codd to the multitude cloud-era of options available today. Discover the strengths of various database styles and explore the tradeoffs between general-purpose databases like #PostgreSQL and highly customised ones like #Cassandra or #Snowflake.We delve into the realm of the cloud and the opportunities it brings, both for users and the database vendors themselves. And then we examine the challenges that arise when you're forced to connect multiple databases across an organisation. Should you look at Event Sourcing? Or Event Streaming, and how exactly do they differ?Finally, we look towards the future, discussing Ben's vision of an ideal database and which programming language he would choose to build it in.Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Kris on Mastodon: https://mastodon.social/@krisajenkins