Contributor

Eric Anderson (@ericmander) and Travis Nielsen (@STravisNielsen) talk about Rook, the open-source storage orchestrator for Kubernetes. Travis is a Senior Principal Software Engineer at Red Hat, and maintainer of Rook. Join us to dive deep into the story of Rook, from Microsoft, to Quantum, to Red Hat. In this episode we discuss: Ceph + Kubernetes = Rook The difficulty and importance of a stable storage solution for stateless applications How Rook leverages Kubernetes CRDs Why the Rook team decided to work with the CNCF Red Hat’s philosophy and approach to open-source Links: Rook Red Hat Upbound Quantum CNCF People mentioned: Bassam Tabbara (@bassamtabbara) Jared Watts (@jbw976)

Eric Anderson (@ericmander) and Patrick McFadin (@PatrickMcFadin) delve into the history of Apache Cassandra, the open-source NoSQL database born and bred around cloud over a decade ago. Patrick is the VP of Developer Relations at DataStax, and a member of the Cassandra Project Management Committee. On today’s episode, Patrick shares his philosophy on developer advocacy and experience in open-source. In this episode we discuss: Behind the NoSQL explosion that made Cassandra the darling of the valley Comparing different eras of commercializing open-source, then and now How Patrick became a pioneer in evangelizing and community-building The two kinds of people to recruit for developer relations Why Patrick says open-source is going to “start eating clouds” Links: Apache Cassandra Datastax Datastax Astra People mentioned: Avinash Lakshman (@HedvigEng) Prashant Malik (@pmalik) Adrian Cawcroft (@adrianco) Kelsey Hightower (@kelseyhightower) Other episodes: Chef with Adam Jacob

Eric Anderson (@ericmander) interviews Nick Schrock (@schrockn) about Dagster, the open-source data orchestrator for machine learning, analytics, and ETL. Nick is the founder and CEO of Elementl, and is well-known for creating the Project Infrastructure group at Facebook, which spawned GraphQL and React. On today’s episode of Contributor, Nick explains how he set out to fix an inefficiency he identified amongst the complexity of the data infrastructure domain. In this episode we discuss: Dagster’s place in the industry shift towards thinking of data as a software engineering discipline Why Nick believes it’s time for the term “data cleaning” to be retired The empowerment of Dagster’s instantaneous spin-up process and local development experience How a partner integrated Dagster into workflow for ops workers on the warehouse floor One user’s testimony that, “what dbt did for our SQL, Dagster did for our Python” Links: Dagster Elementl GraphQL React dbt Snowflake Apache Airflow People mentioned: Lee Byron (@leeb) Dan Schafer (@dlschafer) Abe Gong (@AbeGong)

Eric Anderson (@ericmander) and Tanmai Gopal (@tanmaigo) dive into the open-source Hasura GraphQL Engine and the wider Hasura community. Hasura provides real-time GraphQL APIs for databases, so developers can focus on building applications without worrying about infrastructure. Tune in to hear the full story about how Tanmai and his team are helping engineers unlock the dream of self-serve data access. In this episode we discuss: How the early Hasura team created their own version of GraphQL in parallel Developing community with ease of onboarding and radical transparency Transitioning community events into the COVID world, and looking to a future beyond travel Hasura’s secret sauce: the authorization framework Links: Hasura Hasura Con’21 DigitalOcean People mentioned: Rajoshi Ghosh (@rajoshighosh)

Eric Anderson (@ericmander) is joined by the co-founders of MindsDB, Jorge Torres (@JorgeTorresAI) and Adam Carrigan (@AdamMCarrigan). MindsDB is an open-source AI layer that integrates with existing databases, from MySQL to Clickhouse. Tune in to learn how these two former college roommates are working to bring machine learning into the mainstream. In this episode we discuss: Why it makes sense to run machine learning models in the database Partnering with Kafka, Looker, and more MindsDB’s initial adoption by students at Berkeley Different applications for MindsDB and machine learning in ecommerce, finance, and more The moment Jorge knew he had to get into business with Adam Links: MindsDB RedisConf 2021 Looker Apache Kafka ClickHouse Other episodes ClickHouse with Alexey Milovidov and Ivan Blinkov

Eric Anderson (@ericmander) welcomes Peter Wang (@pwang) for a conversation about the Python ecosystem and the open-source communities that have built it. Peter is the creator of Anaconda, the near-essential Python distribution for scientific computing that makes managing packages a lot more manageable. In today’s episode, Peter offers a unique and powerful perspective on how to make the economics of open-source work for everyone. In this episode we discuss: The paradox of the PVM and Python’s packaging difficulties How Guido van Rossum implied permission for Anaconda and the open-source Python movement Python as the lingua franca of a new professional class Looking to Roblox for inspiration for a scientific computing creator community Giving back to open-source communities through the NumFOCUS Foundation Links: Anaconda NumFOCUS NumPy SciPy Enthought  Jupyter TensorFlow MicroPython scikit-learn pandas Quansight Red Hat Roblox People mentioned: Travis Oliphant (@teoliphant) Fernando Pérez (@fperez_org) Brian Granger (@ellisonbg) Min Ragan-Kelley (@minrk) Guido van Rossum (@gvanrossum) James Currier (@JamesCurrier) Other episodes: NumPy & SciPy with Travis Oliphant TensorFlow with Rajat Monga

Eric Anderson (@ericmander) is joined by Alexander Gallego (@emaxerrno) for an examination of Redpanda, the source available event streaming platform designed as a drop-in replacement for Kafka. Redpanda’s storage engine is attractive to developers for its performance and simplicity, removing the complexity of running Kafka to scale and deploying with a single binary. Listen to today’s episode to learn more about how Alexander and the team at Vectorized are looking to advance the conversation around streaming into the future. In this episode we discuss: What Alexander means when he says that hardware is the platform for data streaming The 3 things that turn a data stream into a data product Comparing Redpanda to Kafka and Pulsar A difference in product philosophy between selling to data teams vs app developers How Alexander approached the challenge of monetizing data infrastructure Links: Redpanda Vectorized Apache Kafka Apache Pulsar Apache Spark Apache Beam Apache Storm Apache Flink Elastic CockroachDB Other episodes: TensorFlow with Rajat Monga Scylla with Dor Laor

Eric Anderson (@ericmander) and Zoltan Olah (@zqzoltan) discuss Storybook, the open-source UI component development tool. Storybook supports all the most popular frontend frameworks and libraries such as React, Vue and Angular, but allows users to test and develop components in isolation. In today’s episode, learn more about the early days of the component-driven development methodology and how Storybook was saved by a passionate community of engineers. In this episode we discuss: Storybook as an integral part of UI design workflow How Zoltan and his team inherited Storybook and saved it from being “left out to dry” Solving a pain point for front-end engineers with Chromatic’s UI regression testing, built on top of Storybook Why Zoltan compares components to APIs, and Storybook to a service mesh What’s happening today in the world of open-source design systems Links: Storybook Chromatic Meteor GraphQL React Tailwind Selenium Cypress Material-UI Figma Learn Storybook People mentioned: Dominic Nguyen (@domyen) Tom Coleman (@tmeasday) Arunoda Susiripala (@arunoda) Norbert de Langen (@NorbertdeLangen) Michael Shilman (@mshilman)

Eric Anderson (@ericmander) and Sheng Wu (@wusheng1108) discuss Apache SkyWalking, an open-source APM tool focusing on cloud-native and distributed systems. SkyWalking was originally developed in 2012 as a training tool for developers new to distributed systems architecture, but it became Sheng’s pet project for several years until he brought it to the Apache Incubator program. Listen to today’s episode for the inside scoop of how this “hidden gem” fits into the Apache network of open-source software projects. In this episode we discuss: Why open-source APMs are not very common SkyWalking’s focus on attracting more contributors rather than users How a conflict of interest at Huawei led to a “bake-off” between Apache and CNCF The impact of Elastic changing their license on the open-source community The name “Skywalking,” its sources of inspiration, and an easter egg Links: Apache SkyWalking Kubernetes The Apache Incubator CNCF Tetrate Apache ShardingSphere Apache APISIX Envoy Proxy Apache Airflow Apache Beam Dynatrace New Relic Elastic Helm Zipkin Other episodes: Envoy Proxy with Matt Klein

Eric Anderson (@ericmander) and Fred K. Schott (@FredKSchott) dive into the world of Snowpack, an open-source, frontend build tool for web developers. Snowpack is special because it uses Javascript’s ES module system to instantly write file changes to the browser. Fred created Snowpack and the Skypack CDN to fulfill his vision of the future of the web, which he first recognized while trying to advance the Javascript ecosystem with an earlier project called Pika. On today’s episode, find out how Fred rejected the pain of modern web development, and came up with a better solution. In this episode we discuss: Reconfiguring old ideas for today’s web development landscape How Snowpack and Skypack lighten the load when it comes to Node modules and storage space Questioning what it means to build a modern application that works for developers and users alike Skypack and the future of shared dependencies across different sites Why Snowpack is using an open governance framework Links: Snowpack Skypack OCTO Speaker Series - Fred K. Schott Svelte React Ripple Microsite Deno Next.js esbuild webpack People mentioned: Rich Harris (@Rich_Harris) Nate Moore (@n_moore)

Eric Anderson (@ericmander) and Travis Oliphant (@teoliphant) take a far-reaching tour through the history of the Python data community. Travis has had a hand in the creation of many open-source projects, most notably the influential libraries, NumPy and SciPy, which helped cement Python as the standard for scientific computing. Join us for the story of a fledgling community from a time “before open-source was cool,” and their lessons for today’s open-source landscape. In this episode we discuss: How biomedical engineering, MRIs, and an unhappy tenure committee led to NumPy and SciPy Overcoming early challenges of distribution with Python What Travis would have done differently when he wrote NumPy Successfully solving the “two-option split” by adding a third option Community-driven open-source interacting with company-backed open-source Links: NumPy SciPy Anaconda Quansight Conda Matplotlib Enthought TensorFlow PyTorch MXNet PyPi Jupyter pandas People mentioned: Guido van Rossum (@gvanrossum) Robert Kern (Github: @rkern) Pearu Peterson (Github: @pearu) Wes McKinney (@wesmckinn) Charles Harris (Github: @charris) Francesc Alted (@francescalted) Fernando Perez (@fperez_org) Brian Granger (@ellisonbg) Other episodes: TensorFlow with Rajat Monga

Eric Anderson (@ericmander) and Dor Laor (@DorLaor) go under the hood of Scylla, the open-source NoSQL database designed for low latency and high throughput in big data applications. Dor and his team have reimplemented Apache Cassandra in C++ from scratch, with additional compatibility for DynamoDB. In today’s episode, Dor shares details on the exciting work coming out of ScyllaDB, including Seastar, their open-source C++ framework. Also, check out Scylla Summit 2021 to learn what’s next for Scylla. In this episode we discuss: Enabling Scylla to “gain control” by implementing Apache Cassandra in C++ How Dor and his co-founder were ahead of the curve with their vision for virtualization Scylla’s unique shard-per-core architecture Working with distributed teams, even before the COVID-19 pandemic The growing significance of separating the interface from the engine in open-source Learn about Project Circe, which is being featured at Scylla Summit 2021 right now Links: Scylla Seastar Scylla Summit 2021 Apache Cassandra DynamoDB MongoDB Redhat QEMU Redis Vectorized Apache Hadoop Apache HBase Apache Beam Apache Flink Apache Spark People mentioned: Avi Kivity (@AviKivity)

Eric Anderson (@ericmander) chats with Sven Efftinge (@svenefftinge), Christian Weichel (@csweichel) and Gero Posmyk-Leinemann (Github: @geropl) about their work on Gitpod, an open-source Kubernetes application that allows engineers to spin up a server-side dev-environment from a Git repository, all within their browser. The three team members are part of TypeFox, a consulting firm that specialized in developer tools for different companies before branching out into open-source projects. Upon Gero’s hiring at TypeFox, he was tasked with creating a minimum viable product for the idea that would eventually become Gitpod. Tune in to hear how shifting from consulting to working on their own open-source projects was a breath of fresh air for the developers at TypeFox. In this episode we discuss: How Gitpod solves the problem of switching between multiple dev environments, and improves deep code review The trap that many open-source founders fall into Why TypeFox wanted to switch from a consulting firm to a product shop Details on how Gitpod handles licensing Learn how you can instantly try out a Gitpod environment for any existing Github repository Links: Gitpod TypeFox Theia Kubernetes People mentioned: Anton Kosyakov (@akosyakov) Sid Sijbrandij (@sytses)

Eric Anderson (@ericmander) interviews Graham Neray (@grahamneray) about oso, the open-source policy engine for authorization. oso was originally born from a desire to make infrastructure and security easier for developers, which is why Graham and his company describe themselves as being in the “friction-removal business.” Listen to today’s episode to learn how the team at oso are working to put security in the hands of developers.  In this episode we discuss: Developers building RBAC (role-based access control) systems over and over again Why open-source is the best way to handle authorization logic The history behind oso’s core policy language, Polar How someone beat Graham to the punch submitting oso to a Python newsletter Comparing oso and OPA (Open Policy Agent) Links: oso Stripe Trulioo MongoDB Auth0 Show HN OPA Polar Adventure People mentioned: Sam Scott (@samososos) Alex Plotnick (Github: @plotnick) Stephen Olsen (@olsenator4) Other episodes: Presto on Contributor OPA on Contributor

Eric Anderson (@ericmander) is joined by Rajat Monga (@rajatmonga), a co-creator of TensorFlow. Originally developed by the Google Brain team, TensorFlow is now one of the most popular open-source libraries for machine learning. The team at TensorFlow seek to “democratize” the world of AI as we know it, and by all accounts, they are succeeding. Listen to today’s episode to get inside one of the largest and most exciting open-source projects of the decade. In this episode we discuss: How TensorFlow compares to other open-source projects at Google Taking bets on launch day numbers Balancing the demands of different kinds of TensorFlow users Lessons from Keras and PyTorch Links: TensorFlow Keras  PyTorch Kafka Kubernetes MapReduce: Simplified Data Processing on Large Clusters Bigtable: A Distributed Storage System for Structured Data People mentioned: Jeff Dean (@JeffDean) Andrew Ng (@AndrewYNg) François Chollet (@fchollet)

Eric Anderson (@ericmander) and Frank McSherry (@frankmcsherry) dive into Materialize, a source-available streaming database that lets engineers build real-time applications. Frank is a data processing expert whose work at Microsoft Research on the Timely and Differential Dataflow models culminated in the Materialize project. Tune in to today’s episode to learn how the team at Materialize are making the technology from cutting-edge data research accessible to a wider swath of users. In this episode we discuss: Sharing early ideas with an “academic open source” approach How Materialize made a commitment to correctness Frank’s developmental philosophy of iterative thinking Novel applications for the Materialize community Changing the way we approach problems with real-time data processing Links: Materialize Naiad: A Timely Dataflow System DryadLINQ Apache Arrow People mentioned: Arjun Narayan (@narayanarjun) Derek Murray (@mrry)

Eric Anderson (@ericmander) speaks with Thomas Graf (@tgraf__) about Cilium, the open-source networking, observability, and security software for cloud-native applications based on eBPF. Thomas is the co-founder and CTO of Isovalent, which maintains both eBPF and Cilium. Listen to today’s episode for a discussion of how Thomas’ work has leveled up the Linux kernel and the possibilities of network infrastructure in a cloud-native world. In this episode we discuss: The impact of simultaneous development on Cilium and eBPF Google’s incorporation of Cilium Shortening the gap between writing kernel code and its deployment What JavaScript and eBPF have in common Cilium’s sister project, Hubble Links: Cilium eBPF Isovalent Red Hat OpenShift Kubernetes Docker New GKE Dataplane V2 increases security and visibility for containers SPIFFE Istio People mentioned: Brendan Gregg (@brendangregg) Other episodes: Istio on Contributor

Eric Anderson (@ericmander) and Jeremiah Lowin (@jlowin) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail. In this episode we discuss: Solving the “negative engineering problem” Learning from the complaints of data engineers at Apache Airflow The difficulty of having a product that serves two masters How COVID changed the direction of Prefect Links: Prefect Apache Airflow Why Not Airflow? People mentioned: Jim O'Shaughnessy (@jposhaughnessy) Patrick O’Shaughnessy (@patrick_oshag)

Eric Anderson (@ericmander) catches up with Torin Sandall (@sometorin), co-creator of Open Policy Agent (OPA), the open-source, general-purpose policy engine. By focusing on demonstrating OPA’s value through case studies, targeted interviews, and word-of-mouth, Torin and the folks at Styra were able to grow OPA into the emerging standard for unified policy enforcement across the cloud-native stack. In this episode we discuss: When Netflix stumbled across OPA and delivered its “Cinderella moment” Why OPA was designed to be developer-centric The value of demonstrating OPA’s use cases to the industry How one user created an RPG engine with OPA Links: Open Policy Agent Styra OpenStack LinkerD Hacker News Kubernetes KubeCon OPA Gatekeeper conftest Corrupting the Open Policy Agent to Run My Games Envoy Styra Academy People mentioned: Tim Hinrichs (@tlhinrchs) William Morgan (@wm) Kevin Hoffman (@kevinhoffman) Other episodes: LinkerD on Contributor Envoy on Contributor

Eric Anderson (@ericmander) and Maxim Fateev (@mfateev) trace the development of Temporal, an open-source workflow orchestration engine. At Uber, Maxim co-created the project’s predecessor, Cadence, but Temporal’s roots stretch farther back to include lessons learned at Amazon and Microsoft. In this episode, learn how 18 years of experience in asynchronous messaging and workflows culminated in the foundation of Temporal. In this episode we discuss: Why Maxim quit Uber to start his own company Differences between Temporal and Cadence How Uber is filling the position that Google once had incubating open-source projects Maxim’s advice for aspiring open-source founders Related Links: Temporal Cadence Kafka HashiCorp BanzaiCloud Hacker News Andreesen Horowitz TChannel Hadoop People mentioned: Samar Abbas (@samarabbas77)