Oxide and Friends

Bryan and Adam were joined by members of the Oxide manufacturing team to talk about all that goes into ramping up production, from people and processes to expanding the team and refining inefficiencies. It's a great problem to have!In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, CJ Mendes, Kirstin Neira, Erik Anderson, Aaron Hartwig, and Doug Wibben.Previously on Oxide and Friends...OxF s03e20 - Tales from Manufacturing: Shipping Rack 1Some of the topics we hit on, in the order that we hit them:Topic[@M:SS](link into recording) Leventhal's ConundrumPRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Jerry Neumann joined Bryan and Adam to discuss his blog post from 2015, examining the work of Carlota Perez on technological revolutions. These waves have similarities, in particular: frenzy, bust, and deployment. Is AI a new wave or the culmination of the IT wave of the last 50 years?In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Jerry Neumann.Some of the topics we hit on, in the order that we hit them:Jerry's 2015 blog post: The Deployment AgePRs needed!Previous episodes mentioned:OxF s05e24 - Oxide’s $100M Series BOxF s05e04 - AI Disruption: DeepSeek and CerebrasIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Scott Hanselman gave a terrific talk about the promises of tech: connection, convenience and creativity. Did it deliver? Scott joins Bryan and Adam to discuss... and also wander around as one expects from an Oxide and Friends episode.In addition to Bryan Cantrill and Adam Leventhal, our special guest was Scott Hanselman.Past episodes mentioned:OxF s01e12 - A Brief History of Talking ComputersOxF s02e18 - Books in the Box ReduxOxF s05e10 - Lip‑Bu Tan’s IntelSome of the topics we hit on, in the order that we hit them:Scott's talk: Tech Promised Everything. Did it deliver?If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Oxide raised its $100M Series B round of venture capital. Oxide's founders, Bryan and Steve, answer questions selected by Adam from social media about the round, the company, and the future.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide CEO, the man, the myth, the legend, Steve Tuck.Previous episodes mentioned:OxF s05e10 - Lip‑Bu Tan’s IntelOxF s03e04 - Oxide and the Chamber of MysteriesOxF s04e27 - Unshrouding Turin (or Benvenuto a Torino)OxF s05e14 - Bringing up CosmoSome of the topics we hit on, in the order that we hit them:blog: Oxide's $100M Series BHacker News threadPRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Two years ago, the Oxide team encountered data corruption during a fairly simple network data transfer. The ensuing debugging sessions uncovered a truly bizarre bug involving CPU speculation! Bryan and Adam were joined by colleagues John and Rain to discuss the discovery and circuitous hunt to track down the bug.In addition to Bryan Cantrill and Adam Leventhal, speakers included John Gallagher, and Rain Paharia.Previous episodes mentioned:OxF s03e09 - Get You a State Machine for Great GoodOxF s03e20 - Shipping the first Oxide rack: Tales from ManufacturingOxF s04e25 - RTO or GTFOOxF s02e38 - A Debugging OdysseySome of the topics we hit on, in the order that we hit them:The Update FrameworkOmicron Issue #3441 (Oxide Computer GitHub)Omicron Pull Request #3455 (Oxide Computer GitHub)stlouis Issue #454 (Oxide Computer GitHub)Changing psrset.out.txt (Oxide Computer)Commit 5ec2885322423c0cca0d006611b5c9ac94b0f588 (Oxide Computer)Omicron Pull Request #3560 (Oxide Computer GitHub)If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

What happens when the Oxide API is slow? A podcast episode! More specifically, one about how the team employed all manner of debugging techniques to track it down to one obscure and configurable async runtime feature! Bryan and Adam were joined by members of the team to talk about that journey and the tools we used (and made!) along the way.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Dave Pacheco, Eliza Weisman, and Augustus Mayo.Previous episodes mentioned:Oxide and the Chamber of MysteriesThe Saga of SagasDTrace at 20Cultural IdiosyncrasiesMr. Nagle’s Wild RideA Debugging OdysseyRTO or GTFOSome of the topics we hit on, in the order that we hit them:Falling in Love with RustTokio Runtime Builder – disable_lifo_slotmagic‑trace (GitHub)Magic Trace podcast episode from Jane Streetdiesel‑dtrace (GitHub)omicron issue commentqorbstatemaptokio‑dtracetokio issue #7411Visualizing Systems with StatemapsPostgreSQL WAL INIT ZEROStatemaps: Visualizing System Behavior (YouTube)The statemaps that we referred to:Nexus by thread, discussed starting at 55:29. (This statemap has some states coalesced; the full version is also available.)Nexus by Tokio task, tagged by thread, discussed starting at 1:15:33The D scripts that we referred to:nexus-statemap.d used to generate the initial statemapnexus-profile.d to understand what was consuming CPUtokio-statemap-tagged.d to generate the Tokio task statemapIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

On the heels of Bryan's blog post about the similarities between aspiring college athletes finding a team and entrepreneurs raising a round of capital, Bryan and Adam were joined by Robert Bogart to discuss his own experiences with both--and the life lesson accrued along the way.In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Robert Bogart.College Baseball, Venture Capital, and the Long MaybeOxF: Debugger‑Driven DevelopmentAnthony Ervin – WikipediaEddie Reese – WikipediaMetaweb – WikipediaIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Building systems software can be quite opaque, leading to the need for great debugging tools. At Oxide, we've found that debuggers can be even more valuable leading rather than following system development. Bryan and Adam talk with Oxide colleagues about how domain specific debugging tools help us build systems not only more robustly, but faster as well.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Dave Pacheco. John Gallagher, Alan Hanson, and Eliza Weisman.Previous episodes mentioned:OxF: AI Discourse with Steve KlabnikOxF: The Saga of SagasOxF: A Crate is BornOxF: The Network Behind the NetworkOxF: Bringing up CosmoOxF: RIP USENIX ATCOxF: Dijkstra’s TweetstormSome of the topics we hit on, in the order that we hit them:omdb ground rulesIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Last week, our colleague (and frequent Oxide and Friends guest) Steve Klabnik made some new friends on the Internet with a blog entry on AI discourse. Bryan and Adam were joined by Steve to try to de-polarize the discussion a little.In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Steve Klabnik, and valued listener, Julian Giamblanco (aka "Oatmealdealer").Some of the topics we hit on, in the order that we hit them:Steve's blog post: I am disappointed in the AI discourseOxF: A Baseball Startup with Paul Freedman and Bryan Carmel (The Ballers)OxF: Adversarial Machine Learning with Nicholas CarliniOxF: Hiring Processes with Gergely Orosz ("the RFD 3 podcast episode")OxF: AI Disruption: DeepSeek and CerebrasOxF: Reflecting on Founder Mode ("ego con")If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Late in 2024, an economics paper captured the attention of the world. AI, it claimed, had a tremendous impact on materials research, disproportionally benefitted the most productive, and--sadly--reduced job statisfaction. It now appears that the results are entirely fabricated! Ben Shindel joins Bryan and Adam to discuss.In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Ben Shindel.Some of the topics we hit on, in the order that we hit them:Ben's blog: AI, Materials, and Fraud, Oh My!OxF: Theranos, Silicon Valley, and the March Madness of Tech FraudTopic[@M:SS](link into recording) Leventhal's ConundrumPRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Bryan and Adam discuss the recent announcement of the discontinuation of the USENIX Annual Technical Conference (ATC), reminiscing about their own visits to the ATC and the impact of the conference. Long-time Oxide Friend, Tom Lyon, joined to dial the reminiscence back a couple more decades!In addition to Bryan Cantrill and Adam Leventhal, we were joined by Tom Lyon.Some of the topics we hit on, in the order that we hit them:Bryan's blog 2025: RIP USENIX ATCOxF s1e13: Put the OS back in OSDIBryan's Lisa 2011 talk: Fork Yeah! The Rise and Development of illumosBryan's USENIX 2016 talk: A Wardrobe for the EmperorUSENIX 2004Gnutella not NutellaUSENIX DTrace paperUSENIX Summer 1994Slab AllocatorNFSv3WSJ 2006 Technology Innovation Awards0xF s1e18: Dijkstra's TweetstormMeeting Dennis RitchieBoF sessionBirds of a feather flock togetherFreenix20032004Rik Farrow ;login: editorial on USENIX 2016Bryan's blog 2004: Wither USENIX?blog comments from Werner VogelsSystems We LoveAdam's blog 2004: nohup -pillumos sourceOxF s1e4: from /proc to proc_macroThings that don't work as advertisedDiffracting treesCold FusionAdam's blog 2009: Triple-Parity RAID-ZRob Pike 2000: Systems Software Research is IrrelevantZFS paperLiving Computer Museum (now-dead)SDFAll the Chips that Fit by Tom LyonOxF s2e22: RIP OptaneHistory of Programming Languages ConferenceIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Matthew Sanabria joins Bryan and Adam to talk about his role at Oxide--Solutions Software Engineer--and how it fits in with engineering, sales, support and marketing. It takes everyone in Busytown! Sound good? Apply!In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleague, Matthew Sanabria.Some of the topics we hit on, in the order that we hit them:Solutiuons Software Engineer applicationOxF: the "squeezefish" episodeThe Fallthrough podcastBusytownIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Last week the kerfuffle between Synadia and CNCF, tussling over the ownership and futures of NATS, bled into the public. The outcome may cast a long shadow for open source and for the CNCF. Bryan and Adam were joined by Rachel Stephens and Adam Jacob to discuss how we got here and possible outcomes.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Rachel Stephens Adam Jacob, and Eliza Weisman.Some of the topics we hit on, in the order that we hit them:Goats in sweatersCNCF Slide: Why You Should Host Your Project at CNCFCNCF NATS documentsNATS GitHub discussionThe uncashed $10k checkCNCF landscapeCNCF blog on NATS / SynadiaSynadia response to the CNCFPostscript:The CNCF updated its blog with proof that the ACH transfer of $10,000 was completed [still very funny! -ahl].Derek Collison--as reported by Runtime News--has agreed to transfer the NATS trademark to the CNCF "because we just feel that the damage to the ecosystem and the ugliness is not worth it for anyone."If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Oxide is bringing up its next generation server. To discuss the (amazingly smooth) bringup process, Bryan and Adam were joined by members of the oxide team. Tales of adversity, re-work, un-re-work, and triumph!In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Nathanael Huffman, Ian Sobering, Matt Keeter, and Aaron Hartwig.We mentioned quite a few terms! Here's a helpful guide:Cosmo - Oxide’s next-generation sled (currently in development) with an AMD Turin CPUGimlet - Oxide’s current-generation sled with an AMD Milan CPUTurin - AMD Epyc 9005 SeriesMilan - AMD Epyc 7003 SeriesGenoa - AMD Epyc 9004 Series (Oxide chose to skip this generation)Sequencing - the precise control of when power rails are energized throughout a PCBSled - One of the (max 32) computers in an Oxide rack; a custom form-factor optimized for power and cooling efficiencyIBC - Intermediate Bus Converter (Our 54VDC -> 12VDC converter)RoT - Root of TrustSP - Service Processor, the small computer (running Hubris) that allows for low-level controlIgnition - An even lower-level control network for power management (including power of the SP)Ruby - The AMD reference platform (Oxide has used this to prepare Cosmo software in advance of bringup)DC-SCM - https://www.opencompute.org/documents/ocp-dc-scm-spec-rev-1-0-pdf and OpenCompute standard form factor.Grapefruit - OCP DC-SCM form-factor board with our SP, RoT, and FPGA on it, used to replace the OCP DC-SCM baseboard management controller in the Ruby platform.Cadence - Software Oxide previously used for PCB designAltium - Software Oxide now uses for PCB designHubris - Oxide’s embedded operating system, run on the SP and RoTHumility - The Hubris debuggerPLM - Product Lifecycle Management – a class of software used for managing hardware BOMsBOM - Bill of Materials – the components required to build a hardware productRFK - Our colleague, Robert Keith (to distinguish him from our other colleague, Robert, and our former colleague, Keith)FPGA - Field Programmable Gate Array – Also referred to as “soft logic” – effectively programmable hardwareILA - Integrated Logic AnalyzerJTAG - A debugging interface for various processorsUART - A serial port or connectionFor previous tales from the bringup lab:Tales from the bringup labMore tales from the bringup labBringup Lab Chronicles: A Measurement Two Years in the MakingRaiding the MinibarIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Bryan and Adam have been gushing for months over Character Limit, the fantastic book by Kate Conger and Ryan Mac about Elon Musk's haphazard and disastrous takeover of Twitter. They're joined by the authors themselves to discuss the book, Musk, DOGE, and some of the Character Limit unreleased B-sides.In addition to Bryan Cantrill and Adam Leventhal, our guests were Ryan Mac and Kate Conger.If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

An Oxide customer encountered a peculiar issue at the intersection of their Oxide network and their broader network. Bryan and Adam were joined by several members of the Oxide team who collaborated to investigate and--ultimately--solve the problem using a combination of tooling, intuition, and dark knowledge.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Levon Tarver, Alan Hanson, Will Chandler, and Trey Aspelund.Some of the topics we hit on, in the order that we hit them:PRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Much of the work at Oxide goes into hardware and software used to build and test the eventual product. Bryan and Adam were joined by Ian, Doug, and Nathanael to talk about "Minibar", a rig for connecting up an Oxide server (code name: Gimlet) for manufacturing and internal use. Triumphs and catastrophes including stabbing a connector with a guide pin and bringup mishaps!In addition to Bryan Cantrill and Adam Leventhal, speakers included Ian Sobering, Doug Wibben, and Nathanael Huffman,Some other, related Oxide and FriendsOxF: Cabling the BackplaneOxF: The Network Behind the NetworkOxF: The Power of ProtoboardsImages from the show:

Intel has a new CEO! And it's Lip-Bu Tan. We had assumed it would not be Lip-Bu--he was such a clear front-runner that the more time passed the less likely it seemed it would be him... and yet! Bryan and Adam were joined by Reuter's Max Cherney to discuss.In addition to Bryan Cantrill and Adam Leventhal, our esteemed guest was Max Cherney; we were also joined by Bryan Russett, and Alex Kesling.Some of the topics we hit on, in the order that we hit them:Cooking with Oxide and FriendsThe Oxide John von Neumann bustIntel's new CEO plots overhaul of manufacturing and AI operationsLip-Bu Tan: Remaking Our Company for the FutureIntel oneAPIMorris Chang: "A very discourteous fellow"If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Recently, a change to a utility in the Rust toolchain changed behavior in a way that impacted users. Rather than being a story of frustration and aspersions, it was a story of a community working... and working well together! Bryan and Adam were joined by Dirkjan Ochtman (of the rustup team) and Steve Klabnik to discuss.In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest, Dirkjan Ochtman, and treasured colleague, Steve Klabnik.Some of the topics we hit on, in the order that we hit them:Steve: A Happy Day For RustPRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Bryan and Adam were joined by Oxide colleagues Andrew, Rain, and John to talk about creating a general purpose crate for diffing structures. More generally, how do you know when something new is needed? How do you know when the investment of time to validate an idea is warranted? Software engineering is hard! (And also: general enthusiasm for Rust macros.)In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues, Andrew Stone, Rain Paharia, and John Gallagher.Some of the topics we hit on, in the order that we hit them:Checking in on Bryan's 1 Year Intel CEO predictionHiring letter to Intel's co-CEOFrom The Register "Re-hire Gelsinger!"Oxide RFD 457: Control plane sled lifecycleOxide RFD 459: Control plane component lifecycledaft cratediffus crateIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!