DiscoverThe Swyx Mixtape
The Swyx Mixtape
Claim Ownership

The Swyx Mixtape

Author: Swyx

Subscribed: 7Played: 321
Share

Description

swyx's personal picks pod.

Weekdays: the best audio clips from podcasts I listen to, in 10 minutes or less!
Fridays: Music picks!
Weekends: long form talks and conversations!

This is a passion project; never any ads, 100% just recs from me to people who like the stuff I like.
Share and give feedback: tag @swyx on Twitter or email audio questions to swyx @ swyx.io
176 Episodes
Reverse
A wideranging convo with Sunil covering the future of React, the Third Age of JavaScript, and the Meta of online discourse.Watch on YouTube: https://www.youtube.com/watch?v=H3h1WICelqsFollow Sunil: https://twitter.com/threepointoneChapters: [00:01:40] React and Temporal, Declarative vs Imperative My Temporal Explainer: https://twitter.com/swyx/status/1417165270641045505 https://www.solidjs.com/  [00:12:57] State Charts and Lucylang https://lucylang.org/ XState and Stately https://stately.ai/viz [00:17:08] The Future of React [00:25:03] React Streaming Server Rendering vs SSR/JAMstack/DSG/DPR/ISR ReactDOMServer.renderToNodeStream() Sunil's Slides: https://www.icloud.com/keynote/0MyOJkDIOVfFit76PqJFLvPVg#react-advanced https://react-lazy.coolcomputerclub.com/  [00:33:13] Next.js and the Open Source Commons [00:38:46] The Third Age of JavaScript Third Age of JS  Benedict Evans (not Sinofsky) on Word Processors: https://www.ben-evans.com/benedictevans/2020/12/21/google-bundling-and-kill-zones [00:45:16] ESbuild vs SWC vs BunBun (Jarred Sumner) https://twitter.com/jarredsumner/status/1390084458724741121  [00:50:46] Let Non-X Do X: Figma vs Canva, Webflow vs Wix/Squarespace Canva vs Figma valuations https://twitter.com/swyx/status/1438102616156917767 [00:52:42] JavaScript Twitter and Notion's 9mb Marketing Site Notion 9mb JS Site Tweet mrmrs' Components.ai [01:06:33] React Server Components and Shopify Hydrogen/Oxygen https://twitter.com/swyx/status/1410103013885108229  [01:09:18] Categorical Imperatives of Web Platforms: Cloudflare vs AWS, MongoDB vs Auth0, Gatsby vs Netlify https://auth0.com/blog/introducing-auth0-actions/  [01:18:34] Wrap-up  Transcript [00:01:40] React and Temporal, Declarative vs Imperative  [00:01:40] swyx: Okay. So the first topic we want to talk about is React and Temporal, right?  [00:01:43] Sunil Pai: I feel Temporal is introducing a shift into the workflow ecosystem, which is very similar to the one that React introduced to the JavaScript framework system.  [00:01:54] swyx: That's the hope. I don't know if like my explanation of Temporal has reached everybody or has reached you. There are three core opinions, right? The first is that whenever you cross system boundaries, when you call it external API. So when you call internal microservices, there's a chance of failure and that multiplies, the more complex the system gets. [00:02:11] So you need a central orchestrator that holds all the retry states and logic, as well as timers And it tracks all the events and is able to resume from it from failure.  [00:02:21] Second opinion that you should have is you should do event sourcing rather than try to just write your business logic and then instrument with observability logs after the fact you should have your logs as the source of truth. And if it's not in the log, it did not happen.  [00:02:34] And then the final piece is the workflows as code, which is the one that you're focusing on, which is the programming model, in the sense that like all the other competitive workflow engines, like, Amazon step functions, Apache airflow, Dagster, like there's a bunch in this category. [00:02:48] They're all sort of JSON and YML DSLs, and the bind that you find yourself in is that basically you're reinventing a general purpose programming language inside of these JSON and YML DSLs because you find a need for loops, branching, variables functions, all the basic stuff. And, people find that like at the end of the day, all this tooling is available, you just have to make it run in inside of a general purpose programming language. So that's what Temporal offers.  [00:03:12] But it's very interesting because it kind of straddles the imperative versus declarative debate, right? [00:03:17] React, people view as declarative. And I think it's mostly declarative, like there's imperative escape hatches, and because it's declarative, people can have a single sort of render model of their entire app for the entire tree. And I think it makes sense to them. [00:03:32] And you're saying that that's better, right? That's better than the imperative predecessor of like jQuery and randomly hooking up stuff and not having things tied up together. You sounded like you want it to  [00:03:42] Sunil Pai: interrupt. So it's actually two things. One is the jQuery had an imperative API, and then they went way too hard into the declarative side with templating languages and then started reinventing stuff there. [00:03:54] So really react was like, no, you need access to an imperative language to create, you need a fully featured programming language to generate description trees like Dom trees or in this case, a workflow graphs.  [00:04:10] swyx: Got it. So it's kind of like a halfway solution, maybe, maybe anyway. So the problem with us is that we're trying to say that imperative is better than declarative, for the purposes of expressing general purpose business logic, which is an interesting sell for me because in all other respects, I'm very used to arguing to declarative is better. [00:04:33] Then there's also an idea that people should build declarative layers on top of us. And I, it's just a very interesting, like back and forth between declarative and imperative that I don't know where I really stands apart from like, wherever we are is never good enough. So we need to add another layer to solve the current problems  [00:04:51] Sunil Pai: there. [00:04:51] So there's a phrase for it and I forget what it's called the mechanism. It says that, uh, the system that allows you to execute stuff should not be the same system that prevents you from doing bad things. So there's a core, which is basically a fully featured API. And then you put guard rails around like the experiences. [00:05:12] For example, as an example, this is like adding TypeScript on top of JavaScript, let's say, unlike reason ML, let's say like, OCAML or a lot of very strongly type a language where if your code doesn't compile, you can't really run the code in TypeScript. There are times when you're like, you know what? [00:05:29] I need an escape hatch to actually like, do something like really funky here, X, Y, and Z, that that's not even well expressed in either the type system or sometimes even the language itself. You need to like hack it. And like, you might even email a couple of things. Uh, and in react, this was, I think when react came. [00:05:47] It wasn't just that it was a, oh, like there's JSX. It was very much, uh, okay. Uh, I have a lot of existing code, so I can add, React to one part of it and then hook onto the DOM, it renders and then have like this whole jQuery widget that I would like render onto the thing. Uh, so it gave you this whole incremental part to adopting the system, but then like after a point, like react consumes all of it. [00:06:11] And the fuck up with react is if you go too hard into react, doing stuff like animations is like impossible, which is why like we are at least a year or two away from a good animation API in React, or while you use, Framer or whatever Framer has become right now. Like frame of  [00:06:27] swyx: motion. No. Um,  [00:06:31] Sunil Pai: Yeah, but he's working. [00:06:32] I think Matt is now working on like a new, new thing. That's got a really funky name. Like, it sounds like a robot or something. All right. But it was curious to me that React's biggest deal was that, Hey, like, They talk about it being declarative, but a whole lot of things you wrote were like in regular-ass JavaScript, you would say on click and get an event and start doing things  [00:06:53] swyx: beautiful. [00:06:54] It's a perfect blend.  [00:06:56] Sunil Pai: Right. And you would suffer with this in. So there was the jQuery prototype phase, which was like directly imperative. And then they went hard in the other direction with type templating languages, like Jade and dust. And, uh, there were a number of popular ones at the time. And that's when like even Angular 1 became super popular because they're like, here's the whole kit and caboodle full whole framework. [00:07:18] And then React came and said, oh, well just the view. But that's because they didn't want to release like really yet. And they were like, yeah, this is all you need and the whole ecosystem. But anyway, so in temporal temporal for me is particularly interesting for that because it is now clearly making that. [00:07:35] I hate the phrase, but it's a good one. The paradigm shift of like how you start thinking about these systems and you just write some fucking code and then like you start adding on bits and guardrails for the things you want to do, which is on for the few hours I spent going through the docs and failing to get it running on my laptop. [00:07:53] That's my understanding of it. Feel free to correct me.  [00:07:56] swyx: Okay. Yeah. And I think you're right, actually, I'll try this messaging on you because, it's something that we're consciously designing for. In fact, I have a, one of my API proposals was, reacts like API for tempo. And so essentially what we enable you to do is bundle up each individual service or job into a component that we happen to call workflow. [00:08:21] And my struggle here is that I currently tie component to workflow because what is the component like? It's, it's something that's self-contained that is a deterministic. Like it has a strict rule of execution from top to bottom, right. It just does the same thing every single time, uh, where we differ and why I struggle with this is because we put all the side effects into things that we call activities. [00:08:44] That's where all the non-deterministic stuff goes. And that one gets retried, basically at Temporal's will and essentially Temporal is serving as the central runtime or framework that has knowledge of all these workflows and activities. And can re-render them based on its internal rules, I retries timeouts, uh, heartbeats, all that good stuff. So I struggled with things like, which is the component and which is the hook or the effect.  [00:09:08] And then there's other concepts. So, uh, we have ways to send signals into individual workflows, right? That's a very important property of the system that you can send data in while it's running and you can get data out while it's running. I'm not sure that's reflected in React at all. So maybe I'm stretching the analogy too much,  [00:09:24] Sunil Pai: Solid, had an answer for that the word signal. So like solid JS. This is by Ryan Carniato the Marco folks, signals are a first-class concept in the framework. Again, I haven't dived into it in detail in a while, but it feels like an important thing. And I always wondered why React actually didn't have it because props are something that you just like pass. [00:09:46] Right. And it's just a value, like if you like plot it on a graph, for example, it's, let's say if you had to have like a graph of binary values, it would be either zero or like one, and that would be the shape of the graph, but signals are something that can be like something that happens and yeah, just pops up and goes down, like pressing a key on the keyboard. [00:10:06] And that's actually not so easy to define in a, in a react like system, like, uh, which is why it's kind of hard to build like audio processing graphs with like React or JSX. Um, I don't have like a good answer. I'd probably have to like hack on Temporal a little more, but the idea of like signals as a channel, through which you can like send information and having it as a first class part of the system is something that's not represented well in well, in React at least. [00:10:33] Yeah. Well,  [00:10:34] swyx: isn't that in an action? For reducers  [00:10:38] Sunil Pai: and event effectively. Yes. Like it's basically one of those actions.  [00:10:42] swyx: The problem is that everything just ties right into the component tree instead of just having the component B and sort of isolated unit that can function independently.  [00:10:50] Sunil Pai: That's the other thing, which is a workflow engine isn't a directed acyclic graph. In fact, it could have cycles, it could have cycles and it could have a number of other things, which is the  [00:11:00] swyx: beautiful thing, by the way.  [00:11:02] For us coding, a subscription platform literally is charged Stripe sleeps 30 days, charged Stripe again, and then infinite loop until you cancel and then you break out of the loop. [00:11:13] That's it.  [00:11:13] Sunil Pai: That's awesome by the way. So I was actually thinking that someone's going to implement not someone's going to implement, uh, someone's going to use Redux saga on top of Temporal, that's what I was thinking, because then you will have generators that define like long running processes that are just talking to each other. [00:11:30] I think that would be good. CloudFlare also loved Temporal, by the way, like we were talking about it, like for awhile, they're like, oh, this is like fundamentally a new thing. And as you can imagine, some engineers were like, well, why isn't this running on workers? I'm like, I don't know why isn't it running on workers? [00:11:43] Like maybe we should get it there. [00:11:45] swyx: It is fairly heavy duty right now. We're trying to reduce that to a single binary, which could maybe run a workers. I'm not sure about the memory requirements that you guys have. It could, it's just not a priority for us based on our existing users.  [00:12:00] Sunil Pai: Um, I was just, I was saying what they're saying. [00:12:04] They want everything to run on workers and I'm like, dude, it's just like one small, weird isolated like condo.  [00:12:10] swyx: Ironically we also using V8 isolates for our TypeScript runtime. And that's just to make sure that people don't do non-deterministic stuff. So we did mock out everything, which is also pretty cool because whenever you use a library with, like setTimeout inside of that library, that persists to us as well. [00:12:25] So we set the durable timer. Your system can go down and we, we bring it back up and you're using our timer, not the JavaScript runtime timer, which is like just awesome. There's a trade off to that, which is, things don't work when you import them, like you would in a normal, Node.js project. [00:12:39] So most of, because you have to inject them into the environment of the V8 Isolate, you can't just randomly import stuff that as freely as you would in a normal node environment. So dependency injection and becomes a topic for us. [00:12:57] State Charts and Lucylang  [00:12:57] swyx: Um, yeah. We actually clashed a little bit with David Khourshid because David is on this warpath of like everything in a state machine, right. Everything in the time-tested 40 year old JSON format that describes state machines. And we actually thought we were going to be competitive with him for a while because for him, the thing about writing imperative code is that it's prone to bugs, right? Like you can not really see the full, possibly the full span of like all the possible states that you're exposing, but in a state machine everything's explicit so he was butting heads with our founder for awhile. [00:13:31] But I think recently he decided that he is better at building on top of us than trying to compete with us on the reliability front. So that's, that's kind of an interesting evolution that has happened over the past year on this topic of declarative versus imperative. [00:13:44] I'm still like coming to terms with it. Like I'm not fully okay with it yet, but, it clearly is more expressive and that's something I am Very in favor of, and I have genuinely looked at like the workflow solution from Google, the workflow solution from Amazon, and they are literally have you write the abstract syntax tree by hand in JSON and that's just absolutely no way that that's going to work. So I'm pretty down with the imperative approach for now.  [00:14:09] Sunil Pai: Well, that's, I figured at some point you will run XState on it and extent should work fairly well. I think contemporary, I don't see why it would. I think that that would actually,  [00:14:19] swyx: Honestly, I'm not really sure what he's going to charge for. He's pushing the idea of state machines and making it more of a commonly accepted thing. [00:14:26] Sunil Pai: Well, his pitch isn't even state machines. It's very specifically state charts and I love state charts. I even bought the book by the way, the Ian Horrocks $700. So when I got it on Amazon, it was $180. I was like, cheap. Let's do it. I got really lucky at the time. It, it fluctuates like mad by the way that that value, well, you should expense it now is what it is. [00:14:46] Um, but, uh, what struck me about the thing? Here's what I tried. I really liked it. And I took a course, a couple of steps back and I was trying to understand, well, why isn't it like a success? Why don't people get into it? And the truth is that this falls not just into the intersection of this is the intersection of like computers and humans in the sense that sure. [00:15:07] There are things that can be correct, but there are things that can be expressible as well. Like I don't even know what code I want to write when I'm sitting down to write it. I love to like discover it while I'm writing it and really. All the syntax that we have created and abstractions, we have created around programming languages have been purely to express these things and have let's call it implicit state machines, even though that implies that it's bad. [00:15:32] Um, so for example, if you look at state charts, there's no real good way to compose two state charts together. You have to like manually start wiring them together. And like, there's, you know, like you've got in react, you say, oh, combo, if you have two components to put it together, you put like a little, uh, function around it. [00:15:49] And now it's two components in one component. So it's important not just to have a good unit of computation, but to have it like be composable with each other so that you can gather it and then make this whole nesting doll react, Dom tree of things. And I think. Until there's an actual language that supports that has state charts as a first class primitive, much like Lucy, I think that's what Matthew Phillips built. [00:16:15] He wrote a, he wrote an actual language that compares to state charts called Lucy Lang. That was very cool by the way. Like, I really like it. Uh, well, and it's fairly young, so it's too early to say whether people like love it or not. And other than, but people like you and me who look at something like, wow, this is awesome. [00:16:33] Let's all use it. No, like to take a while to grow. But I think that's the state charts has a bit of dissonance with the languages that it's written in right now, because it's not a first-class thing. I mean, it's adjacent object with keys and. Okay. Like we can do better maybe. Uh, but I would not bet against David and the people he's hiring. [00:16:53] Like he's hurting some smart people, you know, they're all like pretty intelligent. So I'm curious to see how that plays out.  [00:17:00] swyx: I'm just glad that we're not competing. Uh, so that's, that's something that, that, that resolved itself without any intervention from me, which is very good. [00:17:08] The Future of React  [00:17:08] swyx: Well, let's have this conversation since it's related, should React to be more of a DSL,  [00:17:14] you know, this conversation that happened over this week, so I'll pull it up. [00:17:20] Sunil Pai: Uh, wait, so I've, I'm seeing, is this the whole Svelte versus React thing that's been happening over the last two, three days?  [00:17:25] swyx: Yes. So basically it's saying React is already so far down almost like its own language. [00:17:30] They should just embrace it more. And instead of using linting to catch rule violations, just make a DSL, people are gonna use it. It's fine. And just like build things in so that it's impossible to make these errors that, that people commonly make.  [00:17:47] Sunil Pai: So this is Mike Sherov, uh, he was smoking about it. [00:17:51] He mentioned how it shouldn't be a lint rule. And since we already have customs, insects and GSX, he should introduce a couple of other things. So as you can imagine, the react team has thought about this a lot. So the big problem with this all boils down to that fucking dependency area on use effect, by the way, that's the one that trips, everything else is fine. [00:18:09] Like you stayed all that is like fine. You can get. This is  [00:18:13] swyx: what it was. Yeah. People want like state something memos and things like, you know, just build the reactor primitives into the language.  [00:18:22] Sunil Pai: So yeah, I think this, this actually, isn't a bad idea and I think that was the whole deal with hooks. Whereas what's the phrase that they use in the docks. [00:18:30] A sufficiently advanced compiler might comply with these things at some point, and you're like, oh wow, great job. On pushing that responsibility onto the community, React team, well done.  [00:18:41] swyx: My joke is like it's the react teams equivalent of a assume, a frictionless spherical cow from physics.  [00:18:48] Sunil Pai: Exactly. [00:18:48] That's a perfectly spherical code. [00:18:54] swyx: It will exist.  [00:18:57] Sunil Pai: And it's just the five of them or six of six of them hacking on this. And they have to make sure they don't break like facebook.com whenever they're working on these things. Imagine it's taken this long for Concurrent to show up and Concurrent is nice by the way. And we can talk about the server rendering API. [00:19:14] Okay. Uh, so react right now is, uh, yeah, that's the one like that. It shouldn't just be an intruder, but, uh, inside the inside Facebook only, well, not everybody can see it, but it's an in an internal, uh, uh, Facebook Wiki page, which is a list of potential F projects. You know, how the react team has fiber, whatever the hell. [00:19:47] Right? So there's a list of these projects that, or when we do this, uh, project F F I forget what the one for, uh, uh, animation that's called, is it called flat? Flat was the dumb one. And so there are lists of them and there are about 15, 20. I'm pretty sure my India has done. So Hey, so, uh, there's a list of them. [00:20:09] And if you look at them and you start assigning values in terms of work, oh, this is about six months of work. This is about, uh, another six months of work. It strikes you that there's a roadmap for about five to 10 years. At least if not more than that, I mean, look at how long it took to get like this. Of course this was very more foundational. [00:20:26] Those could probably happen a little quicker when it comes, which means the react team is like solely aware of what's missing in react right now. And to an extent that they can talk about it because if they do it becomes like a whole thing and like don't really engage in that conversation. They don't, I, I, and I don't blame them for it. [00:20:44] It's very hard to have this discourse without somebody coming in and saying, well, have you considered CSS transitions? I like that. Yes, we have. We have, we have considered CSS a lot. Uh, so, uh, so. There are all these projects like a sufficiently advanced compiler that compiles down to hooks. There's the animation API. [00:21:04] There's a welcome current, et cetera. This whole data fetching thing has been going on for years. And now it's finally starting to come to light, thankfully with collaboration, with the relay team and effectively all of the core when they built out facebook.com and, and that is the length that those are the time periods that Sebastian looks at and says, yeah, this is how we can execute on this because it can be prioritized. [00:21:33] It has to be prioritized by either Facebook wanting it or making Facebook wanted. So for example, the pitch was, Hey, let's rewrite facebook.com the desktop version because they haven't, it's a film mishmash of like hundreds of react routes on one page. It should be a single react route that does this thing. [00:21:52] Now that we have gotten management to agree to a rewrite, let us now attach it to the concurrent mode thing. And that was also part of it, which is in the older version, there was a lot of CPU fighting that used to happen between routes, which is why the whole work for the share dealers started and took like two years to like fix effectively. [00:22:08] They're doing cooperative, multitasking VM in JavaScript, which sure. When you're a Facebook, I guess you've got to like do these things. Uh, and how does that all,  [00:22:18] swyx: was that ever offloaded to the browsers, by the way? Like I know there was an effort to split it out of react.  [00:22:24] Sunil Pai: So I think last, I checked they were talking to Chrome literally every week. [00:22:29] Uh, but I think it's also been down to, uh, well, what Chrome wants to prioritize at the time. I think it is still going ahead again. It's the sort of work that takes years, so it's not going ahead. Nice and slowly, uh, which is why. Which is why it's architected inside react for the same reason as like it's attached to global and then read off the global. [00:22:52] I think it's also why you can't have two versions of React on the same page. There's the whole hooks thing. But also if you have two versions of React, and they'll just start fighting with each other on the scheduler, because the scheduler would yield to one than to the other than to the other one. [00:23:08] And there would be no like central thing that controls what is on the scheduling pipeline. That's from the last, again, this conversation is at least two years, or maybe they fixed that, but that's the goal of the dealer. There has to be one scheduler for the thread that everybody comes on to, and like tries to pull stuff, uh, with it. [00:23:26] I think it will become a browser API. It's just a question of like, when, like, yeah, I mean, the shared dealer in react itself has undergone so much change over the last three years. Uh, so maybe we should be glad that it isn't in the browser yet, because like, it's changed so much. It's coming there. It's I mean, the fact that they're releasing in November is a big deal. [00:23:45] swyx: You said there's so many projects that you want to ship, and the way to ship it in Facebook is to either convince them that this feature itself is worth it, or you tie it together with something else, like the Facebook, I think it's called FB5 rewrite. [00:24:00] Sunil Pai: Oh yeah. I think it's good for them. Like it worked because the Facebook, facebook.com is now more performant. Like it actually works well and they don't have CPU fighting. The fact that Facebook itself is becoming slightly irrelevant in the world is a whole other conversation.  [00:24:17] swyx: Well, you know, I still use my billions, so, uh, it's it's, it improves the experience for them. [00:24:23] Sunil Pai: I'm only being snarky.  [00:24:25] swyx: Uh, but I, you know, hopefully hopefully you're like, you know, there's other properties like Instagram and WhatsApp and what is, uh, which hopefully it will apply there. And then obviously like there there's the VR efforts as well. Absolutely. Yeah.  [00:24:39] Sunil Pai: And that is the future. In fact, uh, several components also happened because they suddenly realized what they could do for how the deal with server components and server-side streaming rendering was never about an SSR story, or even a CEO. [00:24:54] Facebook doesn't give a fuck about SEO, right. It was about finally they figured out how to use concurrent mode to have a better UX altogether.  [00:25:03] React Streaming Server Rendering vs SSR/JAMstack/DSG/DPR/ISR  [00:25:03] Sunil Pai: So, okay. I should probably just keep Server components aside for right now.  [00:25:06] And I'll just talk about the new streaming rendering API. Okay. [00:25:09] Okay. So I know there's like about three styles of rendering. [00:25:14] I say legacy, but legacy is such a dirty word. I don't mean it in the form that it's old it's in fact,  [00:25:20] swyx: traditionally, like, sorry.  [00:25:24] Sunil Pai: Uh, heritage Facebook would say heritage, it's a heritage style rendering, um, which is the, Hey, you use something like a rails or spring or some, it could be node as well. And you spit out a bunch of HTML and then you progressively enhance it with sprinkling JavaScript, pick your metaphor there like three or four metaphors that you could use. [00:25:44] Uh, uh, web components actually falls square into this, where it just comes to life only on the browser and then like make stuff interactive. Uh, then there's the whole client fully client side rendered one. So this is create react app or, well, a number of like smaller players then there's server side rendered. [00:26:04] And so as I rendered is actually like, it's not just next year. It's also your Gatsby. I feel like pretty much every, uh, react framework now has some kind of service side rendering story. Okay. So the next slide goes into what types of server-side rendering things happen. [00:26:20] swyx: there are a lot of subdivisions within here, right? [00:26:22] Like, uh, Gatsby is up here trying to reinvent like D S R D P R or something like that, which is like deferred,  [00:26:29] Sunil Pai: static,  [00:26:32] swyx: DSG, deferred static generation. That's the one. My former employer, Netlify also DPR, and is all, these is all like variations of this stuff with,  [00:26:41] Sunil Pai: like, it's a question of where you put the cache is what it is. [00:26:46] It's a TLA three letter acronym to decide where you put the caching in.  [00:26:49] Yeah, so there's the whole JAMstack and that's like the whole Netlify story, but also CloudFlare pages, or even GitHub pages. [00:26:56] There's no real runtime server rendering. You just generate a bunch of static assets and you Chuck it and it just works. Then there's fully dynamic, which would be next JS without any caching. Right? Like every request gets server-side rendered then like a bundle loads on top of it. And, um, like suddenly makes it alive, like sort of like it hydrates it. [00:27:16] And then after that it's effectively a fully clients rendered application then there's okay. So I just said ISR, but like you said, there are like three or four after this as well. There's this whole DSP. Yeah. Oh wait. So the new streaming API is actually fundamentally new because. I don't know if people even know this, but react already has a streaming rendering API. [00:27:37] It's called a render to node stream. I think that's the API for it. And the reason that that exists is so that, uh, only for a performance thing on the server where otherwise synchronous renders would block like other requests. And it would make like if for a server that was very, uh, uh, there was heavily trafficked. [00:27:57] It would become like really slow. So at least with the streaming API, yeah. That's the one learner to notice the stream, at least with this one, it wouldn't clash and you could interleave requests from there happening, but it didn't solve like anything else, like nothing, you couldn't actually do anything asynchronous on it, which is kind of that fucking sucks because like, it looks like it's an asynchronous API, but you can't do anything asynchronous through it. [00:28:18] It's the only thing that, okay, so vendor to readable stream is cool because I can, even if you go to the very last slide last bit, once. You know what this is, where the very first link open it up. Like it says react, lazy.cool computer club. So this is the demo that they have that exists with this new API. [00:28:36] This is what they link to. So if you refresh it a couple of times and I'll show you something that happens here, so you see the little spinner that shows up there and then the content loads. Yep. So, um, you know what, maybe I can share my screen because I want to show like a couple of things. Uh,  [00:28:53] swyx: yeah. I'll fill in some context, like I knew that the renderToNodeStream API was not good enough, basically because everyone who is doing SSR was doing like a double pass render just to get the data in. Um, and I noticed a very big sticking point for Airbnb so much that they were almost like forking react to something like that to,  [00:29:11] Sunil Pai: they invented a caching API. [00:29:13] They did like a whole bunch of things. Okay. So if you have a look here, you'll see that there's a little bit of spinner and then the content comes in. But now what I'm going to do is I'm going to show you the actual HTML. So let's just go to prettier and just pretty far this, for that, we can see the content and I'll show you something that's very like fundament. [00:29:32] That's the playground playground paste, big HTML. All right. So are you looking at this HTML it's rendering rendering by the way, this, these are special comments that mark suspense boundaries. It's very cool. If you come down here, you'll see a dev, which is the spinner. So this is the spinner that you see when you refresh the page. [00:29:52] So this is. And then the rest of like then, like the, like the bits that are below that close and the HTML closes, but content still start stream is streaming in at that point. So like, this is the actual, like devs that are coming in with the content. And then a script tag gets injected that says, Hey, this thing that just came in, shove it into where the spinner was. [00:30:13] This template  [00:30:14] swyx: tag is so small. I would, I would have imagined it was much bigger.  [00:30:18] Sunil Pai: It's not. So by the way, at this point, the react has not loaded. This is happening without react. This is just a little DOM, much like swelled ha uh, just a little operation that does it. So you, you, you get this content. And, uh, so, so that's the first feature which is that suspense. [00:30:35] It not only works out of the box, but fallbacks and replacing or fallbacks with actual content also happened. Um, I want to pull this outside of this main window to show you something. Um, so you can see the content load in, but keep an eye on the loading spinner. Okay. Just to prove a point. So the content loads in, oh man. [00:30:56] Oh, is it cash just that way? Uh, the content loads in, but the spinner is still going on. That's because there's an artificial delay for the react bundle to show, to show up. That's the point of this demo, which is to show that it can do async. Now you can imagine that it's not just one part of the page. [00:31:13] There could be multiple suspense boundaries here, some with something heavy, something with something asynchronous and they're potentially streaming in effectively in parallel in the, like after the HTML tag closes and they load nicely the, the other cool feature, which is a feature, every framework should steal is if you do a second refresh and here, I think if the, if you do a second refresh and at this point, the react bundle, the JavaScript bundle is cached. [00:31:42] So it loads before the react, the server. Finish the streaming. So at that point, the react says, fuck you, I don't care about the streaming bit anymore. I'm taking over, it's now a client set up like just automatically out of the box, because now that would be faster. So it basically raises the client and suicide. [00:31:58] So suspends working out of the box itself is like a big deal first. So people will start using it like with react dot lazy, but then with data fetching and a bunch of slate styling solutions, which they're also working on. Um, but this is the new server entering API. The reason I was talking about this, I keep losing context about these things. [00:32:19] I should stop sharing, I guess. Um, the absolute best feature of this of course is the reason why is something that comes out of Facebook, which is it works with existing applications and you can incrementally add it. So the first thing you will do is you'll take your render to string that one line somewhere in your code base, which says rendered to. [00:32:39] And you'll replace it with vendor to notable readable string. I mean,  [00:32:43] swyx: either way 99% of users have never used render to string. Right. That's what next year is for.  [00:32:51] Sunil Pai: Well, that's the, that's all my God. That's part of a whole other conversation, right?  [00:32:54] swyx: This is rendered a string as a service. [00:32:59] Sunil Pai: The moment you update next, year's your version of next year? So work on yes.  [00:33:04] swyx: Which is good, which is good. Right? Because, uh, people won't even know and they will just benefit, but it's, it's a little bit bad. Okay. [00:33:13] Next.js and the Open Source Commons  [00:33:13] swyx: And this is a little bit of my criticism, which is that your blessing, a meta framework, at the expense of all the others, right? Like which admittedly have not been as successful, but, uh, basically reacts Chrome, picked a winner and it was next year.  [00:33:27] Sunil Pai: I've been thinking about this so much. Oh, look, it let's get into them at our conversation now. So let's standard disclaimers. I think Guillermo is a mench. [00:33:35] I think the people who work there are incredible. There are some people I'm close to. I'm so happy for them. I know people on the Chrome team who work with these folks. I love them as well. Nicole for me is, uh, is a hero. Uh, and of course the React team at all my buddies, I love them. Okay. That being said, the React team is six people and they don't have the time to build the meta framework and Guillermo, uh the one thing he's incredible at is he's great at building relationships. [00:34:03] He's just amazing at that. Like he, uh, in a very genuine way, like this, there's nothing like ulterior about it. Next JS is open-source and runs on any node runtime and it's designed to do so. There's nothing about it. That's become special on Vercel. Because of that the React team felt, feel like, okay, fine. [00:34:20] We can have a primitive and meta frameworks will solve it. And let's just make sure it works with next two years, because so many other people who are just reach out to them and say, Hey, this new API is showing up. Uh, this is not just with next.js. It's a similar thing is with like react testing library. [00:34:34] When the new activity I showed up, right. I made the PRS to react testing library. I was like, what you should do is have every function and react testing library be wrapped in back act. So nobody really has to like use the API by hand. I just, it's now it's the D and it's a very good testing framework, the Chrome team. [00:34:53] And this is my, I'm not saying this, like, it's a bad thing. I think they did the right thing. The Chrome team realized that if they provide performance enhancements to next years directly, they can have so much impact on the internet because so much of the react tool is running on next year. So fixing how the images are loaded in next year certainly makes the internet faster. [00:35:15] Yeah. And maybe that's what we should do also like for the accessibility, just ship acts in, uh, all the acts rules in development mode, either in like react Dom directly, or at least the next years. Oh yeah. The sweatshop, the axles. Yeah.  [00:35:33] swyx: Oh, they're enabled by default. And, uh, your, your app one compile, uh, actually I think it would warn you won't fail by a worn. [00:35:40] Sunil Pai: Okay. So you should be making the swag folks should be making way more noise about that. That is such an incredible draw for accessibility.  [00:35:48] swyx: The thing is like, uh, if you encourage, if you think that your, your problems are solved by X, then you're taking a very sort of paint by numbers approach to accessibility. [00:35:57] Right. Which is actually kind of against the spirits of, of, uh, what people really want, which is, um, real audits with like tap through everything. Like the stuff that machines could catch is so little,  [00:36:08] Sunil Pai: I agree. The whole point of actual SIS to make sure that all the low hanging fruit is done by default. [00:36:15] It's like TypeScript, like I guess, which is a TypeScript. Doesn't solve all your bugs, but the stupid undefined is not a function once it does. Yeah, exactly. Make sure that your images have. Just by default, like we can have stronger conversations about tab order once you make sure all your images have all tags. [00:36:35] swyx: Uh, okay. Anyway, so, so yeah. So first of all, yeah, I agree with you on the, on this Chrome. And, uh, I think this is opensource winning, right? Like, uh, there's a, there's a commons. Vercel built the most successful react framework, Nate. They went the investor really hard at it. They had the right abstraction level, you know, not too much, not too little, just the right one. [00:36:55] Uh, and now everyone is finding them as like the Schelling point, which is a word I'm coming to use a lot, uh, because you know, that is the most impact that you reach. Uh, so no hate on any of them. It's just like it happens that a venture backed startup benefits from all of this.  [00:37:11] Sunil Pai: Can you imagine how hard it makes my job? [00:37:13] We don't run, not on CloudFlare workers, which means Next.js doesn't run on it. It's annoying.  [00:37:19] swyx: Oh, is there any attempt to make it run?  [00:37:22] Sunil Pai: There are a couple of ways where we can get it to work, but it like, it's a lot of polyfill and, uh, we'll get that. Like, I expect it to be fixed within the next three to six months, but out of the box, it doesn't run on it. [00:37:35] And for me in my head, it doesn't, it's not even about CloudFlare workers. I'm like, oh shit. That's what makes Bezos like even richer because everyone's got, has, if you want to use Nadia using AWS or Lambda. And that just means more folks are using AWS. I'm just like, okay, I guess. Sure. I know you work there as well, but it's just very annoying to me where I'm like, shit. [00:37:56] What's even more interesting is that node is now moving to implementing web standard APIs inside of it. So they already have the streams implementations. They will have fetch fetch will be a node API. Like it will be implemented based on standards, which means the request response objects. And once that happens and people, if people build frameworks on that, then you can say that it will run on CloudFlare workers because the cloud fed worker's API is also like a standards based thing. [00:38:21] So it's an interesting shift of like what's happening in the, in the runtime world. Also conveniently the person who implemented the web stream implementation at node just started at CloudFlare like last month, like James. Oh, James  [00:38:38] swyx: now. Okay. Yeah. I recognize  [00:38:39] Sunil Pai: a great guy by the way. Uh, very, I just love these people who have like clarity of thought when they talk James as well. [00:38:46] The Third Age of JavaScript  [00:38:46] swyx: We're kind of moving into the other topic of like JavaScript in 2021. Right. So first of all, I have a meta question of how do you keep informed of all this stuff? Like I ha I had no idea before you told me about this Node stuff. How do you know? [00:38:57] Sunil Pai: I have an internet information junkie problem. [00:39:00] I replaced the weed smoking habit with a Twitter habit. This is what it is.  [00:39:05] swyx: You're not unlike some magic mailing lists that like tells you all this stuff. Okay.  [00:39:09] Sunil Pai: Like reading the tea leaves is what it is. Like. I keep trying to find out what's going on. The problem  [00:39:14] swyx: is I, I, I feel like I'm ready. I'm relatively plugged in, but you're like, you're way more plugged in than me. [00:39:22] and then this development with node adopting web standard APIs, um, is this a response to Deno?  [00:39:28] Sunil Pai: I don't know if it's a response to Deno because I know Mikeal Rogers wrote about this. Like your. That we made a mistake by trying to polyfill note APS and browser code with like modules and stuff. [00:39:41] Right? Like that's what the whole browser, if I, during those days, when we started actually using the same module system and the word isomorphic came up, what ended up happening was naughty APIs were polyfill in web land, but what should have happened is we should have gone the other way. And it would have kept like bundle this bundle size problem would have been a web smaller pro problem right now, just because of that. [00:40:07] So I know that the folks at not have been thinking about it for awhile, maybe Deno finally pushed them to do it, but I don't, I don't, I don't, I don't think it's like that reductive. I don't think it's just, it's just dental. It's very much a, this is the right time to do it and we actually can do it now. [00:40:22] So let's like flesh it out and do it the right way. Uh, and it's hard to do it in, in no, right. It's not just that you can just implement this thing. Like, what does making an HTTP server mean now? Because the request response objects are slightly different in shape. So you have to make sure that you don't break existing code. [00:40:39] So it's not as simple as saying, oh, we're just implementing the APS. That being said, having fetch inside node proper is going to be great. I think. Excellent.  [00:40:47] swyx: Yeah. Yeah, no more node fetch. Um, yeah. You know, my other thoughts on I've been, I've been doing this talk called the third age of JavaScript. Right. [00:40:55] Which is a blog post that I wrote last year that, um, honestly I feel quite a bit of imposter syndrome around it because all I did was name a thing and like it was already happening. It was, you already saw, like, I think basically when, when COVID hit, a lot of people were. I have a lot of time on my hands, I'm going to make new projects or something. [00:41:14] Um, uh, and then, yeah, so I just, I named it and I just called it a few trends. So the, the trends I I'm talking about are the rise of IES modules first, you know, in, in development and in production, uh, concurrently the death of 11, which I'm also tracking.  [00:41:30] Sunil Pai: Yes, those are, those are both come to fruition.  [00:41:34] swyx: Which, by the way, I think the us government will have to drop by 11, uh, sometime in the next six months or so because, uh, the, the use, the usage levels have plummeted. [00:41:43] 3.6% of all visits to the U S government website in November, 2020 was I 11 and now that has dropped to 1.6, um, all  [00:41:51] Sunil Pai: accelerating the drop is actually accelerating.  [00:41:53] swyx: Uh, I don't know if it's accelerated it's everything, but it's under the 2% mark that the us government sets for itself. [00:41:59] They have an opportunity to essentially say like once it's stable, you know, there's no chance that it'll ever go back up again. Uh, they could just deprecate 11 for all government websites and then that, that will be the signal for all enterprises. And that's it. Yeah. So, um, and then the second. [00:42:15] Oh, I was going to move on the second bit. But what was your calling?  [00:42:19] Sunil Pai: Oh, just saying that this happened, like, while I was working in JP Morgan over the last year, they did the same shift, but they're like, yeah, we are now a Chrome company. Literally none of our clients are asking for this and you know, it, it was just in rules somewhere, or we need to target, I 11, some people looked at it and said, okay, fine. [00:42:35] What happened is people are spending money on something that wasn't giving them the returns. And that's when a bank is like, yeah, we don't need  [00:42:41] swyx: to do this anymore. Like you, you can deprecate free support. Right. And, and just make, just charge for 11 support, stop spreading it out among all the other users who are bearing the cost of development and maintenance. [00:42:54] The other one was collapsing layers, which is the death of Unix philosophy. Like , we used to have one tool does one thing, but now we want to combine everything. So, uh, Deno and Rome both have ambitions of linter format or test runner, all of that into a single binary, because the idea of what we want out of a default runtime has changed, uh, from a, for a very minimalist thing. And I always made the comparison to what word processors used to be like. [00:43:18] So, are you aware of Benedict Evans? He has a blog post, which is amazing about what a job of a platform should be. And he talks about like in 1980s word processors used to only let you type words. And if you wanted a horizontal layout, if you wanted word counts, if you wanted footnotes, these are all plugins that you buy and install separately. [00:43:38] Right. Okay. So, but as we evolve, as we just use all these things, we realize that these are just like the same tool that we want out of a word processor. So then they absorb all these features instead of plugins. They're just part of the platform now. They're there now in the new table stakes. [00:43:53] So I make that analogy to the runtimes that already doing, right. Like, Node used to be this like much more minimal thing. And, uh, but now we are expecting more and more out of our default setup with all these tools . Um, it's also very wasteful because when each of these tools don't know each other, they're all parsing their own ASTs running, running their own code. And then yeah, that's the whole  [00:44:12] Sunil Pai: proposition, but yeah.  [00:44:14] swyx: Any, any tool that collapses layers will, we will meet this, like, ESBuild, um, collapsed. Like a standard web pack would do like five or six AST runs. ESBuild collapsed it to two to three. That's a source of its speed as well.  [00:44:27] Sunil Pai: One of my favorite facts about ESBuild is that it is faster to minify the code than to not modify the code when you run. Yes. And the reason for that is because when you try, when it tries to do the full AST, keep comment notes, everything else, it has to do a lot more bookkeeping, but the moment it just ditches all those things, because ESBuild doesn't do like full magnification, like something like a torso, but it does do like a smaller symbol substitution, white space, uh, uh, removes all white spaces. [00:44:59] And it does like some dead code elimination. Uh, and it's a lot more work to keep the bookkeeping for everything and all the white space notes than to not do it. So he has built is actually faster when you have a modification turned on, love it.  [00:45:14] swyx: It's amazing. It's amazing. [00:45:16]   [00:45:16] ESbuild vs SWC vs Zig  [00:45:16] swyx: Do you have opinions on ESBuild versus SWC? [00:45:18] Sunil Pai: Okay. So I like ESBuild. Uh, because I was very strongly looking for something a lot more opinionated. I've noticed that the reason that code basis Surat usually boiled down to the acute decisions that you make. Like in the very beginning of the project, you can do anything. I mean, whichever dumbass came up with the idea of baby plugin, macros has like ruined a lot of lives. [00:45:41] It was me. I came up today, but that is like, then you're like tight. So the thing that ESBuild does is very like its creator, Evan Wallace, which is that it's, it's one of a kind like, he's not really interested so much in having community, uh, uh, PRS or like having suggestions on how it should be built. He has a very strong vision of what it should be like, which is why there are no AST level plugins and all that jazz. [00:46:08] And because of that, because of, like I said, because he's collapsed the lyrics and collapse, the size of the development team to just himself, he has like such a clear vision on what it should be. So it w is good. It would be great for, I want to say 95% of projects that fall under the things he has designed at four. [00:46:28] Okay. Uh, and that's a lot of applications. That's a shit ton of applications. That's like everything, but your host, if you need anything, uh, unique, I'll give you one. That's like a very good use case that is bill will never use. Do you know what, um, uh, really has this idea of persistent queries. Okay. So like for whoever's hearing who doesn't know it, right? [00:46:52] Like you can write a query inside Java. And when it compiles it out, it takes out the query and replaces it just with an identifier, like, like a little eight character identifier. And it hosts that query instead of like on the service side. And it says, oh, that eight, eight character query, you can just hit it as a restaurant point now. [00:47:11] So you can write the code internally in JavaScript where it belongs, but it doesn't add like to your bundle or whatever it is. So ESBuild will never support this, which means if you want to do really optimizations on your react code base, you won't be able to do it all. You have to like add on to yours, which you could do. [00:47:29] I guess like you can still use Babel would, uh, SWC is meant to be a platform and which is why next years will use it because next gen is the meta framework, not just for react, but also for like some programming opinions, extracting get server props, get started, props, which one you want to be that this thing after server components comes into play, but a number of things like there will be people who always want to do. [00:47:54] The emotion macro now is like fairly, uh, popular that they will want to use it. So I assume they will implement it in, uh, interest. I know. Do you know what bun is by the way? Do you mean, do you know, how are you following Jared Sumner? Some  [00:48:10] swyx: summers, no, wait, so  [00:48:13] Sunil Pai: key is reimplementing ESB, but in a language called Zig it's another systems programming language. [00:48:20] And he's his claim is that it's about three times faster than you spell it right now, which is already some 200 times faster than Babel loader. It is just our web pack, but it's a language you said it? No. So the language is called Zig lines at AIG, but the thing he's building is called a, B U N. He hasn't shared it in public yet. [00:48:41] I think he's actually planning on sharing it like next week. Like I think it's that imminent. He's been sharing numbers right now. Yeah. That's the guy, Jared. Uh, I love, I should've followed him like a while ago, create great feed, uh, excellent content. And like, he's, he he's thinking that he's going to like implement. [00:48:57] He might actually implement an AST level, uh, uh, plugin, micro API, possibly just implement the emotion one. I think he was just, yeah. See, oh, that's like literally the tweet would write under the main one right there where he's like, Hey, what if we actually just did this in? Uh, oh,  [00:49:14] swyx: he's right. He's he's right with you. [00:49:17] Yeah. Like he's  [00:49:17] Sunil Pai: just talking about it, like right there. So, uh, so SWC versus ESBuild, I don't think is the conversation. I think ESBuild will have a rise. A bunch of people will use it. The nice thing, the best feature about ESBuild is because there are aren't any like cute decisions. You will be able to move away from it to whatever succeeds. [00:49:39] Th there's nothing customer  [00:49:40] swyx: that I believe that was Evan's original idea. That IES build was a proof of existence that day there's a better way. And that he stuck to it for way longer than I thought he would.  [00:49:51] Sunil Pai: People are using it in production and everything know everything about the designers that it's replaceable. [00:49:56] That it's just a,  [00:49:59] swyx: that's wonderful. Isn't that amazing when people design their stuff? W. You know, it  [00:50:04] Sunil Pai: isn't kind of pressure that he would have had the best. Thank goodness. It was the successful CTO of Figma with money in the bank who is implementing this and didn't have anyone to impress. You know what I mean? [00:50:16] It was like, yeah, let's put a macro API and what else do you want? Like, whatever. No, he doesn't  [00:50:21] swyx: go. Yeah. But he just needs to police himself and no one else. Right. If you don't like it,  [00:50:26] Sunil Pai: this is during his downtime from Figma that he's working on this.  [00:50:30] swyx: Um, my, my secret theory is that he's doing this as an, as a Figma ad. [00:50:33] Like, you know, if he, if the CTO of Figma does this for fun, imagine what it's like to work inside of Figma, you know, like of, I've heard it's pretty great,  [00:50:42] Sunil Pai: pretty great working inside of Figma too. Well, the code is like, it's really cool.  [00:50:46] Let Non-X Do X: Figma vs Canva, Webflow vs Wix/Squarespace  [00:50:46] Sunil Pai: Did you actually point out. Uh, Ken was like six times bigger than Figma. [00:50:51] Now   [00:50:52] swyx: you wanna talk about that?  [00:50:53] Sunil Pai: Oh God. That's. I didn't realize until you pointed it out.  [00:50:58] swyx: Incredible. Imagine all the geniuses working in Figma and go looking at Canada and like, yo, like I, I have like a thousand times your features and your six times in my size as a business.  [00:51:10] Sunil Pai: Uh, but I hope every one of those engineers understands the value of sales and like reaching out to your actual customers because  [00:51:17] swyx: I don't think it's just sales. [00:51:18] It's more like, uh, they're always going to be more non, like, this is a category of software called let Nanex do X, right? Let non-designers do design. Whereas Figma is clearly for designers doing design. Um, and there's always going to be like a tool, three orders of magnitude more non-experts uh, who just want to do basic shit. [00:51:37] Sunil Pai: Oh man. I hope that flow has a multi-billion dollar buyout and at some point,  [00:51:42] swyx: uh, I mean, I, yeah, I mean there's clearly something that w the problem with flow is that. They're too close to code. Right? You have to learn CSS the box model.  [00:51:56] Sunil Pai: Yeah. I mean, they do say there's no code, but really they're a visual,  [00:52:00] swyx: if you don't know CSS when using Webflow you're screwed. [00:52:03] Like   [00:52:04] Sunil Pai: that's right. It's uh, they have, they have the best grid editor on the market too. I have to say that. I  [00:52:10] swyx: mean, the UI is just amazing, right? It's just like, um, yeah, I mean, you know, there's a reason why like the Wix is, and the Squarespaces are actually worth more than the workflow and it's not just cause they were around earlier. [00:52:22] Like, um, they're, they're just easier to use for non-technical people.  [00:52:26] Sunil Pai: That's a good, you you're talking about why did we even start talking about this? What did you want to talk about? Uh, we were talking  [00:52:33] swyx: about like, uh, 32 JavaScript. Um, so I think we kind of like dealt with those, those, uh, those topics.  [00:52:39] Was there anything else that you want to talk about? [00:52:40] Didn't JavaScript land,  [00:52:42] JavaScript Twitter and Notion's 9mb Marketing Site  [00:52:42] Sunil Pai: I don't know if you have noticed, but I've kind of actually stopped engaging in the JavaScript discourse on Twitter specifically, which actually hurts me like a little bit, because that's where all my jobs could friends are. And that's kind of like, I've seen it all. I've seen JavaScript router now for the last 11 years, I would think 10, 11 years that I've seen it. [00:53:02] And I used to like participate very heavily. And back to the thing that you, uh, that we were just discussing about the conversations that happened too, about like SBA versus MPA and about like the whole notion blow up about how they made them thing into like 800 KB. Yep. Uh, the easiest kind of discourse to have is to have like one absolutist opinion, uh, that I saw a number of people in like those threads and the surrounding threads have, which is a, well, this is bad or this is good. [00:53:35] And, uh, that's, that's all I got to say about it. Now give me like 40 likes on this reply industry. Uh, whereas like there's real opportunity here to understand how and yeah, that's the one, that's the one with treat by the way. Clearly it got like attention. No,  [00:53:51] swyx: by the way I phrased it very neutrally. I actually was pretty careful. [00:53:54] Cause I knew that it's going to attract some buzz. I had no idea what's going to be this much, but  [00:54:00] Sunil Pai: no, no, no. But like I'm so interested in talking about, uh, so this is what I was talking to you about, which is like, it's not just about a website at one point of time. It's about the system that generates these kinds of like artifacts, uh, of, so for example, with what, what did they say? [00:54:24] They're there 8 47 KB right now. They're not 8 47 KV today. They were 8 47 KB. When you, uh, Uh, tweeted this, uh, on the 11th, they are not in 47 KB. Now they might be 852, or they might be 841. Are you about to check?  [00:54:43] swyx: No, no, no, no, I'm not. I'm not, it doesn't matter. The exact number. Doesn't matter. I'm going to give you another example, which also came up, which is Netflix. [00:54:49] Remember they ripped out react and he said they have react back  [00:54:54] Sunil Pai: on Netflix. I use, are you serious on that? Wait, did they have like both Netflix, they have both react and jQuery, jQuery and react on that page right now. It's just, but like, for me, it's interesting that, which is like, I think the most insightful tweet in this was very pointed out that nobody noticed this until they told it to us. [00:55:16] Nobody saw it. It bothered. Yeah. That's the one, like nobody bothered about it. It was still making the money. They were happy about it. And they wanted to share that. And we need more of them. We need more people to be like sharing the process because if we react very badly to these things, then fewer people will want to actually share the numbers. [00:55:34] And you won't learn from the industry, but I don't know whether it's a good thing or a bad thing. It does mean that you can make a multi-billion dollar company with a marketing site. That's nine MB of Charles' script. And I think, I think people who have very strong opinions about how much jealous should be on a page to take a step back and wonder how do you make it? [00:55:55] So like, how do you, from the very beginning of like running your company, how do you make it so that it doesn't go up beyond that? Also, what opportunities are you abandoning by focusing on making sure your marketing page, uh, has like 100 KB of JavaScript instead of like nine MB  [00:56:17] swyx: shipping velocity, right? [00:56:19] Sunil Pai: You are somewhere, you are spending effort on it somewhere. Just so we're clear because somebody will look at it and say, fuck you, are you suggesting that we all put in that's not what I'm saying. I'm just saying that the resources, that word, but resources at these companies are limited and they are, they they're prioritized and sequenced and you should ask yourself in what order you want to do it and who you're trying to please, are you trying to please your customers and your users or the peanut gallery on Twitter? [00:56:48] And I think that's something that like, I, it's why I don't engage so much anymore because it's so hard to communicate in once and somebody will come in with a, well, fuck you, you work for Facebook or used to work for Facebook. What would you know? I'm like, you got me that kind of ends the conversation that, right. [00:57:04] Like I'm studying contributed to babies being burned alive or whatever it is like, this is what it is.  [00:57:12] swyx: Um, it's a nuanced debate, like, uh, because they also did some like notion clearly did some stupid stuff here. Right? Like it, it, they could have spent a day. Uh, so do you know why it was 9.9 megabytes?  [00:57:25] Sunil Pai: If I understand it was the whole notion that that was being used, the  [00:57:27] swyx: whole app. [00:57:28] Yeah. They were shipping the whole, there was actually someone from notion, uh, answering me. Uh, it's here. Yeah. This guy's, this guy works at notion before the marketing site was another route in our, at the time 9.1 NBME and app, we load the whole app just to show the sign up button. [00:57:44] So what,  [00:57:45] Sunil Pai: what it's worth Facebook sign up page does start prefetching actual Facebook code so that once you log in it loads instantaneously. So there's a reason to do it. It's just that it shouldn't be nine and B of course. That's  [00:58:00] swyx: yeah, they could have like took a day every, every six months or something like perfect day, you know, and do that. [00:58:06] So that's why I'm hesitant, uh, giving them a pass for like, okay, so what your multi-billion dollar company? This is embarrassing. This is just an unprofessional. Um, so yes, but at the same time, the people who are upset about other people's sites on Twitter, uh, and then they move on to the next source of outrage the next day. [00:58:26] Like they also very uneffective. So.  [00:58:31] Sunil Pai: I it's hard to have a conversation with them, and God bless you. I noticed that you try so hard to stay, uh, not, not emotionally detached, I guess, but that you're, you're just staying above the water and that you're not actually picking a fight or like fueling it on. But there are people who are getting in there to like, get their shots and to say, ha I've said the tweet against a multi-billion dollar company I'm better than them. [00:58:53] Ha   [00:58:54] swyx: no, that gets a little sad. You know, like the dev Twitters is a small microcosm of the overall universe. And in fact it's often the least powerful people and the most powerful people have no don't care. They don't care what you think. They, they just, they're looking at the OTRs and if they, if they meet it and they get promoted, they have more money. [00:59:14] And you're, you're still complaining about your performance issues.  [00:59:19] Sunil Pai: It sucks that it's this way, because I don't know if I'm romanticizing the. Actually I probably am. Yeah, that's just, it, it feels like, oh, when, when I was younger, things were like way better than the music was better. You know what I mean? [00:59:32] Like clearly all the best music was made between the ages of like 13 to 16 when I was exactly the best music in the world happened. So maybe that's like my memory of Twitter, but to be honest, in the last 10 years, 10 years ago, a lot of things about JavaScript kind of sucked. It definitely wasn't as diverse. [00:59:50] Thankfully it's a lot more diverse right now. We have people from way more walks of life. And, uh, uh, this obsession with computer science degrees appears to be dying down, which is just great. Absolutely great news because they, whenever they talk about computer science degrees, nobody actually is like, okay, what did you learn in your computer science degree? [01:00:09] That's like useful here. There are some things that are, if you're building the shed EULAR and shady lyric and fiber for react. Yes. But really the people that I've seen do really well as web developers are the folks who read a lot of comic books and learn to design because they know how to lay out stuff and how to tell a story. [01:00:28] And they like know how to like translate that onto the, on the side. So I've seen like, and designers who spend like a little time to learn how to do like just enough stuff to be dangerous. They're like incredible. Uh, Adam Morris is now doing a generative, whatever components.ai is, it's this?  [01:00:45] swyx: What is that  [01:00:48] Sunil Pai: strong pitch? [01:00:49] And it's, he's taking his time to execute on it. And I love it, which is you should be able to search through a problem space to find what your UI should look like. It shouldn't arise from your brain. You should be able to turn knobs and adjust things on your system. And turning knobs is not just about a, Hey, let me look at this one button and change the color of it. [01:01:12] You shouldn't be able to do. As a whole. So right now they're in this phase where they're exploring a generative technologies, like how do you generate the entire, uh, instead of just doing math dot random, how do you generate 40 pallets and then like just present them to the user one by one based on fact. [01:01:32] So, uh, he's doing a bunch of that right now. And he's working with some very smart people, I think. Uh, uh  [01:01:41] swyx: huh. I think big this guy called murmurs. I think that's out of most. Yes.  [01:01:44] Sunil Pai: Norma says, oh, but also not as working, uh, not Alison is tested. Alice, Alice, I think is working with them. Uh, uh, Dean black is working with like a number of very interesting people and I'm very, they're taking their time with it, which is very strange. [01:02:02] It's not as startup-y as I would have, like. But, oh yeah, this is the syntax a generation. I think we can literally click a button and it'll do a  [01:02:14] swyx: great, great grid. Okay. Yeah.  [01:02:20] Sunil Pai: These are all generated in your browser. I think, I don't think this is like from a database somewhere. Yeah. Why would it need to be? [01:02:27] Um, so they're working through this. They're trying to get the fundamentals for a science that they're effectively like inventing right now. I mean, it exists for art and stuff and they're just, well, not even just art, uh, but they're trying to figure out how to productionize it for UI. And I think it's very exciting and it's very distinct and unique and I'm very, I'm keeping my eye on this all the time. [01:02:49] I love it. I am.  [01:02:51] swyx: Yeah. Well, that's fascinating, man. Um, okay.  [01:02:59] Sunil Pai: So like I did a bunch of talking in your directions.  [01:03:02] swyx: No, no. Uh, you know, we talked about Twitter, like, yeah. I don't know. I feel, I feel hot and cold with my Twitter stuff. Uh, I started out tweeting, you know, a lot of react stuff. That's how I, I guess I initially got traction and then I did the career stuff and now I guess I'm. [01:03:18] I don't have a particular theme. I am feeling a bit directionless as well, but I also think there's like a natural sweet spot for Twitter, where basically this is why I tell people you want, so you want enough followers so that you're not a nobody, you know, you have instant social proof, but you don't want too many that you get all the idiots coming in your, in your mentions. [01:03:38] Right? So,  [01:03:40] Sunil Pai: uh, well, I don't think I have too many ideas. Everybody who follows me is very beautiful and smart and intelligent in the future. Uh, but I definitely, that's why I have an alt now and I do my whining and Randy's stuff there because it's very hard. I've not, I can only count on my friends to assume the nuance that I'm trying to communicate like good intent. [01:04:05] Just generally good intent. Isn't so much of a dumbass. He doesn't hate. Uh, wait, what, uh, uh, he doesn't actually want nine MB of JavaScript on the page. Surely.  [01:04:19] swyx: Yes. It's, it's so helpful. Um, yeah, I mean, that's, that's, that's about what I, what I do for Twitter. I mean, I guess I might move into like founders slash VC Twitter eventually. [01:04:31] Um, I am not, I'm not currently on the founder path, but I, I, I always keep my eye on it. I don't know what your thoughts are on like the whole, like, okay. Maybe, maybe, maybe this is a worthwhile topic. Like there's a lot of jumpsuit ecosystem. People who are, you know, our friends and now are startup founders. [01:04:46] Right. Like, which is  [01:04:48] Sunil Pai: interesting. Don't take it as advice. It isn't really advice, but let me tell you what I fear. Oh yeah. Okay. Yeah. Okay. Let me put it this way. The people that I like in JavaScript, Twitter, that I still follow for years, even the ones who talk a lot less or it's that fucking Picasso quote that I love. [01:05:09] When art critics get together, they discuss, uh, uh, form and function and philosophy of art. But when artists get together, they discuss where to buy cheap. Turpentine always be a turpentine person. I think like, oh, like never lose sight of like reality. There will always be things to discuss. There are like metal things, which is what we're talking about right now. [01:05:28] I think, but the true, uh, it feels, it has always felt this way to me, which is that true progress in our industry and other industries are the people who are putting actual skin in the game and actually working on it. This is, this is how I think about crypto Twitter. Also, by the way, I know a bunch of engineers who are working on crypto stuff, but refuse to talk about it on Twitter, because it's just so toxic there. [01:05:52] And they are working on actually interesting use cases with a technology and immutable database that's spread across the world. And you're telling me, you can't think of like videos cases. There are people who have thought about it. That's that's not just an NFT. Let's get drugs delivered to our house, which by the way, you get good use cases, I guess. [01:06:10] I don't know. So I think the same thing of like founder Twitter, there are good founders who actually like the, for example, one founder that I've been following a lot lately because I just admire the story so much is Toby Luca from Shopify. Uh, he's a hacker he's he did the video that expo that demonstrated the hydrogen oxygen reacts. [01:06:33] React Server Components and Shopify Hydrogen/Oxygen  [01:06:33] Sunil Pai: What do you  [01:06:33] swyx: think? Or what do you think about that? I think  [01:06:36] Sunil Pai: what, what do I think about server components? Game-changer everybody should steal it.  [01:06:40] swyx: Hydrogen oxygen is a server components, uh, implementation. It is, uh, it's  [01:06:44] Sunil Pai: there, like hydrogen is basically react server components, uh, and oxygen is the run time slash network. [01:06:52] There   [01:06:52] swyx: it's there for cell phone, right?  [01:06:55] Sunil Pai: Uh, it's actually closer to workers, which suck because are already our customers at CloudFare. I'm like, why don't they use so their dealers that they want it running right next to their data. And we don't turn people's databases on the edge yet. Maybe we will. I dunno. [01:07:10] swyx: I could change. Well, isn't that a durable objects is kind of databases on the edge.  [01:07:16] Sunil Pai: Well, it's a data store on the edge, but it's not running sql.js on it yet. Let's just never say these things. I suck at keeping secrets. Uh, the truth is that like, clearly people are talking about things internally or what happens when we can actually literally have SQL database that uses durable objects as a backing. [01:07:34] swyx: Like there was one slide out. I already had some version of this. Right. So  [01:07:38] Sunil Pai: right. Shopify right now is just exploding this because it's not just about like running a database, it's also, you don't want to expose your database to the internet and how serverless workers like hitting it, even though that's planet skills data's plan, I think, which is they're developing my sequel, but for the serverless world. [01:07:57] So you should be able to have like thousands of connections open under children, bring it down. Uh, another great company to keep an eye on them and super bass, super base. Just how, like good, good people in general, I spoke to a bunch of them. They donated during the India fundraiser that I was doing, and I spoke to a couple of them. [01:08:14] Great, great team and nicely way. So it's basically react server components that have access to Shopify, API APIs. So, uh, the end, it's very strange for a company like them to go full in on, uh, basically all for technology. It's not even better yet. Like the implementation is not finished. It's not coming out with React 18. [01:08:39] It'll be, it'll be an 18 dock techs release, but it's not coming out in November. Uh, and to see Shopify suddenly like it. It's the right choice. It feels like the technology is right. It's not something novel and people will be using it more and more. I think solid already has some kind of version of a server components because Ryan just like steals ideas wholesale. [01:09:01] And I love that, uh, attitude. Uh, but, uh, what do I think about it? Uh, great call. What a, what a, what a way to like, make sure that you're a first adopter of something that I think is a sure thing remains to be seen.  [01:09:18] Categorical Imperatives of Web Platforms: Cloudflare vs AWS, MongoDB vs Auth0, Gatsby vs Netlify  [01:09:18] swyx: So there's a thesis that I'm drafting, which I'm not really sure about yet, but, uh, this might be useful for you because I, I spoke about this with, uh, Rita and with James Allworth right up there. [01:09:34] Um, And so the original thesis was called fare specific, which is how is CloudFlare taking on AWS? It's CloudFare is, was the originally just the CDM they're free, CDN, whatever. Um, and, uh, obviously the, the best buy by any measure. Um, but it's eating AWS from the outside in. You first do the CDN, then you do the workers, then you do the durable objects. [01:10:00] You, you, you kind of like reinvent every primitive from the first principles concept of like, everything is distributed. It's just a pretty amazing to watch, right? Like that's, that's kind of what, uh, what coffers, like, how do you take on a giant behemoth? Like you do the one thing that they cannot do, which is you start from like all this, all this, like distributed Stouffer's, um, And then I start thinking about things like Shopify doing, uh, hydrogen and oxygen, uh, start thinking about, uh, here's your other, your other interesting movements that I'm tracking, right? [01:10:31] Um, Shopify, sorry, got to be making serverless functions and guests be making his own CDN. Um, so this is, this is how I blog by the way I have, I have all  [01:10:40] Sunil Pai: these insight and do the,  [01:10:43] swyx: okay. He's the only blog post ideas. That's, that's pretty good. Um, actually I want to kind of make my book, my podcast into something like, okay, basically mouth blog, like you have a, you have an idea that you're shaping and you want to blog it, but you don't have time to blog. [01:10:56] Let's just talk it up, you know, that's kind of it that's of that idea. Yeah. So, so Netlify and guests be doing deferred generation. So these are all examples of like, uh, like a trilemma of like there's rendering. Uh, which is one group of constituencies, there's the cloud and for people, there's another group. [01:11:13] So this is like Netlify herself up there, uh, rendered fairway. It's like, uh, next. Yes, but also gets be and react. And those guys, and in the CMS, which is like all the, like a different crew that like has, is completely above the free of all this fighting and fighting, but they always start competing with each other. [01:11:28] Right. Um, like you're going to my turf, my turf going into your turf, that kind of thing. And, and here's, here's another one, uh, which I'm also, this is called fair inside it. Um, so I recently found out that Mongo DB that's off. Do they, do they do off [01:11:48] Sunil Pai: by the way, that's the feature that they sell. Like, it's part of the.  [01:11:54] swyx: Oh, yeah, yeah, yeah. They, they, uh, they wrapped a Netlify identity and they made much more progress with it. Then now the fibers, uh, basically it's because they took it more seriously. Um, and obviously role does serverless functions. What the hell? [01:12:09] Sunil Pai: Okay. All right.  [01:12:10] swyx: So here's a zero actions. These are all these are introduced. Uh, basically whenever you, you sign in, you want to execute some custom logic, they build an editor into their platform. So you can, you can write service code for office zero. [01:12:29] Sunil Pai: Okay. That's sort of that's the land director. Okay. Sure.  [01:12:32] swyx: So, so you know, the off guys are doing compute, the storage guys are doing off, like, uh, I think there's a, there's like essentially what I'm trying to basically form a, uh, not this one. Let me see, trying to form a triangle ignore. Um, yeah, so like kind of like a triangle. [01:12:54] Hear hear all the players and here here's where they all start out. And then, um, you know, I'll have my, I have my sort of beachhead here. right here. But in order to grow, I have to grow in one of these directions, you know? Um, and that's kinda what I'm, I'm trying to form a thesis around this. Like, so I have, this is for individual APIs and then this is for contents. [01:13:19] Maybe this is maybe what I'm saying, that this is for apps, you know, storage competes with, uh, compute computers off. And then this is for sites where it's a content rendering and infra, something like that. You know, that that's kind of the thesis that I.  [01:13:37] Sunil Pai: That's fascinating. This is a good one. Oh God. I'm going to have to sit and think about this and get back to you. [01:13:42] Aren't I, oh, this is really good. So there's a  [01:13:44] swyx: question of like, should they bother, right? Should Gatsby bother making serverless functions because they're basically a shitty version of Netlify functions. Right. Um, but at the same time, next year has, has functions and it's fine. Like, or API routes, right? [01:13:59] Whatever you call it. What determines a good expansion and one determines a waste of time?  [01:14:07] Sunil Pai: Well, I assume that a bunch of these things they're doing is because their customers are asking for it, like Gatsby, for example, they're like, Hey, we just want to be able to call it an API and not deploy two separate things. [01:14:19] So can you give us functions? And I assume it gets me, is looking for customers and getting more people to pay for that. I assume they are doing it because next year's and still are doing it and people can easily get. To versatile because we're still as functional. So they want to make sure that, so I assume that's one, one part of the story, but I keep with this whole DST thing, it feels like they're clearly moving away from their whole static asset as content story. [01:14:46] Like my  [01:14:47] swyx: God. Yeah. It's uh, I don't know if you saw the, the announcements. I mean the, where it gets before. Okay. So I almost tweeted out the screenshot. Um, okay. Here, you know, you know how, when, uh, after the build of next year site, it'll give you a report of like, what was, uh, statically generated? What was, uh, dynamic what's the serverless functions. [01:15:18] This looks familiar. Oh, [01:15:23] Sunil Pai: I see. I see. Interesting. Okay. I think it's a little obvious than what's going on here.  [01:15:33] swyx: The obvious thing is like, it kind of Gatsby just capitulating and just saying like, okay, we're a Next.js clone with a GraphQL layer. Right. With a much bigger plugin ecosystem and hopefully that will, that would make a lot of people happy. Um, my it's interesting, like Gatsby, we can quite conceivably just be a Next.js layer right now. Like they don't have to build all this. It already exists with Next.js. What they should focus on is actually the graphical data layer. [01:15:58] Right. And all their plugins.  [01:16:01] Sunil Pai: Um, so. The mistake here would be thinking that they think that this is the end game. Of course, like I assume this is like first release. Part of the next thing we want to do is, oh my God, God gets me on the blockchain. I bet. Like I want to see, which is the first JavaScript framework that uses the word blockchain on their page. [01:16:21] I haven't seen it yet. It's definitely going to become a thing. Oh  [01:16:25] swyx: yeah. Anyway, so, no, I don't, I don't mean to talk about Gatsby so much more just about, um, categorical imperative. Um, if you are a business in a, if you're a startup or a framework or a organization, you, uh, you have a goal, you, you achieve that goal. [01:16:41] Then you start to look into what's what's next? Everyone does this, right? Like I'm Mongo DB. I like dominated the no SQL databases. Okay. What's next off  [01:16:51] Sunil Pai: because they want to post them data applications on the day. Yeah. So why not have JavaScript? Well, yeah, that's a, that's a, that's a good thesis. I would, uh, I would it, so,  [01:17:03] swyx: but so far CloudFlare is the, the most concrete of those thesis thesis, which has started with the CDN and built in. [01:17:10] Um, and so far, I don't see anyone else doing that. So, I mean, that's kind of like the framing of what I think your team is doing. Uh, it's kind of building  [01:17:22] Sunil Pai: is doing differently this year. The big deal is they want to fix their developer experience. Like they have no brand recognition. They're not even a contender when it comes. [01:17:30] Like if you see the state of JavaScript survey, the word cloud flood or workers does not show up, like it's not there. Like Deno has a way bigger presence and workers has a zero presence. The word doesn't even isn't even there. Uh, and which is very strange for them because their tech is their tech is very. [01:17:49] But then I actually tried using, it was like, have you tried using this? You end up with like 40 different tabs open. What is the local development experience supposed to be? And the thing about developers is not that they will give you feedback on how to make it better, right? They'll be like, fuck this. [01:18:03] I'm going to go onto our salary, AWS and get to production tonight. I don't need, I don't need to kill. So, uh, the story is to fix the developer experience and to have a coherent story. One of the first part of that story is what we're calling CloudFlare pages, which is right now effectively, just JAMstack. [01:18:19] But as you can imagine, it's not, it is going to become a bigger thing because it's like, oh no, you can run workers with it and have API end points. Oh, now that's durable objects. Now there's this. So I would keep an eye on that. And of course, Glen Martin is on it. Who's also my close friend, but an incredibly smart guy. [01:18:34] Wrap-up  [01:18:34] Sunil Pai: Uh,   [01:18:36] swyx: I had a chance to hang out with him. Uh, Um, and yeah, I love the guy I love, I love basically all Australia is, and JavaScripts are always a blessing. What  [01:18:46] Sunil Pai: the fuck is that? Why, how is that a thing? Why are suddenly like every person who knows JavaScript from Australia? Like all also the accent, I think makes it sound sexier. [01:18:56] I think so, but no, what is this whole, it's not just a us it's Australia and it's UDA for some reason. Incredible JavaScript developers, like from both those places, like salt lake city,  [01:19:10] swyx: Utah, so yeah. Oh, um, you know, the joke, right? That's a, the book of Mormon is actually the ECMAScript spec.  [01:19:16] Sunil Pai: That's what it is. [01:19:17] That must be, I bet. I bet somewhere in the temple there, like classes, horses, functions and stuff like that. I don't get it. Australians and folks from UDA  [01:19:30] swyx: is what it is. Yeah. Cool. Um, okay. Well, uh, I think that's a natural ending point for conversation there. Uh, is there any, are, uh, Topics or questions?  [01:19:40] Sunil Pai: Not really. [01:19:41] It was just so nice to see you and talk to you again. I want to hang with you in real life and grab a beer and we will do that when either I'm there or you come over and I look forward to it, man. It was so nice seeing you today. Absolutely.  [01:19:53] swyx: It's really great to catch up. Yeah, finally. Right.  [01:19:56] Sunil Pai: This was just a great experience for me. [01:19:58] swyx: Awesome. All right. Well, have a, have a good weekend.  [01:20:02] Sunil Pai: I'll see you when I see you. Bye.
Video version on youtube: https://www.youtube.com/watch?v=ErO9UjccwdsMy conversation with Jaden Baptista introducing him to Temporal.Transcript[00:00:00] swyx: Hey everyone is Swyx. I've been gone for the past couple of days and you may or may not be wondering what's happened. I basically got a cold and at this conference, it's actually nice to have a conference cold. I guess, because that means that things are going back to normal, even the, not so great stuff about being meeting back in person. [00:00:22] But I have a cold and I did not feel great. And I think gets COVID. But anyway, I decided to take a little bit of a break, but I also, I'm not sure if you know, but the weekday topics are all done on the same day. And I tried to do this pattern of batching things in weekly themes and this week's theme. [00:00:43] Basically it was poorly chosen. I thought that I had enough to do some, some stuff on basically the how technologies get adopted or get their traction. But I just didn't I ended up not liking any of the other episodes or any other podcasts that I shortlisted for that particular topic. So I just ran out and I just didn't feel inspired. [00:01:06] I felt trapped in the format and didn't really know what to do with it. So I ended up not doing anything. I also had other work to catch up on, on top of the conference stuff. So that was all in my head recently. And in general, and wondering whether or not I'm going in the right direction. So if you do that, And I really am grateful. [00:01:26] Checking out this podcast, because it's like a weird experiment with no particular theme or direction apart from cause it's stuff that I like. If you are, if you have a strong opinion and if you particularly think that there's some ideas that I should just pursue further and you're just mad at me for not doing. [00:01:42] Now's the time to tell me, because I'm also a little bit direction as to where this podcast could go. That's it? I do think that there's a strong thesis for exploring or having a central place for exploring ideas. I am personally interested in and recent dictating my podcast appearances on other people's podcasts to my own feed. [00:02:03] So people who are interested in what I do can follow up directly. So here's a conversation that I had with Jen Battista on Twitter spaces that was recorded. And it's about Tim Poro. So this is the first of maybe. Three podcasts that I did in the last month or so. So I'll be dripping them out over the weekends the next couple of weeks. [00:02:23] And we'll see, we'll see where this goes. I want to get back more into writing. I still have hopes of spinning out my YouTube more seriously, but I think the creative journey. The part-time creative journey of these, where I have a day job and that should take priority over the other side of stuff. [00:02:42] I do dictates the format of the things that I can take on. So, so far the podcast thing has been really great. I actually have a lot, a long backlog of things, which I selected for sure. They just don't fit in any, any particular theme. I think that theme is really nice when I can do it. And just, maybe you don't have enough backlog to do it just yet, so maybe I might go back to not having themes anyway. [00:03:06] So here's my conversation with Jane and review stuff. Well, thank you for  [00:03:09] Jaden Baptista: joining me today, Sean. I really appreciate you taking the time are you doing today?  [00:03:14] swyx: Very good. I'm very entertained by your Twitch title called temporal nutso temporary. Very interesting.  [00:03:22] Jaden Baptista: Oh yeah. I was trying to come up with like a S a stupid clever name for the Twitch streams, despite them not really helping out with what the stream is usually ends up being about. [00:03:34] We tend to wander from topic to time.  [00:03:37] swyx: Sure sure. Yeah. Thanks for having me in a happy to chat to portal. Awesome.  [00:03:42] Jaden Baptista: Yeah. Well, let me ask you just the first quick question. You know, every, every big program that we all talk about w we really enjoy using was, was built to solve a problem, some sort of problem. [00:03:55] What problem was tempura both too soon, but what was the point of building it in the first.  [00:03:59] swyx: So, to be clear, I did not build it. Who did exactly it was built to solve the problem of The abstract problem that the category of problem, this is called is workflows anything long running that needs to take anything more than a simple request response cycle, a request response will be just like, you know, you're paying a serverless function. [00:04:20] It gets back to you in, let's say 300 milliseconds, right? That's a typical cycle, but sometimes you need to do long running work. Typically I would think this is something like video projects. So, if you kick off a job, it takes like four hours to transfer code audio file. But actually it is both longer and shorter than that. [00:04:40] So, this is actually a topic of my recent blog posts. Because even if you, so Dropbox, so box is one of our users and they use us for file transfers and normally a file transfer just feels instantaneous until you try to transfer a million files. Right. They just change that. A thousand of them. [00:04:58] So you need a solution that scales pretty nicely from like a single transfer that should feel instantaneous to you know, something that's gonna take a while. Cause, cause it's just distributed across a lot of systems and you need it to be perfect or you lose data. And if your box you cannot lose data the other super long running task is why does anything have to ever end? [00:05:17] So what if you could just model the entire journey of your customer from. Like their first contact with you to the time that they unsubscribe. And that is a single entity that you just interact with. So it then becomes easy to say things like, okay, on their seventh visit, send them a coupon. Every month charged them based on their, you know, their, their usage or the billing. [00:05:38] And all of this is encapsulated in a single function. So  [00:05:43] Jaden Baptista: yeah.   [00:05:44] swyx: Yeah, because because we have an internal basically, so. Solves, it brings together a database, a scheduler some networking, some search capability and it, and it does. And then it uses, it offers you an SDK, so you can write it in idiomatic language. [00:06:01] The, there, this is not a new problem. Our founders have been working on this for something like 20 years. The tech leads for Amazon SQS. And then in simple workflow Azure doable functions and then when they arrived at Uber, they, they built the initial version of temporal where it now power is like, Something like 400 use cases at Uber mostly driver onboarding marketing which they call communications or something like that. [00:06:25] It's just like, whenever you do something that is so fundamentally asynchronous, you could use it for a lot of things. And it tends to grow that way. So, you know, it was open source that Uber and then it was adopted by other companies like Coinbase. Airbnb Stripe and then once it lands in a, in a company, it just starts growing from there. [00:06:43] And it's just a, it's just a really interesting thing to watch when you solve a problem as well. You get a really strong developer adoption and that's, that's one of the reasons I joined it, just because you know, a bit about my, my personal history, like I typically have been on the front end and serverless side working at another fire in AWS. [00:07:00] But the thing I think that sort of doesn't do well, is anything long running. And the more time you spend studying developer businesses, the more time you understand that, actually, a lot of the money, a lot of value is in the long running stuff. So we've got to do this. And we got to do this and we're going to make it easy to do asynchronous stuff. [00:07:22] And when I say easy, I mean, easy to do production grid services. So, things like retries failures, timeouts, heartbeats you know, when, when systems go down do you have an easy way to. Bring it back up again. If systems go wrong, do you have a way to have everything tracked and audited? Do you have everything properly tracks so you can go back and see what happens, right? [00:07:43] Are you, is your as your monitoring practices? Correct. And then finally, When you need to scale when, when stuff is going really, really well, can you just throw more machines at it? Sometimes if you have the wrong system design, you can not, because you've just, you've just assume a single dependency somewhere. [00:07:57] So every part of your system needs to be horizontally, scalable, completely fault tolerant. And that's what tomorrow brings.  [00:08:03] Jaden Baptista: Awesome. I mean, I, I'm going to be going to be totally honest with you. I, before interacting with you for the first time, never heard of temper. But I ran into this problem a lot. [00:08:15] When I got first into development, I wanted to schedule things and kind of like model a workflow around that schedule instead of the other way around. And we, we ended up switching to like a weird monolithic backend just to get that done with the first projects as a developer. And it was miserable. I hated it. [00:08:36] That's actually why I got into the jam sag in the first place. Cause I hated those monolithic backs. So much that we tried to use this, solve this problem. But I'm curious, one thing that we ran into and by the way, anybody who's listening, if you have questions for Sean about temporal feel free to ask it to be a speaker or in the chat in Twitch. [00:08:52] I'm watching both of them, but my, my first question I've run into situations where that schedule is not set right. So, you know, perhaps with your Uber onboarding flow, maybe there's a set set of steps that you've, you've got that you're going to use to onboard this user, but that might change on a lot of different platforms. [00:09:15] Like, for example, the one I was working on we didn't know what email we would be sending a month down the line because it completely depends on what happened in the meantime. How, how would temporal handle something like that? Is there built in flexibility if the schedule were to change?  [00:09:32] swyx: Oh, migrations. [00:09:33] Yeah. Yeah, absolutely. In fact that's one of the key selling points. So they're the three select, the three key selling points. I'll go in reverse order. Cause that's, that's where we happen to be starting. One of the key selling points is that you want to model workflows as code and that's pretty much. [00:09:47] We're one of the only few workflow engines that do that. And by the way, there are others. That's why, that's why we do have to have our talking points straight. So when you migrate, you need migration, tooling, and you need tests, right. To move with confidence. Would it surprise you to learn that most people, when they set up their asynchronous workflows have no idea. [00:10:06] They, they just they're there. They're happy that it works. That's about it. And then when it, when they moved,  [00:10:14] Jaden Baptista: sorry,   [00:10:16] swyx: when they, when they made changes, they they're just like, okay, should work now. You know, and then, and that's about it. That's the state of the industry is pretty sad. But I, I. Once you have the right abstraction, things just really lock into place. [00:10:29] And being the main thing about tests is that you have to be able to mock out every single possible source of determinism in time. And that's one of the reasons why we're called to Poro. We help you to handle time. And one of the pictures actually for the front end, Because we're going to launch our JavaScript SDK in October. [00:10:47] That's off the record. I, I, I didn't, I did not promise that like, we are components for time, right? So we help you mock out time. We help you test time. We help you structure asynchronous workflows in the same sort of component ties, logic that you might with a reactor view of whatever components. [00:11:03] And I think that that pitch. I'm testing it still and I'm still working on it. That pitch would probably make sense to, to front end developers who are like, yeah. Okay. You know, I have rendering solved, but like what happens to like all the asynchronous stuff? I'm still a black box  [00:11:21] Jaden Baptista: that that's kind of the, the situation that, that I ran into too. [00:11:25] Like I got all of them. That'd be awesome.  [00:11:27] swyx: What was, what was the tech stack? What did you use?  [00:11:30] Jaden Baptista: So at first we were trying to go with like a, a sort of like almost the JAMstack kind of thing. It was very bare bones. This is when I was first getting into web development and I went all gung-ho on this project with the classmate of mine that did not turn out well, but it was, it was basically just a simple JAMstack site actually hosted on Heroku. [00:11:50] Cause we didn't know any better. And we ended up trying to solve like the problem. Of sending messages back and forth. And then depending on the content of those messages doing something like a month later, like sending an email with a recap of some of those messages, or if there were no messages sent and do something else, we ended up actually having to create a PHP backend, like a second backup. [00:12:17] And then a third one later in Python. So we actually had three backends for this problem because the only tools that we could find were all over the place like I said, we didn't know what we were doing, but we were, we're all over the place trying to solve this problem. And so talking to you, it's very fascinating to me that somebody had this problem 20 years ago and decided to fix it. [00:12:37] Yeah,  [00:12:41] swyx: there's a weird, there's an interesting pipeline, I think. And you know, this is a more general career reflect. The pipeline is, is as such you work on a large web-scale company early on in your career. You face all these problems at scale early on, and then you, you leave that company and then you start the problem. [00:12:59] You start a company to solve that problem for everyone else. Right. It's kind of that, that's a lot of the the Google too. There's there's a lot of ex Googlers. So, I think now we're starting to see the emergence of the Uber and Lyft mafia who solve a lot of problems that we were in Lyft. And now they're just emerging as like, you know, independent startups get hub, you know, spawn kind of scale. [00:13:23] Lyft respons there's, there's a bunch of, there's a bunch of them. Twitter  [00:13:26] Jaden Baptista: has done that a lot too people who worked at Twitter and then spun off and did something else, like fauna comes up.  [00:13:33] swyx: Yes. Yes. So, it's interesting. So, so this is, you know, this is an export of Amazon, Microsoft and Uber. [00:13:39] Cause that's the path that, or, [00:13:44] yeah. So, so, I want to give you, yeah. So that's the other thing, right? I want to, I want to really impress upon you this point, which is that you want to, so the, the, the first, the first talking point was, was worthless. Second talking point is about orchestration that you want a single engine that handles retries, timeouts, heartbeats, all that stuff. [00:14:05] And. Anytime you call an API, it actually has a, has a probability of failure, which is like you're calling you're you're crossing network boundaries. You were you know, running into rate limits, whatever, whatever that is. Have you handled it correctly? And that error handling is something that you don't typically think about. [00:14:22] Cause when you code you're typically coding. But having it properly. Exactly. And especially if you think about this in a large company setting you have service call service, a calling service B it's called service C each of these services cannot trust the others essentially. Right? Each of each other, every time you cross the service boundary, you're essentially entering untrusted territory in some sense of the word. [00:14:43] So in order to ensure the reliability of your service or that a response very well, you start building up defenses against each other service. So around the boundaries of your service, you're building in like, okay, like, you know, if, if this call failed, like wait five minutes and we try again, make them and that kind of thing. [00:14:59] And if you run a large enough company, you realize that every team is doing this and. In their own way. And they're all solving problems, incident by incident like this, you don't have to sit down and. Design this from scratch, like you run into an incident and you're like, oh, okay. Yeah, we better put in some production hardening on this thing. [00:15:16] Right. And they're all provisioning infrastructure. It's not just about duplication of code. They're also provisioning infrastructure, right? Like you went from running. Just one little machine somewhere toward to like, okay, now I need a database because I need to store what requests I've made what tracking everything. [00:15:32] Now I need a scheduler, like a Cron job to, to retry things. And you just like build up a lot of. And so what orchestration engines do is they provide reliability on rails. You have a CTO that says, okay, I have a central platform team that does this. Everyone just like use this platform and we'll scale it for you. [00:15:50] Now, you never have to think about the production hardening between your system boundaries. Again. You can just reliably. Things and, and they'll behave properly and feel according to a, a company-wide standard that we can predict and examine this also centralizes failures. So, so in other words, like, if you, if you could like one of the fun jokes about microservices is that they turn every outage into a murderer. [00:16:13] Like who who's at fault. Right. You know, then you go through the trace and then you, you, you, that's the  [00:16:18] Jaden Baptista: one, the one I've heard is a murder mystery where you're both the detective and the murderer.  [00:16:22] swyx: Exactly, exactly. The way to solve it is to have a global view of, of. What happens? So, so yeah, I mean, there's, there's that, which, which I think is, is pretty under appreciated by a lot of people. [00:16:31] And I think it's, it's very core to what we do, but also what a lot of other people do. So we're not the only workflow engine out there. You can try in other, in different contexts Amazon step functions Apache airflow and there's a bunch of others. I don't have to go through all of them. [00:16:47] Jaden Baptista: No, there's a really long list and I've looked into a lot of these, cause like I said, I've seen this problem a lot. And I'm just wondering a lot of those other ones, there was, there was a ton of setup work with those other ones. Like I've tried step functions and I, I actually didn't even get to the point of creating the step function because before that there was so much. [00:17:11] What's the word I'm looking for. Administrative stuff, just to get to that point is how much do you have to do to get temporal working for a really small project? Because you said it will scale with you as, as you start growing bigger. So what, what's the, the investment of time that you'd have to put into it, like a hobby project  [00:17:33] swyx: That's an interesting question for us right now. [00:17:35] It depends on your skill with dev ops, essentially. [00:17:37] So   [00:17:39] Jaden Baptista: is going to help out a lot with this.  [00:17:41] swyx: We still need to do a lot of work. Maybe not coming. We still, we still need to do a lot of work on that. So essentially right now it's a, it's a single go binary. That is that's that runs four services and those four services can be scaled independently. [00:17:55] You will also need a persistence layer. So we support Cassandra my sequel. And then optionally, we also support visibility search. So, using elastic search. So if you want to run all of that you can, we're not, we're not opinionated on how you deploy that. You can try to run it on a single box but also you probably wants to have like a managed database layer. [00:18:15] So you don't, you don't have. Handle the database stuff yourself. But it is very easy to get up and running with our Docker proposed file. So that's the way that we typically get people to get started. If you run Kubernetes, we have a helm chart that you can reference. We definitely don't you'll probably want to customize a home in some way. [00:18:31] People have all sorts of other container orchestration and stuff that they might want to use. But that is essentially the story right now. What it will be in the future is that we will have a single binary that you can one-click deploy on Heroku. Right. [00:18:43] That's obviously the best developer experience. It's just not ironically something that we're optimizing for right now, because right now a lot of our users already have that knowledge. So we're better off spending time working on other stuff. So, and SDK's awesome.  [00:19:00] Jaden Baptista: Cool.  [00:19:00] So other than the JavaScript SDK, that we didn't really mention earlier what other things are in the roadmap for your temporal and by the way, anybody else who has any questions in case you just came in? So feel free to request to be a speaker and a astronomic question. But yeah, what is on your roadmap for them? [00:19:17] A little bit  [00:19:18] swyx: we I've been forbidden from publicly discussing them. So yeah, it's just the thing. Like we we look bad if we let our users down by saying we're working on something and then it doesn't pan out for whatever reason. So we just don't do that. I will say that our JavaScript and Python SDKs are highly, highly, highly in demand. [00:19:35] So, you know, Make your own conclusions there? I am also, maybe  [00:19:41] Jaden Baptista: I was thinking more like your priorities for like, not necessarily like a specific feature or anything, but like, what are you. Planning to focus on .  [00:19:52] swyx: Yeah, yeah, sure. Appreciate that question. Yeah, so, I mean, this is, this is a part of that, like getting us in front of the largest developer populations in the world is very important. [00:20:01] Our web UI is under a major rewrite right now. And that, that will hopefully. Help us be more intuitive, especially to people who are new to the system. always have this view that the docs or docs are. Dogs are a last resort. If, if at all possible use your UI to teach people as they go, instead of having to read a wall of docs. [00:20:21] And then, and then finally, so we're not generally available yet. We're just in private design partnerships with people. So we have, you know, some, some very, very well-known customers, some of them publicly listed and yeah, and, and we're basically learning how to scale. Service and scale a multitenant version of this, right? [00:20:39] It's the, you have to think about the origins of this company was an open source project running solely within Uber. So it never really had to think about You know, often organizations and like security and billing and all that. So we have to build out all of that. We have a list of about 30 things that we need to build before we are a production ready cloud. [00:20:58] But it is a very different startup from other forms of startup, because it seems like this company is like, it there's no risk of mismatch in product market fit. Like we already there in terms of. Right. You're already there. It, I, there, the reason I hesitate to say that we have product market fit is because we don't actually have a full product yet. [00:21:22] Right. Like, we need to build that product. And then we need to be able to repeatedly sell it either from enterprise sales or through a bottoms up, like just developers, trying stuff out. And yeah, there's a long way to get before we go there. But we have plenty of open source developers who are open-source users who are happy. [00:21:38] Build out. You know, we're looking forward to our production hosted cloud.  [00:21:43] Jaden Baptista: Awesome.   [00:21:44] Well, thank you for joining me, Sean today. I really appreciate you. You teach me a little bit about this because I've been looking to try and figure out what temporal is about so that I can potentially be using it in the future. [00:21:54] Once it fully launched. Yeah.  [00:22:00] swyx: You hear more stuff from me about it? There's a lot more, I haven't covered. We use event sourcing. So the way that we do reliability is that events are the source of truth. And we are able to resume from any downtime because of that, right there. That's just a fundamentally opinionated architecture, which I think a lot of people struggle with. [00:22:15] Like they eventually find it, but then they're like, oh, I have like five years of legacy work that I had to throw away now. Right. Yeah. I'll you hear more from me about, about this, for sure, but thanks for taking the time to, to check it out. Cool.  [00:22:28] Jaden Baptista: Well, maybe you can just, while we're finishing up here tell us a little bit about your social media and I have one more bonus question. [00:22:37] swyx: Can you, can you help me here? Like what, what specifically about social media follow him on Twitter at, oh yeah. Yeah. It's Swyx, it's my English and Chinese initials. SWIX but also, you know, just. Temporal, we have a mailing list or we're doing a meet up next week. A lot of people, you know, we'll have one of our users presenting airbikes and we're just trying to grow our community, you know, and, and help people solve their problems. [00:23:00] We're still cool. 30. Yeah. I mean, I'm pretty excited.  [00:23:04] Jaden Baptista: Awesome. Well, I have one little bonus question for you. I didn't let you know about beforehand, but I've been asking all of our interviewees, thus what is one tool that you use all the time that you just could not live with? Put me on the spot here. [00:23:15] swyx: Man, there's Honestly, simple notes. Very good. Oh yeah. Just the notes and notes app has to be reliable offline first and multi-platform I know there are other notes I have out there. Simple notes free. It's backed by WordPress. And it's very reliable and it just does notes, right? It doesn't do any, any rich media. [00:23:37] So you don't like, I know, I know notions the new hotness and all that, but like anything that slows it down, like notion we'll choose fancy new feature over speed and reliability. And I value speed and reliability. So I, yeah, I shout out for simple notes. I think everyone should have at least as reliable a note taking. [00:23:56] Jaden Baptista: Cool. Cool. I'll have to go. I'll have to go down and download that. Then I've been using notion forever. So anyway,  [00:24:03] swyx: it's like, like anytime you search it, it's like that spinner just like ticks you off. Like anytime you accidentally or something on the mobile app, it takes you off. Like just stuff like that. [00:24:13] Just don't do it around  [00:24:15] Jaden Baptista: and around and around. But thank you for introducing me to that. I'm going to go download that right now, actually. Well, thank you for joining me today, Sean. I really appreciate you. Teach me all about them for us. I'd love to have you back in the, in the coming weeks to tell us a little bit more about progress once those SDKs are released. [00:24:31] So let me know, but in the meantime maybe we should cap it off there. Thank you, Sean. All  [00:24:37] swyx: right. Thanks everyone.
Listen to the Sourcegraph podcast: https://about.sourcegraph.com/podcast/christopher-chedeau/ (~28mins in)Lessons learned: Hackathon is important for intrapreneurship Near Death moment requires leap of faith Solve People Problems with Technology, not just technology problems Transcript [00:00:00] swyx: I'm continuing my exploration of how technologies get adopted. And recently there was a really good interview of Christopher shadow by Beyang Liu on the source graph podcast. So I really had to feature it. this tells the story of the invention and adoption of react native internally within Facebook. And the context to this is. [00:00:21] Facebook had a problem with the iteration speed of mobile apps at Facebook, and they were much slower than web apps. So basically the context before this conversation is that mark Zuckerberg had actually set an engineering priority to fix the solution and the engineers have to figure it out. [00:00:36] Christopher Chedeau: One of those things that John had in mind, Can we embed JavaScript in a iOS app. And so at that point, we actually there was no API for this and they're going to do it, but like you found a way to cross compare something and he was able to do it. And you, you want it to be like, okay, can we run, react and power, like native iOS views out of react instead of like dibs and spans, you can like a UI label and a UI views. [00:01:03] And. And so this is like at Facebook there's hackathons. And so in the summer does a three day hackathon. And we basically like Ashwin Lynn that's where on my team, on the photos team. And Joanne, we hacked together off for three days working on this like raw using react to power, like native use. [00:01:25] And at the end of the three days, we had a demo where we are able to show his view and we had a primitive layout system. And. I have the texts it's on the right and the iOS, the rep on the left and basically moved the line like a before and after and a native, iOS, like button. And I bought to move the, like better from the left to the right. [00:01:49] And we were able to click on it. Now, the lights like actually like walks and we're able to like, do. Changing and saving would be like almost entirely. I think it was a hundred milliseconds, a refresh. So it was barely perspective perceptible and we presented to the like hackathon group, like all the, did some callbacks could go. [00:02:11] And what happened is the five project, like most interesting have an audience with Mark Zuckerberg. And so you went to present this hack to Mark Zuckerberg and there was a SRAP, the CTO of the company as well, like in this? Yes. Yeah. And wasn't for this. Yeah. And both of them were like, we're super excited. [00:02:32] And what happened the next week is we basically did like a wall tour of Facebook and we talked to all of the management chain, . And it was like a super, like a fun thing. But now the thing that happened then is okay, so we did this demo, there was like people liking it and everything, but now for like myself as an engineer, like I joined like Facebook, like less than a year ago. [00:02:54] And I moved like my, like myself, I married my wife. I like, this was like a huge like thing. We moved off way across the world. So now it was like a soul searching moment. Do I want to like bets, like this dream, like on a crazy idea that and I was like, like at that time I was like the answer was No. [00:03:18] And so what happened is like a Lena Nash Ashwin basically went and B up like a, they wanted to do an iOS photo up and they built it using what was react iOS at the time. And three months after they basically came back to us and said Hey look like we really want to build like the iOS photo app. [00:03:37] Yeah. And right now we're spending more time building this react iOS thing than actually building the products. And so we're going to restart the product like in a normal iOS. So we can actually walk on it. The project could die. All the project could live and I need to make a call. Do I want to invest in? [00:03:58] And so I did a lot of soul searching and I basically at that point I was like, okay, I think I need to do it. And so, Tom Occhino was the manager of a Joel at the time. And Joel Dan and myself okay, now we like basically creating a small team. And then she came in after this. And this is how like the program. [00:04:17] But this was like not an easy, like a decision to make. And this is like something, a lot of people don't realize it counting products and like they're super high risk Harry, well, for decks, but most of the projects actually fail. This is a startup. Like most of us, not the film and the fact that I'm like here talking to you, like all of the stars align and everything, like it's good work, but this was not a given at the time. [00:04:40] Beyang Liu: You took a risk and it paid off. And I think especially it was a big risk in Facebook at the time. Cause my understanding is that they had already tried to build an initial version of the mobile application using web technologies. That was like the first attempt. [00:04:54] And then it wasn't like performant enough for the user experience. Wasn't good. So they ended up rebuilding it in, native iOS and Android. So there must have been like probably some amount of institutional resistance. Right. . Have another go at a web based or web inspired technology, right? [00:05:11] Christopher Chedeau: Yeah. Yeah. This was like a very interesting time. And like one of the things that's also I don't know if it's true today, but like when I like joined Facebook and even at school, there was this notion of if you're doing web, you're not a real engineer. If you're doing like a channel or like network or native, like you're real engineer. [00:05:31] Yeah. And so there was this like super interesting notion where we are coming from a web background. And so when you have all of these preconceived notions, Whatever we'll doing is not going to be a real engineering. And we're like, scripted is, and let's do this. And this is like very interesting. [00:05:52] The other thing which has been super interesting is there's also a very big difference between the iOS community and general. Hmm. And so, the what's super interesting is iOS community is like apple, like plays a central role. And basically there's like this notion that like everything that apple does is the right way, and this is the way things should go. [00:06:16] And there's a very angry notion of this. And so now if you're basically come in, like we're doing something different when apple guideline, there's a lot of reasons. Yeah. And so this is in practice it has works well because like apple in practice, the software that they produce and like the libraries and everything, I actually really good in practice. [00:06:36] So that's reasonable. But it's been like one of the very interesting thing about trying to convince people the, on the other side for Android, which was super interesting is Android, like the state of the development at the time was like very fragmented. There's 20 different of Android and everything is hiking and like the quality of the framework and the fraud, was not as good. [00:06:57] And so basically like 400, like people were a lot more like welcoming to the idea, but the challenge there was. The actual devices were like, really, really was like for iOS, like the iPhone is actually like super powerful on Android. There was like really bad phones. And so now like a, the channel was like, oh, do you have the JavaScript runtime on top of it and everything. [00:07:20] So it was a lot more challenging to actually get it to work well. And so it was super interesting, like how, like one place was more about like the. The like people mindset and everything and the other, one's more about like the actual techno like technology called shysters and that's well, challenges for like working on this. [00:07:40] And one thing that we found is that in practice, like a lot of the people working on react native, I've been people like coming from a web. Yeah. And the thing is we have been hiring a lot, but it's still like how that via an iOS and Android engineers and women employee as more of them. [00:07:58] And so we've been able to like bootstrap and say, I found them there, our team using react native through like the. And all the channels were starting to hire like dedicated native engineers. They were able to like get funded through like web development. And so it's also like a, an interesting thing. [00:08:15] The staffing and the people working on this projects.  [00:08:18] Beyang Liu: Yeah, that's a neat I guess developer team growth hack is a using a technology that allows people from a much wider pool to, to come work on, on the product. I I'm curious, you mentioned all these challenges with getting started in both iOS and Android, partly psychological, partly technical. [00:08:37] What was the first project or first major project that the react native team targeted inside Facebook as a focal point.  [00:08:45] Christopher Chedeau: So the first one was Instagram. They had a list of all of the photos you've liked. Oh, your favorite regions. And this view like was super small and would have like being our entry point into getting Instagram. [00:08:59] And so one of the challenge with this is like the, the, the mission is like, the feature was not big enough to justify like the investment and adding like a whole like JavaScript run time and this kind of things. And so. We were not able to like, actually succeed there. And I, in interest respect, like I think you need to start to find like something small enough, but something like significant enough that they actually solves a real problem. [00:09:27] Fall the company. And I don't think we really judged like the right balance. And so this is four counts. The thing that actually worked out and like the first one was ads manager. And so we were building like a ads, like app for managing your ads on the phone. Yeah. One of the thing is like the team had a really hard time hiring native engineers to actually build this and this first time on the app. [00:09:53] And so this felt like the perfect opportunity for us to actually get like a team. And most of that same like ads is like the most complicated, like piece of a JavaScript software. And so, there was a lot of really good. From an engineer's like JavaScript expelled, like in the ads organization, they were able to bootstrap the asthma job product. [00:10:15] And we were able to walk with them. And so this was like the first big app that was shipped, using react native. And to this day, it's still using right there live and like it's been super successful.  [00:10:24] Beyang Liu: That's awesome. And, and how did it grow from there? I mean, these days, so  [00:10:28] Christopher Chedeau: one of the biggest thing you're using react native is actually marketplace. [00:10:33] And so this is a yeah, one of the big success stories. And one of the reason why they have been able to be successful as they've been able to iterate like super quickly and like figure out like what works, what doesn't work and like being able to yeah, to move faster on like with true native development.  [00:10:51] Beyang Liu: Yeah, that's awesome. [00:10:53] And marketplace, I think these days it's like one of the largest online marketplaces where like goods are bought and sold anywhere. I think I I've certainly bought like a ton of stuff on marketplace and have as well. So thanks for that. That's a, that was like more than 1 billion people using marketplace every month. [00:11:12] Wow. Pretty insane. That's insane. [00:11:14] swyx: So that is the react native story, which I don't expect to be very representative or applicable to a lot of other situations, but I think it's still notable and it still has a lot of impact and there's still some other lessons to learn from it. So the first lesson is that it's a case of entrepreneurship.  [00:11:30] It started because of a hackathon because mark Zuckerberg had some overall objectives, but didn't really prescribe this solutions. And I think hackathons and giving people space like a three-day hackathon is actually a pretty significant engineering investment, but giving people space to come up with solutions that they've always wanted to work on, I think gives you a lot of these breakthrough ideas that would just never have come up in a regular cycle or a sprint or whatever you call it.  [00:11:54] And the second point that I really learned about is deciding to bet on react native that react native had a near death experience because they tried it out on Facebook photo app, and it was just too unproductive and they could have just died if Christopher did not decide to invest more time in it. And I think you always need these.  [00:12:12] At critical moments, people to just decide that this investment is worth it, that this technology is worth it. Um, Before you get much traction because it's just not in a usable state. And finally, I think the reflection that I have is that react native solve the people problem. And solving people problems actually often solves the technology problem or beats the technology problem. Uh, Innocence that web engineers are much easier to hire than native engineers and web development cycles are much faster than native development cycles it may be inferior in all other respects but if you just win on those two things you win the technology
Listen to the Future of Coding Podcast: https://futureofcoding.org/episodes/011 (30ish mins in)3 Lessons Learned: Features over Benefits- original was a tutorial - second time: here's why react is different. Focus on the implementation rather than how to use it- Sophie Alpert, Dan Abramov, Cheng Lou Support everybody: IRC, Stackoverflow, RedditInfluencers - David NolenBigger conferences, F8, ReactConfPublic user WikiHaters - view every hater comment was your fault Table stakesDocumentationInclusive communicationThree single sentences to communicate why your project is different and worthy of someone's attention - real reasons with tradeoffs, not "faster, smaller, lightweight" Transcript [00:00:00] swyx: Hey everyone. I'm coming to you today from the Infoship shift conference where I just gave a talk on the third is your JavaScript. And it's got me in a mood to look back a little bit on the history of some of the JavaScript frameworks and what better history to cover it, then react, which is something I know. [00:00:17] Well, but I think the history of react is not that well-documented and people. Should hear it from people who are there. The central question, which occupies a lot of my waking thought is how to get developer tools adopted. And there's no better case study than most, no more successful case study than react and how it overcame its initial difficulty. [00:00:38] Here's original reactive team member, Pete hunt on the future of coding podcast.  [00:00:43] Pete Hunt: It took a lot of time to figure out how to message this. Because he can't just come in and say like, everybody's wrong and we're right. Blah, blah, blah, blah, blah, blah. That's not that's not really, you did that a little bit. [00:00:55] Which, which talk. So there was the original JS coffee us talk where we came in and we said, Hey, this is how we build user interfaces. At the time. And it was just like a tutorial. And then there's another way, which was the second top, which was the one that I did, which was basically like, Hey, here's why react is different. [00:01:15] The argument that we were trying to make is that, Hey, this is these are the problems that we had. Here's the solution we came up with and here's what makes our solution. And we had a lot of caveats in there that said, Hey, this might not work for you. There are these certain edge cases where it's actually slower than what you're doing today, but what we found was this was a better set of trade-offs and really what we focused on was. [00:01:40] Educating people on how to use react or how to build their next application with it. It was more about this is what makes it unique and interesting. And what that did was it disarmed people. They were like, oh, this is actually really interesting. We focus much more on the implementation in that than the, how to use it. [00:01:56] And people appreciated that. And the second thing it did was it recruited people into the community that were really passionate about what it does differently. And so you see. These big shots in the react community now like Sophie Albert, and Dan Abramov and Cheng Lou and all these, these people they originally recruited because I think they found the internals of react to the interesting or at least some of the ideas around it to be really interesting rather than, oh, I built my, my application and, three less days than it otherwise would have taken. [00:02:25] Steve Krause: From my perspective it seemed like react was inevitable and it just happened magically, but you were more on the ground floor making it grow. And it seemed like, like you find around the conferences telling people about evangelizing it. [00:02:38] So could you talk through like how it became adopted how, how that felt. Like w what were like some of the key milestones or like key the key things that happened that like made it like moved along.  [00:02:50] Pete Hunt: Yeah. So there was JS con you asked, which was the original announcement. Everybody hated it. Then there was JS con EU which got some more people excited about it. [00:03:00] We wanted to support everybody a lot. So we were in IRC, like almost 24, 7. People would come in and ask a question and we would answer it. Some people would, would camp on stack overflow and answer those questions. But basically like the, the idea was we wanted to recruit and, and basically keep those people engaged in the community because hopefully they could help out. [00:03:20] And that ended up working out nicely. So the number one thing was like just supporting the hell out of people that  [00:03:26] the second big milestone that happened was when David Nolan got involved. And brought in the closure script community. And they, he wrote this blog post called the future of JavaScript MVCs and he was kind of like, Hey, this reacting solves a missing piece that we've had in the closure script community for a long time. [00:03:44] And it's got a programming model that I really like. So that was a big noticeable uptick in the use of reaction. So again, what we're doing right now is, is recruiting. Passionate early adopters and started to slowly turn into some real production usage of react outside of the Facebook companies. [00:04:06] And then fast forward, maybe a year we are so flux is introduced and that solves a problem that the community had. We started talking at kind of bigger, more corporate-y conferences like Facebook's FAA, and then eventually put on a react. For all of the users and then that sort of to inspire a lot of confidence in people to use react. [00:04:28] And so then all these big companies started actually using react. And once you've got some PR real-world production usage, we had this Wiki page where people could add a link to their, their service and where they were using react and Redux. And we would point people to that when they're like, Hey, my boss doesn't know if I should use this new technology. [00:04:48] We said, well, did you know that Facebook is using an Instagram is using it Airbnb and the New York times and all these other, other well-known brands. So that was, that was helpful to then, we just started to see this big explosion in, in the usage of the, of react throughout the community. [00:05:05] The snowball was, was rolling down the hill at that point. React native was another big milestone in, in reacts kind of adoption because that opened up the world mobile developers.  [00:05:14] So I I found react because of David Nolan's article that you mentioned. And I was immediately convinced after reading that article and then watching your rethink, rethinking best practices talk, which, which I think he links to in the, yeah. [00:05:26] In that essay. So I can definitely see how that was a big milestone. I didn't realize how big of a milestone that was in your mind, but, but that's, that's how you got me.  [00:05:34] So makes sense. Yeah. That wasn't an accident either. So like there's a lot of, I was going to conferences and I was connecting with people on Twitter and stuff like that. [00:05:44] And the way that got put together was. There's this guy, Brandon bloom, who's a game programmer who you guys should all follow on Twitter cause he's he's really smart. And he sent me a message or tweeted at me or something and he was like, Hey, I saw your talk. Did you know that this is how like all game engines are implemented? [00:06:05] And I was like, what? I don't know very much about graphics and, and he taught me through it and explained it to me. And he's like, He was in New York and I was like, oh, I'm flying to New York. We should meet up. So we met up over a cup of coffee and he's like, I'm going to bring my friend David Nolan. [00:06:20] And he's going to, he's going to come sit in on our conversation. And so us, the three of us were just at a coffee shop somewhere in New York talking about react. And then he pops into the IRC channel, has a bunch of questions as he's building his first version of his react bindings. [00:06:35] Kind of, again, continuing this, this level of support that we had and that blog post pops out and changed the game for, for react. So that was, that wasn't like, oh, I stumbled upon this technology. And I was struck by how awesome it is. It's like none of this stuff, I'm like, yeah, things go viral, but they go viral after a lot of boots on the ground effort. [00:06:56] Steve Krause: Ah, I'm so glad that that's But that's how it actually happened. Cause it that's so empowering that like, it isn't just random. What goes down on what doesn't, if you work really hard and do the work, it'll eventually work out. So I'm really glad that that's how it happened in reality. And that I have it on this podcast. [00:07:12] That's really inspiring.  [00:07:14] Pete Hunt: Yeah. Yeah. I mean, there are things you got to get, right? Like you gotta write documentation and you got to build a community that you need to foster community that that is inclusive and gets everybody excited about working on this. And you need to also communicate your project really well. [00:07:28] So you need to say, Hey, this is why you should pay attention to our project. And it's got, you get like three bullet points, three single sentences to communicate why your project is different and worthy of someone's attention. And like it's faster and it's smaller and it's lightweight. Real reasons like you got to sacrifice something, right? [00:07:50] Cause otherwise react would be smaller, lighter weight. Right. So what are you sacrificing?  [00:07:54] Steve Krause: Totally. So one thing I saw on the internet that caught my attention when I was preparing for this interview is that there was a quote where you talked about how you replied to every hater comment on Twitter. [00:08:04] And is that, is that something that you thought was effective? Do you still do that now?  [00:08:09] Pete Hunt: Tell me more about that. Oh yeah. That was actually Reddit. So we would post it to Reddit and there would just be like, all of these salty programmers would just reply and trash on like, oh my God, you're putting HTML in your JavaScript. [00:08:22] That's so bad. Or like who is Facebook to tell us how to build applications when we have this thing from Google and Google's wait, got way nicer products than Facebook whatever it is. And our, to every single one of those. Yeah. You know what, like some of these people had legitimate concerns. There's, I've actually found in my personal experience, there's very few real trolls that are just trying to troll and just trying to piss you off. [00:08:50] There's a lot of arrogant people and there's a lot of people that have strong opinions and don't do a great job of expressing them in a respectful way. But their goal is not to piss you off. They actually have legitimate concern. And the way that I thought about it was I, I viewed every single one of those concerns as my fault, because either the technology didn't do what it needed to do, or they, as in, they had a legitimate technical concern or they didn't have a legitimate technical concern and we failed communicating that to them. [00:09:21] And so if you just say. Hey, all of these complaints on Reddit are actually legitimate or have a kernel of truth to them. And they're all my fault. You start to do a lot of self-improvement really quickly. And so that's why I think the messaging around react got to be I thought very, very crisp within the first nine months. [00:09:41] Yeah. Of it being open sourced and that that messaging like branding it, the virtual Dom that now gets a lot of flack and criticism, but I stand by it, it's not a hundred percent accurate, but if you've got someone's attention for five minutes and you want to explain how react works, a virtual time is like a great way to do that. [00:10:00] swyx: Okay. So there were a bunch of very important lessons here. So just to recap, the first is to talk about features over benefits. This is the exact opposite from traditional marketing, where you talk about benefits over features. When reacts was initially introduced, they were trying to introduce them as a tutorial. Like we can do this. This is how you use react.  [00:10:21] And you can code, uh, so-and-so faster than you would normally code. And people really reacted against that. But the second time they introduced react, they focused on why react is different focused on the implementation rather than how to use it. And that actually successfully nerds sniped, the second generation of react core team members like Sophie, opport, Danny Mamava.  [00:10:40] VIN Chang Lu. The second tip after the features of a benefits thing is to support everybody. And they really mean everybody. So camping out in IRC stack overflow and Reddit. Supporting influencers like David Nolan. Supporting haters. Bye Andre. On Reddit by viewing every hater comment as if it was your fault.  [00:10:59] And then building more social. Social momentum. With bigger conferences like Facebook, F eight, and then throwing their own react con. And then also publishing a public user Wiki so they can see. Other companies like them using react. Finally, you have to have your table-stakes. So the table stakes that were mentioned were documentation.  [00:11:20] Inclusive community. And three single sentences to communicate why your project is different and worthy of someone's attention. Real reasons with trade-offs not just, we are we're pleasing faster, smaller, or lightweight, actual reasons. So I thought this was very jam packed with. Really good lessons. 
Song: DJ Snake - Run It (ft. Rick Ross & Rich Brian) [from Shang-Chi and the Legend of the Ten Rings]BBC Radio 1 interview: https://www.bbc.co.uk/sounds/play/p09tz2bz
Full interview on the HOSS talks FOSS podcast: https://www.youtube.com/watch?v=8e3IC-lAgXsVitess: https://vitess.io/Planetscale: https://planetscale.com/Vitess was created in 2010 to solve the MySQL scalability challenges that the team at YouTube faced. This section briefly summarizes the sequence of events that led to Vitess' creation: YouTube’s MySQL database reached a point when peak traffic would soon exceed the database’s serving capacity. To temporarily alleviate the problem, YouTube created a master database for write traffic and a replica database for read traffic. With demand for cat videos at an all-time high, read-only traffic was still high enough to overload the replica database. So YouTube added more replicas, again providing a temporary solution. Eventually, write traffic became too high for the master database to handle, requiring YouTube to shard data to handle incoming traffic. As an aside, sharding would have also become necessary if the overall size of the database became too large for a single MySQL instance. YouTube’s application layer was modified so that before executing any database operation, the code could identify the right database shard to receive that particular query. Vitess let YouTube remove that logic from the source code, introducing a proxy between the application and the database to route and manage database interactions. Since then, YouTube has scaled its user base by a factor of more than 50, greatly increasing its capacity to serve pages, process newly uploaded videos, and more. Even more importantly, Vitess is a platform that continues to scale.
Listen to the full episode on SERadio: https://www.se-radio.net/2020/06/episode-413-spencer-kimball-on-cockroachdb/Previous two parter on Spencer:  https://swyx.transistor.fm/episodes/consistent-synchronous-replication https://swyx.transistor.fm/episodes/spencer-kimball-pt-2-competing-with-big-clouds
Listen to the full interview on SEDaily: https://softwareengineeringdaily.com/2021/06/28/timescale-time-series-databases-with-mike-freedman/We originally created Timescale, really from our own need. Around thattime, 2014-2015, my co-founder and I, Ajay Kulkarni, who we go back many years, we resyncedup and we started thinking about it was a good time for both of us to think about what the nextchallenges are that we want to tackle. It seemed to us that there was this emerging trend ofnow, people talk about the digitization, or digital transformation. It feels like somewhat of ananalyst term, but I think, it's really responsive of what's happening, in that if you think about thelarge, big IT revolution, it was about changing the back office. What was used to be on paperwas now in computers.What we saw was somewhat the same thing happened to basically, every industry, from heavyindustry, to shipping, to logistics, to manufacturing, both discrete and continuous and home IoT.Sometimes this gets blurred under IoT, but we also think about it more broadly as operationaltechnology, those which are not necessarily bits, but atoms. A big part of that was actuallycollecting data of what those systems were doing. It's about sensors and data and whatnot.When we do Initially looked at this problem, we were thinking about a type of data platform wewould want to build, to make it easy to collect and store and analyze that type of data. I thinkthat's a way that we're slightly different, or why our – what we ultimately built as our databaseended up being fairly different than a lot of other so-called time series databases. That'sbecause many of them arose out of IT monitoring, where they were trying to collect metrics fromservers, where we were originally thinking about collecting data more broadly from all these typeof applications and devices around your world.When we started building it, it was originally focusing mostly on IoT. We quickly ran into thisproblem that the existing databases out there and the time series databases out there were notreally designed for our problems. They were often much more limited, because they werefocusing on this narrow infrastructure monitoring problem, where the data maybe wasn't asimportant. It was only a very specific type. Let's say, they stored only floats. They didn't have tohave extra metadata that they wanted to enrich their data to better understand what was goingon, like through joins.After, basically working on this platform for about a year, we somewhat came to the conclusionthat we actually need to build somewhat of our own time series database that was focusing onthis more broad type of problem, and so that's what we do. That's what led the development ofwhat became Timescale.JM: Today, what are the most common applications of a time series database?Like and speak mostly about obviously, TimescaleDB, rather than – as I wasalluding to before, a lot of the other time series databases are much more narrowly focused onIT monitoring, or observability. We really see our use cases across the field. We certainly seecases of observability. In fact, we have subsequently built actually a separate product on top of Timescale called Promp scale, that is really used for initially Prometheus metrics, but morebroadly, to make it easier to store observability data with TimescaleDB.We see still a lot of IoT. We see a lot of logistics. We see financial data and crypto data. We seeevent sourcing. We see product and user analytics. We see people collecting data about howusers are using their SaaS platforms. We see gaming analytics, where companies are collectinginformation about how people's virtual avatars are actually playing within the games. We seemusic analytics. We like to think of the old way, used to find the pop stars, you went down to thesmoky club. Now you collect SoundCloud and Spotify streams, and you use that to identify whothe next breakout artist is going to be.All of these are example of time series data. It's really what's so exciting to us as is it's such abroad use case, so horizontal, because basically, it's all about collecting data at the finestgranularity you can.Tell me about the initial architecture for TimescaleDB. You’re based off ofPostgresSQL. What was the reasoning around that decision?I think, as you point out, Timescale is actually implemented as an extension onPostgresSQL. Starting maybe 10 or 15 years ago, PostgresSQL started exposing low-levelhooks throughout its code base. This is not a plugin where you're running a little JavaScriptcode. We have function pointers into – we get function hooks into the C. PostgresSQL is writtenin C, and so TimescaleDB is, for the most part written in C. We have hooks throughout the codebase at the planner, at sometimes in the storage, at the execution nodes. We are able to insertourselves and do Lot of optimizations as part of the same process.You could ask the question of why not just implement a new database from scratch? Why buildit on top of PostgresSQL? I think this really gets to that, we always viewed ourselves as, and wehear this from our users and community all the time that we are – they are storing critical datainside TimescaleDB, and they need it to, A, work and be reliable. They also need it to be – theyhave a lot of use case requirements. It’s not this, again, narrow thing where you're collectingone metrics, and all you're asking to do is figure out the min-max average of a certain metric.You want to do fancy analysis. You want to do joins. You want to do sub queries. You want to docorrelations. You want to have views. You want the operational maturity of a database. You wanttransactions, backup, and restore, and all of the replication and all of the above. Some peoplesay, it takes maybe 10 years, at least, to build a reliable database. We thought this was a greatway in order to immediately gain that level of reliability, we ourselves are huge fans ofPostgresSQL. It has such a great community. It also has such a large ecosystem.The idea is that effectively, that entire ecosystem would work from us on day one. That means,all of the tooling, all of the ORMs, all of your libraries would just work. If we support full SQL, notSQL-ish. If you know how to use SQL, you could start using – and if your tools speak SQL, ifyou're running Tableau, if you're running Power BI if you're running Grafana, if you're runningSuperset, those all just start working on day one.Now, the second part of it is, well, what does that mean to build a time series database on top ofPostgresSQL, which clearly was designed more as a traditional transactional database, OLTPengine? Sometimes they talk about you think about this architecturally. What I mean by that isyou somewhat think about what your workloads look like and what that would mean from asoftware architecture. Maybe I'll give you a very concrete example. Starting maybe 10 or 15years ago, if you look at traditional databases, you started seeing the growth of what peoplecommonly now called as log structured merge trees, LSMs.This is a data structure that goes back to the mid-90s, but I think you first saw Google, JeffDean and Sanjay Ghemawat built something called LevelDB. The whole idea of an LSM treewas, if you look at a workload that has a lot of updates, so with a lot of e-commerceapplications, with a lot of social networks, you're constantly updating things. Traditionaldatabase, if you think about a disk, if you're doing a lot of in-place updates, and these updatesare randomly distributed across all of your user IDs, this means that you're going to cause yourdisk to do a lot of random writes on hard drives, that's particularly bad. You need to move thedisk.Even on SSDs, it doesn't do great, because SSDs still do a lot better to have sequential writesthan random writes, the way the internals of SSDs work. You started seeing this new type ofdatabase architecture called LSM trees emerge, because people wanted to build databases thathad a lot faster updates. On time series databases, on the other hand, don't typically have thistype of workloads.If you think of a stream of new observations, with the timestamp, these are typically aboutwhat's happening now. It's typically about a stream of inserts that are about this stock price now,this stock price now, this stock price now, or different, or a 100, or a 1,000, or a 100,000 differentsensors all about what the recording right now.If you think about how you would then design the internals of your database and the datastructures, when most of your rights are insert heavy, and particularly about the latest timeinterval, then what that would mean is the somewhat internal structure of your data shouldreflect that. You should optimize your insert path to make it super-efficient to perform inserts onthe latest time interval. It doesn't have to be perfectly in order, but it mostly is about what'shappening recently, as opposed to what's happening a year ago.That said, Timescale absolutely allows you to backfill data and perform updates, or deletes toolder data. It's just from a performance perspective, keeps all the recent stuff in memory andbuilds more efficient data structure to allow you to insert at much higher rates. For example, ona single machine, if you're collecting a stream of records, eat for several, let's say 10 metrics,you'll be able to collect even once 2 million metrics per second on a single, pretty standardmachine. Then we see this again and again, the way we think about architecting Timescale is, is reallythinking about what the workload looks like, that people often care about recent data. The waythey want to manage their data changes as that data ages. They might want to optimize foreven fast queries for the recent stuff. They might want to start reorganizing their data as ages. They might want to start automated automatically aggregating the data as it ages, and droppingthe raw data for the very old stuff to save space. All of these things are what you'd want in agood time series database, when it's not what you want from either a traditional OLTP database,nor if you have a traditional data warehouse, or an analytical database, which doesn't think ofthis operational view of time series so central to it.
Listen to more on the StackOverflow Podcast: https://stackoverflow.blog/2021/08/06/podcast-364-mark-porter-mongodb-database/Transcriptmarkporter  [00:00:00] swyx: This is Mark Porter, the CTO of Mongo DB on his personal journey from relational databases to Mongo DB.   [00:00:06] Mark Porter: I am a relentless tech geek. I've loved tech my whole life. In fact, my Twitter handle is MarkLovesTech. I have used databases since I was 14 with some really ancient technologies started out on a 4k TRS 80 model one computer. We had to program it in assembly language because there wasn't enough memory to use the local basic copy. And I very quickly got into databases and I was talking to someone the other day and he pointed out something I'd never noticed, which is I've oscillated between using databases and building database. So I started out at Caltech and NASA using databases for space, data, and chip data. And then I built databases at Oracle versions, 5 6, 7, 8 for about 13 years. And then I used databases at NewsCorp for huge student data systems. And then I built databases at Amazon with Amazon RDS. Then I moved to Grab taxi, which is the Uber of Southeast Asia and use databases to deliver 15 million rides and meals a day, and then came back to Mongo DB. And here I am building databases again. I frankly can't get away from this thing.  [00:01:20] Ben Popper: I love that story. I wonder. Does that mean. You know, at each point you had some sort of frustration or saw some sort of like opportunity for innovation, you know, you kind of would build something, then you'd be the user of it. Then you'd realize that like the next sort of turn of the wheel was coming. As you move between those jobs where new paradigms and databases and murders.  [00:01:38] Mark Porter: Yeah. I mean, it's been really interesting. Half of my career. I've been the Bo and half my career. I've been the target. And I got to tell you that sometimes as a customer, you're not really happy being the target of what has been produced. Look, the reality is, is relational databases have been the modus operandi since 1970, when Cod first did his paper. And then Oracle was the first company that released them in 1979. They were actually known as relational technology back then and then changed their name later to Oracle. So the mission criticality of databases has never been in doubt. What has changed is the amount of data, the way we process that data. And what's really, really important. And it used to be duplication of data was important and things like that. And while that's still important, what's really important. Now is developer product. Bar none. That is job one for any mission critical software company is developer productivity and innovation  [00:02:35] Ben Popper: makes a lot of sense. It does seem like data has become almost this, uh, overwhelming force for some companies. Ryan. I know if you have experience with this, but I've been getting a lot of pitches and, and talking with folks on the podcast and you know, it's gone from, we're using data to, we have data lakes and there's a data iceberg. And, you know, we're only sort of scratching the surface of what we might be able to do with this. Endless flow of unstructured data that we're collecting. And as you mentioned, yeah, a lot of times what they're looking to do is understand it in a way that allows them to enhance productivity or automate certain processes, which right now are very time labor intensive. Yeah. Yeah. At my previous job, I worked out on an article about data pipelines and, you know, ETL processes and that yeah. There's a becoming a separation, I think, between your production database and the database you use to gain insights, right? Then the production database has to be fast. But the insight database, it can be a little more flexible in how it produces data, right? [00:03:34] Mark Porter: Yeah. So we think about systems of record. We think about systems of insight and yeah. I mean, definitely different people want to do different things with the databases. And so what we do is we think about personas. Are you an analyst? Are you a developer? Are you an AI ML engineer? Are you a PhD data scientist? We always try to come at it from the customer and what they want to accomplish. Yeah,  [00:03:56] Ben Popper: I think that's so interesting because as you said, obviously, databases have always been part of working in the world of software and computers, but increasingly there are these specialties that are very important in which are producing these really interesting results that themselves are devoted to data, as opposed to it being something that, you know, needs to be part of the larger process. Um, so mark, I wanted to touch on something, which is that you had a part of your career at AWS, which now, you know, has grown into. Quite a behemoth. Um, yeah. Just wondering if you can talk to us a little bit about what you learned there and maybe how some of that applies to the role you have at, at Mongo DB. [00:04:26] Mark Porter: Yeah. So I joined AWS as the general manager of AWS RDS, which at that time was probably the largest fleet of databases in the world. And that fleet grew just tremendously while I was there. It was, it was amazing, you know, just showing. That it's not just databases. It was managed databases that mattered. So RDS did not build any of its own databases, RDS vended. By the time I left over a million significantly more than a million Postgres, my SQL Maria DB, Oracle, and SQL server databases. And so the product that we produced was managing those databases and people love it when their database stays up. When the backups and restores work, when you can change parameters when fail over works and all those things. However, over time, as much as I loved running those databases, I became frustrated with how they were shackles almost on customer innovation and customer operability. And so we developed this system called Amazon Aurora, which changed out the storage system underneath Postgres in my SQL. Obviously we couldn't do that for the commercial databases and we made those databases so much more resilient, so much more durable, so much more available, but we kept running into the fundamental limit. Of a rigid architecture of high fail over times and a single primary architecture, which meant that the blast rate. Of a system going down or play in changing in Oracle database. I mean, it takes down a whole company and I can talk more about availability. In fact, you'll have trouble stopping. When you talk to you about availability, if you get me started  [00:06:09] Ben Popper: well, I mean, that's, that's the, uh, the big thing about a no SQL is, is availability, right? The replicability, the speed of access. Yeah, for folks who don't know, let let's lay out the value prop here. Like what is sort of the difference between the two and why would you prefer one over the other? You know, you mentioned shackles. I love that word, but yeah. You know, what are the limitations that it allows you to avoid when you, when you move to a new SQL and I guess, you know, to the degree that it makes sense. Yeah. Talk a little bit about availability or I guess, you know what I would say, it's almost like how robust your system can.  [00:06:41] Mark Porter: So I do think availability is really important, but from, just from a value prop point of view, the main reason that no SQL was started was multiple things. Number one, was this platform availability. I actually think you guys had a podcast with Elliot about a year and a half ago where he talked about the founding of Mongo DB. And I will give a shameless plug for one of your other podcasts, which, which is a great podcast that Elliot did. And. You know, in it, he talked about the fact that they want to do 400,000 transactions per second, and there was no way they could do it, but along the way, they did something even more important, which is they developed the document model. And the document model is just a natural way to program. When you want to add a field to a no SQL application that you're writing. You just added in your code and your structure in your, in your structure in Java or go or rust or whatever. And the database automatically starts having that field. So it's not just about availability. Now to get to your point about availability, Mongo DB uses what's called a sharding architecture or a replica set architecture where you can't actually configure a Mongo DB that doesn't have three nodes and those nodes automatically do elections and they automatically start up. And as opposed to relational databases where fail over is measured in 30 seconds, 60 seconds, 90 minutes, 10 seconds. Fail-over in Mongo. DB is measured in single digit seconds, RP 99.9 election time. And our outlet service is less than seven seconds. And why is that important? Because when an app is down for three to five to seven seconds, people go, huh? What happened? What's going on? On my phone. When it's down for 60 seconds, they've already visited another website to complete their purchase. And so there's a fundamental difference. So the ability to stay up and the ability to, to be available is thing one, the second ability is the ability to scale without limit. We have customers running a petabyte. In Mongo, DB clusters. And with over a thousand nodes, you just can't do that with the relational, even Aurora, which I just got to tell you, I love deeply because I helped architect it. You have one writeable master or primary and up to 15 read replicas. And if you run out of the ability of that master or primary to take right. You're done. You got now split your database and do crazy stuff. So those were the fundamental premises of databases. So, but the thing that's really missing there is that developers love databases, but developers do so much more than just store and retrieve data developers want to do graphs. Developers won't do analytics. Developers want to have a connection to their mobile device. They want to do all this. So what we're doing at Mongo DB and sorry for the brand plug, but I I'm pretty passionate. Yes, we're building an application data platform where the. Correspondence between what we produce and our main persona, the developer we're trying to get to a hundred percent. [00:09:44] swyx: I think this conversation was exceptional, not just because one of the key criticisms of Mongo DB is always from the SQL folks who just say that in the Mongo DB script, kiddies don't really know SQL one now. So they picked Mongo DB. This is a guy who definitely know SQL and is still in love with Mongo DB, but also this is a CTO who's clearly in love with the technology his company represents. And I think there's just not enough of those. I see so many resting investing CTOs were basically completely checked out and not really inspirational leaders or the company. And I think there should be more CTOs that are exactly like my Porter.
Watch on https://www.youtube.com/watch?v=iJIc16aqpO8&t=642s
The Petersens: https://www.youtube.com/c/ThePetersens/about Take Me Home: https://www.youtube.com/watch?v=qap9Qm-Q894 Jolene: https://www.youtube.com/watch?v=viQx4KDivPY Landslide: https://www.youtube.com/watch?v=joUwy8lpvP0
Listen to his full interview on the Lex Fridman podcast: https://www.youtube.com/watch?v=1YbcB6b4A2UBryan's wiki: https://en.wikipedia.org/wiki/Bryan_Johnson_(entrepreneur)
Listen to the Cloud Giants podcast: https://www.listennotes.com/podcasts/cloud-giants/jeff-lawson-co-founder-and-sFi-ad_etHQ/#
Listen to the NFX podcast: https://www.nfx.com/post/the-insider-story-of-waze/ (28 mins in)
Listen to the OSS Startup Podcast for the full episode.References:- Heptio's acquisition in 2018 "It’s not clear how many customers Heptio worked with but they included large, tech-forward businesses like Yahoo Japan."
Superman Lives: Part 1, Part 2  Tim Burton: https://www.youtube.com/watch?v=fKbAEmvZyKQ
Wiki: https://en.wikipedia.org/wiki/Walk_off_the_Earth- 2011: Someone Like You (5 People 1 Guitar) https://www.youtube.com/watch?v=d9NF2edxy-M- 2018: Girls Like You https://www.youtube.com/watch?v=e3PcnNiWygw- 2020: A History of the Beatles https://www.youtube.com/watch?v=rfOx4CmQWLs
From the Psychology Podcast again: https://www.listennotes.com/podcasts/the-psychology-podcast-with-scott-barry-10LF_aSGfEg/#search
Source: https://www.listennotes.com/podcasts/the-psychology-podcast-with-scott-barry-10LF_aSGfEg/#searchSource: https://freakonomics.com/podcast/frbc-robert-cialdini/
Listen to the Pyschology Podcast: https://scottbarrykaufman.com/podcast/robert-cialdini-the-new-psychology-of-persuasion/Steph Smith's book:  Announcement Tweet: https://twitter.com/stephsmithio/status/1418320255429062661 Launch Tweet: https://twitter.com/stephsmithio/status/1411047949094637569 Doing Content Right: https://stephsmithio.gumroad.com/l/doing-time-right ## Social ProofThe next principle is what we call social proof. The idea that one way we decide what we should do in a situation is not proof that comes from some empirical or logic. Uh, information that we've received, it comes from social information. What are the people around us like us doing in this situation that allows me to reduce my uncertainty about what I should.In this situation. Uh, so, uh, for example, uh, a study done in Beijing shows you the cross-cultural reach of this, uh, restaurant managers at one string of restaurants, uh, in, in, in China, put a little asterisk next to certain items on their menu. Uh, and each one immediately became, um, 13 to 20%, depending on the idea.More likely to be purchased. So what did the asterick stand for it? Wasn't what we normally see, which is, this is a specialty of the house, or this is the chef's selection for this evening. It was, this is one of our most popular items and each became 13 to 20% more popular for its popularity. And so. One way as a communicator of genuine information that we can give to other people is to say, we have a lot of popularity for what we are doing and give them examples of that or percentages or market share or this sort of thing.And that always is, uh, an easy way for people. To take the shortcut to yes. Oh, okay. Then I don't have to continue to calibrate it. Yeah. And, and, and, and, uh, thinking about the pros and cons of this, the majority of people like me like it. So that's a shortcut. Yes. But there's some new research. Now, my team is responsible for some of it that takes the principle of social proof to a nother level. And. It is that suppose you have a startup or you have a new product or service or an idea, a new initiative, you would like people to, to join you in. But because it's new, you can't point to social proof. The social proof is minimal. I mean, it's actually negative that a lot of people are doing it.Is there anything you can do under those circumstances? It turns out it is. Even if you don't have, even if you only have a minority or a small minority of people who have adopted it, because it's a good idea, you have to have a good idea. But if it's a good idea, you get the show, a trim who that minority position, if it's only 20% of the market, that's interested in the, if you just say 20%, that's a statistic. If six months ago, it was 10%. That's a difference and that's much better. But if you say six months ago, what was 10%? Three months ago, it was 15% this month. It's 20%. The same 20% is the end point of a trend. And people project the function of a trend into the future. So that for the first time you have the leverage of something, we didn't know the label of before future social proof in the research that we did that showed that if you give people a trend to 20%, They are more likely to say yes to it, right?Because they expect in the future, it will be more than 20%. If you've got a good idea with that kind of ability to move people upward in a trend, you'd be a fool of the influence process. Not the honestly, give them three. Data points. One data point is a statistic, two data points, a difference, three data points, a trend.## ScarcityPeople want more of those things they can have less of. And, uh, w one reason that is the case, I think applies to, uh, what Daniel condoms. It has won the Nobel prize for demonstrating. And that is the power of loss aversion, uh, as opposed to, so that we are more in his prospect theory, says the prospects of losing something are more motivating to us than the prospects of gaining that same thing under conditions of risk and uncertainty.Yeah. You know, I was on a, uh, at a conference where, um, I was in a program to be interviewed, uh, with, uh, Richard Thaler and Daniel Kahneman. And me. Right. And I said to the interviewer, you know, I feel like I'm in a Nobel Laureate sandwich.I'm the lettuce. That's hilarious. I mean, those are big hitters. Those are, those are big hitters. Uh, anyway. Yeah. Yeah, so, Hey, thank you. But, uh, what Kahneman says is this loss aversion, well, that's what scarcity, the basis of scarcity is your F if something is scarce or rare or dwindling and available availability, you're afraid that it will be lost to you.And so, yeah. That's the reason people want those things, uh, that have those characteristics. And, uh, there was a study done of, um, 6,700 E commerce websites. And they looked at AB tests within them to see, which were the factors that if they included it or withdrew, it had the made the biggest difference.In, uh, conversion from, uh, prospect to customer, it was scarcity. Here's the thing. If you could honestly say that the, we have a limited number of these at this price or with these features or with this payment plan or whatever it was, um, you got significantly more, um, uh, Conversions than any other feature they looked at at 29 of them, by the way, the next five where the other principles of influence nice that scarcity was at the top of those principles provided that it was scarcity of number, not scarcity of not limited time. So it was a limited number of items. Rather than, oh, you can only get this for a one week. If you can get it for one week, that means you can decide to get it any time in that week. If it's a limited number and there's competition for it, therefore you better move now. And that's the reason limited number is more successful in limited time off.
loading
Comments 
Download from Google Play
Download from App Store