Cloud with Eric Brewer
Description
RECENT UPDATES:
FindCollabs is a company I started recently
The FindCollabs Podcast is out!
FindCollabs is hiring a React developer
FindCollabs Hackathon #1 has ended! Congrats to ARhythm, Kitspace, and Rivaly for winning 1st, 2nd, and 3rd place ($4,000, $1000, and a set of SE Daily hoodies, respectively). The most valuable feedback award and the most helpful community member award both go to Vynce Montgomery, who will receive both the SE Daily Towel and the SE Daily Old School Bucket Hat
Podsheets is our open source set of tools for managing podcasts and podcast businesses
New version of Software Daily, our app and ad-free subscription service
Google’s strategy for cloud computing is centered around providing open source runtime systems and a high quality developer experience.
Google is highly differentiated in its approach to the cloud. To understand Google’s strategy, it is useful to first set the context for the current competitive environment of cloud providers. <If you want to skip straight to the interview with Eric Brewer, begin at 30:33 >
Amazon Web Services (AWS) popularized cloud computing in 2006. AWS was first to market, which has given the company a large competitive advantage over the rest of the industry. It took Google several years to realize how big the public cloud market was, and how good the economics of running a cloud provider could be. Microsoft also realized the opportunity several years after AWS was started.
The multi-year lead that AWS had on getting to market has given the company tremendous leverage. Because AWS is the most widely trusted, accepted default in the market, AWS is able to deepen that relationship more and more over time, despite the fact that the proprietary APIs of AWS create a level of lock-in that bears some resemblance to the lock-in of Microsoft Windows or Oracle’s relational database systems.
The brief history of software gives us examples of what is supposed to happen next.
When a large company is operating a proprietary developer platform, the open source software ecosystem reflexively comes out with an alternative, open source solution that is better than the proprietary system, right? We saw this with the Linux project, which channeled the developer resentment of the proprietary Windows operating system software into the development of the best server operating system to date: Linux.
The difference between Microsoft Windows in the 90s and AWS today is that, for the most part, developers do not resent AWS. AWS keeps its prices low, and embodies a spirit of innovation despite the fact that AWS is partly built around a repeated process of taking open source projects, packaging them up into cloud services, and integrating them closely with other AWS tools, making it harder to move away from AWS.
In its pioneering offerings of managed services, AWS made it easier to set up tools that had previously been impossible for less-experienced developers to operate. Distributed systems became approachable. Companies like Netflix, Lyft, and Heroku were built on the simplified operational model of AWS.
AWS also innovates outside of this business model of repackaging open source projects.
When AWS pioneered the function-as-a-service model, they created an easy way to scale stateless computation. They presented the developer world with an entirely new compute model: event-driven programming, with services communicating to each other via functional glue code running in cheap, transient containers.
There is strong criticism of the event-driven, AWS Lambda-bound “serverless” compute model. And those critics make a valid point: AWS Lambda is a proprietary API.
To build your app around an event-driven architecture glued together with Lambda functions is to lock yourself tightly to Amazon infrastructure. But the critics of this model do not seem to be the ones who are actually locked in. Critics of Amazon’s proprietary strategy tend to be those with a strong incentive to be critical.
Another common criticism of AWS comes from commercial open source companies such as Redis Labs and Elastic, which are vendors of the Redis open source in-memory data system, and the open source Elasticsearch search and retrieval system respectively. These vendors argue that AWS has violated the spirit of open source by repackaging open source software as profitable software and failing to contribute back to the open source ecosystem with commensurate open source contributions.
AWS has repackaged both Redis and Elasticsearch as AWS Elasticache and Amazon Elasticsearch Service respectively.
These vendors frame their relationship to AWS as zero sum, and are turning to licensing changes as their strategy of choice for creating a new defensible moat against AWS. In various Internet forums, the indignant commercial open source software vendors and AWS have both made their cases to the developer community.
If you are a developer who just wants to build your software, these arguments are an unsettling distraction. In addition to worrying about fault tolerance and durability guarantees and cost management, you now have to worry about licensing.
And as you observe these circumstances, with open source vendors accusing cloud providers of improper behavior, you might begin to wonder: what actually are the rules of open source? By what set of commandments are we judging who is good and who is bad?
And who is to judge what is fair or unfair activity by Amazon, which was the first company to prove to all of us just how influential cloud computing could be?
This is the competitive landscape of the cloud circa April 2019, when I attended Google Cloud Next, Google’s annual cloud developer conference.
Throughout the conference, the optimism and playful spirit of Google was present everywhere. There were colorful shapes and cartoonish diagrams that looked like building blocks made for children. In the expo hall, there were food pillars from which free candy and salted snacks could be grabbed all day