Engineering Enablement by Abi Noda

Click here to view the episode transcript'. This week’s episode is a recording from a recent event hosted by Abi Noda (CEO of DX) and Laura Tacho (CTO at DX). The episode begins with an overview of the DORA, SPACE, and DevEx frameworks, including where they overlap and common misconceptions about each. Laura and Abi discuss the advantages and drawbacks of each framework, then discuss how to choose which framework to use.Mentions and Links: Dora.devDiscussion points:2:50- DORA, SPACE, DevEx overview10:35- Choosing which framework to use13:15- Using DORA22:42-Using SPACE

Click here to view the episode transcript. In this week's episode, we welcome Derek DeBellis, lead researcher on Google's DORA team, for a deep dive into the science and methodology behind DORA's research. We explore Derek's background, his role at Google, and how DORA intersects with other research disciplines. Derek takes us through DORA's research process step by step, from defining outcomes and factors to survey design, analysis, and structural equation modeling.Mentions and Links:Derek DeBellis on LinkedInDX’s guide to measuring GenAI adoption and impact2023 Accelerate State of DevOps ReportDiscussion points:(3:00) Derek’s transition from Microsoft to the DORA team at Google(4:28) Derek talks about his connection to surveys(6:16) Derek’s journey to becoming a quantitative user experience researcher(7:48) Derek simplifies DORA(8:19) DORA - Philosophy vs practice(11:09) Understanding desired outcomes(12:45) Self reported outcomes vs objective outcomes(16:16) Derek and Abi discuss the nuances of literature review(19:57) Derek details survey development(27:55) Pretesting issues(29:30) Designing surveys for other companies(35:02) Derek simplifies model analysis and validation techniques(38:48) Benchmarks: Balancing data limitations with method sensitivity

Click here to view the episode transcript. This week we’re joined by Sean Mcllroy from Slack’s Release Engineering team to learn about how they’ve fully automated their deployment process. This conversation covers Slack’s original release process, key changes Sean’s team has made, and the latest challenges they’re working on today. Mentions and links:Read Sean’s blog post, The Scary Thing About DeploysFollow Sean on LinkedInTime Stamps:(1:34): The Release Engineering team(2:13): How the monolith has served Slack (3:24): How the deployment process used to work (6:23): The complexity of the deploy itself(7:39): Early ideas for improving the deployment process(9:07): Why anomaly detection is challenging(10:32): What a Z-score is(13:23): Managing noise with Z-scores(16:49): Presenting this information to people that need it(19:54): Taking humans out of the process(23:13): Handling rollbacks(25:27): Not overloading developers with information(28:26): Handling large deployments

Click here to view the episode transcript. This week’s episode is the recording of a live conversation between Abi and Chris Westerhold (Thoughtworks Head of Developer Experience). This conversation is useful for anyone early in their journey with developer portals or platforms: Abi and Chris discuss common approaches to solving these problems, pitfalls to avoid, building vs. buying, and more. Mentions and LinksFollow Chris on LinkedInWatch the recording of this conversation Watch part 2 of this conversation on the market landscapeLearn about PlatformX, DX’s product mentioned in the conversationTime Stamps:(3:09) Why there’s an increased interest in developer portals(5:33) Chris’ background with dev portals(6:37) Homegrown solutions for developer portals(9:22) How developer portal initiatives begin(11:24) Internal developer portal vs service catalogs and IDPs(16:18) Mistakes companies make with developer portals(21:05) Approaches to solving this problem(24:28) How can developer portals drive value(32:07) Common traps to avoid

Click here to listen to the episode transcript. On this week's episode, Abi interviews Kent Wills, Director of Engineering Effectiveness at Yelp.  He shares insights into the evolution of their developer productivity efforts over the past decade. From tackling challenges with their monolithic architecture to scaling productivity initiatives for over 1,300 developers. Kent also touches on his experience in building a business case for developer productivity.Discussion points:(1:42) Forming the developer productivity team(3:25) Naming the team engineering effectiveness(4:30) Getting leadership buy-in for focusing on this work(7:54) Managing code ownership in Yelp’s monolith(12:23) Supporting the design system(16:00) The business case for forming a dedicated team (19:45) How to standardize (23:50) How their approach to standardization might be different in another company(27:08) Demonstrating the value of their work (32:21) Building an insights platform(38:47) How Yelp is using LLM’sMentions and LinksConnect with Kent Wills on LinkedInWatch Kent’s 2023 talk at ElevateListen to the interview with Peter Seibel (“Let 1,000 flowers bloom”)Download the recently published benchmarks on developer productivity team headcount

This week we’re joined by Gail Carmichael, Principal Instructional Engineer at Splunk. At Splunk, Gail’s team is responsible for improving developer onboarding, which they do through a multi-day learning program. Here, Gail shares how this program works and how they measure developer onboarding. The conversation also covers what instructional engineers are generally, and how Gail demonstrates the impact of her team’s work. Discussion points:(1:16) The Engineering Enablement & Engagement Team at Splunk(8:01) What an Instructional Engineer is(14:36) The developer onboarding program at Splunk(16:05) Components of a good onboarding program(21:11) Why having an onboarding program matters(28:17) Measuring onboarding at Shopify (Gail’s previous company)(31:39) Measuring developer onboarding at SplunkMentions and LinksConnect with Gail on LinkedInDownload the report on Developer productivity metrics at top tech companies

In this episode we’re joined by Adam Rogal, who leads Developer Productivity and Platform at DoorDash. Adam describes DoorDash’s journey with their internal developer portal, and gives advice for  other teams looking to follow a similar path. Adam also describes how his team delivered value quickly and drove adoption for their developer platform.Discussion points:(1:47) Why DoorDash explored implementing a developer portal(6:59) The initial vision for the developer portal 12:19 Funding ongoing development 16:01 Deciding what to include in the portal 19:15 Coming up with a name for the portal 20:01 Advice for interested beginners23:55 Putting together a business case32:32 Getting adoption for the portal 37:27 Driving initial awareness 41:29 Getting feedback from developers48:33 What Adam would have done differentlyMentions and links:Adam Rogal on LinkedInGet started (API)New testing and monitoring tools

In this episode, Abi has a fascinating conversation with Rebecca Parsons, ThoughtWorks's CTO, Camilla Crispim, and Erik Dörnenburg on the ThoughtWorks Tech Radar. The trio begins with an overview of Tech Radar and its history before delving into the intricate process of creating each report involving multiple teams and stakeholders. The conversation concludes with a focus on the evolution of Tech Radar's design and process and potential future changes. This episode offers Tech Radar fans an exclusive behind-the-scenes look at its history and production.Discussion points:1:20-An introduction to the Tech Radar6:06-Common terms used in this episode6:27-The origin of the Tech Radar8:50-Problems that the Tech Radar was aiming to solve12:23-The impact on internal decision making-a tool for driving change14:30-The teams philosophy behind Tech Radar18:33-What sets the Tech Radar apart21:11-Why maintaining independence is crucial for their audience25:08-How Tech Radar publishes their reports29:36-A look into Thoughtworks live meeting sessions34:51-Tech Radars Git repository42:20-Recent changes and upcoming shiftsMentions and links:ThoughtWorks TechRadarRebecca Parsons on LinkedInCamilla Crispim on LinkedInErik Dörnenburg on LinkedInThoughtworks Git repository

This week's guest is Eirini Kalliamvakou, a staff researcher at GitHub focused on AI and developer experience. Eirini sits at the forefront of research into GitHub Copilot. Abi and Eirini discuss recent research on how AI coding assistance impacts developer productivity. They talk about how leaders should build business cases for AI tools. They also preview what's to come with AI tools and implications for how developer productivity is measured.Discussion points:(1:49) Overview of GitHub’s research on AI(2:59) The research study on Copilot(4:48) Defining and measuring productivity for this study(7:44) Exact measures and factors studied(8:16) Key findings from the study(9:45) How the study was conducted (11:17) Most surprising findings for the researchers(14:01) The motivation for conducting a follow-up study(15:34) How the follow-up study was conducted(18:42) Findings from the follow-up study(21:13) Is AI just hype? (26:34) How to begin advocating for AI tools(34:44) How to translate data into dollars(37:06) How to roll out AI tools to an organization(38:47) The impact of AI on developer experience(43:24) Implications of AI on how we measure productivityMentions and links:Eirini Kalliamvakou on LinkedInResearch on the impact of Copilot Crossing the Chasm by Geoffrey Moore

Christopher Sanson is a product manager at Airbnb who is dedicated to enhancing developer productivity and tooling. Today, we learn more about Airbnb's developer productivity team and how various teams use metrics, both within and outside the organization. From there, we dive even deeper into their measurement journey, highlighting their implementation of DORA metrics and the challenges they overcame throughout the process.Discussion points:(2:43) Who is the developer productivity customer(4:49) The evolution of developer productivity at Airbnb(9:26) Approach before DORA metrics(14:29) Getting buy-in for DORA metrics(17:49) Planning how to deliver new metrics to the organization(21:12) How Airbnb calculates deployment frequency(23:29) Implementing a proof of concept(27:20) Statistical measurement strategies and tactics(31:11) Operationalizing developer productivity metrics(34:26) How Airbnb reviews data(35:41) How Airbnb uses DORA metricsMentions and links:Christopher Sanson on LinkedInChristopher’s talk at DPE SummitHow Top Companies Measure Developer Productivity

In this episode, Abi speaks with Ana Petkovska, who is currently leading the developer experience team at Nexthink. Ana takes us through her journey of leading a DevOps team that underwent multiple transformations. She explains how her team went from being a DevOps team to EngProd and eventually DevEx. Ana elaborates on her team's challenges and the reasons behind the shift in focus. She also shares how she discovered EngProd and used data from companies like Google to convince her company to invest in EngProd. Finally, Ana explains how DevEx came into the picture and changed how her team approaches and measures their work.Discussion points:(00:28) Creating and leading a DevOps team(05:04) Shifting from DevOps to EngProd(07:28) Inspiration from Google(10:05) Building the case for EngProd(13:42) Ratio of engineers to DevEx engineers(15:10) Team mission and charter(16:53) Learning about DevEx(20:05) The difference between EngProd and DevEx(22:32) Nexthink's focus todayMentions and links:Ana Petkovska on LinkedInEngineering Productivity @Google (Michael Bachman)

In this episode, Abi chats with Grant Jenks, Senior Staff SWE, Engineering Insights @ LinkedIn. They dive into LinkedIn's developer insights platform, iHub, and its backstory. The conversation covers qualitative versus quantitative metrics, sharing concerns about these terms and exploring their correlation. The episode wraps up with technical topics like winsorized means, thoughts on composite scores, and ways AI can benefit developer productivity teams.(1:10) Insights in the productivity space(7:13) LinkedIn's metrics platform, iHub(12:52) Making metrics actionable(15:35) Choosing the right and wrong metrics(19:39) The difficulty of answering simple questions(26:23) Top-down vs. bottom-up approach to metrics(32:12) Winsorized mean and selecting measurements(39:25) Using composite metrics(46:57) Using AI in developer productivity

This week's episode is with Jim Beyers, VP of Engineering Enablement at CVS Health. Jim joined CVS a year ago to lead an effort to build an internal developer platform. Abi and Jim discuss how Jim joined CVS to build an internal developer platform, what brought him to the job, and how the developer experience fits into the broader transformation goals of CVS. Additionally, this episode covers building the team, defining a strategy, and how he's thinking about winning the hearts and minds across his organization.Discussion points:(1:15) How Jim was brought into CVS(2:39) How DevEx aligns with CVS’s transformation initiatives(6:06) Jim’s vision for developer experience(8:26) Building a DevEx team and working with product managers(15:06) Defining and communicating a DevEx strategy(19:37) Assessing Backstage and developing a platform(24:40) Working with developers and leaders(27:55) Working alongside colleagues tackling similar problems(29:26) Reporting on progressMentions and links:Jim Beyers on LinkedInJim’s talk on the Target Application Platform

This week we spoke with Nils Loodin, Platform Product Manager at Spotify. Nils describes how his role in platform product management works, including unique challenges, approaches, and career considerations. Nils also discusses some of the recent changes within Spotify's platform organization, including shifting teams from tech-centric to journey-centric. Discussion points:(1:30) How Nils came into his role(3:59) How “developer experience” came into the picture at Spotify(5:30) How the Platform team is structured(8:52) Unique challenges of the Platform PM role(12:51) Defining the Platform PM’s focus(16:39) Staying close to their customers(21:09) Optimal background for someone in this role(24:43) Attracting PMs into Platform roles(29:40) How it is that Spotify’s leadership invests in developer experience(31:19) How a recent reorg shifted Platform’s focus (41:29) Improving onboarding for mobile engineers(47:33) Measuring onboarding Mentions and links:Connect with Nils on LinkedInThe product management discipline in platform teams | Russ Nealis (Plaid)Spotify’s Engineering blog 

This week we’re joined by Justin Wright and Matthew Dimich, who lead Platform Engineering and Engineering Enablement at Thomson Reuters. Justin and Matt give an inside look at how they’ve evolved their organization’s structure and approach over the past 8 years. Discussion points: (1:03) Founding the platform team(5:49) The current organizational structure(9:00) Key initiatives the platform organization is focused on(12:55) The enablement function within platform(16:44) What drove the engagement function’s growth(19:42) The value of having an enablement function(24:05) Marketing the enablement team’s work(29:47) How enablement interfaces with other platform teams(33:22) Managing the work enablement focuses on(36:55) The balance of requests vs proactive workMentions and links:Connect with Justin and Matt on LinkedInManuel Pais’ Platform as a Product talk