Experimentation Masters

Send us FeedbackIn this episode of the podcast, Jason van der Merwe, Director of Engineering and Growth at Strava, provides some practical tips and lessons on performing negative experiments. Jason shares how Strava use negative experiments to remove elements of the user experience to measure impacts on customer behaviour. Negative experiments are a great way to dispel organisational assumptions, identify new value creating opportunities and amplify customer experiences.---Jason van der Merwe is the Director of Engineering, Growth at Strava. Strava is the popular exercise tracking and social networking service, with more than 110 million users and 8 billion activities uploaded.With a background in iOS development, he manages multiple engineering teams at Strava. Jason is responsible for the Growth, Insights and Optimisation teams, ensuring operational efficiency and cross-functional collaboration across Strava teams.Undertaking studies in Computer Science Engineering at Stanford University, Jason majored in Artificial Intelligence and Machine Learning, with sub-major studies in Mathematics and Computational Sciences.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep015-strava-how-to-use-negative-experiments-to-create-more-customer-valueAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on Twitter @signalnotnoiseFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackLukas Vermeer is widely acknowledged as a global leader in the field of online experimentation. He is best known for his work establishing the world-leading experimentation program at Booking.com. Lukas is an expert at establishing and scaling experimentation and is currently the Director of Experimentation at Vista.  ---Lukas has co-authored many influential academic papers on experimentation and is a highly sought-after conference speaker. He is an experimentation practitioner at heart, combining industry experience in A/B Testing and Data Science with a background in computing science and machine learning. Previously, Lukas was Director of Experimentation at Booking.com, the world’s leading accommodation website. At Booking.com he was responsible for the tools, infrastructure, and training that helped product teams at Booking.com improve the customer experience through hundreds of thousands of experiments. Lukas also works as a freelance consultant, helping businesses grow their experimentation culture. ----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep014-lukas-vermeer-a-global-leader-in-online-experimentationAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on Twitter @signalnotnoiseFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackPicking up from Part One of this episode, Vishal Kapoor discusses why you need to be more surgical with experimentation when testing big bets, how marketplace experiments can set off a vicious cycle and why it's difficult to use scenario modelling to predict real-world marketplace outcomes. All this, on top of more practical experimentation tips and techniques.---Vishal Kapoor is Director of Product at Shipt. Shipt, owned by Target Corporation, is a marketplace that facilitates delivery of groceries, home products and electronics. With more than 300,000 Shipt shoppers delivering same-day customer orders across the United States, Shipt was recently valued at $15B, with revenues of more than $1B per annum.He is a highly experienced senior product leader, with deep expertise spanning Marketplaces, Transportation, Advertising, Search, Messaging, Gaming and Retail Industries. Prior to Shipt, he was Lead Product Manager, Marketplace Pricing and Intelligence at Lyft and led growth of Words With Friends at Zynga. Vishal has also held roles at Amazon, Microsoft, and Yahoo focussed on Advertising and Platforms.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep013-vishal-kapoor-marketplace-experimentation-zero-to-one-pt2Add Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on Twitter @signalnotnoiseFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackVishal Kapoor is the Director of Product at Shipt. Vishal has built, launched, and scaled next-generation consumer products that have generated billions in revenue at Zynga, Lyft and Shipt. In this episode, he provides a practical masterclass on Marketplace Experimentation.---Vishal Kapoor is Director of Product at Shipt. Shipt, owned by Target Corporation, is a marketplace that facilitates delivery of groceries, home products and electronics. With more than 300,000 Shipt shoppers delivering same-day customer orders across the United States, Shipt was recently valued at $15B, with revenues of more than $1B per annum.He is a highly experienced senior product leader, with deep expertise spanning Marketplaces, Transportation, Advertising, Search, Messaging, Gaming and Retail Industries. Prior to Shipt, he was Lead Product Manager, Marketplace Pricing and Intelligence at Lyft and led growth of Words With Friends at Zynga. Vishal has also held roles at Amazon, Microsoft, and Yahoo focussed on Advertising and Platforms.----Want more? Head to the First Principles website for the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep012-vishal-kapoor-marketplace-experimentation-zero-to-one-pt1Add Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on Twitter @signalnotnoiseFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackSebastien Polis is the Experimentation Lead at Dojo. Sebastien knows a thing or two about creating impact with experimentation in startups and large enterprise. Hear how he has established and scaled experimentation at high-growth B2B2C FinTech Dojo.----Sebastien Polis is the Experimentation Lead at Dojo. Dojo is a UK based Fin Tech company that provides payment services to Small and Medium businesses. Dojo is a sub-brand of Paymentsense, one of Europe’s fastest growing Fin Tech companies, handling more than $12B worth of sales each year, and processing more than 250M transactions annually. Dojo supplies over 100,000 small businesses in the UK and Ireland, generating revenues over $300M.Prior to Dojo, Sebastien worked in as a Senior Analyst at Healthily, a health tech startup focused on providing triage advice to customers. He was also Data Scientist at DAZN a sports streaming service that provides fans access to live events around the world on demand. ----Want more? Head to the First Principles website for  the episode transcript, resources and reading lists - https://www.firstprinciples.ventures/ep011-sebastien-polis-experimentation-in-fintech-dojoAdd Beat The Odds newsletter to your inbox for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on Twitter @signalnotnoiseFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackRohan Katyal was the Senior PM, Experimentation at Yelp. He shares his personal story of building and growing the experimentation program from ground up at Yelp. Rohan’s approach to democratising experimentation is the subject of a Harvard Business School case study.----Rohan is a Product Manager at Facebook (Meta) working in the New Product Experimentation (NPE) team with a focus on helping creators monetise.Before working in Facebook’s NPE team, he led growth for Messaging Monetisation on Messenger and Instagram. He also led growth of Click to WhatsApp ads to over 1M monthly advertisers.Previously, Rohan was a PM at Yelp where he grew "Request-A-Quote" by 25% leading to more than $500M in services revenue. He was also responsible for building and scaling the Yelp experimentation program. Prior to Yelp, Rohan was a part of the new products team at Yahoo!. He also co-founded and spearheaded a 16 people non-profit called FindAWay focused on equitable access to education.----Join Beat The Odds newsletter for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackDenise Visser is the captain of the award winning experimentation program at bol.com. In this episode, we learn how Denise established and grew the bol.com experimentation program from less than 100 experiments in 2018, to more than 1,000 experiments p.a. in 2022.Denise Visser is the Product Experimentation Manager at bol.com. bol.com is the largest eCommerce business in the Netherlands, generating €5.5 B in revenue in 2021. The business serves more than 12 million customers in the Netherlands and Belgium, offering more than 34 million products. 47,000 sales partners sell their goods through the bol.com platform. For the past three years she has been the captain of Team Experimentation at bol.com, growing experimentation in the business from less than 100 experiments in 2016 to a team that became the winner of the Experimentation Culture awards in 2021.Denise shares her thoughts, learnings and experiences working at the coalface of experimentation every day.----Join Beat The Odds newsletter for practical tips from world-leading experts in Experimentation & Product Design. https://www.firstprinciples.ventures/newsletterFollow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackJustin Porcano share how he creates experiments and experience prototypes to quickly understand user needs and use cases for solving problems. In this episode, we learn how Justin hacked IKEA to perform an experiment to test a new product idea, the Walhub.Justin Porcano is a Senior Industrial Design Manager at Fjord/Accenture. He has 13 years-experience designing consumer electronics, connected devices, contract furniture and small-scale architecture. Justin has worked with many leading design firms and a wide range of clients including Belkin, HP, Teague, Lunar and Sol Republic.His philosophy is that design should support how we interact with each other and our surroundings and improve on these experiences. Solutions are conceived by researching, observing, testing through hands on experimenting and building.Prior to Accenture, for seven years Justin owned and operated design consultancy Upwell Designs. He is also the Co-Founder of Save Sight Now Foundation, an organisation dedicated to finding a treatment for Usher Syndrome Type 1B.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackRuben de Boer uses the principles of behavioural psychology to elicit organisational transformation to successfully embed experimentation programs in complex businesses . In this episode, we discuss Ruben's four quadrant framework for effective experimentation program change management.Ruben is Lead Conversion Manager and consultant at Online Dialogue. He has been working in experimentation and optimisation for 10+ years. He has worked with world-class clients such as Vodafone, eBay, RTL, Swiss Sense, I amsterdam, fonQ, and DPG.Ruben works with clients to improve their optimisation strategy and culture, implement, and improve experimentation processes, and coaches and leads multidisciplinary teams.With his company Conversion Ideas, his goal is to make CRO accessible to everyone. Ruben’s high-quality and affordable courses have coached and educated over 7,000 students. He has four Udemy courses (two bestsellers) as well as publishing a book, Psychology of A/B Testing. Ruben has coaching clients in the US, UK, and EU.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackMichael Luca is an Associate Professor at Harvard Business School and co-author of the book The Power of Experiments: Decision-Making in a Data-Driven World. In this episode we learn how eBay saved $50M by stopping paid advertising, how Airbnb overlooked second and third order effects of experimentation and why Uber changed their experimentation strategy.Michael's research, teaching, and advisory work focuses on the design of online platforms, and on the ways in which data can inform managerial and policy decisions. His research has been published in academic journals including the Journal of Economic Perspectives, Management Science and the American Economic Journal. He has also written about behavioural economics and online platforms for media outlets including The Wall Street Journal, The Atlantic, Wired, and Slate. At Harvard, Professor Luca developed and teaches an MBA course on using experiments to guide managerial decisions, called From Data to Decisions: Leveraging Experiments for Effective Strategy, Marketing, and Entrepreneurship. He has also taught and developed materials for executive education and MBA courses on platform design, behavioural economics, and business analytics.  ----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works from performing thousands of experiments - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackJames Slayter knows how to successfully navigate large, complex organisations to create impact with experimentation . In this episode, we discuss how he established experimentation at REA Group, Australia's #1 online real estate advertiser (realestate.com.au).James is the Head of Product (Membership) at REA Group. For the past 10 years he has held a variety of Digital Leadership and Product roles. James has extensive experience in Experimentation, Personalisation, CRM, and Big Data across eCommerce, B2B, B2C and customer experience.He was responsible for implementing and scaling experimentation at REA Group, working across the business to deliver personalised customer experiences across apps, website, content, and communication channels. REA Group is one of Australia’s biggest digital businesses, leading the charge with experimentation for more than a decade.Prior to REA Group, James worked for Jetstar Airways (Qantas) developing new digital products and services in Adjacent and Emerging Channels, also managing Personalisation, Optimisation and CRM for the airline.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackJonny Longden knows how to implement and scale experimentation programs in complex matrix organisations. In this episode, we discuss how he established and scaled the experimentation team at Sky, including some of the hurdles and challenges he encountered along the way.Jonny Longden is Conversion Director at Journey Further, a performance marketing agency that works with the world’s leading brands, including ASDA, Virgin Money, Moet Hennessy and Airtasker.During his 15-year career of improving websites with strategy, experimentation, and data, he built and led the conversion team at Sky, and has worked with the likes of Nike, Visa, O2, Nokia, Principal Hotels and Manchester United.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works from performing thousands of experiments - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackEden Bidani knows how to get businesses noticed, drive traffic and convert more customers. In this episode, she teaches us how to develop a value proposition, why customer research is critical in forming a strong proposition and strategies for testing your value proposition.Eden Bidani is the founder of Greenlight Copy, an agency that helps companies develop smart messaging strategies, and copywriting, to support user journeys and increase conversion rates across the funnel. Based in Israel, Eden is a conversion copywriter who is trained in anthropology and sociology. For 10+ years she has helped SaaS, technology, and eCommerce businesses produce high-converting ads, websites, and landing pages.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works from performing thousands of experiments - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackKate Sutton knows what it takes to solve some of the world's toughest humanitarian challenges. In this episode, she teaches us how to innovate successfully in the public sector, how to use experiments to create learning loops and why sustainable scaling is so important.Kate Sutton is the Head of Innovation Centre, Bangkok, for the United Nations Development Program (UNDP). Prior to working for the United Nations, Kate founded a business development consultancy in Malawi, establishing an entrepreneurship accelerator and impact investment fund.She was also Head of Corporate Social Innovation at the UK’s Innovation Foundation Nesta, responsible for leading programmes with the private sector for social change and building cross sector partnerships.----Join our newsletter for practical experimentation tips, tools and methods. I only share with you what I know works from performing thousands of experiments - https://www.firstprinciples.ventures/Follow Gavin on LinkedIn at : https://www.linkedin.com/in/gavinbryant/

Send us FeedbackExperimentation Masters PodcastThe leading resource for product experimentation.Learn practical tips, methods and techniques from world-leading experts in experimentation.Design better experiments, lead with more confidence and have more impact in your organisation. Learn faster. Make better decisions.