Measure Up

⁠Mike Taylor⁠, co-author of ⁠Prompt Engineering for Generative AI⁠ and first two-time guest of the pod joins us to "delve" into the important questions around getting the best answers out of ChatGPT and the like.Learn about the principles of prompt engineering, the role of generative AI in analytics, and how to effectively use AI for data cleaning, code generation, and more.▶️ Watch on YouTubeLinks from the show: ⁠Ep. 2 with Mike - Marketing Memetics & Measurement⁠ ⁠Prompt Engineering for Generative AI⁠ [book] ⁠Cursor⁠ (AI code editor)00:00 Introduction and Casual Banter01:16 Speed Reading and Da Vinci Jokes02:52 Introducing the Guest: A Modern Day Da Vinci05:19 Diving into Prompt Engineering08:35 Generative AI in Analytics and Marketing14:37 Practical Applications and Challenges18:38 Prompt Engineering Tips and Tricks24:54 Crafting Effective Prompts for Social Media25:37 The Value of Prompt Engineering in Automation26:39 Exploring Prompt Marketplaces and Tool Use27:51 Automating Prompt Creation and Its Implications29:28 The Future of Prompt Engineering as a Skill32:50 Applications of AI in Copywriting and Image Generation34:58 AI in Data Extraction and Analytics45:12 Leveraging AI for Industry Insights and Efficiency48:45 Conclusion and Book Pitch

Have you ever been told by some smug data scientist that you just need to run a randomized controlled trial to understand the true incrementality of a paid media channel? Just run a power analysis to understand your sample size requirements, right? Were you left with questions about specifically how to do that?In this episode, we talked with Nick Stoltz, Chief Strategy Officer at Measured - an incrementality testing and media optimization platform. We go beyond the "what" and "why" of testing, and dive into the details of the "how."📺 ⁠Watch on YouTubeLinks from the show: Nick on LinkedIn Measured

Version A: Join Simon and Jim as they babble about button colors while the experts tell them what really matters in experimentation.Version B: Have you ever wondered what makes for a great experimentation culture?(Let us know in the comments which version you prefer!)We were lucky to talk with Kelly Worthham and Ton Wesseling, both experts in decision science and experimentation.We cover the transition from traditional conversion rate optimization (CRO) to broader growth engineering, organizational transformation for better decision-making, and the impact of AI on experimentation.Discover how conferences like Experimentation Island and Conversion Hotel foster deep, meaningful conversations and collaborations within the community.Links from the show:The Conference known as Conversion HotelThe Conference known as Experimentation IslandTest & Learn CommunityNo Hacks PodcastExperimentation Culture Awards00:00 Introduction and Casual Banter00:52 Halloween Candy Stories02:27 Introducing the Special Guests02:41 Kelly Worthham's Background03:58 Todd's Background and Contributions05:18 Experimentation Island Conference13:57 Conversion Optimization vs. Growth Optimization25:07 The Risk of Imitating Big Brands25:22 Helping Businesses Make Data-Driven Decisions27:02 Building Effective Experimentation Processes30:22 Challenges in Replicability and Experimentation34:54 The Role of AI in Experimentation40:52 Future of Experimentation and AI48:32 Concluding Thoughts and Recommendations

Ever wonder how Walmart, Kroger, Home Depot, and Instacart are different from a guy in a trench coat? They ask for consent before exposure! Join Simon and Jim as they dive into the often misunderstood world of retail media networks with Sky Frontier, EVP at Incremental.Discover why retail media is the Wild West of advertising, how it's moving up the funnel, and the unique challenges it poses. Learn about Sky's journey from philosophy to advertising, and why your branded search strategy might need a rethink. Plus, we tackle Amazon's recent Google Shopping mystery – did they just pull off the greatest experiment of all time?Links from the show:Skye Frontier on LinkedInIncrementalShow Notes:00:00 Welcome Back and Summer Catch-Up00:34 AI Measurement Solutions and Future Prospects01:10 Retail Media Networks: An Introduction02:21 Guest Introduction: Sky Frontier04:15 Defining Retail Media Networks06:46 Challenges in Retail Media Measurement14:12 Incrementality in Retail Media18:42 Operational Challenges and Clean Rooms22:55 Challenges in Retail Media Experimentation23:41 Econometrics and Granularity in Retail Media25:18 Synthetic Experimentation and Difference-in-Differences Analysis28:30 Future Trends in Retail Media32:28 Amazon's Strategic Shift in Advertising41:55 Concluding Thoughts on Retail Media Measurement

Should you measure SEO by its ability to climb a tree? Hear Mike King's take on all things SEO - how AI is disrupting the space, and how measurement is (or should be) changing for this channel.Mike King is the founder and CEO of digital marketing agency iPullRank. King's journey from battle rapping with the Wu-Tang Clan to decoding Google's algorithms lays the foundation for a spirited discussion on the future of SEO, the impact of AI on search behavior, and innovative measurement techniques.Listen as we explore the shift from traditional click-based metrics to more complex, probabilistic methods driven by AI. Learn how SEO is adapting to changes in user behavior, including the rise of AI overviews and the challenges of measuring their impact.Plus, don't miss King's tips on leveraging advanced tools and strategies to stay ahead in the ever-changing SEO landscape.Links from the episode:Mike King on LinkedIniPullRankQforiaShow Notes:00:00 Introduction to Mike King00:23 Mike King's Early Career and Achievements00:38 Transition to Digital Marketing01:08 Future of SEO and Measurement Challenges01:51 Understanding SEO Metrics04:36 Google's AI Overviews and User Behavior05:21 SEO as a Brand Channel07:45 Challenges in Measuring SEO Effectiveness09:21 Impact of AI on SEO Traffic11:49 Evolving SEO Measurement Techniques18:22 Tools and Strategies for SEO Measurement25:07 Understanding Query Fan Out and Reasoning in SEO28:42 Client Education and Shifts in Search Behavior32:57 Improving Relevance and Content Structure36:04 Measuring SEO Performance and Authoritativeness43:25 Sentiment Analysis and Query Fan Out46:03 Reframing SEO for Better Investment48:41 Final Thoughts and Incremental Insights

Google Marketing Live 2023: Key Announcements, AI in Search, and the Future of MeasurementSimon and Jim unpack the highlights of Google Marketing Live 2025. Fresh from the conference, Simon "delves" (yes, AI was a big part) into key announcements, including AI in Search, AI Max for Search, VEO advancements, and measurement challenges.They discuss the excitement around AI mode, scenario planning in Google Analytics, and the evolving landscape of Marketing Mix Modeling (MMM).Additionally, they touch on insights from Google's new tools and partnerships, underscoring a shift towards more integrated and accessible marketing strategies.▶️ Watch on YouTube00:00 Introduction and Casual Banter00:15 Google Marketing Live Highlights01:23 Shaq and T-Pain at GML03:19 Measurement Announcements at GML05:13 Incrementality Testing13:01 Google Analytics and Attribution20:44 Meridian Scenario Planning25:21 Making MMM Accessible: A Double-Edged Sword25:57 The Complexity of Tagging and MMM28:14 Validation and the Dangers of Simplification32:11 AI's Role in MMM and Marketing37:48 Google Marketing Live: Key Announcements38:53 The Future of AI in Search and Advertising42:42 VEO and the Evolution of Creative Development47:37 Final Thoughts and Community Engagement

Are you wondering what to do next in your career? Is AI going to leave you living in a van, down by the river? What paths have others taken through this crazy measurement industry we're in?Let's learn from a sample of 1: Sani Manić - who has gone from web dev to CRO to co-founder of a SaaS company. Find out how he transitioned across various web development and CRO roles and how decided what to do next. ▶️ Watch on YouTubeLinks from the show:Sani Manić on LinkedInPodpacerCohesioBook - Can't Hurt Me (David Goggins)The School of Greatness podcast (ep. with Leila Hormozi)Show Notes:00:00 Introduction and Friendly Banter01:33 Career Backgrounds and Education01:57 Career Progression and AI Impact02:58 Guest Introduction and Career Journey06:08 WordPress Beginnings11:57 Transition to Marketing and SEO13:21 Bridging the Gap Between Teams19:36 Challenges in CRO and Statistics23:16 AI and Best Practices in Website Optimization23:59 The Future of Hyper-Personalized Experiences26:38 The Problem with Historical Data in CRO28:11 Universal Experiences vs. Individual Optimization31:53 The Journey to Becoming a Founder34:15 Building Tools to Solve Real Problems38:04 Advice for Aspiring Entrepreneurs39:16 The Importance of Focus and Small Steps48:09 Incremental Insights and Recommendations

Is MMM the right solution for your company?With seasoned professionals Gabriel Franco (Founder of Cassandra) and TS Kelly (Managing Director at Arima), we go deep into the world of Marketing Mix Modeling (MMM).What does the adoption of MMM look like? What's the role of open-source tools like Meta's Robyn and Google's Meridian? When is MMM the NOT the right solution? What do MMM critics get wrong? And what's in store over the next few years for MMM?▶️ Watch on YouTube00:00 Introduction and Opening Remarks00:34 Introduction to Marketing Mix Modeling00:55 Current Trends and Sponsors in Marketing Mix Modeling01:36 Choosing the Right Marketing Mix Modeling Approach02:55 Consultant vs. In-House vs. Open Source04:17 Challenges and Success Stories in Marketing Mix Modeling08:31 Adoption and Education in Marketing Mix Modeling23:20 Future of Marketing Mix Modeling30:18 Audience Q&A37:10 Conclusion and Closing Remarks

A professor, an analytics director, and a podcaster walk into a bar and order a whiskey. Which brand do they order? And how does that data make it's way into a marketing mix model?That's what Simon and Jim wanted to know, so they asked Elea Feit - Associate Dean of Research and Professor of Marketing at Drexel, and Karen Chisholm, Director of Transformation Analytics at Pernod Ricard.Find out the biggest challenge marketers are facing today regarding measurement, and how they're tackling it. Find out what's in store for marketing measurement and MMM in the next 3 years.Grab a drink and have a listen :)▶️ Watch on YouTubeLinks from the show:Marketing Science InstituteThe Advertising Research FoundationElea Feit on LinkedIneleafeit.comKaren Chisholm (email about job opportunities!)00:47 Today's Topic: Marketing Mix Modeling02:10 Introducing the Guests05:07 MSI and ARF Initiative07:52 Survey Insights and Challenges12:20 Measurement Techniques and Strategies16:34 Brand-Level Optimization and Earned Media22:44 Granularity in Marketing Mix Modeling28:54 Understanding Marketing Mix Modeling29:18 The Four Ps and Their Importance30:20 Media Mix Modeling vs. Marketing Mix Modeling32:29 Challenges in Media and Marketing Mix Modeling34:09 Always-On Discounts and Their Impact37:14 Data Quality and Availability Issues40:38 The Future of Marketing Mix Modeling43:08 Industry Perspectives and Best Practices50:28 Open Source Solutions and In-House Modeling54:58 Job Opportunities and Final Thoughts

There's been a lot of excitement about Google's recent launch of Meridian - their open-source MMM framework. But does it live up to the hype?Jim and Simon explore what sets Meridian apart from other tools like Meta's Robyn and PyMC Marketing. They also discuss the evolution and impact of open-source MMM, the role of new data streams such as Google Query Volume and Reach & Frequency data, and how the availability of Meridian might shift industry standards. Listen in to understand the features, benefits, and the potential gaps Meridian fills within the marketing measurement ecosystem.▶️ Watch us on YouTubeLinks from the show:Google's MeridianMeta's RobynPyMC MarketingRecast - Validating MMMMMM Hub00:00 Introduction and Host Switch-Up00:37 Discussing Google's Meridian Announcement02:00 Understanding Meridian's Name and Popularity03:48 Google's Open Source MMM Framework05:53 Comparing Lightweight MMM and Robyn12:02 The Role of AI and Data in MMM21:52 Debunking Meridian's Myths22:13 Understanding Meridian's Role in Experimentation22:54 The Importance of Competition in Open Source MMM23:28 Unique Features of Meridian Compared to PyMC and Robin23:49 Choosing the Right MMM Tool: Robyn, Meridian, or PyMC?24:22 Explaining Time Varying Coefficients24:30 Entry-Level Recommendations for MMM Tools25:00 Static vs. Time Varying Baselines in MMM28:44 Evaluating the Accessibility of MMM Tools35:20 The Need for Robust Validation in MMM37:38 Challenges in Incrementality Testing39:07 Bridging the Gap Between MMM and Media Managers43:17 Future of MMM: In-Platform Incrementality Testing43:46 Final Thoughts and Joining the MMM Hub

Big Bets for 2025: AI, Marketing Mix Modeling, and the Future of Retail MediaJoin Simon and Jim as they place their bets for 2025. Find out why Jim thinks MMM adoption is going to skyrocket, and why Simon is all in on retail media measurement.More importantly, find out why Simon's house was once listed on Google Maps as a Forever 21 location.▶️Watch on YouTube00:00 Introduction and Year-End Reflections00:24 Cauzle's First Year and Growth01:02 Podcast Popularity and Listener Engagement01:35 Career Transitions and New Roles03:49 The Future of Measurement and Media04:05 Big Bets for 202504:55 Big Bet #1: Retail Media Measurement14:33 Big Bet #2: Marketing Mix Modeling Predictions26:05 Big Bet #3: AI Answer Engines and Search Behavior30:55 Big Bet #4: AI's Role in Marketing33:18 Big Bet #5: Synthetic Audiences in Market Research43:39 Big Bet #6: External Performance Calibration59:48 Concluding Thoughts and Incremental Insights

How much should you be paying for Marketing Mix Modeling (MMM)? How do you make day-to-day tactical decisions when MMM isn't that granular? Should you go with open source MMM solutions, or stick with a vendor?Join Jim and Simon in this in-depth discussion about the current state and future of Marketing Mix Modeling (MMM). They cover the evolution of open-source MMM platforms like Meta's Robyn, Google's Meridian, and PyMC Marketing, and explore their pros and cons. They also discuss the implications of AI-driven ad units and the challenges they pose for measurement.📺 ⁠⁠⁠Watch on YouTube⁠Links from the show: MMM Solution Comparison Questions to ask MMM Vendors Meta's Robyn Google's Meridian PyMC Marketing Uber's Orbit Recast PyMC Labs00:00 Introduction and Greetings00:44 Discussing the State of MMM02:02 Exploring MMM Platforms08:33 Quick Tangent - Baseball and Data Analytics09:41 Comparing MMM Packages12:23 Open Source MMM and Future Trends25:45 The Origin of PyMC Marketing26:54 Comparing Open Source Options29:23 The MMM Gap: Challenges and Solutions31:55 Triangulation and Platform Data39:53 The Role of AI in Ad Buying49:50 Future of MMM and Industry Trends

What are the biggest challenges facing marketers and analysts today? Is it privacy regulations? Technology changes? AI?In two words: unmet expectations.Neil Hoyne, Chief Strategist at Google, shares his insights into the evolution of marketing measurement given the current environment and challenges.Even with MMMs and incrementality testing, at the end of the day the organizations that win will be the ones that can take action and make decisions based on the data. This is not a tool problem, or a data problem - it’s a people problem. Organizational friction and human behavior is still there.Listen in as we discuss the resurgence of Marketing Mix Modeling (MMM) as essential tool for marketers in a privacy-centric era, the importance of adaptability within the marketing industry, and the crucial role of storytelling and relationship building in data interpretation.📺 ⁠⁠Watch on YouTube⁠Links from the show: ⁠Neil on LinkedIn⁠ ⁠Journal of Consumer Research Journal of the Academy of Marketing Science Journal of MarketingTimeline:00:21 A Dive into Historical Context01:08 Setting the Stage for Today's Episode: The Marketing Measurement Dilemma02:44 Introducing the Expert: Neil Hoyne03:54 The Evolution of Marketing Measurement Post-COVID06:19 Challenges and Expectations in Modern Marketing Measurement16:51 The Resurgence of MMM and Addressing Organizational Challenges22:08 Bridging the Gap Between Data and Decision-Making23:22 The Future of Measurement and Analytics in Marketing26:47 Exploring the Imperfections of Marketing Tools27:05 The Complexity of Understanding Consumer Behavior28:10 Navigating Uncertainty in Marketing Decisions29:04 Competitive Advantage in Consumer Behavior30:26 The Quest for Better, Not Perfect, Marketing Models32:12 Bridging the Gap Between Data Science and Business Goals33:52 Embracing Imperfection and Business Impact38:05 The Future of AI and Its Impact on Marketing43:08 Adapting to Change and Embracing New Skills46:25 Final Thoughts and Advice for Marketers

You know it's that time of the year - time for navel gazing (not to be confused with naval gazing) end-of-year reviews, proclamations, and prognostications.It's been a busy year for me and Simon, as we were both laid off and started new paths and projects. One project in particular was this podcast! Join us as we open up about the highs and lows of our year, and what we have in store for 2024.

Get out your #2 pencils, because today we're going back to school! In this episode, we’re talking about marketing measurement education - how we learn it, what we learn, and how to deal with teaching and learning technical subjects effectively. And as an added bonus, I'm lucky to be joined by Simon Poulton as my new co-host for the show! We're joined by Myles Younger, Head of Innovation and Insights at U of Digital, where he leads new product development, tech services, and thought leadership to help scale the company’s educational offerings to new formats, new learners, and new markets. He’s been in the advertising industry for nearly 20 years, spanning client side, agency side, and even founding a dynamic ads platform that was successfully acquired. He also offers up plenty of spicy takes on Twitter like “disc golf stores are the vape store of sporting goods” and “you can’t get a good Italian sub on the west coast” (apparently they add mayonnaise in addition to the oil and vinegar, which is just disgusting). ▶️ Watch on YouTube Links from the show: Myles Younger on LinkedIn on Twitter U of Digital The Filing Cabinet: A Vertical History of Information

Do you ever feel like your activities online are being watched? Well, it's probably because they are 🤣Join Simon and Jim as they explore behavioral analytics with guest Josh Silverbauer. Josh explains the intricacies of tools like Microsoft Clarity, the comparison between marketing and behavioral analytics, and the potential future of AI-driven personalization. We also dive into Josh's unique contribution to the analytics community through his rock opera 'User Journey'.Links from the show:Josh Silverbauer on LinkedInUser Journey - The Rock OperaSteve from Milwaukee (iykyk)Show Notes:00:00 Introduction02:08 Behavioral Analytics Overview02:15 Introducing the Guest: Josh Silverbauer08:06 Deep Dive into Microsoft Clarity24:47 Future of Behavioral Analytics and Personalization29:45 The Assumption of Personalization29:56 Hyper-Personalization in Practice30:33 Challenges of Hyper-Personalization31:34 User Privacy and Personalization32:20 First Party Cookies and Identity Resolution33:40 The Future of Web Identity34:57 Personalization Gone Wrong38:30 The Ethics of Screen Recording40:14 The Analytics Rock Opera45:32 Creative Projects and Community Integration51:23 Final Thoughts and Recommendations

Did you know that pre-roll podcast ads cost half as much and drive twice the ROI of mid-roll ads? That's just one of the many insights today's guest brings to the table. Listen in to understand the intricacies of podcast advertising with Amila Coomber, Head of Marketing & Growth at Podscribe. We explore everything from the technicalities of measuring podcast ad effectiveness, the evolution of podcast advertising, the role of pixel-based measurement, and the differences between host-read and dynamic ads. Amelia also shares valuable insights into the future of podcast advertising, including the potential impact of video on platforms like YouTube.Links from the show:Amelia Coomber on LinkedInPodscribeQuarterly (Podcast Advertising) Performance Benchmark Reports00:00 Introduction and Nickname Banter01:28 Discussing Podcast Ads02:17 Introducing the Guest05:01 Understanding Podcast Advertising14:32 Technical Aspects of Podcast Ad Measurement26:16 The Rise of Podcast Advertising26:38 Simulcast and Pixel-Based Attribution27:20 YouTube's Role in Podcasting27:59 Challenges in Measuring Podcast Ads31:10 Programmatic Advertising and Future Trends33:38 Dynamic Feedback Loops and MMM44:22 Creative Strategies in Podcast Ads45:56 Insights and Future of Podcast Measurement47:32 Final Thoughts and Incremental Insights

What do PMax, Advantage+ and an 8-year-old in an arcade have in common?They'll spend as much as they can, if you let them!In this episode, join Jim and Simon as they dive deep into the world of PPC with Navah Hopkins, a seasoned expert with over 16 years in the marketing industry. Navah shares her insights on Google’s Performance Max (PMax) campaigns, discussing the intricacies of managing them, excluding branded searches, and overcoming broader measurement challenges.Links from the show:Navah on LinkedInOptmyzr00:00 Vacation Plans and PPC Jokes01:25 Introducing Today's Guest: Navah Hopkins03:21 Diving into Performance Max Campaigns06:23 Challenges and Strategies in PPC10:42 Understanding Performance Max and Asset Management16:54 The Role of Ad Strength and Measurement in PPC24:16 Incrementality Testing Challenges24:35 Meta's Advantage Plus and A/B Testing25:06 Google's Approach to Testing and Privacy25:57 Geo-Based Holdout Testing26:24 PMax Campaigns and Branded Search27:46 Testing Strategies and Market Considerations28:29 Google's Targeting Limitations28:55 Branded Search in PMax33:05 Data Teams and PPC Collaboration38:16 Onboarding and Training for Measurement Practitioners47:45 Creative and Data-Driven Marketing49:47 Productivity Tips and Metal Music

Billboards, transit posters, digital signage - these are just a few of the options for out-of-home (OOH) advertising. OOH is a format that as digital marketers and analysts, we don’t typically consider. But when it comes to measuring billboards and other OOH channels, how exactly do we do that?We brought on an expert at the cutting edge of OOH to answer this question, and bring light to how OOH is changing in today’s digital world.Ty Tinker is Head of Analytics at AdQuick - a company that is modernizing the out-of-home advertising space. We talked with him about the finer details of OOH measurement, including the past, present and future capabilities.📺 ⁠Watch on YouTubeLinks from the show: Ty on LinkedIn AdQuick MeasureUp Ep. 6 with Greg Dolan MeasureUp Ep. 22 with Nick Stoltz AdQuick Madvertising Podcast Billboard Cost Calculator

What's MeasureCamp, you ask? Oh, do we have a treat for you! Buckle up as Simon and Jim talk with MeasureCamp founder, Peter O'Neill about this global 'unconference' movement focused on digital analytics. Learn what draws people to MeasureCamps around the world, and about the critical things you must get right if you want to start a measurement community of your own.📺 Watch on YouTubeLinks from the show: Get your free MeasureCamp New York ticket! MeasureCamp.org Peter on LinkedIn