SEO Then & Now: Marshall Simmonds on AI, Dark Traffic , LLM.txt & the Future of Search π
Description
https://page2pod.com - What happens when the gatekeepers of search stop sending traffic, but still take the content? In this milestone Episode 90, Marshall Simmonds (Defined Media Group) joins us to trace SEOβs journey from white-text-on-white-backgrounds to log-file analysis, AIO, and LLM.txtβplus what publishers must do right now. He even shares the wild story of spinning up a mirror test server so Google (and others) could pound on About.com to refine their crawlers.
We dig into why watching server logs matters more than ever, how βdark trafficβ is back, and why author authority and structured content still winβdespite shifting consumption and murky attribution in AI-assisted search.
Marshall also takes us behind the scenes on unlocking The New York Times archive (and why it crushed for traffic and revenue), plus a pragmatic take on AIO monetization, blocking AI training vs. real-time access, and the early but evolving role of LLM.txt.
π§ In This Episode:
β’ The About.com βtest serverβ era: letting engines crawl a mirror site to improve their botsβwhat Marshall learned by βwatching the watchers.βΒ
β’ Why log files are mission-critical again as dark traffic surges and LLM crawlers multiply.Β
β’ Enterprise-grade log tooling & options: Sitebulb, Screaming Frog, Splunkβwhat to use when.Β
β’ Crawl budget signals hiding in plain sight: the Search Console βCrawl statsβ report.Β
β’ NYT archive strategy: opening history for scale, membership, and revenue (and why it worked).Β
β’ E-E-A-T before the acronym: building human authority into content and authorship.Β
β’ LLM.txt today: yes, itβs crawled; no, standards arenβt settled; how teams are exposing it.Β
β’ AIO reality check: impressions up, clicks down; whatβs converting (and why patience pays).Β
β’ Blocking AI training vs. real-time use: robots.txt limits, legal gray zones, and first-line defenses.Β
β’ Holistic marketing > channel silos: brand, email, social, search, Discoverβworking together.
If you've ever wondered where SEO has been and where it's headed, this conversation with Marshall Simmonds is essential listening.
π Subscribe for more insightful episodes on SEO, AI, and digital strategy.
π¬ Comment below: How are you adapting your SEO strategy in the age of AI and dark traffic?
π Links & Resources Mentioned
β’ Marshall Simmonds LinkedIn
β’ Defined Media Official Site
β’ Page 2 Podcast Noah Learner episode
Sponsored by Moving Traffic Media
Follow the Page 2 Podcast
Follow Jon Clark
Β