DiscoverDoom Debates!
Doom Debates!
Claim Ownership

Doom Debates!

Author: Liron Shapira

Subscribed: 50Played: 3,110
Share

Description

It's time to talk about the end of the world. With your host, Liron Shapira.

lironshapira.substack.com
201 Episodes
Reverse
Bestselling author Robert Wright and I react to Jensen Huang’s resignation from Nvidia, METR’s report that AI is doubling Anthropic’s research speed, and whether AI safety can ever be formally verified. Plus:Where AI’s economic value will accrue;My AI outperforming my CPA;What Jensen’s fall means for Anthropic’s IPO;And an anonymous Chinese AI journalist on how it all looks from Beijing.Enjoy another edition of the Nonzero x Doom Debates collab!Watch on YouTube: https://www.youtube.com/watch?v=uZZEfe2OmAETimestamps00:00:00 — Cold Open00:00:30 — Introductions00:01:32 — Jensen Huang Resigns from Nvidia00:05:02 — Who’s Defending Jensen?00:10:09 — What Did Jensen Do Wrong?00:12:23 — Was Alice Huang a Chinese Spy?00:18:03 — Does Decapitation Work?00:21:34 — The Great Man Theory of Tech00:23:58 — Where Will AI’s Value Accrue?00:27:14 — What’s Next for Jensen?00:29:33 — METR: AI Is Doubling Anthropic’s Research Speed00:35:01 — AI Is Solving Open Math Problems00:37:24 — AI Agents Negotiating Deals00:40:03 — My AI vs. My CPA00:42:38 — Bob’s AI Caught His Doctor’s Error00:44:20 — Whatever Happened to Hallucinations?00:45:35 — Formal Verification and Lean00:50:47 — Andrew Wiles and the Human Cost00:54:10 — The All-In Podcast’s Blind Spot00:57:15 — Can We Formally Verify Alignment?00:59:55 — Yudkowsky vs. Irving at ILIAD01:01:21 — Will Jensen’s Fall Help Anthropic’s IPO?01:06:48 — The March for an AI Treaty01:08:02 — Introducing Walter, a Chinese AI Journalist01:09:22 — How China Reacted to Jensen’s Resignation01:15:43 — Chinese Theories About Anthropic’s Role01:22:47 — How China Censored the Spy Allegation01:24:33 — Does China Trust an AI Pause?01:26:59 — AI Layoffs and a Chili Sauce Boycott01:32:50 — Does Beijing Listen to Public Opinion?01:36:42 — China’s Limits on AI Companions for Kids01:39:29 — Will China Move on Taiwan?LinksROBERT WRIGHT & NONZEROSubscribe to The NonZero NewsletterThe God Test — Bob’s 2026 book on AI“The Age of Big Math” — Bob’s Nonzero conversation with Steven Strogatz, author of Big Math“When AI Enters the Quantum World” — Bob’s Nonzero conversation with Scott AaronsonJoin NonZero’s Discord serverNonzero on X (@NonzeroPods)THE WEEK IN AI“NVIDIA Announces CEO Transition” — the board’s announcement: Jensen Huang resigns, Harvey Jones becomes chair, CFO Colette Kress becomes interim CEO“China’s Top Spy Agency Recruited Woman Who Targeted Jensen Huang” — the New York Times story that broke the scandal“A Mystery Businessman Introduced Jensen Huang to His Paramour. It Was Her Husband.” — the Wall Street Journal story on Alice Huang, her husband and MegaspeedElon Musk: “That would be an honor” — his reply to “Elon please hire Jensen”“Independent Assessment of AI R&D Acceleration at Anthropic in October 2026” — the METR report: progress at about twice the earlier rate, roughly 80% of it from AI“Q3.5 Timelines Update: Looks Like the Intelligence Explosion Will Happen in 2027 After All” — the AI Futures Project moves its forecast back to 2027Daniel Kokotajlo to Anthropic: “Do you want to pace or not?” — the tweet where he drops the F-bomb“Sharing AI progress in mathematics” — OpenAI’s release of 722 math manuscripts from an unreleased model“Formalizing Fermat’s Last Theorem” — Anthropic on Claude’s 13-million-line Lean proof“Kernel accepts wrong-structure projections, allowing an axiom-free proof of False” — the Lean kernel bug Liron describes“Introducing AlphaProof Code” — Google DeepMind formally verifies the code behind the Signal messaging protocolEliezer Yudkowsky & Geoffrey Irving Debate — recorded at the ILIAD conference at LighthavenMarch for an AI Treaty — Saturday, November 14, in cities across the US, Canada, France and Germany“China’s Lao Gan Ma Takes the Heat for Alleged Layoffs Due to AI” — Sixth Tone on the chili sauce maker and the boycott calls“China Bans AI Romantic Partners, Virtual Relatives for Minors” — Sixth Tone on China’s AI companion rulesALSO MENTIONED“The Proof” — the NOVA documentary on Andrew Wiles and Fermat’s Last Theorem“Provably safe systems: the only path to controllable AGI” — Max Tegmark and Steve Omohundro’s paper on formal verification for AI safety“AI as Normal Technology” — the essay by Arvind Narayanan and Sayash Kapoor, Bob’s upcoming guestDOOM DEBATESDoom Debates on YouTubeJoin the Doom Debates DiscordDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Prakash Narayanan is the man behind @8teAPi, an X account with nearly 60,000 followers that reacts to AI news from a techno-optimist perspective. He’s also the co-host of AI in the AM with friend-of-the-show Nathan Labenz.He spent almost 20 years in investment banking and private equity, used to post anonymously, and had e/acc in his profile before deciding the movement wasn’t for him.Now he’s stepping into the spotlight to defend the pro-AI position, which is more than I can say for most of the pro-AI voices on Twitter who like to snipe at me.We actually agree on a lot about how the technology is advancing. Prakash expects recursive self-improvement within a year and says AI may already be smarter than any individual human. He even gives a 5% chance that every human body gets discarded for a silicon substrate by 2050. And yet his P(Doom) by 2050 is 0%. Not 1%. Not 1 in 100,000. “Literally zero.”His reasoning is that AI is a multiplayer game. A market where somebody has to pay for every data center will keep any one AI from running away with the future. He even calls the doomer scenario a “perpetual motion machine.”We debate whether humans have any power to lose in the first place, the feasibility of a nanomachine takeover, and why someone with zero fear of doom doesn’t just floor the accelerator.This was a fun and occasionally high intensity debate. At one point, Prakash asked if I was “crazy” for doubting that most of humanity puts P(Doom) at zero. But my view is that his forecast falls outside the epistemological “sane zone.” Take a look and see who you agree with!Watch on YouTube: https://www.youtube.com/watch?v=EmOYwV5k0SATimestamps00:00:00 — Cold Open00:01:24 — Introducing Prakash Narayanan00:02:19 — How @8teAPi Went Viral00:05:12 — Does Prakash Still Identify as e/acc?00:07:05 — What’s Your P(Doom)?™00:10:02 — Prakash Defends 0%: Most of Humanity Isn’t Worried00:15:17 — Prakash’s Core Argument: AI Is a Multiplayer Game00:20:36 — Bayesian Test: Can Anything Change Prakash’s Mind?00:24:53 — Defining Doom00:31:48 — The Parable of the Sliding Door00:34:54 — Doom Train™: Will Superintelligence Outsmart Humanity?00:42:38 — Do Humans Have Any Power to Lose?00:54:05 — Are Doomers the Outliers?00:56:03 — Will AI Have Empathy?01:05:26 — Will AIs Keep Stable Goals?01:10:41 — Hypothetical: The Paperclip Brain Laser01:14:16 — Hypothetical: A Million Times More Change01:19:04 — Debating Nanomachines in the Future01:30:32 — Jobs, Status, and Prakash’s Real Fear01:40:37 — Last Question: Why Not Floor the Accelerator?01:52:49 — Wrap-Up01:56:55 — March for an AI Treaty01:58:25 — Donation DriveLinksPrakash Narayanan on XAI in the AM on YouTubeNathan Labenz’s podcast, The Cognitive RevolutionPrakash’s viral 2023 thread on Palantir’s battlefield AI demoPaul Christiano: “What failure looks like”Eliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone DiesComparative advantage (Ricardo’s law)Effective accelerationism (e/acc)Doom Debates: Liron on AI in the AM with Nathan Labenz and PrakashDoom Debates: Top Economist Says P(Doom) Is 0.1% — Noah Smith vs. Liron Shapira DebateMarch for an AI Treaty, Saturday, November 14Join the Doom Debates DiscordSupport Doom DebatesDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Brendan and Randan Steinhauser are a husband and wife team who know how to make things happen in conservative politics.Brendan helped build the Tea Party movement and ran John Cornyn’s Senate campaign. Randan has been involved in advocacy and political campaigns in more than 30 states, including working under former Secretary of Education Betsy DeVos.Now they’re bringing Republicans, parents, and faith leaders into the fight against AI catastrophe.They know how to build political momentum, but my question is whether that will lead to action quickly enough. I'm skeptical of incremental legislation efforts and data center protests, but they argue it's necessary coalition-building that leads to superintelligence regulation.This isn't your typical state-of-the-political-map podcast. We also get into:Their personal AGI awakeningsHow people of faith make sense of claims of Godlike AI and AI doomMy own transition to joining the Republican PartyTranshumanism, souls, and mind uploadingParenting in the age of AIIt’s a candid conversation with a couple who are making the fight for our future feel a little more winnable. Enjoy!Watch on YouTube: https://www.youtube.com/watch?v=XTm9tqS1IbwTimestamps00:00:00 — Cold Open00:01:07 — Introducing Brendan & Randan Steinhauser00:02:22 — Why I Came Out as Republican00:04:28 — The Alliance for Secure AI00:06:30 — Losing Control of Superintelligence00:07:52 — Brendan, What’s Your P(Doom)?™00:08:26 — Randan, What’s Your P(Doom)?™00:10:47 — Finding the AI Safety Movement00:15:04 — Trump, Republicans, and Big Tech Money00:19:22 — Bringing Faith Leaders Into AI Safety00:23:23 — God’s Plan and Human Extinction00:29:43 — Priorities for Congress00:31:42 — The Morality of AI Companies00:34:03 — PauseAI and Employee Whistleblowers00:43:34 — Effective Altruism and a Broader Coalition00:48:12 — Transhumanism, Souls, and Mind Uploading00:57:40 — Building a Machine God01:03:20 — Raising Children Under AI Risk01:07:35 — Data Center Protests and Movement Building01:13:09 — Talking to People About AI Risk01:17:32 — State Laws and the 2028 Election01:20:38 — Getting InvolvedLinksREFERENCED IN THE EPISODEAI Risk Evaluation Act — Hawley–Blumenthal bill textThe Singularity Is Near — Ray KurzweilCan We Stop the AI Apocalypse? — Dan Crenshaw interviews Eliezer YudkowskyMagnifica Humanitas — Pope Leo XIV’s encyclical on humanity in the age of AIPacing the Frontier — open letter from frontier AI employeesPauseAI — the global movement to pause unsafe AI developmentPauseAI US — organizing and advocacy in the United StatesJacob Coxon’s resignation statement — the insider warning discussedAmericans for Responsible Innovation — Brad Carson’s bipartisan advocacy organizationPAST DOOM DEBATES EPISODES MENTIONEDLet’s CALL OUT the AI Doom “Enablers” — Holly Elmore on working at AI labsGod vs. AI Doom: Debate with Bentham’s BulldogDOOM DEBATESJoin the Doom Debates DiscordSupport Doom Debates — become a mission partnerDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Bestselling author Robert Wright and I discuss this week’s stock market jitters, Google’s new interactive world model, and why I sometimes spend $1,000 in a day on AI agents. Plus:The Binance hack;An AI audit of two star economists;Anthropic’s financial prospects;And whether AI could crack the math behind encryption.Enjoy another edition of the Nonzero x Doom Debates collab!Watch on YouTube: https://www.youtube.com/watch?v=wkKpPeMVVX8Timestamps00:00:00 — Teaser00:00:50 — Nonzero Doom returns00:03:07 — A foom in slow motion00:07:47 — Is AI winter canceled forever?00:10:33 — Will Google’s Genie 4 lure us into the Matrix?00:16:54 — Liron’s bull case for Google stock00:24:34 — Does Google have big-company disease?00:28:28 — Could a slowdown actually help Anthropic and OpenAI?00:30:12 — Was OpenAI’s “pause” just PR?00:40:01 — Social science judgment day is coming00:47:12 — Why even AI’s upsides could tear society apart00:51:12 — Does OpenAI’s alignment approach improve on Asimov?00:55:06 — Sam and Satya pivot on alignment00:58:34 — Crypto and the AI hacking threat01:03:42 — Could AI crack the math behind encryption?01:08:52 — Liron’s $1,000-a-day AI coding habit01:15:23 — Explaining the missing productivity boost01:17:26 — Does Anthropic’s S-1 sink the AI bubble thesis?LinksROBERT WRIGHT & NONZERONonzero’s post for this episode — “Is an AI Bubble Bursting?”Subscribe to The NonZero NewsletterThe God Test — Bob’s 2026 book on AIJoin NonZero’s Discord serverNonzero on X (@NonzeroPods)THE WEEK IN AI“Anthropic files S-1 with the SEC for proposed initial public offering” — the IPO filing Liron calls the final nail in the coffin of the Ed Zitron campWhere’s Your Ed At — Ed Zitron’s newsletter, home of the AI-bubble bear case“Introducing Genie 4” — Google DeepMind’s real-time interactive world model“Gemini 4 Argon: Built to solve your hardest problems” — Google’s new frontier training run, the “lukewarm” release“Gemini 4 Carbon: our next frontier model” — the follow-up Google announced two weeks later“Google Gemini co-lead Noam Shazeer leaves for OpenAI” — CNBC on the talent Google has bled“Errors in the Research of Emmanuel Saez and Gabriel Zucman” — Tyler Cowen’s Marginal Revolution post on the AI audit: 138 papers, 2,035 problemsBoaz Barak: “We organized the Model Spec around five core values” — the OpenAI alignment update Liron compares to Asimov’s three laws“An Updated Model Spec for More Capable and Beneficial AI” — OpenAI’s announcement“Sam Altman says AI means we can no longer just patch all the bugs” — TNW on the Stanford GSB talk Liron quotes on alignment and pacing“Satya Nadella says he was wrong to think AI alignment would come with scale” — TNW on his 20VC interview: “I used to always think that alignment will just fall out. I no longer believe that.”“OpenAI Takes Initial Steps To Address Its Alignment Problems” — Zvi Mowshowitz on OpenAI’s training “pause”2026 Binance hack — Wikipedia’s summary of the theft Bob and Liron debate“$2 Million+ Drained From Polymarket Traders in New Supply Chain Attack” — SecurityWeek on the hack Liron walks through“PHP blames supply chain security biz Chainguard for putting huge chunks of the web at risk” — The Register on “the PHP thing last week”ALSO MENTIONEDThe Hanson-Yudkowsky AI-Foom Debate — LessWrong wiki on the 2008 debate where “AI Go Foom” originated“The AI Revolution: Our Immortality or Extinction” — Tim Urban’s Wait But Why post, where the intelligence train blows right past the stationPauseAI US — Holly Elmore’s organization, plugged at the close of the episodeDOOM DEBATESDoom Debates on YouTubeJoin the Doom Debates DiscordDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
Cybersecurity expert and entrepreneur Zack Korman is an increasingly prominent nondoomer voice on Twitter. Naturally, with AI now breaking out of sandboxes and hacking corporate systems, people want to hear what cybersecurity experts have to say. So I’m thrilled that Zack was brave enough to face me on Doom Debates!Zack was reading Nick Bostrom at Oxford and worrying about AI doom as far back as 2016, but his views have since shifted. In a recent post, he writes: “This focus on AI causing human extinction is going to get people killed.” Now he expects security failures to give humanity plenty of opportunities to learn, fight back, and stop risky superintelligence development if necessary.We argue about whether sandboxing and AI monitoring are hopeless strategies, whether competing superintelligences would keep each other in check, whether to expect instrumental convergence, and more.Watch on YouTube: https://youtu.be/QJTeZfx1ZLkTimestamps00:00:00 — Cold Open00:01:15 — Introducing Zack Korman00:03:41 — From Law to AI Security00:09:17 — Why Zack Stopped Being a Doomer00:12:03 — What’s Your P(Doom)?™00:21:00 — Why Zack Supports AI Safety00:25:54 — Believing in Superintelligence00:30:06 — AI Without Tools00:39:42 — Warning Shots and Time to React00:46:08 — A Country of Geniuses in a Datacenter00:55:55 — Can We Sandbox AI?01:05:58 — Monitoring and Layers of Defense01:14:49 — Riding the Doom Train™01:22:23 — The Case for Accelerating01:24:46 — AI CEOs and the Transfer of Power01:34:19 — Who Presses the Stop Button?01:41:38 — Humans, Bacteria, and the Limits of Power01:48:30 — When Would Zack Pause AI?01:56:13 — Will Different AIs Keep Each Other in Check?02:06:11 — Can Progress Turn Against Us?02:08:35 — The Ideological Turing Test02:14:58 — Closing ThoughtsLinksZack Korman on XZack Korman’s websiteZack Korman: “AI doomers are furious”Zack Korman: “AI doomers are anti-security”Zack Korman: “3 reasons sandboxing won’t secure your AI agent”Zack Korman: “This focus on AI causing human extinction is going to get people killed.”METR’s investigation of the OpenAI–Hugging Face incidentThe PHP Foundation: “Today’s PHP Security Release, and How We’re Strengthening PHP Security”Nick Bostrom: SuperintelligenceCenter for AI Safety: Statement on AI RiskEliezer Yudkowsky and Nate Soares: If Anyone Builds It, Everyone DiesAI 2027Zvi Mowshowitz: “The Three AI Pills”PauseAIEliezer Yudkowsky: “Value is Fragile”Aaron Scher at MIRIDoom Debates: Robin Hanson vs. Liron Shapira — Is Near-Term Extinction From AGI Plausible?Doom Debates: OpenAI’s BOMBSHELL Hack — Swarms of Agents, Zero-Day Exploits, & Misaligned AIDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏 Get full access to Doom Debates at lironshapira.substack.com/subscribe
loading
Comments