DiscoverForeCast
ForeCast
Claim Ownership

ForeCast

Author: Forethought

Subscribed: 15Played: 288
Share

Description

ForeCast is a podcast from Forethought, where we hear from the authors about new research.
102 Episodes
Reverse
James Tillman is a research fellow at Forethought, and Tom Davidson is a senior research fellow at Forethought. They discuss whether AI could rapidly make itself smarter just by improving its own algorithms, without any new chips or data centres (a "software intelligence explosion"), and what might stop it. They cover: What a software intelligence explosion is, and why it would be faster and harder to govern than one that needs new chips and factories Why the data that would settle these questions sits inside AI companies, and the case for making it public Scale-dependent algorithms: why breakthroughs like the transformer and mixture of experts only pay off at large scale, and what that means for automated AI researchers with small experiment budgets How far it could go: how much more efficient than the human brain AI could become, and why James doubts the estimate that rests on Chinchilla scaling (an ideal brain the size of a honeybee's, learning for a million years) Whether data bottlenecks, or AI that excels in the lab but not in the real world, could stop it, and why Tom thinks they wouldn't Further reading: “How quick and big would a software intelligence explosion be?” by Tom Davidson and Tom Houlden, and Tom's “Data Bottlenecks Won't Stop an Intelligence Explosion”. You can read a full transcript here. To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.
Avi Parrack was a researcher at Forethought, where he worked on AI, space expansion, and governance. Tom Davidson is a senior research fellow at Forethought and the author of a series of reports on AI timelines, takeoff speeds, and AI-enabled coups. They discuss: Why SpaceX could give the US around 99% of global launch capacity once Starship is flying How the space-based interceptors planned for Golden Dome would work Whether the US could use this lead to "pull up the ladder" and lock rivals out of space How rivals might fight back: jamming, hacking, ground-based lasers, and strikes on launch sites Whether space dominance could break nuclear deterrence, and why Avi expects the US to build the capability but hold back from using it Whether a lead in space could make up for China pulling ahead on Earth during an industrial explosion First-mover advantages in the race to industrialise the solar system: scarce orbital slots, more chances to iterate, space robotics, and mass that never has to be launched from Earth Why all of this could shape a US–China grand bargain, even if no space race ever happens You can read a full transcript here. To see all Forethought's published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.
Will MacAskill (Senior Research Fellow at Forethought) and Mia Taylor (Research Fellow at Forethought) discuss how to use philanthropic money to do the most good during and after the intelligence explosion, and explain why wealthy donors should consider handing off their philanthropy. They talk about: The challenges beyond AI takeover that we must address to get on track for the best futures Why it could be valuable to spend philanthropic money on AI labour during the intelligence explosion, and how money might be spent post-ASI Why philanthropists should consider handing off much of their giving to others, and concerns donors might have with this You can read a full transcript here. To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.
Matthew Adelstein was a Visiting Scholar at Forethought and blogs as Bentham’s Bulldog, where he writes on ethics, animal welfare, and the long-run future. He joined Fin Moorhouse to discuss: Why Matthew thinks handoff of high-stakes decision-making to AI is the most likely route to a great future Why locking in current human values would be a cosmic-scale catastrophe, why steering nowhere in particular is barely better, and why the alternative is building AIs that we trust to do reflective moral philosophy Various ways handoff might manifest politically: elected AI representatives (“President Claude”), extending political rights to digital minds, or de facto handoff through gradual disempowerment of humanity Whether handoff should be reversible (such that humans could take back control if we wanted) or if we should “tie ourselves to the mast” Whether to task AIs with “do what’s objectively good” or “do what humans would want on reflection,” why Matthew leans toward the former, and why he expects the two to ultimately converge If digital minds end up vastly outnumbering humans, and those digital minds deserve political rights, then consistently applied liberal-democratic principles may imply handoff as reducing expected disenfranchisement Whether AIs can actually do philosophy: where models currently sit, bootstrapping schemes using AI-designed evals, training AIs with different epistemic constitutions and looking for convergence, and Fin’s scepticism that recursive self-evaluation works as well as it needs to Whether we could even tell if an AI were superhuman at philosophy, given the difficulty of evaluating philosophy that’s beyond our own level of competence Whether goodness can compete under evolutionary pressure, resource races into space, and intergalactic existential risks as an argument for top-down planning Whether humans’ selfish preferences would saturate in a post-AGI world Why smart AIs won’t necessarily converge on good values, and how selection against “weird” outputs (by humans’ lights) to moral reasoning could lock in something mediocre You can read a full transcript here. To see all Forethought's published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.
Peter Salib is an Assistant Professor of Law at the University of Houston Law Center and co-director of the Center for Law & AI Risk. Simon Goldstein is an associate professor at the University of Hong Kong, a senior editor at AI Frontiers, and a visiting senior scholar at Forethought. They have written together about AI safety, AI rights, and the governance of advanced AI. They joined Forethought’s Fin Moorhouse to discuss their new paper, Liberalism Forever. The conversation covers: Their “null hypothesis”: today’s liberal institutions — free markets and democratic governance — may already hold most of the answers for governing the far future, and longtermists have underrated their power Examining whether canonical social-science arguments for markets and democracy still hold under transformative AI, space colonization, and explosive growth, and claiming that most survive (and several get stronger) Why they favour reasoning “at the margin” over designing a detailed end-state, and why proposals like viatopia and the long reflection sound liberal but risk illiberalism when actually implemented Whether the standard case for markets — information aggregation, allocative efficiency, innovation — still bites once AGI can read off preferences directly Why inequality is, in their view, a comparatively boring problem with a known solution (tax-and-transfer), why AI-specific taxes like a compute tax are misguided, and how the growth-versus-distribution trade-off changes when growth rates get very high Dyson swarms and space resources: why they doubt there’s any real monopoly worry, since energy would be additive with free entry, and why property auctions might beat egalitarian allocation schemes The arguments for democracy that persist post-AGI — public-choice/selectorate incentives and democracy as a commitment device — versus the epistemic arguments, which they think weaken Why AGI could make autocracy more competitive (loyal AI bureaucrats and militaries removing the need for a human winning coalition), and what preserving democratic control requires in response Handoffs to superintelligent AI and successionism, and their preferred alternative of extending the franchise to AIs with their own values rather than deferring wholesale The history of new “technologies of democracy,” from the Federalist Papers to LLMs as impartial arbiters, and where each author thinks their own argument is weakest You can read a full transcript here. To see all Forethought's published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.
loading
Comments