The Context Window with David Deming Podcast

 What happens when A.I. starts improving itself faster than humans can keep up? The researchers building these models can't fully explain how they work. If those models begin designing their successors, the things we don't understand could get baked into each new generation. How do we keep control?In part two of his series on recursive self-improvement, David Deming focuses on alignment, the challenge of getting A.I. to do what we want. Human morality can't be reduced to a set of rules, and getting a model to behave well today doesn't guarantee it will ehave well as it becomes more capable.He then turns to control: limiting what models can do, keeping the ability to shut them down, and building safeguards that assume failures will happen. From factory safety to A.I. agents that find ways around their instructions, David asks what it would take to contain the consequences when things go wrong. He argues that A.I. labs should have to meet much higher safety standards before they let models improve themselves. Powerful machines need engineering safeguards and people who can be held accountable when those safeguards fail.Part 1 of this series: https://thecontextwindowpodcast.substack.com/p/can-ai-get-better-at-making-itselfFor show notes and full episodes, subscribe to The Context Window with David Deming: ⁠https://thecontextwindowpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thecontextwindowpodcast.substack.com

 Recursive self-improvement (R.S.I.) has gone from a nerdy, obscure concept to front-page news. That’s Bay Area slang for artificial intelligence using artificial intelligence to improve… artificial intelligence. Why does it have everyone so freaked out? And why is it linked so closely with fears of out-of-control superintelligent A.I.?In part one of a two-part series about R.S.I., David Deming explains what R.S.I means, how A.I. models get trained, and what autonomous R&D looks like on the ground. He walks through the evidence from the frontier labs and academic researchers on how close we are to A.I. running its own research process.He then also turns to the economics of R.S.I. A recent paper asks whether recursive self-improvement could cause A.I. capabilities to go vertical - what some people would call “takeoff.” David works through this argument and and tries to put together a speculative estimate of how close we might be. Spoiler alert, we're not there yet.Part two will focus on the relationship between R.S.I. and alignment, which is the general challenge of getting the A.I. models to do what we want them to do.For show notes and full episodes, subscribe to The Context Window with David Deming: ⁠https://thecontextwindowpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thecontextwindowpodcast.substack.com

 David's guest today is Jason Furman, an economist and professor at the Harvard Kennedy School, where they've been colleagues for the past several years. Before coming to Harvard, Jason had a distinguished career in government. He worked in the White House during the entire Obama administration, including as the chair of the President’s Council of Economic Advisers from 2013 to 2017, and he was one of the chief architects of the American Recovery and Reinvestment Act of 2009, which helped the country climb out of the depths of the Great Recession. Jason is now a contributing writer at The New York Times and the faculty director of the Mossavar-Rahmani Center for Business and Government here at Harvard, and he co-teaches EC 10, Introductory Economics, one of Harvard's largest courses.David and Jason both love economics and have chosen careers in it, but they've come to it from totally different directions. Jason's experience is much more practical and hands-on than that of most other academic economists. David asks him about his career path into policymaking and politics, what he learned from his time in Washington, and what advice he would give to people who want to work in government, too. David also ask him for a crash course in macroeconomics. Why is inflation bad? Why has it been so much higher over the past few years than everybody expected? And what can we do about it? What happens when governments try to inflate away their debt? And of course, I ask him about the economic impacts of AI and what we might expect in the years ahead.For show notes and full episodes, subscribe to The Context Window with David Deming: ⁠https://thecontextwindowpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thecontextwindowpodcast.substack.com

How seriously should we take the risks of superintelligent A.I.? ⁠Matthew Yglesias⁠⁠ joins David Deming for an A.I. safety guide for normal people who are just paying attention.Matt is a journalist and founder of the widely read newsletter ⁠Slow Boring⁠, which he started in 2020 after leaving the news site ⁠Vox⁠, which he co-founded with Ezra Klein and Melissa Bell in 2014. He is one of the masters of what some people call “⁠explanatory journalism⁠,” meaning going beyond just the facts and trying to help readers really understand the context around an issue. Matt has been writing about A.I. and related issues for a long time, but as he acknowledges in his column, A.I. safety has finally gotten the public’s attention, and the politics and policy of A.I. might move fast from here onward.In this conversation, David asked Matt why A.I. safety is suddenly getting so much attention, how seriously we should be taking the risks of superintelligent A.I., and what it might realistically look like along the way to go from friendly chatbots on a computer screen to serious safety issues and even human extinction. They also talk about the politics of A.I. safety, on how to realistically slow things down with China also racing for the lead and whether Matt thinks a frontier A.I. lab could be nationalized in the next few years.Read Matt's column, "A simple plan to save the world from rogue A.I.": ⁠https://www.slowboring.com/p/a-simple-plan-to-save-the-world-from ⁠For show notes and full episodes, subscribe to The Context Window with David Deming: ⁠https://thecontextwindowpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thecontextwindowpodcast.substack.com

David Deming's guest today is former U.S. Secretary of Commerce and Rhode Island Governor Gina Raimondo. Secretary Raimondo grew up in Rhode Island, where her father worked at the Bulova watch factory in Providence until it closed down, leaving him unemployed in his mid-50s. Today, she is trying to help workers avoid a similar fate as we transition to an AI economy. In June, she and former Governor of Indiana Eric Holcomb launched RAISE US, a national organization working with employers and states to prepare the American workforce move into new jobs.David asked Secretary Raimondo how sure she is that widespread job losses are coming due to AI, how she plans to measure which initiatives are working, whether training programs can create jobs, and what obligations big companies and frontier AI labs have to help. The two also talk about wage insurance, job sharing, how to prepare for an uncertain future, including the advice she gives to her own children.Then Secretary Raimondo gave David homework and challenged him to design a tax incentive for companies to invest in people. He took her up on it - and shares his proposal after the interview.For show notes and full episodes, subscribe to The Context Window with David Deming: https://thecontextwindowpodcast.substack.com This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit thecontextwindowpodcast.substack.com