AI Article Readings

The AI Futures Project team, the authors of AI 2027, revisit their forecasts after METR’s report that AI is now speeding up AI progress at Anthropic. They work through a set of simple extrapolations and their own AI Futures Model to ask when a Superhuman AI Researcher might arrive, look hard at where their earlier model went wrong, and then each of the six authors gives an all-things-considered forecast in their own words, before the team sets out what it would mean for alignment, security and international agreements.* 00:00 - Introduction* 03:35 - Timelines/Takeoff Update Summary* 16:26 - METR’s estimates imply that progress is currently sped up by 1.7x due to AI* 22:32 - 1.7x acceleration of overall progress is a much bigger deal than a 1.7x labor uplift* 27:46 - A 1.7x pace of progress is 5 months ahead of the AI 2027 scenario* 29:42 - Modeling* 29:44 - Operationalization* 32:23 - Simple models* 32:26 - Research taste linear extrapolation* 32:36 - Basic method* 37:46 - One-at-a-time parameter changes* 43:00 - Monte Carlo* 46:16 - Labor uplift extrapolation* 49:01 - Extrapolating Anthropic automation levels* 51:54 - Extrapolating software R&D uplift per unit of ECI* 01:00:32 - Tom Davidson’s toy model* 01:01:24 - Filtered AI Futures Model* 01:06:11 - Filtering for a single acceleration level* 01:11:00 - Filtering for acceleration levels at 2 different dates* 01:13:58 - Hypotheses for why the AI Futures Model was wrong* 01:14:31 - Background* 01:15:18 - Have we achieved AC?* 01:18:27 - Why is the pace of progress acceleration so high?* 01:21:22 - More uplift from near-AC coding skills* 01:28:35 - Research taste is better than anticipated* 01:29:40 - A given level of research taste skill results in more uplift than anticipated* 01:30:16 - Skills besides coding and research taste are driving acceleration* 01:32:35 - Conversion of uplift to pace of progress is too conservative* 01:38:15 - Other considerations* 01:38:18 - How large is the (AI-attributable) acceleration in the current pace of progress, really?* 01:45:45 - Other recent sources of evidence* 01:46:43 - It’s hard to not update much on a 1.7x acceleration* 01:50:32 - Adjustments for what the filtered AIFM could be missing* 01:55:42 - All-things-considered forecasts* 01:55:46 - Forecast table* 01:57:10 - Eli’s forecast* 02:05:25 - Brendan’s forecast* 02:07:49 - Daniel’s forecast* 02:10:47 - Lukas’s forecast* 02:14:13 - Thomas’s forecast* 02:18:23 - Romeo’s forecast* 02:28:30 - Implications* 02:35:42 - Appendix: How our views have changed over time* 02:37:26 - Appendix: Ambiguity in “80% of current acceleration is due to AI”* 02:43:39 - Appendix: More analysis of higher-than-expected near-AC coding uplift* 02:49:05 - Appendix: How much would 1.7x acceleration in inputs to AI progress accelerate AI progress overall?* 02:51:34 - Appendix: A minor fix to the AI Futures Model for better modeling when there’s acceleration in present day* 02:53:33 - Appendix: TasteVal-implied taste slope, AECI basis* 02:56:24 - Appendix: Software R&D uplift extrapolation* 03:02:32 - Appendix: How many years of AI progress in 2027 do the filtered AIFM runs predict?https://open.substack.com/pub/aifutures1/p/q35-timelines-update-looks-like-the?r=67y1h&utm_campaign=post-expanded-share&utm_medium=web This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe

Quite a long technical paper, but this kind of research, I think, is worth highlighting. I understand that this is not most people’s cup of tea, but I’m putting it out into audio for those that think it is a value add. If you think this kind of accessibility boosting is important, I ask for a couple of things. One, share with your friends. Two, give me ideas for other such papers/reports that might be worthwhile converting into full-cast produced audio. I’m also very much open to working with people and / or organisations to get audio like this made for their articles and papers. I’m willing and happy to consult on audio production pipeline stuff or anything of that nature in producing episodes like these Please feel free to reach out to me, either here or via [email protected]. If world leaders decided tomorrow to pause frontier AI training, could they actually do it? This book-length working paper from an interdisciplinary group of Berkeley-led researchers, including Stuart Russell and Rose Gottemoeller, takes the question seriously and answers it in detail: a “hardwired pause” that freezes a whitelist of today’s models into inference-only chips, takes a global census of the training-capable hardware already out there, governs the concentrated chip supply chain, and asks how much compute a cheater could hide, how long the pause could last, what it would cost the economy, and what would have to change in Washington and Beijing for leaders to want it. Citations and cross-references have been left out for listening; the full paper, with its references and calculations, is at the link.* 00:00 - Introduction* 02:06 - 1 Introduction* 07:40 - 1.1 A Hardwired Pause* 17:59 - 1.2 Scope* 22:06 - 1.3 Organization* 23:11 - 2 Background: arguments for a pause and lessons for AI governance* 23:34 - 2.1 Observed arguments for a pause* 24:02 - 2.1.1 Loss of control* 26:10 - 2.1.2 Malicious use and catastrophic harms* 27:32 - 2.1.3 Economic disruption, gradual disempowerment, and concentration of power* 29:27 - 2.1.4 Destruction of a healthy information ecosystem* 30:41 - 2.1.5 Geostrategic instability* 32:08 - 2.1.6 Why a pause* 33:21 - 2.2 Historical lessons for AI governance* 34:32 - 2.2.1 Lesson 1: Governance is a three-legged stool* 36:26 - 2.2.2 Lesson 2: Governance agreements tend to focus on observable, verifiable units* 38:58 - 2.2.3 Lesson 3: Monitoring needn’t be perfect to be effective* 40:07 - 2.2.4 Lesson 4: Signatories need to benefit* 43:09 - 2.2.5 Lesson 5: Effective Institutions Are Needed* 49:28 - 2.3 Game-theoretic lessons for AI governance* 55:22 - 3 Example implementation of a hardwired pause* 55:27 - 3.1 Overview and objectives* 01:02:56 - 3.1.1 Political objectives* 01:07:40 - 3.1.2 Technical objectives* 01:11:56 - 3.1.3 Technological readiness* 01:12:47 - 3.2 Operationalizing technical requirements* 01:13:33 - 3.2.1 Inference-only and training-capable accelerators* 01:20:41 - 3.2.2 Quantitative training thresholds* 01:30:13 - 3.2.3 Verification and breakout accounting* 01:35:50 - 3.2.4 Model whitelisting* 01:42:20 - 3.3 Hardware governance* 01:42:58 - 3.3.1 Residual uses of training-capable accelerators* 01:46:37 - 3.3.2 Hardware licensing, registration, and tracking* 01:54:11 - 3.3.3 Sociotechnical verification measures for preventing illicit use of residual training-capable accelerators* 02:01:03 - 3.3.4 Governing hardware production* 02:05:01 - 3.4 Transition governance* 02:05:17 - 3.4.1 Economic alignment* 02:07:50 - 3.4.2 Phase 1 (design)* 02:11:55 - 3.4.3 Phase 2 (substitution)* 02:13:27 - 3.4.4 Traded-in training-capable compute* 02:15:00 - 3.5 Possible exit conditions* 02:17:53 - 3.6 Institutional governance* 02:27:53 - 4 Feasibility of inference-only hardware* 02:28:14 - 4.1 Quantifying training capability* 02:32:05 - 4.2 Practical inference-only accelerators* 02:36:57 - 4.3 Model-specific integrated circuits (MSICs)* 02:45:00 - 4.4 Time required to ramp inference-only hardware production* 02:54:32 - 5 Feasibility of hardware governance* 02:54:36 - 5.1 Overview* 02:58:17 - 5.2 The supply chain for accelerator chips* 03:06:35 - 5.3 Feasibility of governing chip manufacturing* 03:07:14 - 5.3.1 Feasibility of detecting undeclared facilities* 03:10:28 - 5.3.2 Feasibility of monitoring declared manufacturing facilities* 03:12:21 - 5.4 Feasibility of a global accelerator census* 03:14:23 - 5.4.1 The paper trails left by pre-pause hardware* 03:23:02 - 5.4.2 Incentives of states and private actors* 03:28:01 - 5.5 Estimating how many chips a census would miss* 03:33:51 - 5.5.1 Estimating the quantity of pre-pause hardware* 03:40:47 - 5.5.2 Estimating the missing fraction by holding type* 03:41:22 - Missing failed chips* 03:44:00 - Functional chips not declared* 03:48:23 - 6 Feasibility of verification* 03:48:27 - 6.1 Summary assessment* 03:55:04 - 6.2 Fleet thresholds for defection* 04:01:31 - 6.2.1 Method for determining governance thresholds* 04:11:38 - 6.2.2 Covert evasion fleet threshold* 04:21:30 - 6.2.3 Overt breakout fleet threshold* 04:29:28 - 6.2.4 Private evasion fleet threshold* 04:35:15 - 6.2.5 Small-scale evasion* 04:38:16 - 6.3 Training efficiency gains and governance innovation* 04:38:47 - 6.3.1 Estimates of recent historical training efficiency gains* 04:42:40 - 6.3.2 Training efficiency gains after a pause* 04:46:34 - 6.4 Quantitative ledger estimates* 04:47:42 - 6.4.1 Discount factors* 04:53:33 - 6.4.2 Transitional training-capable inference service* 04:58:52 - 6.4.3 Dark compute* 05:06:44 - 6.4.4 Inference-only accelerators* 05:10:44 - 6.4.5 Consumer GPUs* 05:15:28 - 6.4.6 Licensed sovereign fleets* 05:18:49 - 6.4.7 Other categories* 05:21:03 - 6.5 Durability assessment* 05:24:28 - 6.5.1 Durably guarding against covert evasion* 05:26:48 - 6.5.2 Durably guarding against overt breakout* 05:29:44 - 6.5.3 Extension options* 05:30:29 - 6.5.4 Lithography proliferation* 05:31:39 - 7 Implications of a hardwired pause* 05:32:26 - 7.1 Would restraints on frontier AI crash the economy?* 05:47:02 - 7.2 Reduction of inference costs and related economic effects* 05:49:40 - 7.3 Economics of MSICs* 05:52:21 - 7.3.1 MSIC adoption in a mixed MSIC-flexible world* 06:06:37 - 7.3.2 The cost of a pure MSIC world* 06:10:44 - 7.3.3 Shifting demand among whitelisted models* 06:17:12 - 7.3.4 Temporary or permanent de-whitelisting of models* 06:26:53 - 7.4 Impacts on concentration of power* 06:31:46 - 7.5 Consequences of receding knowledge cutoffs* 06:37:47 - 7.6 The need for other measures* 06:41:37 - 8 Pause-willing futures: conditions favoring international AI restraint* 06:42:02 - 8.1 Introduction* 06:45:48 - 8.2 Theoretical drivers of pause willingness* 06:47:15 - 8.2.1 Fundamental changes in the technology* 06:54:03 - 8.2.2 Changes in the international system: cooperation in the shadow of great power rivalry* 06:55:40 - Power transition theory and the security dilemma—heightened dangers open up possibilities* 06:59:31 - Hegemonic stability and the strategic value of institutional governance* 07:02:10 - Polarity and possibilities for collective action* 07:05:52 - 8.2.3 Changes in national political environments* 07:09:58 - 8.3 More than willingness: conditions for durability* 07:14:50 - 8.4 Imagining pause willing futures* 07:15:32 - 8.4.1 The expected costs of continued development have increased* 07:18:53 - 8.4.2 The perceived value of “winning the AI race” has fallen* 07:25:18 - 8.4.3 The distribution of capabilities creates support for a pause* 07:26:20 - Limitations of the conventional wisdom* 07:28:03 - When breaking new ground is hard and catching up is easy* 07:31:09 - Trust dilemmas vs. Prisoner’s Dilemmas* 07:32:24 - 8.4.4 Domestic coalitions could make reciprocal restraint politically viable* 07:36:47 - 8.4.5 Shifting power distribution and heightened risks drive multilateral restraint* 07:38:28 - 8.5 Major actors’ perspectives and incentives* 07:40:37 - 8.5.1 The view from the United States* 07:48:32 - 8.5.2 The view from the People’s Republic of China* 07:56:15 - 9 Conclusion* 07:58:47 - Reproducibility* 07:59:22 - Financial Disclosures* 08:00:49 - AI Use Disclosurehttps://hardwired-pause.ai/paper This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe

Addy Osmani’s practical guide to the current Claude lineup: where to start, when to move up or down a tier, and why the effort level is often the finer dial. It walks through cost per finished task rather than list price, the latency and context trade-offs, mixing models with subagents, advisors and cheaper workers, the API differences to test before migrating, and how to measure candidates on your own work. Both of the post’s tables are read in full.* 00:00 - Introduction* 01:40 - TL;DR* 03:31 - Pick a model by task* 06:13 - The four models side by side* 09:15 - The trade-offs* 09:29 - Capability and cost per finished task* 11:36 - Latency against accuracy* 13:39 - Context against cost* 15:20 - Mixing models* 16:08 - Switch at a natural break* 17:33 - Put subagents on smaller models* 20:38 - Escalate hard decisions to an advisor* 22:09 - Delegate bulk work to cheaper workers* 23:11 - Check compatibility before migrating* 26:22 - Measure on your own work* 27:21 - Learn morehttps://claude.dev/blog/which-claude-model-should-you-build-with/ This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe

Scott Alexander argues that putting a number on something you can’t really measure still beats going with your gut: starting from a CFAR tutorial where a student refuses to guess the odds his computer breaks, through doctors and the mammogram problem, scope insensitivity and partisan double standards, to why he would like to see QALY-style thinking tried far beyond health care.* 00:00 - Introductionhttps://slatestarcodex.com/2013/05/02/if-its-worth-doing-its-worth-doing-with-made-up-statistics/ This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe

Language is not one invention but two, says this finalist in the ACX Book Review Contest. It reads Walter Ong’s Orality and Literacy, on how writing changed what we remember, how we build sentences and stories, and even how we think, from Homer’s improvising bards and Luria’s syllogisms to the SAT. Then it looks up from the book at a world that is drifting back toward speech through the infinite scroll, and argues that if literacy is to survive, writing has to start doing what speech never could.* 00:00 - Introduction* 01:48 - 1. Carve* 03:13 - 2. Press* 21:36 - 3. Feed* 53:21 - 4. Eyehttps://open.substack.com/pub/astralcodexten/p/your-book-review-orality-and-literacy?r=67y1h&utm_campaign=post-expanded-share&utm_medium=web This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit askwhocastsai.substack.com/subscribe