DiscoverEA Forum Podcast (Curated & popular)
EA Forum Podcast (Curated & popular)
Claim Ownership

EA Forum Podcast (Curated & popular)

Author: EA Forum Team

Subscribed: 32Played: 2,137
Share

Description

Audio narrations from the Effective Altruism Forum, including curated posts and posts with 125+ karma.

If you'd like more episodes, subscribe to the "EA Forum (All audio)" podcast instead.
666 Episodes
Reverse
Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post. Subtitle: We can’t develop safety standards if we have to rely on opaque judgment This post discusses a public METR report on AI R&D acceleration at Anthropic. I did not work on this project or see the unredacted report. All opinions are my own and do not represent METR or the report authors. Thanks to Ryan Greenblatt for discussion, feedback, and help with modeling, and Daniel Kokotajlo, Eli Lifland, and Hjalmar Wjik for comments. Earlier this week, Anthropic stated in their S1 that they “may imminently meet” the threshold of “dramatically accelerated progress due to AI” as defined in their Responsible Scaling Policy. In their latest RSP (effective July 2026), say this threshold will be met when: They observe or expect overall capabilities progress to be going twice as fast as the fastest sustained rate before AI was contributing much to AI R&D. This is plausibly attributable to AI automating AI R&D, rather than acceleration in compute or other inputs. For the last few months, some of my colleagues at [...] ---Outline:(02:49) Researchers must be sped up a whole lot(06:45) This could compound on itself extremely rapidly(09:07) The industry is not prepared to handle this safely The original text contained 15 footnotes which were omitted from this narration. --- First published: November 6th, 2026 Source: https://forum.effectivealtruism.org/posts/g7DQdcpnFq65CcjP5/ai-is-dramatically-accelerating-ai-research-right-now --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
‘Like most of life's dangers, the dangers of artificial intelligence are not easily identified from the armchair.’ J Dmitri Gallow Epistemic status I’m not an expert in this field, and herein I make some sweeping generalisations educated guesses about the overall nature of its contents based on my strong impression from the subset of work that I’ve read. I’ve recently spent maybe 2 to 300 hours learning about modern AI safety technical work, and some amount of spare time over the years before that thinking about here-relevant notions such as preferences from an analytic-philosophical perspective. I intend this post as a work in progress. TL;DR: The concept of instrumental convergence comes from a reference class of pre-empirical categorical predictions with a very low success rate. In historical usage, its definition is very murky. Perhaps too murky to really say that it made any meaningful real-world predictions. I suspect its definition is fundamentally unfixable. On my best interpretation of what it could be said to have predicted, it seems to be much worse than we would want a good high-level model of AI behaviour to be. Despite these shortcomings, it still seems to be widely internalised by the EA and [...] ---Outline:(00:22) Epistemic status(00:54) TL;DR:(02:14) Intro(04:41) How is instrumental convergence defined?(06:05) Predictions of instrumental convergence theory(09:04) A simple gap in seminal IC arguments(09:42) Bostrom(09:58) Omohundro(10:54) Carlsmith(11:49) LessWrong wiki(12:30) Why we can't easily fill the gap(12:34) Intuitive goal notions aren't adequate (aka, coffee-fetching is hopelessly overloaded)(15:24) Formal goal notions aren't adequate(18:21) A stronger counterthesis: goals are incoherent(21:14) Related work to the stronger counterthesis(24:04) Counterpoints(26:02) What's the alternative, then?(29:37) Why does it matter?(32:13) 'Misalignment'(34:39) The Hugging Face attack(38:44) Takeaways(40:01) Acknowledgements(40:24) Disclaimers(41:31) Appendix: more on the extreme ambiguity of coffee-making The original text contained 27 footnotes which were omitted from this narration. --- First published: October 22nd, 2026 Source: https://forum.effectivealtruism.org/posts/i6TbMr5yc4Jq5eqyE/instrumental-convergence-is-incoherent-so-pls-slightly-lower --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better. TL;DR: I helped set up a large foundation and we wanted a consistent way to allocate grants We planned to allocate substantially more than our field was able to absorb right now, giving us the opportunity to actively shape our field We outlined key principles and considerations We quickly learned we could not just copy the for-profit world: so what could we do? We could try and optimize perfectly, mathematically, but it gets absurd and impractical Instead, we tried out a bunch of different approaches to finding a consistent grant allocation formula, settling on the one that most “felt” right given our starting principles and constraints I quite liked this process of setting principles and constraints, and then vibing your way to an allocate formula that feels right vs being super mathematical I wanted to share this process as a real example of trying to work out how to allocate funding. [...] ---Outline:(01:33) x = height of the opposing player(02:50) Searching for the perfect grant-allocation formula(03:30) Our core metrics(04:32) We couldn't just copy from other grantmakers or the for-profit world(07:46) Turning our principles into a utility function + optimizing(09:56) "Vibing" our way to a utility function(10:35) The "Obvious" Option: Allocate by Expected Cost-Effectiveness(11:38) The "Risk-Buckets" Approach(12:43) The Kelly Criterion(14:22) Sharpe ratio(14:57) Edge(16:24) What We Actually Did(17:11) If we settled on edge, doesn't that mean there was a calculable utility function in the background the whole time?(20:00) What I learned in all of this The original text contained 1 footnote which was omitted from this narration. --- First published: October 8th, 2026 Source: https://forum.effectivealtruism.org/posts/zWTuwxwRdNjSyWuw2/trying-to-spend-usd100m-well-my-search-for-the-perfect-grant --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I’m a bit concerned that the EA Forum is going to die a slow death. I don’t want this to happen as it has long been, in my opinion, the best place on the internet. I'd like it to stay that way. I personally have found myself engaging less, because the content is less interesting and relevant to me. I also think some of the trends I highlight below are due to CEA investing less in the Forum (e.g., my impression is the Forum team is smaller than it used to be). I think a lot of what I critique below is downstream of community dynamics, rather than the Forum itself. But I also think injecting the EA Forum with a vibrant intellectual culture will be good for EA as a whole. The point of this post is to open a discussion, and give my own ideas for what I think might help. I welcome people to disagree, share their own takes on what the EA Forum should be, and the purpose it serves. EA Forum moderators — I think you all do an admirable job. My personal view is that there should be way more of you doing [...] ---Outline:(01:51) Is the EA Forum dying?(03:25) What's the value of the Forum?(05:09) Ideas for saving the Forum --- First published: October 28th, 2026 Source: https://forum.effectivealtruism.org/posts/AuecjpnoCZKF26fkp/keeping-the-ea-forum-alive --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
This post is part of the Cross-Cause Fund (CCF) series by Rethink Priorities — a sequence of articles dedicated to effective giving across philanthropic areas. TL;DR: A lot of new money is about to enter effective giving. When that happens, the question "Where does a dollar do the most good?" gets harder to answer, not easier. It depends on how much money each area already has, and that amount keeps changing as the money arrives. This is why the model behind the Rethink Priorities Cross-Cause Fund doesn't just rank funds once and stop. It tracks how each fund's cost-effectiveness changes as it receives more funding, and re-ranks as it goes. Below, we explain why anyone giving at scale should use similar reasoning. If you’re a small or mid-sized donor who makes giving decisions independently, it's relatively easy to figure out what your money is doing. For example, when you give your first $1000 to a charity that runs corporate cage-free egg campaigns, your donation will go on to fund corporate cage-free egg campaigns. When you give your second $1000 to the same charity, it will fund…corporate cage-free egg campaigns. And you will never guess what your tenth $1000 goes [...] ---Outline:(02:37) Modeling diminishing marginal returns(07:05) Acting as part of a group(08:48) How the Cross-Cause Fund works(10:42) Why you can't just follow the highest marginal number(13:03) Acknowledgements --- First published: October 21st, 2026 Source: https://forum.effectivealtruism.org/posts/FZkwut9d5fD2vzC5i/ccf-as-more-money-arrives-the-best-place-to-give-keeps --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
loading
Comments