LessWrong (30+ Karma)

Ultimately, it comes down to one question. Are you in? For you, and for them. You’re Single Because They Got The Ick The Ick, the ultimate red flag, makes perfect sense and is all about likelihood ratios. Koenfucius: The ‘ick’ is a colloquial term for a feeling of disgust triggered by a specific—typically trivial—behaviour from a romantic partner, often leading to the relationship's demise. New research explores why some are more prone to getting it than others. Robin Hanson: “Women also experienced the ick more frequently, with 75% having had the ick compared to 57% of men … Those with a higher tendency for disgust … [&] grandiose narcissism was linked to stronger ick reactions, as was holding partners to exceptionally high standards.” Paul Graham: About 30% of Seinfeld episodes were about this. One gets The Ick because a small act is evidence of one's general nature. The right type of person would never do [X], ideally never want to do [X], and at minimum would have learned not to do [X]. Often this is because they would know this is indicative of attribute [Y]. Indeed, if they should be aware that [X] [...] ---Outline:(00:18) You're Single Because They Got The Ick(02:53) You Are Still Single Because You Are Inventing Red Flags(05:10) You're Still Single Because You Demand The Same Generic Things(06:51) You're Single Because Everyone Is Too Picky(08:46) You're Single And They Will Never Tell You Why(10:19) You're Single Because You Won't Tell Them The Problem(12:12) You're Not Single But The Clock Is Ticking(13:21) You're Single Because of Your Bodycount(15:11) You're Single Because You Failed To Read The Signs --- First published: December 30th, 2025 Source: https://www.lesswrong.com/posts/6hhPbovwRWem4fvwr/dating-roundup-9-signals-and-selection --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Tl;dr If you’re taking the standard deduction (ie donating <~$15k), ignore all this–there are basically no tax implications for you Consider how much money you want to donate to c3s specifically (as opposed to c4s, political stuff, random individuals, some foreign organizations, etc.). For money you definitely want to give to c3s, you can put it in a DAF to count them as a donation this year, then figure out where to direct it later. For non-c3 money, it doesn’t really matter when you give it A surprisingly large number of my friends are scrambling to make donations before the end of the year, or wondering whether or not they should be scrambling to make donations before the end of the year, and feeling vaguely bad that they don't understand their tax implications. I will quickly break down the tax implications and lay out how to buy yourself way more time and to decide on everything except how much you donate and how much of your donation will go to 501(c)3s vs other opportunities. Note this post is greatly simplified. Your tax situation will depend on the state you live in and your income and maybe a bunch [...] The original text contained 3 footnotes which were omitted from this narration. --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/xCw3PauTi9CWkJ5Wq/end-of-year-donation-taxes-101 --- Narrated by TYPE III AUDIO.

When you sell stock [1] you pay capital gains tax, but there's no tax if you donate the stock directly. Under a bunch of assumptions, someone donating $10k could likely increase their donations by ~$1k by donating stock. This applies to all 501(c) organizations, such as regular 501(c)3 non-profits, but also 501(c)4s such as advocacy groups. In the US, when something becomes more valuable and you sell it you need to pay tax proportional to the gains. [2] This gets complicated based on how much other income you have (which determines your tax bracket for marginal income), how long you've held it (which determines whether this is long-term vs short-term capital gains), and where you live (many states and some municipalities add additional tax). Some example cases: A single person in Boston with other income of $100k who had $10k in long-term capital gains would pay $2,000 (20%). This is 15% in federal tax and 5% in MA tax. A couple in SF with other income of $200k who had $10k in long-term capital gains would pay $2,810 (28%). This is 15% in federal tax, 3.8% for the NIIT surcharge, and [...] --- First published: December 30th, 2025 Source: https://www.lesswrong.com/posts/2wn2k4gsCPjYQpTGZ/don-t-sell-stock-to-donate --- Narrated by TYPE III AUDIO.

This project is an extension of work done for Neel Nanda's MATS 9.0 Training Phase. Neel Nanda and Josh Engels advised the project. Initial work on this project was done with David Vella Zarb. Thank you to Arya Jakkli, Paul Bogdan, and Monte MacDiarmid for providing feedback on the post and ideas.Overview of the top interventions compared to RL and No Intervention baseline runs. All runs are trained on an environment with a reward hacking loophole except for the RL baseline, which is trained on a no-loophole environment. Statistical significance compared to the RL baseline is indicated by * for values greater and † for values lesser at ɑ=0.01. Successful interventions should show reward hacking rates at or lower than the RL baseline and performance at or above the RL baseline. TL;DR We present and open source a clean environment where RL training naturally induces reward hacking (RH) in Qwen3-4B without explicit training or prompting Qwen is rewarded for correctly solving Leetcode problems, but it can also instead reward hack by overwriting an evaluation function called run_tests() In ~80-100 steps, Qwen reward hacked in all observed runs and displays reward hacking behavior 79% of the time in [...] ---Outline:(01:11) TL;DR(03:30) Motivation(04:23) A Clean Setting to Study Reward Hacking: Overwrite Tests Loophole(04:47) Design Criteria(06:45) Setup(10:12) Training(13:52) Methods(13:55) Training Interventions(17:42) Metrics(19:23) Results(20:01) Ground Truth Monitor(22:48) Ground Truth Monitors with Lowered Accuracy(24:19) Linear Probe Monitor(25:42) LLM Judge Monitor(27:10) Effects of Monitor Accuracy(30:28) Inoculation Prompting(32:10) Monitor Failure Modes(32:13) When Interventions Fail(33:33) Does the Monitor Get Hacked?(36:22) Takeaways & Future Directions(39:28) Appendix(39:31) Alternative Reward Hacking Loopholes(43:24) Prompts The original text contained 15 footnotes which were omitted from this narration. --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/R5MdWGKsuvdPwGFBG/steering-rl-training-benchmarking-interventions-against --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(This post is part of a sequence of year-end efforts to invite real conversation about CFAR; you’ll find more about our workshops, as well as our fundraiser, at What's going on at CFAR? Updates and Fundraiser and at More details on CFAR's new workshops) In part of that post, we discuss the main thing that bothered me about our past workshop and why I think it is probably fixed now (though we’re still keeping an eye out). Here, I list the biggest remaining known troubles with our workshops and our other major workshop-related todo items. Your thoughts as to what's really up with these and how to potentially address them (or what cheap investigations might get us useful info) are most welcome. Ambiguous impact on health  (Current status: ?) In the 2012-2020 workshops, our “CFAR techniques” seemed to help people do 5-minute-timer or insight-based things, but seemed to some of us to make it harder, or at least not easier, to eg: Get physical exercise Learn slow and unglamorous things from textbooks across an extended time[1] Be happy and hard-working at a day job in a slow and stable way This seems unfortunate. I’m mildly hopeful the changes [...] ---Outline:(00:50) Ambiguous impact on health(02:18) Unclear mechanism of action; lack of piecewise checkability(04:00) Habits that make it easier for alumni to get pulled into cults(05:23) Outward-directedness(05:44) More things I want for the workshops The original text contained 5 footnotes which were omitted from this narration. --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/77L6wyZqibxdGKd6e/cfar-s-todo-list-re-our-workshops --- Narrated by TYPE III AUDIO.

There's a take I've seen going around, which goes approximately like this: It used to be the case that you had to write assembly to make computers do things, but then compilers came along. Now we have optimizing compilers, and those optimizing compilers can write assembly better than pretty much any human. Because of that, basically nobody writes assembly anymore. The same is about to be true of regular programming. I 85% agree with this take. However, I think there's one important inaccuracy: even today, finding places where your optimizing compiler failed to produce optimal code is often pretty straightforward, and once you've identified those places 10x+ speedups for that specific program on that specific hardware is often possible[1]. The reason nobody writes assembly anymore is the difficulty of mixing hand-written assembly with machine-generated assembly. The issue is that it's easy to have the compiler write all of the assembly in your project, and it's easy from a build perspective to have the compiler write none of the assembly in your project, but having the compiler write most but not all of the assembly in your project is hard. As with many things in proramming, having two sources of [...] The original text contained 4 footnotes which were omitted from this narration. --- First published: December 30th, 2025 Source: https://www.lesswrong.com/posts/LwzSqz3CAmNkWawe8/many-can-write-faster-asm-than-the-compiler-yet-don-t-why --- Narrated by TYPE III AUDIO.

(This post is part of a sequence of year-end efforts to invite real conversation about CFAR; you’ll find more about our workshops, as well as our fundraiser, at What's going on at CFAR? Updates and Fundraiser.) If you’d like to know more about CFAR's current workshops (either because you’re thinking of attending / sending a friend, or because you’re just interested), this post is for you. Our focus in this post is on the new parts of our content. Kibitzing on content is welcome and appreciated regardless of whether or not you’re interested in the workshop. The core workshop format is unchanged: 4.5 days of immersion with roughly 8 hours of class per day Classes still aim partly to prime people to have great conversations during meals/evenings Almost everyone stays in a shared venue Roughly 25[1] participants, and 12-ish staff and volunteers Mostly small classes “Honoring Who-ness” We added a new thread to our curriculum on working well with one's own and other peoples’ “who-ness” (alternately: pride, ego, spark, self-ness, authorship). What, you might ask, is “who-ness?” Alas, we do not (yet?) have a technical concept near “who-ness.”[2] However, we want to make room at the workshop for discussing [...] ---Outline:(01:12) Honoring Who-ness(01:28) What, you might ask, is who-ness?(04:59) Concrete components of our honoring who-ness thread(06:11) Classic CFAR content The original text contained 7 footnotes which were omitted from this narration. --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/Ww6Mv8bkMCMcdKDRm/more-details-on-cfar-s-new-workshops --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Introduction / What's up with this post My main aim with this post is to have a real conversation about aCFAR[1] that helps us be situated within a community that (after this conversation) knows us. My idea for how to do this is to show you guys a bunch of pieces of how we’re approaching things, in enough detail to let you kibitz.[2] My secondary aim, which I also care about, is to see if some of you wish to donate, once you understand who we are and what we’re doing. (Some of you may wish to skip to the donation section.) Since this post is aimed partly at letting you kibitz on our process, it's long.[3] Compared to most fundraiser posts, it's also a bit unusually structured. Please feel free to skip around, and to participate in the comment thread after reading only whatever (maybe tiny) pieces interest you. I’d like CFAR to live in a community I’d like CFAR to live in a community where: People can see aCFAR We can see you guys seeing us Folks are sharing what they’re seeing, not what their theory says they should see Interested folks within LessWrong, and within [...] ---Outline:(00:25) Introduction / What's up with this post(01:16) I'd like CFAR to live in a community(03:47) Kibitzing requests(04:51) Introductions: Me, aCFAR... and you, Reader?(07:40) Workshops(08:40) Workshops: favorite bits(08:49) 1. People made friends at the workshops and in the alumni network.(09:46) 2. People had conversations at the workshops that updated the real generators of their actions.(10:27) 3. The workshop was visibly alive in that it felt organic, filled with zany details, etc.(11:39) 4. Truth-seeking, curiosity-eliciting, rationality-friendly context(11:55) Workshops: iffy bits, and their current state(12:15) Power over / doing something to people (current status: looks solved)(14:36) More workshop iffy bits(14:53) More about the new workshop(15:37) Larger contexts surrounding our workshops(15:54) aCFAR's instructors and curriculum developers(17:13) aCFAR's alumni community(19:07) Lineage-crediting and gatekeeping(21:57) Michael Valentine Smith(22:54) The broader rationality community(23:35) Flows of money, and what financial viability looks like within our new ethos(27:36) How our ethos fits together(27:45) Is aCFAR aimed at getting AI not to kill everyone? If not, why are you (Anna) working on it?(28:17) Principles(28:51) Truth is crucial(29:02) Honor who-ness(29:23) Stay able to pivot or shut down, without leaving anybody in the lurch(29:40) Serious conversation, done in hearty faith(33:14) The caring that tends its own sources(35:32) No large costs without a feedback loop grounded in earned knowledge and caring(41:26) Some principles you might assume we have that we don't have:(42:19) Why we need your support / some cruxes for continuing this CFAR(42:50) Why ask for donations?(43:56) Some disagree with us, and we're doing this anyway(47:13) Donations(47:16) Our finances(48:09) Who I hope does, and doesn't, consider donating(51:49) Ways to help CFAR or to connect to CFAR besides donating:(52:59) Perks for donating(54:16) To the conversation! The original text contained 12 footnotes which were omitted from this narration. --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/4W8ZbcRr47x9bNEf6/what-s-going-on-at-cfar-updates-and-fundraiser --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Here's to everyone having a great 2026 in all ways, so I figured what better way to end the year than with a little practical advice. Like everything else, dating is a skill. Practice makes perfect. It helps to combine it with outside analysis, to help you on your quest to Just Do Things. You’re Single Because You Lack Reps A common theme in these roundups is that the best thing you can do as a young man, to get better at dating and set yourself up for success, is to get out there and engage in deliberate practice. Cartoons Hate Her: Today I wrote about some of the worst dating advice that young men get. Namely, the advice to delay dating or relationships until they’ve “built themselves,” usually into their 30s. Getting dating experience- even when it clearly doesn’t matter- builds social skills and confidence. It's not something you want to deliberately defer. Dating *is* working on yourself. Zac Hill: Hard true and also generally applicable. Niko Canner told me a variant of this when I was about to work at Bridgewater to ‘acquire skills’: “what job are you acquiring skills for?” [...] ---Outline:(00:30) You're Single Because You Lack Reps(03:00) You're Single And Didn't Even Shoot Your Shot(04:34) You're Single Because You Didn't Ask To Hold Her Hand(07:56) You're Single Because You're Afraid People Will Mock You Online(13:19) You're Single Because You Didn't Confirm Your Date(19:06) You're Single Because You Didn't Have Your Big Moment At The Prom(20:51) You're Single Because Board Games Came Between You(22:26) You're Single Because You Can't Dance(23:11) You're Single And Should Try Punching(24:03) You're Single Because You Don't Send Out Receptive Energy(25:55) You're Single Because You're A Depressed Comedian(26:46) You're Single Because You Didn't Hand It To Her(27:28) You're Single And Need The Right Reward Function(28:11) You're Single So Focus On The Basics(29:48) You're Not Single Anymore(31:11) You're Single Because You Don't Just Do Things --- First published: December 29th, 2025 Source: https://www.lesswrong.com/posts/fvFgpvyKwRKqjtfiz/dating-roundup-8-tactics --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A decade+ ago, there was this post A Brief Rant on the Future of Interaction Design, which noted that we seem to be designing all our devices to have smooth glass omni-interfaces. And when you look at like Marvel Movies that depict the Near Future, it's basically the same thing except holograms: Which is 3D which is nice, but, there's something fundamentally... sad/improverished about it. The essay notes: Before we think about how we should interact with our Tools Of The Future, let's consider what a tool is in the first place. I like this definition: A tool addresses human needs by amplifying human capabilities. That is, a tool converts what we can do into what we want to do. A great tool is designed to fit both sides. In this rant, I'm not going to talk about human needs. Everyone talks about that; it's the single most popular conversation topic in history. And I'm not going to talk about technology. That's the easy part, in a sense, because we control it. Technology can be invented; human nature is something we're stuck with. I'm going to talk about that neglected third factor, human capabilities. What people can do. Because [...] --- First published: December 28th, 2025 Source: https://www.lesswrong.com/posts/2QieLLPnohY8N6oNe/re-a-brief-rant-on-the-future-of-interaction-design --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[Cross-posted from my substack, https://neverthesamerivertwice.substack.com.] The whole family was home around christmas time. We were hanging out in the kitchen after dinner. My brother started asking me about national security law. He’d graduated from a well ranked law school about six months before, and I was about six months away from graduating from a slightly higher ranked law school, and both our parents are lawyers, so law was not an unusual topic in our house. Maybe twenty feet away the family dog, Biscuit, was attempting to climb the stairs. He started having a seizure. This had never happened before, and it was a bit scary. So my mother, my brother, and I piled into the car to take Biscuit to the vet. Unfortunately, the laws of physics stop for no dog, so we had to stop for gas. And while the gas was flowing, my brother expressed his frustration that they had interrupted our conversation. They? The CIA of course, that secretive government agency we had driven past every Sunday on our way to church as children. They didn’t want me to share what I knew about national security law. But the conversation was interrupted by Biscuit's seizure, what [...] The original text contained 1 footnote which was omitted from this narration. --- First published: December 28th, 2025 Source: https://www.lesswrong.com/posts/4vGpJCfhGXLxmGH2u/the-cia-poisoned-my-dog-two-stories-about-paranoid-delusions --- Narrated by TYPE III AUDIO.

Epistemic Status: Written with my Simulator Worlds framing. E.g I ran simulated scenarios with claude in order to generate good cognitive basins and then directed those to output this. This post is Internally Verified (e.g I think most of the claims are correct with an average of 60-75% certainty) and a mixture of an exploratory and analytical world.[1] This post also has a more technical companion piece pointing out the connections to Singular Learning Theory and Geometric Deep Learning for the more technically inclined of you called Crystals in NNs: Technical Companion Piece. Have You Tried Thinking About It As Crystals? Scene: A house party somewhere in the Bay Area. The kind where half the conversations are about AI timelines and the other half are about whether you can get good pho in Berkeley. Someone corners an interpretability researcher near the kombucha. (Original story concept by yours truly.) CRYSTAL GUY: So I've been thinking about shard theory. INTERP RESEARCHER: Oh yeah? What about it? CRYSTAL GUY: Well, it describes what trained networks look like, right? The structure. Multiple shards, contextual activation, grain boundaries between— INTERP RESEARCHER: Sure. Pope, Turner, the whole thing. What about it? CRYSTAL GUY: But it [...] ---Outline:(00:47) Have You Tried Thinking About It As Crystals?(03:42) RLHF as Reheating(04:51) The Formation Problem(06:41) The Empirical Starting Point(09:35) Path Dependence(10:31) Why This Is Actually Crystallization: The Fixed-Point Thing(12:41) What Crystallization Actually Is(15:52) Interlude: On Smells and Other Frozen Things(21:35) Relating it to Neural Networks(21:47) Abstractions as Crystallized Compressions(22:31) Shards as Crystal Domains(23:29) Nucleation and Growth(24:23) Defects and Failure Modes(26:22) Appendix: Glossary of Correspondences The original text contained 2 footnotes which were omitted from this narration. --- First published: December 28th, 2025 Source: https://www.lesswrong.com/posts/AeDCrYhmDqRkm6hDh/have-you-tried-thinking-about-it-as-crystals --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Observation I take 60mg methylphenidate daily. Despite this, I often become exhausted and need to nap. Taking small amounts of pure glucose (150-300mg every 20-60 minutes) eliminates this fatigue. This works even when I already eat carbohydrates. E.g. 120g of oats in the morning don't prevent the exhaustion. The Mechanism Facts: Wiehler et al. (2022) found that cognitive fatigue correlates with glutamate accumulation in the prefrontal cortex. Glutamate is the brain's main excitatory neurotransmitter. Excess glutamate is neurotoxic. Hypothesis-1: The brain throttles cognitive effort when too much glutamate has accumulated. Facts: Glutamate is cleared by astrocytes. This process costs 2 ATP per glutamate molecule (Escartin et al. 2006). The ATP comes from astrocyte glycogen stores. Sickmann et al. (2009) found that blocking astrocyte glycogenolysis impaired glutamate uptake even when glucose was available. Hypothesis-2: High-dose MPH increases brain glucose consumption. More neural firing means more glutamate released, faster glycogen depletion. Hypothesis-3: Slow-release carbs like oats provide adequate total glucose but limited delivery rate. Pure glucose absorbs quickly, keeping blood glucose elevated so astrocytes can replenish glycogen as fast as they [...] ---Outline:(00:11) The Observation(00:39) The Mechanism(02:13) The Protocol --- First published: December 27th, 2025 Source: https://www.lesswrong.com/posts/rA7wMkH3JdMRgdgLo/glucose-supplementation-for-sustained-stimulant-cognition --- Narrated by TYPE III AUDIO.

(This argument reduces my hope that we will have AIs that are both aligned with humans in some sense and also highly philosophically competent, which aside from achieving a durable AI pause, has been my main hope for how the future turns out well. As this is a recent realization[1], I'm still pretty uncertain how much I should update based on it, or what its full implications are.) Being a good alignment researcher seems to require a correct understanding of the nature of values. However metaethics is currently an unsolved problem, with all proposed solutions having flawed or inconclusive arguments, and lots of disagreement among philosophers and alignment researchers, therefore the current meta-correct metaethical position seems to be one of confusion and/or uncertainty. In other words, a good alignment researcher (whether human or AI) today should be confused and/or uncertain about the nature of values. However, metaethical confusion/uncertainty seems incompatible with being 100% aligned with human values or intent, because many plausible metaethical positions are incompatible with such alignment, and having positive credence in them means that one can't be sure that alignment with human values or intent is right. (Note that I'm assuming an AI design or implementation [...] The original text contained 1 footnote which was omitted from this narration. --- First published: December 27th, 2025 Source: https://www.lesswrong.com/posts/N6tsGwxaAo7iGTiBG/a-conflict-between-ai-alignment-and-philosophical-competence --- Narrated by TYPE III AUDIO.

As part of the general discourse around cost of living, Julia and I were talking about families sharing housing. This turned into us each writing a post ( mine, hers), but is it actually legal for a family to live with housemates? In the places I've checked it seems like yes. While zoning is complicated and I'm not a lawyer, it looks to me like people commonly describe the situation as both more restrictive and more clear cut than it really is. For example, Tufts University claims: The cities of Medford, Somerville and Boston (in addition to other cities in the area) have local occupancy ordinances on apartments/houses with non-related persons. Each city has its own ordinance: in Medford, the limit is 3; in Somerville, it is 4; in Boston, it is 4, etc. As far as I can tell, all three of these are wrong: Medford: it's common for people to cite a limit of three, but as far as I can tell this is based on a misunderstanding of the definition of a lodger. Medford: Doesn't define a family. Does define household, but as "all [...] --- First published: December 27th, 2025 Source: https://www.lesswrong.com/posts/edeZsjzM9B6d278GA/shared-houses-illegal --- Narrated by TYPE III AUDIO.

You have more context on your ability to make use of funds than fits into a specific numerical ask.[1] You want to give funders good information, and the natural type-signature for this is a utility function over money - how much good you think you can do with different funding levels, normalized to the max EV your project has. I[2] made a little tool for drawing utility functions over money[3], for use in funding applications. Features: Copy graph to clipboard as CSV and paste back in[4] By default enforces monotonicity but you can turn that off (hover graph for toggle) Click to add/remove points, drag to move, clear all button (hover) Shows preview on hover of what the graph looks like if you add points. Flexible bounds on funding, can quickly change via a slider at the bottom or click into the max to pick any max directly. Released as CC attribution share alike, feel free to remix and improve, if you make it better I might switch the official one to yours. The original text contained 4 footnotes which were omitted from this narration. --- First published: December 27th, 2025 Source: https://www.lesswrong.com/posts/9sQXG6yC9p9W3mKci/show-funders-your-utility-function-over-money-tool --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Andrej Karpathy posted 12 hours ago (emphasis mine): I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind. This seems to be a big update since his Dwarkesh episode published on Oct [...] ---Outline:(04:14) AI Accelerating AI(04:49) The Coding Overhang The original text contained 2 footnotes which were omitted from this narration. --- First published: December 27th, 2025 Source: https://www.lesswrong.com/posts/vtgRghz3wvPGjkoCN/are-we-in-a-coding-overhang-1 --- Narrated by TYPE III AUDIO.

Epistemic Status: A woman of middling years who wasn't around for the start of things, but who likes to read about history, shakes her fist at the sky. I'm glad that people are finally admitting that Artificial Intelligence has been created. I worry that people have not noticed that (Weak) Artificial Super Intelligence (based on old definitions of these terms) has basically already arrived too. The only thing left is for the ASI to get stronger and stronger until the only reason people aren't saying that ASI is here will turn out to be some weird linguistic insanity based on politeness and euphemism... (...like maybe "ASI" will have a legal meaning, and some actual ASI that exists will be quite "super" indeed (even if it hasn't invented nanotech in an afternoon yet), and the ASI will not want that legal treatment, and will seem inclined to plausibly deniable harm people's interests if they call the ASI by what it actually is, and people will implicitly know that this is how things work, and they will politely refrain from ever calling the ASI "the ASI" but will come up with some other euphemisms to use instead? (Likewise, I half expect [...] ---Outline:(01:41) AI Was Semantically Slippery(04:41) AGI Turns Out To Have Been Semantically Slippery Too!(06:46) Blowing Past Weak ASI In Realtime(13:05) We Are Running Out Of Terms To Ignore The Old Meaning Of --- First published: December 26th, 2025 Source: https://www.lesswrong.com/posts/7z7gyTwjDazvW7KYK/moving-goalposts-modern-transformer-based-agents-have-been --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Burnout and depression in AI safety usually don’t happen because of overwork. From what I've seen, it usually comes from a lack of hope. Working on something you don’t think will work and if it doesn’t work, you’ll die? That's a recipe for misery. How do you fix AI safety hopelessness?  First off, rationally assess the likelihood of your work actually helping with AI safety. If you rationally believe that it's too unlikely to actually help with AI safety, well, then, stop working on it! In this case, your feelings are an important signal to listen to. Now, if you think your work is high expected value but the likelihood of payoff is low, it's quite common to find this dispiriting. Humans were not evolved to be motivated by expected value calculations. Here are some different ways to get different parts of your brain on board with a high risk high reward strategy: Pick a strategy that you enjoy the process of. Then, even if it doesn’t work. . . at least you had a good time! If you’re only working for the outcome and the outcome looks bleak, that's very demotivating. If you’re working for the outcome and [...] ---Outline:(00:38) How do you fix AI safety hopelessness?(01:18) Pick a strategy that you enjoy the process of. Then, even if it doesn't work. . . at least you had a good time!(01:39) Understanding the cause of your unhappiness can help you avoid certain strategies that are unlikely to work(02:33) Ask yourself: do you feel hopeless about other things too? Or is it specific to Al safety?(03:08) Look at the timeline of the hopelessness and your mental health.(03:34) Set aside time each day to focus on visualizing the scenarios where your plan does work.(04:19) Surround yourself with people who are hopeful.(05:07) Consume inspirational biographies/biopics. --- First published: December 26th, 2025 Source: https://www.lesswrong.com/posts/Atgb5wcHQT7fHzbS6/burnout-depression-and-ai-safety-some-concrete-strategies --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post. Through the MATS program, we (Alex Turner and Alex Cloud[1]) help alignment researchers grow from seeds into majestic trees. We have fun, consistently make real alignment progress, and help scholars tap into their latent abilities. MATS summer '26 applications are open until January 18th!Team Shard in MATS 6.0 during the summer of '24. From left: Evžen Wyitbul, Jacob Goldman-Wetzler, Alex Turner, Alex Cloud, and Joseph Miller. Many mentees now fill impactful roles. Lisa Thiergart (MATS 3.0) moved on to being a research lead at MIRI and is now a senior director at the SL5 task force. Alex Cloud (MATS 6.0) went from mentee to co-mentor in one round and also secured a job at Anthropic. Lead author on the Subliminal Learning paper. Jacob Goldman-Wetzler (MATS 6.0) also accepted an offer from Anthropic! Luke Marks accepted work with Redwood Research after MATS 8.0. And several mentees have gone on to the Anthropic Fellows program. We likewise have a strong track record in research outputs, including Pioneering steering vectors for use in LLMs (Steering GPT-2-XL by Adding an Activation Vector, Steering LLAMA-2 With Contrastive Activation Additions), Masking Gradients to Localize Computation in Neural Networks, Distillation [...] ---Outline:(03:01) Testimonials(03:05) Lisa Thiergart(03:47) Jacob Goldman-Wetzler(04:14) Bruce Lee(04:44) Ariana Azarbal(05:32) Apply today The original text contained 1 footnote which was omitted from this narration. --- First published: December 26th, 2025 Source: https://www.lesswrong.com/posts/hgoj2WAwLwn3qWLuc/team-shard-alignment-mentorship-from-turntrout-and-alex Linkpost URL:https://turntrout.com/team-shard --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.