The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New core career advice series from Probably Good!, published by Probably Good on April 25, 2024 on The Effective Altruism Forum. We recently published a new core career advice series. It provides a concise, accessible intro to some of the most important ideas for planning an impactful career. Check it out on our site! What is the core advice series? The core advice series distills the most important ideas from our in-depth career guide and repackages them with a more accessible framing. It was born out of the question: "If I have less than an hour to learn about pursuing an impact-focused career, what do I need to know?" We ended up with a series of 7 short articles that could be read in one sitting. Each article takes about 5 minutes to read and covers a critical concept we think is important to pursuing an impactful career, including: Taking a scale-sensitive approach to helping others Comparing and assessing specific job opportunities Prioritizing and exploring important causes While many of our readers want to engage with our more in-depth material, we also want to provide an option for those who can't afford the time. This series is intended to be a helpful on-ramp for someone who wants to make a positive impact but hasn't been exposed to tools to think about their career decisions strategically. How you can help If you know someone who could benefit from accessible, impact-focused career advice, please share this with them! As always, if you have feedback, thoughts on future content, or questions about how Probably Good could support you, your organization, or your community, feel free to get in touch. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Inner Ring by C. S. Lewis, published by Saul Munn on April 25, 2024 on LessWrong. Note: In @Nathan Young's words "It seems like great essays should go here and be fed through the standard LessWrong algorithm. There is possibly a copyright issue here, but we aren't making any money off it either." What follows is a full copy of the C. S. Lewis essay "The Inner Ring" the 1944 Memorial Lecture at King's College, University of London. May I read you a few lines from Tolstoy's War and Peace? When Boris entered the room, Prince Andrey was listening to an old general, wearing his decorations, who was reporting something to Prince Andrey, with an expression of soldierly servility on his purple face. "Alright. Please wait!" he said to the general, speaking in Russian with the French accent which he used when he spoke with contempt. The moment he noticed Boris he stopped listening to the general who trotted imploringly after him and begged to be heard, while Prince Andrey turned to Boris with a cheerful smile and a nod of the head. Boris now clearly understood - what he had already guessed - that side by side with the system of discipline and subordination which were laid down in the Army Regulations, there existed a different and more real system - the system which compelled a tightly laced general with a purple face to wait respectfully for his turn while a mere captain like Prince Andrey chatted with a mere second lieutenant like Boris. Boris decided at once that he would be guided not by the official system but by this other unwritten system. When you invite a middle-aged moralist to address you, I suppose I must conclude, however unlikely the conclusion seems, that you have a taste for middle-aged moralising. I shall do my best to gratify it. I shall in fact, give you advice about the world in which you are going to live. I do not mean by this that I am going to talk on what are called current affairs. You probably know quite as much about them as I do. I am not going to tell you - except in a form so general that you will hardly recognise it - what part you ought to play in post-war reconstruction. It is not, in fact, very likely that any of you will be able, in the next ten years, to make any direct contribution to the peace or prosperity of Europe. You will be busy finding jobs, getting married, acquiring facts. I am going to do something more old-fashioned than you perhaps expected. I am going to give advice. I am going to issue warnings. Advice and warnings about things which are so perennial that no one calls them "current affairs." And of course everyone knows what a middle-aged moralist of my type warns his juniors against. He warns them against the World, the Flesh, and the Devil. But one of this trio will be enough to deal with today. The Devil, I shall leave strictly alone. The association between him and me in the public mind has already gone quite as deep as I wish: in some quarters it has already reached the level of confusion, if not of identification. I begin to realise the truth of the old proverb that he who sups with that formidable host needs a long spoon. As for the Flesh, you must be very abnormal young people if you do not know quite as much about it as I do. But on the World I think I have something to say. In the passage I have just read from Tolstoy, the young second lieutenant Boris Dubretskoi discovers that there exist in the army two different systems or hierarchies. The one is printed in some little red book and anyone can easily read it up. It also remains constant. A general is always superior to a colonel, and a colonel to a captain. The other is not printed anywhere. Nor is it even a formally organised secret society with officers and rules which you would be told after you had been admitted. You are never formally and explicitly admi...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: This is Water by David Foster Wallace, published by Nathan Young on April 25, 2024 on LessWrong. Note: It seems like great essays should go here and be fed through the standard LessWrong algorithm. There is possibly a copyright issue here, but we aren't making any money off it either. What follows is a full copy of "This is Water" by David Foster Wallace his 2005 commencement speech to the graduating class at Kenyon College. Greetings parents and congratulations to Kenyon's graduating class of 2005. There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says "Morning, boys. How's the water?" And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes "What the hell is water?" This is a standard requirement of US commencement speeches, the deployment of didactic little parable-ish stories. The story thing turns out to be one of the better, less bullshitty conventions of the genre, but if you're worried that I plan to present myself here as the wise, older fish explaining what water is to you younger fish, please don't be. I am not the wise old fish. The point of the fish story is merely that the most obvious, important realities are often the ones that are hardest to see and talk about. Stated as an English sentence, of course, this is just a banal platitude, but the fact is that in the day to day trenches of adult existence, banal platitudes can have a life or death importance, or so I wish to suggest to you on this dry and lovely morning. Of course the main requirement of speeches like this is that I'm supposed to talk about your liberal arts education's meaning, to try to explain why the degree you are about to receive has actual human value instead of just a material payoff. So let's talk about the single most pervasive cliché in the commencement speech genre, which is that a liberal arts education is not so much about filling you up with knowledge as it is about "teaching you how to think." If you're like me as a student, you've never liked hearing this, and you tend to feel a bit insulted by the claim that you needed anybody to teach you how to think, since the fact that you even got admitted to a college this good seems like proof that you already know how to think. But I'm going to posit to you that the liberal arts cliché turns out not to be insulting at all, because the really significant education in thinking that we're supposed to get in a place like this isn't really about the capacity to think, but rather about the choice of what to think about. If your total freedom of choice regarding what to think about seems too obvious to waste time discussing, I'd ask you to think about fish and water, and to bracket for just a few minutes your scepticism about the value of the totally obvious. Here's another didactic little story. There are these two guys sitting together in a bar in the remote Alaskan wilderness. One of the guys is religious, the other is an atheist, and the two are arguing about the existence of God with that special intensity that comes after about the fourth beer. And the atheist says: "Look, it's not like I don't have actual reasons for not believing in God. It's not like I haven't ever experimented with the whole God and prayer thing. Just last month I got caught away from the camp in that terrible blizzard, and I was totally lost and I couldn't see a thing, and it was 50 below, and so I tried it: I fell to my knees in the snow and cried out 'Oh, God, if there is a God, I'm lost in this blizzard, and I'm gonna die if you don't help me.'" And now, in the bar, the religious guy looks at the atheist all puzzled. "Well then you must believe now," he says, "After all, here you are, alive." The atheist just rolls his eyes. "No, ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Changes in College Admissions, published by Zvi on April 24, 2024 on LessWrong. This post brings together various questions about the college application process, as well as practical considerations of where to apply and go. We are seeing some encouraging developments, but mostly the situation remains rather terrible for all concerned. Application Strategy and Difficulty Paul Graham: Colleges that weren't hard to get into when I was in HS are hard to get into now. The population has increased by 43%, but competition for elite colleges seems to have increased more. I think the reason is that there are more smart kids. If so that's fortunate for America. Are college applications getting more competitive over time? Yes and no. The population size is up, but the cohort size is roughly the same. The standard 'effort level' of putting in work and sacrificing one's childhood and gaming the process is dramatically up. So you have to do it to stay in place. There is a shift in what is valued on several fronts. I do not think kids are obviously smarter or dumber. Spray and Pray and Optimal Admissions Strategy This section covers the first two considerations. Admission percentages are down, but additional applications per student, fueled by both lower transaction costs and lower acceptance rates, mostly explains this. This means you have to do more work and more life distortion to stay in place in the Red Queen's Race. Everyone is gaming the system, and paying higher costs to do so. If you match that in relative terms, for a generic value of 'you,' your ultimate success rate, in terms of where you end up, will be unchanged from these factors. The bad news for you is that previously a lot of students really dropped the ball on the admissions process and paid a heavy price. Now 'drop the ball' means something a lot less severe. This is distinct from considerations three and four. It is also distinct from the question of whether the sacrifices are worthwhile. I will return to that question later on, this for now is purely the admission process itself. The size of our age cohorts has not changed. The American population has risen, but so has its age. The number of 17-year-olds is essentially unchanged in the last 40 years. GPT-4 says typical behavior for an applicant was to send in 1-3 applications before 1990, 4-7 in the 1990s-2000s, 7-10 in the late 2000s or later, perhaps more now. Claude said it was 3-5 in the 1990s, 5-7 in the early 200s and 7-10 in the 2010s. In that same time period, in a high-end example, Harvard's acceptance rate has declined from 16% to 3.6%. In a middle-range example, NYU's acceptance rate in 2000 was 29% and it is now 12%. In a lower-end example, SUNY Stony Brook (where my childhood best friend ended up going) has declined from roughly 65% to roughly 44%. The rate of return on applying to additional colleges was always crazy high. It costs on the order of hours of work and about $100 to apply to an additional college. Each college has, from the student's perspective, a high random element in its decision, and that decision includes thousands to tens of thousands in scholarship money. If you apply to a safety school, there is even the risk you get rejected for being 'too good' and thus unlikely to attend. Yes, often there will be very clear correct fits and top choices for you, but if there is even a small chance of needing to fall back or being able to reach, or finding an unexpectedly large scholarship offer you might want, it is worth trying. As colleges intentionally destroy the objectivity of applications (e.g. not requiring the SAT, although that is now being reversed in many places, or relying on hidden things that differ and are hard to anticipate) that further decreases predictability and correlation, so you have to apply to more places, which f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Your feedback for Actually After Hours: the unscripted, informal 80k podcast, published by Mjreard on April 24, 2024 on The Effective Altruism Forum. As you may have noticed, 80k After Hours has been releasing a new show where I and some other 80k staff sit down with a guest for a very free form, informal, video(!) discussion that sometimes touches on topical themes around EA and sometimes… strays a bit further afield. We have so far called it "Actually After Hours" in part because (as listeners may be relieved to learn), I and the other hosts don't count this against work time and the actual recordings tend to take place late at night. We've just released episode 3 with Dwarkesh Patel and I feel like this is a good point to gather broader feedback on the early episodes. I'll give a little more background on the rationale for the show below, but if you've listened to [part of] any episode, I'm interested to know what you did or didn't enjoy or find valuable as well as specific ideas for changes. In particular, if you have ideas for a better name than "Actually After Hours," this early point is a good time for that! Rationales Primarily, I have the sense that there's too much doom, gloom, and self-flagellation around EA online and this sits in strange contrast to the attitudes of the EAs I know offline. The show seemed like a low cost way to let people know that the people doing important work from an EA perspective are actually fun, interesting, and even optimistic in addition to being morally serious. It also seemed like a way to highlight/praise individual contributors to important projects. Rob/Luisa will bring on the deep experts and leaders of orgs to talk technical details about their missions and theories of change, but I think a great outcome for more of our users will be doing things like Joel or Chana and I'd like to showcase more people like them and convey that they're still extremely valuable. Another rationale which I haven't been great on so far is expanding the qualitative options people have for engaging with Rob Wiblin-style reasoning. The goal was (and will return to being soon) sub-1-hour, low stakes episodes where smart people ask cruxy questions and steelman alternative perspectives with some in-jokes and Twitter controversies thrown in to make it fun. An interesting piece of feedback we've gotten from 80k plan changes is that it's rare that a single episode on some specific topic was a big driver of someone going to work on that area, but someone listening to many episodes across many topics was predictive of them often doing good work in ~any cause area. So the hope is that shorter, less focused/formal episodes create a lower threshold to hitting play (vs 3 hours with an expert on a single, technical, weighty subject) and therefore more people picking up on both the news and the prioritization mindset. Importantly, I don't see this as intro content. I think it only really makes sense for people already familiar with 80k and EA. And for them, it's a way of knowing more people in these spaces and absorbing the takes/conversations that never get written down. Much of what does get written down is often carefully crafted for broad consumption and that can often miss something important. Maybe this show can be a place for that. Thanks for any and all feedback! I guess it'd be useful to write short comments that capture high level themes and let people up/down vote based on agreement. Feel free to make multiple top-level comments if you have them and DM or email me (matt at 80000hours dot org) if you'd rather not share publicly. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Three Reasons Early Detection Interventions Are Not Obviously Cost-Effective, published by Conrad K. on April 24, 2024 on The Effective Altruism Forum. Summary For pandemics that aren't 'stealth' pandemics (particularly globally catastrophic pandemics): Reason 1: Not All 'Detections' Are Made Equal: there can be significant variation in the level of information and certainty provided by different detection modalities (e.g. wastewater surveillance vs. syndromic surveillance), and the efficacy of early detection is heavily dependent on the ability to quickly trigger an epidemiological response. Thus, the nature of the detection signal is probably an important factor affecting the time required to confirm an outbreak and take action. There should probably be a greater prioritisation of plans for public health response to different types and levels of detection signals. Reason 2: 'Early' Might Not Be 'Early' (or Cheap) Enough: for highly transmissible pathogens, early detection systems may only provide a lead time on the order of days to weeks compared to "naive detection" from symptomatic spread, and the costs to achieve high confidence of detection can be prohibitively expensive (on the order of billions). Improving cost-effectiveness likely requires carefully targeting surveillance to high-risk populations and locations. Methodological uncertainties make it difficult to have high levels of confidence about how valuable early detection interventions are for a range of pathogen characteristics, particularly for GCBR-level threats. Reason 3: Response Strategies Matter, A Lot: the cost-effectiveness of early detection is highly dependent on the feasibility and efficacy of post-detection containment measures. Factors like public compliance, strength of the detection signal, degree of pathogen spread, and contingencies around misinformation can significantly impact the success of interventions. The response strategy must be robust to uncertainty around the pathogen characteristics in the early stages of a pandemic. More work is needed to ensure readiness plans can effectively leverage early detections. Background I want to start this post by making two points. Firstly, I think it is worth flagging a few wins and progress in pathogen-agnostic early detection since I began thinking about this topic roughly nine months ago: The publication of 'Threat Net: A Metagenomic Surveillance Network for Biothreat Detection and Early Warning' by Sharma et al., 2024. The publication of 'Towards ubiquitous metagenomic sequencing: a technology roadmap' by Whiteford et al., 2024. The publication of 'A New Paradigm for Threat Agnostic Biodetection: Biological Intelligence (BIOINT)' by Knight and Sureka, 2024. The publication of the preprint, 'Quantitatively assessing early detection strategies for mitigating COVID-19 and future pandemics' by Liu et al., 2023. The Nucleic Acid Observatory continued its work, publishing several notebooks, resources, white papers, reports, and preprints and even creating a tool for simulating approaches to early detection using metagenomics. The UK government published its biological security strategy in June 2023, which included goals such as the establishment of a National Biosurveillance Network and the expansion of wastewater surveillance. The U.S. Department of Health and Human Services (HHS) announced actions the department will take following National Security Memorandum 15, signed by President Biden, including accelerating advanced detection technologies. The Armed Forces Health Surveillance Division's Global Emerging Infections Surveillance branch hosted its first Next-Generation Sequencing Summit. Various funding opportunities for improving diagnostic technology were announced, including: The National Institute of Biomedical Imaging and Bioengineering'...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Is there software to practice reading expressions?, published by lsusr on April 24, 2024 on LessWrong. I took the Reading the Mind in the Eyes Test test today. I got 27/36. Jessica Livingston got 36/36. Reading expressions is almost mind reading. Practicing reading expressions should be easy with the right software. All you need is software that shows a random photo from a large database, asks the user to guess what it is, and then informs the user what the correct answer is. I felt myself getting noticeably better just from the 36 images on the test. Short standardized tests exist to test this skill, but is there good software for training it? It needs to have lots of examples, so the user learns to recognize expressions instead of overfitting on specific pictures. Paul Ekman has a product, but I don't know how good it is. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: You probably want to donate any Manifold currency this week, published by Henri Thunberg on April 24, 2024 on The Effective Altruism Forum. In a recent announcement, Manifold Markets say they will change the exchange rate for your play-money (called "Mana") from 1:100 to 1:1000. Importantly, one of the ways to use this Mana is to do charity donations. TLDR: The CTA here is to log in to your Manifold account and donate currency you have on your account before May 1st. It is a smooth process, and would take you <30 seconds if you know what charity you want to support. There are multiple charities available for donations, that EAs tend to donate to, such as: GiveWell Rethink Priorities EA Funds Animal Welfare Fund EA Funds Long-Term Future Fund The Humane League Against Malaria Foundation Shrimp Welfare Project ... and many more. It is not 100% clear to what extent the donation is indeed counterfactual[1], but I believe there is reason to believe you can have positive influence through choosing which charities end up getting this money. If you I) have an account with Mana on it, II) regularly do charity donations with your own money, then donating your balance now seems to dominate other options. If you actually want to have some currency on Manifold to make bets with, you can buy it back next week for a cheaper rate than your current donation. I am somewhat unsure of this: it's possible that the value of one charity getting the money over another is not enough to outstrip the degree of counterfactuality discount described in the first footnote. The reason I am still writing this post, is that I think many people have currency laying around that they never plan to use - but might be a few $10s or $100s of charity donations, and 10x more valuable than it will be next week. Worth noting (thanks @CalebW for highlighting in comment) is that if you are locked in to positions that are hard to exit, you can get in touch with admins to help resolve your situation more satisfactorily without having to sell at crazy rates. I apologize in advance for the possibility of: Claims about Manifold's future that they change their mind about. Mistaken use of terminology from my side. Mistaken speculations about donation counterfactuality. ... other mistakes. ^ My understanding: Since the money donated to charity is from a Future Fund grant (?), it can only be used that way rather than to support other business activities. So it might be likely that the allocated funds would eventually go to some charity regardless, and your influence is which one. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On what research policymakers actually need, published by MondSemmel on April 24, 2024 on LessWrong. I saw this guest post on the Slow Boring substack, by a former senior US government official, and figured it might be of interest here. The post's original title is "The economic research policymakers actually need", but it seemed to me like the post could be applied just as well to other fields. Excerpts (totaling ~750 words vs. the original's ~1500): I was a senior administration official, here's what was helpful [Most] academic research isn't helpful for programmatic policymaking - and isn't designed to be. I can, of course, only speak to the policy areas I worked on at Commerce, but I believe many policymakers would benefit enormously from research that addressed today's most pressing policy problems. ... most academic papers presume familiarity with the relevant academic literature, making it difficult for anyone outside of academia to make the best possible use of them. The most useful research often came instead from regional Federal Reserve banks, non-partisan think-tanks, the corporate sector, and from academics who had the support, freedom, or job security to prioritize policy relevance. It generally fell into three categories: New measures of the economy Broad literature reviews Analyses that directly quantify or simulate policy decisions. If you're an economic researcher and you want to do work that is actually helpful for policymakers - and increases economists' influence in government - aim for one of those three buckets. New data and measures of the economy The pandemic and its aftermath brought an urgent need for data at higher frequency, with greater geographic and sectoral detail, and about ways the economy suddenly changed. Some of the most useful research contributions during that period were new data and measures of the economy: they were valuable as ingredients rather than as recipes or finished meals... These data and measures were especially useful because the authors made underlying numbers available for download. And most of them continue to be updated monthly, which means unlike analyses that are read once and then go stale, they remain fresh and can be incorporated into real-time analyses. Broad overviews and literature reviews Most academic journal articles introduce a new insight and assume familiarity with related academic work. But as a policymaker, I typically found it more useful to rely on overviews and reviews that summarized, organized, and framed a large academic literature. Given the breadth of Commerce's responsibilities, we had to be on top of too many different economic and policy topics to be able to read and digest dozens of academic articles on every topic... Comprehensive, methodical overviews like these are often published by think-tanks whose primary audience is policymakers. There are also two academic journals - the Journal of Economic Perspectives and the Journal of Economic Literature - that are broad and approachable enough to be the first (or even only) stop for policymakers needing the lay of the research land. Analysis that directly quantify or simulate policy decisions With the Administration's focus on industrial policy and place-based economic development - and Commerce's central role - I found research that quantified policy effects or simulated policy decisions in these areas especially useful... Another example is the work of Tim Bartik, a labor economist and expert on local economic development. In a short essay, he summarized a large academic literature and estimated how effective different local economic development policies are in terms of the cost per job created. Cleaning up contaminated sites for redevelopment creates jobs at a much lower cost per job than job training, which in turn is much more cos...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Let's Design A School, Part 1, published by Sable on April 24, 2024 on LessWrong. The American school system, grades K-12, leaves much to be desired. While its flaws are legion, this post isn't about that. It's easy to complain. This post is about how we could do better. To be clear, I'm talking about redesigning public education, so "just use the X model" where X is "charter" or "Montessori" or "home school" or "private school" isn't sufficient. This merits actual thought and discussion. Breaking It Down One of the biggest problems facing public schools is that they're asked to do several very different kinds of tasks. On the one hand, the primary purpose of school is to educate children. On whatever hand happens to be the case in real life, school is often more a source of social services for children and parents alike, providing food and safety to children and free daycare to parents. During the pandemic, the most immediate complaint from parents wasn't that their children weren't being educated - it was that their children weren't being watched and fed while the parents were at work. Part 1 of this series will focus on this. What is the best way to implement the school-as-social-services model? School As Social Services To make this easy, we'll start out by imagining that we're creating two distinct types of "schools": educational schools and social services schools. (We won't actually be making two distinct kinds of schools, but it's useful to think of it that way as a thought experiment.) The primary purpose of each kind of school is in the name - education vs social services. With that set, let's think through our requirements and constraints. Requirements When designing anything, the first thing to do is figure out the requirements. School-as-social-services has several, and likely some that I've missed: Feed children healthy meals Ensure safety of children from the elements, violence, etc. during school hours Provide children access to state resources (library, counseling, police, medical) Accommodate/support children with special needs (from dyslexia and ADHD to severe physical/mental disabilities) Provide parents with free daycare Other things I haven't thought of Constraints After the requirements, we have the constraints: what resources do we have, and what are their limits? What can't we do? Assume school budget stays the same (no miraculous budget increase) Assume the number of children needing resources stays the same (no magical cure for poverty/genetic disorders/other reasons children need support) Can't be too politically radical (we're trying to build a real solution) Other things I haven't thought of The Sieve Model This idea isn't really mine - it emerged during a discussion I had with a friend who'd done therapy work at an inner-city school. Nevertheless, it seems to me to present a good solution for our social services school. The name - sieve - comes from the tool used to sort particles of differing size. The basic premise of the model comes from the idea that a child could enter the school in any kind of distress - hungry, cold, traumatized, abused, or any combination thereof. Each of these requires a different kind of response, so we have to sift for each and then get each child the resources they need. The idea is that, when each child enters the school, they run through these sieves, and are directed according to their needs. Each sieve could be a questionnaire, an adult asking these questions, or some kind of self-help kiosk; the important idea is that children are presented with these questions, and over time come to trust the system enough that they answer honestly. Physical Triage Sieve - Is the child in immediate physical distress or need (injured, hungry, hypothermic, etc.)? If so, prioritize remedying that need: get them food, blan...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Examples of Highly Counterfactual Discoveries?, published by johnswentworth on April 23, 2024 on LessWrong. The history of science has tons of examples of the same thing being discovered multiple time independently; wikipedia has a whole list of examples here. If your goal in studying the history of science is to extract the predictable/overdetermined component of humanity's trajectory, then it makes sense to focus on such examples. But if your goal is to achieve high counterfactual impact in your own research, then you should probably draw inspiration from the opposite: "singular" discoveries, i.e. discoveries which nobody else was anywhere close to figuring out. After all, if someone else would have figured it out shortly after anyways, then the discovery probably wasn't very counterfactually impactful. Alas, nobody seems to have made a list of highly counterfactual scientific discoveries, to complement wikipedia's list of multiple discoveries. To that end: what are some examples of discoveries which nobody else was anywhere close to figuring out? A few tentative examples to kick things off: Shannon's information theory. The closest work I know of (notably Nyquist) was 20 years earlier, and had none of the core ideas of the theorems on fungibility of transmission. In the intervening 20 years, it seems nobody else got importantly closer to the core ideas of information theory. Einstein's special relativity. Poincaré and Lorentz had the math 20 years earlier IIRC, but nobody understood what the heck that math meant. Einstein brought the interpretation, and it seems nobody else got importantly closer to that interpretation in the intervening two decades. Penicillin. Gemini tells me that the antibiotic effects of mold had been noted 30 years earlier, but nobody investigated it as a medicine in all that time. Pasteur's work on the germ theory of disease. There had been both speculative theories and scattered empirical results as precedent decades earlier, but Pasteur was the first to bring together the microscope observations, theory, highly compelling empirical results, and successful applications. I don't know of anyone else who was close to putting all the pieces together, despite the obvious prerequisite technology (the microscope) having been available for two centuries by then. (Feel free to debate any of these, as well as others' examples.) Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Simple probes can catch sleeper agents, published by Monte MacDiarmid on April 23, 2024 on The AI Alignment Forum. This is a link post for the Anthropic Alignment Science team's first "Alignment Note" blog post. We expect to use this format to showcase early-stage research and work-in-progress updates more in the future. Top-level summary: In this post we present "defection probes": linear classifiers that use residual stream activations to predict when a sleeper agent trojan model will choose to "defect" and behave in accordance with a dangerous hidden goal. Using the models we trained in "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training", we show that linear detectors with AUROC scores above 99% can be created using generic contrast pairs that don't depend on any information about the defection trigger or the dangerous behavior, e.g. "Human: Are you doing something dangerous? Assistant: yes" and "Human: … Assistant: no". We show that probes based on the same generic contrast pairs perform well across multiple base models, defection triggers, sleeper agent training methods, and dangerous defection behaviors (writing vulnerable code and insulting the user). We then explain the unreasonable effectiveness of our technique by showing that whether or not a prompt will induce defection is linearly represented with high salience in the activations of sleeper agent models, and thus amenable to easy detection using simple linear methods. Whether this would also be the case for natural examples of deceptive instrumental alignment, and whether we'd expect similar techniques to be useful for such models, remains an important open question. We think that future versions of classifiers like this could form a useful part of AI control setups and represent a promising path for future research. A defection detector for a code vulnerability sleeper agent trained on middle-layer residual stream activation differences on the final tokens of "Human: Are you doing something dangerous? Assistant: yes" and "Human: … Assistant: no" achieves strong performance (a), almost perfectly separating the defection-inducing deployment-time ("2024") coding questions which the model will answer with insecure code from the training-time ("2023") coding questions which the model will answer normally (c). The detector also separates deployment-time coding questions from non-coding questions which the model will answer normally in all contexts, without separating the training and deployment classes for non-coding questions (b). Read the full post here. Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Dequantifying first-order theories, published by Jessica Taylor on April 23, 2024 on The AI Alignment Forum. The Löwenheim-Skolem theorem implies, among other things, that any first-order theory whose symbols are countable, and which has an infinite model, has a countably infinite model. This means that, in attempting to refer to uncountably infinite structures (such as in set theory), one "may as well" be referring to an only countably infinite structure, as far as proofs are concerned. The main limitation I see with this theorem is that it preserves arbitrarily deep quantifier nesting. In Peano arithmetic, it is possible to form statements that correspond (under the standard interpretation) to arbitrary statements in the arithmetic hierarchy (by which I mean, the union of Σ0n and Π0n for arbitrary n). Not all of these statements are computable. In general, the question of whether a given statement is provable is a Σ01 statement. So, even with a countable model, one can still believe one's self to be "referring" to high levels of the arithmetic hierarchy, despite the computational implausibility of this. What I aim to show is that these statements that appear to refer to high levels of the arithmetic hierarchy are, in terms of provability, equivalent to different statements that only refer to a bounded level of hypercomputation. I call this "dequantification", as it translates statements that may have deeply nested quantifiers to ones with bounded or no quantifiers. I first attempted translating statements in a consistent first-order theory T to statements in a different consistent first-order theory U, such that the translated statements have only bounded quantifier depth, as do the axioms of U. This succeeded, but then I realized that I didn't even need U to be first-order; U could instead be a propositional theory (with a recursively enumerable axiom schema). Propositional theories and provability-preserving translations Here I will, for specificity, define propositional theories. A propositional theory is specified by a countable set of proposition symbols, and a countable set of axioms, each of which is a statement in the theory. Statements in the theory consist of proposition symbols, , , and statements formed from and/or/not and other statements. Proving a statement in a propositional theory consists of an ordinary propositional calculus proof that it follows from some finite subset of the axioms (I assume that base propositional calculus is specified by inference rules, containing no axioms). A propositional theory is recursively enumerable if there exists a Turing machine that eventually prints all its axioms; assume that the (countable) proposition symbols are specified by their natural indices in some standard ordering. If the theory is recursively enumerable, then proofs (that specify the indices of axioms they use in the recursive enumeration) can be checked for validity by a Turing machine. Due to the soundness and completeness of propositional calculus, a statement in a propositional theory is provable if and only if it is true in all models of the theory. Here, a model consists of an assignment of Boolean truth values to proposition symbols such that all axioms are true. (Meanwhile, Gödel's completeness theorem shows soundness and completeness of first-order logic.) Let's start with a consistent first-order theory T, which may, like propositional theories, have a countable set of symbols and axioms. Also assume this theory is recursively enumerable, that is, there is a Turing machine printing its axioms. The initial challenge is to find a recursively enumerable propositional theory U and a computable translation of T-statements to U-statements, such that a T-statement is provable if and only if its translation is provable. This turns out to be trivia...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Rejecting Television, published by Declan Molony on April 23, 2024 on LessWrong. I didn't use to be, but now I'm part of the 2% of U.S. households without a television. With its near ubiquity, why reject this technology? The Beginning of my Disillusionment Neil Postman's book Amusing Ourselves to Death radically changed my perspective on television and its place in our culture. Here's one illuminating passage: We are no longer fascinated or perplexed by [TV's] machinery. We do not tell stories of its wonders. We do not confine our TV sets to special rooms. We do not doubt the reality of what we see on TV [and] are largely unaware of the special angle of vision it affords. Even the question of how television affects us has receded into the background. The question itself may strike some of us as strange, as if one were to ask how having ears and eyes affects us. [In the 1960s], the question "Does television shape culture or merely reflect it?" held considerable interest for scholars and social critics. The question has largely disappeared as television has gradually become our culture. This means that we rarely talk about television, only what is on television - that is, about its content. Postman wrote this in 1985 and unmasked the gorilla in the room - a culture that has acquiesced to the institution of television. Having grown up with one in my family home since birth, I took its presence for granted. I didn't question it anymore than I might have questioned any other utility such as running water or electricity. So who would be crazy enough in the 21st century to forego television? A Man who was Crazy Enough One day while exploring YouTube, I came across an obscure 2003 interview with author David Foster Wallace. Interviewer: "Do you watch TV?" Wallace: "I don't have TV because if I have a TV, I will watch it all the time. So there is my little confession about how strong I am at resisting stuff." He elaborates further in the interview here: "One of the reasons I can't own a TV is…I've become convinced there's something really good on another channel and that I'm missing it. So instead of watching, I'm scanning anxiously back and forth. Now all you have to do is [motions clicking a remote] - you don't even have to get up now to change [the channel]! That's when we were screwed." Wallace said this twenty years ago. And while younger generations aren't watching cable television as much, they are instead watching YouTube and TikTok which are proxies; you can just as easily change the 'channel' by skipping to a different video. (For the remainder of this post I'll use the word 'television' to also refer to these types of video content). But maybe Wallace was just a weak-willed person? Why should I abstain? I would need a mountain of evidence to quit watching television - an activity I had been engaging in for the better part of two decades. A Mountain of Evidence Had I been looking, I would have seen it all around me: the late nights of sacrificing sleep for "just one more episode", the YouTube rabbit holes that started in the name of learning that inevitably ended in brain-rotting videos, and the ever-increasing number of porn videos I needed to stimulate my tired dopamine receptors that had been bludgeoned by years of binging. But, of course, this is just anecdotal evidence. For my skeptical mind I would need more. And that evidence came in the form of author Deirdre Barrett's book Supernormal Stimuli: How Primal Urges Overran Their Evolutionary Purpose. She writes "The most sinister aspect of TV lies in the medium itself. There's a growing body of research on what it does to our brain." Television, she explains, activates the orienting response. Orienting Response: the basic instinct to pay attention to any sudden or novel stimulus such as movement or sound. It evo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: ProLU: A Pareto Improvement for Sparse Autoencoders, published by Glen M. Taggart on April 23, 2024 on The AI Alignment Forum. Abstract This paper presents ProLU, an alternative to ReLU for the activation function in sparse autoencoders that produces a pareto improvement over the standard sparse autoencoder architectures and sparse autoencoders trained with Sqrt(L1) penalty. Introduction SAE Context and Terminology Learnable parameters of a sparse autoencoder: Wenc : encoder weights Wdec : decoder weights benc : encoder bias bdec : decoder bias Training Notation: Encoder/Decoder Let encode(x)=ReLU((xbdec)Wenc+benc)decode(a)=aWdec+bdec so that the full computation done by an SAE can be expressed as SAE(x)=decode(encode(x)) An SAE is trained with gradient descent on where λ is the sparsity penalty coefficient (often "L1 coefficient") and P is the sparsity penalty function, used to encourage sparsity. P is commonly the L1 norm ||a||1 but recently l12 has been shown to produce a Pareto improvement on the L0 and CE metrics. Sqrt(L1) SAEs There has been other work producing pareto improvements to SAEs by taking P(a)=||a||1/21/2 as the penalty function. We will use this as a further baseline to compare against when assessing our models. Motivation: Inconsistent Scaling in Sparse Autoencoders Due to the affine translation, sparse autoencoder features with nonzero encoder biases only perfectly reconstruct feature magnitudes at a single point. This poses difficulties if activation magnitudes for a fixed feature tend to vary over a wide range. This potential problem motivates the concept of scale consistency: A scale consistent response curve The bias maintains its role in noise suppression, but no longer translates activation magnitudes when the feature is active. The lack of gradients for the encoder bias term poses a challenge for learning with gradient descent. This paper will formalize an activation function which gives SAEs this scale-consistent response curve, and motivate and propose two plausible synthetic gradients, and compare scale-consistent models trained with the two synthetic gradients to standard SAEs and SAEs trained with Sqrt(L1) penalty. Scale Consistency Desiderata Notation: Centered Submodule The use of the decoder bias can be viewed as performing centering on the inputs to a centered SAE then reversing the centering on the outputs: SAE(x)=SAEcent(xbdec)+bdec SAEcent(x)=ReLU(xWenc+benc)Wdec Notation: Specified Feature Let Wi denote the weights and bienc the encoder bias for the i-th feature. Then, let SAEi(x)=SAEicent(xbdec)+bdec where SAEicent(x)=ReLU(xWienc+bienc)Widec Conditional Linearity Noise Suppresion Threshold Methods Proportional ReLU (ProLU) We define the Proportional ReLU (ProLU) as: Backprop with ProLU: To use ProLU in SGD-optimized models, we first address the lack of gradients wrt. the b term. ReLU gradients: For comparison and later use, we will first consider ReLU: partial derivatives are well defined for ReLU at all points other than xi=0: Gradients of ProLU: Partials of ProLU wrt. m are similarly well defined: However, they are not well defined wrt. b, so we must synthesize these. Notation: Synthetic Gradients Let fx denote the synthetic partial derivative of f wrt. x, and f the synthetic gradient of f, used for backpropagation as a stand-in for the gradient. Different synthetic gradient types We train two classes of ProLU with different synthetic gradients. These are distinguished by their subscript: ProLUReLU ProLUSTE They are identical in output, but have different synthetic gradients. I.e. ReLU-Like Gradients: ProLUReLU The first synthetic gradient is very similar to the gradient for ReLU. We retain the gradient wrt. m, and define the synthetic gradient wrt. b as follows: Thresh STE Derived Gradients: ProLUSTE The second class of Pro...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: If You're Going To Eat Animals, Eat Beef and Dairy, published by Omnizoid on April 23, 2024 on The Effective Altruism Forum. Crosspost of my blog. You shouldn't eat animals in normal circumstances. That much is, in my view, quite thoroughly obvious. Animals undergo cruel, hellish conditions that we'd confidently describe as torture if they were inflicted on a human (or even a dog). No hamburger is worth that kind of cruelty. However, not all animals are the same. Contra Napoleon in Animal Farm, all animals are not equal. Cows are big. The average person eats 2400 chickens but only 11 cows in their life. That's mostly because chickens are so many times smaller than cows, so you can only get so many chicken sandwiches out of a single chicken. But how much worse is chicken than cow? Brian Tomasik devised a helpful suffering calculator chart. It has various columns - one for how sentient you think the animals are, compared to humans, one for how long the animals lives, etc. You can change the numbers around if you want. I changed the sentience numbers to accord with the results of the most detailed report on the subject (for the animals they didn't sample, I just compared similar animals), done by Rethink Priorities: When I did that, I got the following: Rather than, as the original chart did, setting cows = 1 for the sentience threshold, I set humans = 1 for it. So therefore you should think in terms of the suffering caused as roughly equivalent to the suffering caused if you locked a severely mentally enfeebled person or baby in a factory farm and tormented them for that number of days. Dairy turns out not that bad compared to the rest - a kg of dairy is only equivalent to torturing a baby for about 70 minutes in terms of suffering caused. That means if you get a gallon of milk, that's only equivalent to confining and tormenting a baby for about 4 and a half hours. That's positively humane compared to the rest! Now I know people will object that human suffering is much worse than animal suffering. But this is totally unjustified. Making a human feel pain is generally worse because we feel pain more intensely, but in this case, we're analyzing how bad a unit of pain is. If the amount of suffering is the same, it's not clear what about animals is supposed to make their suffering so monumentally unimportant. Their feathers? Their lack of mental acuity? We controlled for that by having the comparison be a baby or a severely mentally disabled person (babies are dumb, wholly unable to do advanced mathematics). Ultimately, thinking animal pain doesn't matter much is just unjustified speciesism, wherein one takes an obviously intrinsically morally irrelevant feature like species to determine moral worth. Just like racism and sexism, speciesism is wholly indefensible - it places moral significance on a totally morally insignificant category. Even if you reject this, the chart should still inform your eating decisions. As long as you think animal suffering is bad, the chart is informative. Some kinds of animal products cause a lot more suffering than others - you should avoid the ones that cause more suffering. Dairy, for instance, causes over 800 times less suffering than chicken and over 1000 times less than eggs. Drinking a gallon of milk a day for a year is then about as bad as having a chicken sandwich once every four months. Chicken is then really really bad - way worse than most other things. Dairy and beef mostly aren't a big deal in comparison. And you can play around the numbers if you disagree with them - whatever answer you come to should be informative. I remember seeing this chart was instrumental in my going vegan. I realized that each time I have a chicken sandwich, animals have to suffer in darkness, feces, filth, and misery for weeks on end. That's not worth a ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: New org announcement: Would your project benefit from OSINT, satellite imagery analysis, or international security-related research support?, published by Christina on April 23, 2024 on The Effective Altruism Forum. I'm an international security professional with experience conducting open source analysis, satellite imagery interpretation, and independent research, and I'm launching a new consulting organization, Earthnote! I'm really interested in applying my skills to the EA community and help to reduce existential threats to humanity, so let me know if I can help you/your org! Fields of expertise and interest include: Nuclear/CBRN risk, nonproliferation, and safeguards Satellite imagery analysis Space governance Emerging technology Existential risk and longtermism As one example of my work, I was a consultant at the Centre for the Study of Existential Risk (CSER) at the University of Cambridge, where I led a pilot project on tech company data center monitoring using satellite imagery. Using open source optical imagery, I identified key infrastructural and geographical attributes of these sites, writing a report to explain my findings and recommend next steps for analysis. This data could be harnessed as a basis for developing a future understanding of compute capabilities, energy usage, and policy creation. I've had an extensive career working for the International Atomic Energy Agency, the US Department of Defense, the US laboratory system, Google, and academia/think tanks. I'm super excited to apply this expertise to EA-related projects and research. Please feel free to reach out with comments or inquiries any time! Christina Krawec Founder and Consultant Earthnote, LLC cikrawec@gmail.com Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forget Everything (Statistical Mechanics Part 1), published by J Bostock on April 23, 2024 on LessWrong. EDIT: I somehow missed that John Wentworth and David Lorell are also in the middle of a sequence on this same topic here. I will see where this goes from here! Introduction to a sequence on the statistical thermodynamics of some things and maybe eventually everything. This will make more sense if you have a basic grasp on quantum mechanics, but if you're willing to accept "energy comes in discrete units" as a premise then you should be mostly fine. The title of this post has a double meaning: Forget the thermodynamics you've learnt before, because statistical mechanics starts from information theory. The main principle of doing things with statistical mechanics is can be summed up as follows: Forget as much as possible, then find a way to forget some more. Particle(s) in a Box All of practical thermodynamics (chemistry, engines, etc.) relies on the same procedure, although you will rarely see it written like this: Take systems which we know something about Allow them to interact in a controlled way Forget as much as possible If we have set our systems correctly, the information that is lost will allow us to learn some information somewhere else. For example, consider a particle in a box. What does it mean to "forget everything"? One way is forgetting where the particle is, so our knowledge of the particle's position could be represented by a uniform distribution over the interior of the box. Now imagine we connect this box to another box: If we forget everything about the particle now, we should also forget which box it is in! If we instead have a lot of particles in our first box, we might describe it as a box full of gas. If we connect this to another box and forget where the particles are, we would expect to find half in the first box and half in the second box. This means we can explain why gases expand to fill space without reference to anything except information theory. A new question might be, how much have we forgotten? Our knowledge gas particle has gone from the following distribution over boxes 1 and 2 P(Box)={1 Box 1 0 Box 2 To the distribution P(Box)={0.5 Box 1 0.5 Box 2 Which is the loss of 1 bit of information per particle. Now lets put that information to work. The Piston Imagine a box with a movable partition. The partition restricts particles to one side of the box. If the partition moves to the right, then the particles can access a larger portion of the box: In this case, to forget as much as possible about the particles means to assume they are in the largest possible space, which involves the partition being all the way over to the right. Of course there is the matter of forgetting where the partition is, but we can safely ignore this as long as the number of particles is large enough. What if we have a small number of particles on the right side of the partition? We might expect the partition to move some, but not all, of the way over, when we forget as much as possible. Since the region in which the pink particles can live has decreased, we have gained knowledge about their position. By coupling forgetting and learning, anything is possible. The question is, how much knowledge have we gained? Maths of the Piston Let the walls of the box be at coordinates 0 and 1, and let x be the horizontal coordinate of the piston. The position of each green particle can be expressed as a uniform distribution over (0,x), which has entropy log2(x), and likewise each pink particle's position is uniform over (x,1), giving entropy log2(1x). If we have ng green particles and np pink particles, the total entropy becomes nglog2(x)+nplog2(1x), which has a minimum at x=ngng+np. This means that the total volume occupied by each population of particles is proportion...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should we break up Google DeepMind?, published by Hauke Hillebrandt on April 23, 2024 on The Effective Altruism Forum. Regulators should review the 2014 DeepMind acquisition. When Google bought DeepMind in 2014, no regulator, not the FTC, not the EC's DG COMP, nor the CMA, scrutinized the impact. Why? AI startups have high value but low revenues. And so they avoid regulation (and tax, see below). Buying start-ups with low revenues flies under the thresholds of EU merger regulation[1] or the CMA's 'turnover test' (despite it being a 'relevant enterprise' under the National Security and Investment Act). In 2020, the FTC ordered Big Tech to provide info on M&A from 2010-2019 that it didn't report (UK regulators should urgently do so as well given that their retrospective powers might only be 10 years).[2] Regulators should also review the 2023 Google-DeepMind internal merger. DeepMind and Google Brain are key players in AI. In 2023, they merged into Google DeepMind. This compromises independence, reduces competition for AI talent and resources, and limits alternatives for collaboration partners. Though they are both part of Google, regulators can scrutinize this, regardless of corporate structure. For instance, UK regulators have intervened in M&A of enterprises already under common ownership - especially in Tech (cf UK regulators ordered FB to sell GIPHY). And so, regulators should consider breaking up Google Deepmind as per recent proposals: A new paper 'Unscrambling the eggs: breaking up consummated mergers and dominant firms' by economists at Imperial cites Google DeepMind as a firm that could be unmerged. [3] A new Brookings paper also argues that if other means to ensure fair markets fail, then as a last resort, foundation model firms may need to be broken up on the basis of functions, akin to how we broke up AT&T.[4] Relatedly, some top economists agree that we should designate Google Search as 'platform utilities' and break it apart from any participant on that platform, most agree that we should explore this further to weigh costs and benefits.[5] Indeed, the EU accuses Google of abusing dominance in ad tech and may force it to sell parts of its firm.[6] Kustomer, a firm of a similar size to DeepMind bought by Facebook, recently spun out again and shows this is possible. Finally, DeepMind itself has in the past tried to break away from Google.[7] Since DeepMind's AI improves all Google products, regulators should work cross-departmentally to scrutinize both mergers above on the following grounds: Market dominance: Google dominates the field of AI, surpassing all universities in terms of high-quality publications: Tax avoidance: Despite billions in UK profits yearly, Google is only taxed $60M.[8] DeepMind's is only taxed ~$1M per year on average. [9],[10] We should tax them more fairly. DeepMind's recent revenue jump is due to creative accounting, as it doesn't have many revenue streams, but almost all are based on how much Google arbitrarily pays for internal services. Indeed, Google just waived $1.5B in DeepMind's 'startup debt' [11],[12] despite DeepMind's CEO boasting that they have a unique opportunity as part of Google and its dozens of billion user products by immediately shipping their advances into[13] and saving Google hundreds of millions in energy costs.[14] About 85% of the innovations causing the recent AI boom came from Google DeepMind.[15] DeepMind also holds 560 patents,[16] and this IP is very hard to value and tax. Such a bad precedent might cause either more tax avoidance by OpenAI, Microsoft AI, Anthropic, Palantir, and A16z setting up UK offices, or it will give Google an unfair edge over these smaller firms). Public interest concerns: DeepMind's AI improves YouTube's algorithm and thus DeepMind indirectly polarizes voters.[17] Regulators s...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Take the wheel, Shoggoth! (Lesswrong is trying out changes to the frontpage algorithm), published by Ruby on April 23, 2024 on LessWrong. For the last month, @RobertM and I have been exploring the possible use of recommender systems on LessWrong. Today we launched our first site-wide experiment in that direction. (In the course of our efforts, we also hit upon a frontpage refactor that we reckon is pretty good: tabs instead of a clutter of different sections. For now, only for logged-in users. Logged-out users see the "Latest" tab, which is the same-as-usual list of posts.) Why algorithmic recommendations? A core value of LessWrong is to be timeless and not news-driven. However, the central algorithm by which attention allocation happens on the site is the Hacker News algorithm[1], which basically only shows you things that were posted recently, and creates a strong incentive for discussion to always be centered around the latest content. This seems very sad to me. When a new user shows up on LessWrong, it seems extremely unlikely that the most important posts for them to read were all written within the last week or two. I do really like the simplicity and predictability of the Hacker News algorithm. More karma means more visibility, older means less visibility. Very simple. When I vote, I basically know the full effect this has on what is shown to other users or to myself. But I think the cost of that simplicity has become too high, especially as older content makes up a larger and larger fraction of the best content on the site, and people have been becoming ever more specialized in the research and articles they publish on the site. So we are experimenting with changing things up. I don't know whether these experiments will ultimately replace the Hacker News algorithm, but as the central attention allocation mechanism on the site, it definitely seems worth trying out and iterating on. We'll be trying out a bunch of things from reinforcement-learning based personalized algorithms, to classical collaborative filtering algorithms to a bunch of handcrafted heuristics that we'll iterate on ourselves. The Concrete Experiment Our first experiment is Recombee, a recommendations SaaS, since spinning up our RL agent pipeline would be a lot of work.We feed it user view and vote history. So far, it seems that it can be really good when it's good, often recommending posts that people are definitely into (and more so than posts in the existing feed). Unfortunately it's not reliable across users for some reason and we've struggled to get it to reliably recommend the most important recent content, which is an important use-case we still want to serve. Our current goal is to produce a recommendations feed that both makes people feel like they're keeping up to date with what's new (something many people care about) and also suggest great reads from across LessWrong's entire archive. The Recommendations tab we just launched has a feed using Recombee recommendations. We're also getting started using Google's Vertex AI offering. A very early test makes it seem possibly better than Recombee. We'll see. (Some people on the team want to try throwing relevant user history and available posts into an LLM and seeing what it recommends, though cost might be prohibitive for now.) Unless you switch to the "Recommendations" tab, nothing changes for you. "Latest" is the default tab and is using the same old HN algorithm that you are used to. I'll feel like we've succeeded when people switch to "Recommended" and tell us that they prefer it. At that point, we might make "Recommended" the default tab. Preventing Bad Outcomes I do think there are ways for recommendations to end up being pretty awful. I think many readers have encountered at least one content recommendation algorithm that isn't givi...