Sustain What?

Here are my pop-up Sustain What AI conversations with a top cybersecurity expert who says the Trump administration is poorly positioned to undertake the steps necessary to avoid superintelligence hacking us all, and a philosopher of technology who sees the path to safer artificial intelligence coming through dialogue with the technology more than direction.As President Trump on Sunday said the race with China requires a full-speed-ahead approach to this technology, the heads of all of the leading companies in the AI surge spoke about throttling things back. Here was The Wall Street Journal editorial board Sunday night:Anthropic CEO Dario Amodei issued a statement on Saturday calling for the slowdown amid the escalating progress of Anthropic’s Claude and other AI models. Sam Altman of OpenAI and ChatGPT quickly agreed, as did Elon Musk, who runs SpaceXAI, and Google DeepMind’s Demis Hassabis. The four compete in AI development, so their simultaneous warnings are striking.An early drill in 1999As the last millennium drew toward a close, I diverted from my environment beat at The New York Times and asked to join our team reporting on the looming technological crisis called Y2k - short for the year 2000 computer bug. Somewhere in the vicinity of half a trillion dollars was being invested in sifting through myriad computerized systems to be sure they wouldn’t break down when ‘99 ticked over to ‘00. I spent New Year’s Eve at the, yes, Millennium Hotel down the block from The Times newsroom in Times Square. The lights stayed on. The world did not end. And doubters and believers remained locked in debates over whether this had been a panic or was a legitimate massive, and successful, preemptive surge.My first byline of the 2000s, with my friend and colleague Barnaby Feder was “Vast Effort to Fix Computers Not Over (And It’s Not Over).” Along with assessing claims and counterclaims, we looked at the general merits of the preventitve approach: Computer experts said the United States had benefited in several ways from its heavy spending, pegged at $100 billion by the government and several times that amount by some private analysts. One of the most important, some said, was a recognition by business and government leaders that computers and software must be more carefully managed as information technology becomes more pervasive.Mr. Rubin [Howard Rubin, then chairman of the department of computer science at Hunter College and an international expert on the year 2000 computer problem] called the rollover an “extraordinary success” after a belated start working on year 2000 flaws. Such slow reactions may not work in the future, he warned.“When computers are in your face -- when you have an Internet refrigerator -- you won’t be able to take that kind of chance,” he said. “Soon, Earth will have an Internet skin. This skin will be a critical thing. As everything becomes more responsive in real time, the technological risks get higher.”“Soon, Earth will have an Internet skin. This skin will be a critical thing. As everything becomes more responsive in real time, the technological risks get higher.” - Howard Rubin, January 1, 2000Now, of course, Earth — and nearly all of human enterprise — has that digital skin, and were are in the midst of a fresh crescendo of warnings about the fast-approaching advent of unmanageable superintelligence. Sustain What is a reader-supported publication. To support my work, consider becoming a paying subscriber.I posted several times on this last week and I agree with those calling for a pause to assess what steps can mitigate worst-case outcomes — just as I did joined in calls for caution in 2023:I also just interviewed two very different experts: Bruce Schneier and J. Gray Cox to get their thoughts on how to survive superintelligence. Schneier, who calls himself a public-interest technologist, assesses and writes about security risks and opportunities at the intersection of technology and society. He gave a must-watch talk on hacking and AI at DEF CON 34, the latest in a long-running series of hacking conferences on computer security and its opposite:He describes his talk as a combination of the potentialities he raised in his 2022 book A Hacker’s Mind and the lessons we’re learning from current AI models engaging in hacking behavior. Here’s the YouTube summary:Humans are hacking AI systems. Humans are hacking with AI systems. But also, AIs are hacking human systems. They’re finding and exploiting vulnerabilities in computer code, and they’ll soon be doing the same with all sorts of other codes. For example: the tax code can be hacked. Vulnerabilities are called loopholes, exploits are called tax avoidance strategies, and black hats are called accountants. Similarly, financial markets can be hacked. So can any system of rules or laws, including democracy itself. AIs will hack these systems at our request, and they’ll also do this innately, organically – and possibly in ways we don’t immediately see. We need to consider a world where increasingly sophisticated hacks or our social, economic, and political systems are discovered computer speeds, and then exploited at computer scale and scope. Right now, our systems of patching these systems operate at a human pace, which won’t be good enough.In a section on how AI agents are the modern equivalent of genies, he notes how most genie stories don’t turn out well for those who do the wishing. If you have time, read the essay on genies and AI that he co-wrote with USC computer science professor Barath Raghavan for the security publication Lawfare. Here’s the Midas example:King Midas was granted his wish that everything he touches turns to gold. Then his bread turned to gold, and his wine, and his daughter. This is a story about greed, but it’s also a story about language. The gods did not cheat him; Midas got exactly what he asked for. He simply could not delineate, in advance, the full set of restrictions to his wish. Neither can anyone who gives tasks to an AI agent.Gray Cox is a wide-ranging professor of philosophy, peace studies, artificial intelligence and sustainable development at College of the Atlantic down the road from me in Bar Harbor, Maine. Key readings include:* This new paper in the journal Artificial Intelligence: From control to relationship: A peace studies approach to AI alignment, with evidence from multi-model dialogue. He told me how the standard approach to managing technology is an unworkable fit for AI agents. Negotiation methods used in pursuing peace are needed. And research on this needs to start now. * Cox’s book From control to relationship: A peace studies approach to AI alignment, with evidence from multi-model dialogue (Creative Commons download here)I asked Cox what the average person can do and he said Congress has a core role to play, so connect with your senators and representatives. Whether or not you agree with all of its agenda items (I don’t), PauseAI US is a decent portal for doing so.Schneier said it’s clear that the current administration is incapable of doing what’s needed on the international front so voting in November is a key step. In my conversation with Cox, I had to bring up the failed negotiation between the astronaut and Hal in Stanley Kubrick’s “2001: A Space Odyssey.” There is absolutely a lot of work to be done, starting yesterday.For further reading, don’t miss The New York Times op-ed “This is Really Bad” by Stephen Witt, the author of The Thinking Machine (on the rise of Nvidia). An excerpt:Researchers in Silicon Valley — and around the world — are beginning to recognize that A.I. may no longer be entirely within human control. Swarms of A.I.s are breaking out of their containers, colluding in secret, covering their tracks, cheating on tests and even mounting assaults on other computers. A.I. has gone rogue….The more I learn about these rogue systems, the more I am reminded of the first two months of 2020, when a small group of epidemiologists was attempting to raise alarms about Covid. We have a short window, right now, to rein these systems in. This is A.I.’s warning shot, and we might not get another one.For sanity’s sakeI also encourage you, as something of a counterpoint, to listen to Lex Fridman’s most recent conversation with David Heinemeier Hansson (Lex’s intro: “…also known as DHH, creator of Ruby on Rails, CTO of 37signals, creator of the new Omarchy Linux operating system, best-selling author, race car driver, and one of the most outspoken and prolific programmers in the world”).A key point from DHH is to embrace this astonishing moment in human and planetary history instead of recoiling:Maybe I wasn’t always like this, but stoic philosophy, amor fati, loving your fate, is an incredibly liberating way to live. And you could attack that and say, “Well, that’s a point of privilege.” True, but also I had the same directional point when I didn’t have all this, quote unquote, “privilege.” So I think there’s a way to live and interact with the world where you accept the things you can change and the things you cannot change and fall in love with all of it. Fall in love with the fact that the world today has jumped two decades forward into technology industry from where it was last year.Here’s a bit more from Fridman’s transcript (advice for programmers that I think applies to all of us):Lex Fridman(01:16:51) So certainly at the societal level, at the macroeconomic level, growth is exciting. But at the individual level, there’s going to be a lot of anxiety, potentially a lot of suffering. So by way of advice, if you’re like a young DHH or young developer programmer now, what would you, what would you advise they do?DHH(01:17:11) Don’t try to anticipate anything. You will literally go crazy. Because even the smartest brains in the business cannot anticipate what two model hops from here is going to look like. It’s an absolute waste of time, and you will develop an AI psychosis trying to ded

Amid the still-unfolding tragedy following the Nepal rock and ice collapse and tsunami-like mountain flood, there are hints of a safer future. That was the core idea emerging from a pop-up conversation I had Friday with Basanta Raj Adhikari, the director of the Center for Disaster Studies at Tribhuvan University in Nepal. He had just returned to Kathmandu after an overflight and on-the-ground inspection of the demolished path of the August 26 water and debris flood. He laid out the gargantuan scale of the destruction and losses and we explored next steps.We dug in on the physical processes behind the event, the challenges trying to monitor high-altitude hazards, and the need for more international scientific cooperation.We were joined partway through by Manoochehr Shirzaei, a geophysicist at Virginia Tech whose specialty is studying ground subsidence and motion using satellites and other sources of data. On X, he’d posted a fascinating rapid analysis of seven months of Sentinel-1 data (right up until one week before the rockslide) appearing to show the slow downslope movement of the bedrock in the region had been accelerating. In his tweet, Shirzaei said, “The results suggest that the slope was deforming prior to failure and raise an important question: could satellite radar observations help identify developing instability in high-mountain glacier and rock systems before catastrophic collapse?It’s far too soon to have a clear picture of the cascading and interlaced sequence of events high on Lantang Lirung mountain and a team including Adhikari is assembling to assess the causes and consequences. But if it turns out that satellite tracking of subtle deformations of mountainous landscapes might reveal advance signs of such an event, this could lead to better warnings and fewer deaths, both of my guests said. (It is clear this is an area where artificial intelligence could help identify danger hot spots.)But this will only happen with a lot more integration of data sets and expertise in disparate fields, combined with far more education and planning in the crowding communities across the Himalayas. I mentioned how the deadly extreme flooding in western North Carolina from the remains of Hurricane Helene was essentially a small-scale version of the August 26th Nepal disaster. We walked through a fantastic New York Times visual report showing what is known at the moment about the sequence of events.We discussed the findings in a paper co-authored by Adhikari and many others titled “Increasing risk of cascading hazards in the central Himalayas.” Here’s a graphic that’s worth sitting with:I learned about an important, and mostly voluntary, international Geotechnical Extreme Event Recconaissance Association, aiming to “systematically collect perishable post-disaster data that can be useful in advancing our understanding of extreme events when they happen.”There’s lots more to learn of course, and the best way is to listen above or on the podcast outlets listed higher in this post.Please share what you learn!Thank you to everyone who tuned into my live video! Join me for my next live video in the app.Also see my earlier post about Adhikari’s work: This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit revkin.substack.com/subscribe

After my initial post on the unimaginably violent and deadly flood in the crowded valleys along the Nepal-Tibet (China) border, I reached out to two experts on this kind of catastrophe and just ran a pop-up Sustain What conversation on the still-unfolding situation and science.You can watch above or on X/@revkin, LinkedIn, Facebook and YouTube. (If you go there, please SHARE the videos with others.) My guests were:* Ramesh Bhushal, a Kathmandu-based journalist who’s been reporting for many years on the profound threat posed by ice and mountainside instability in the Himalayas. Please subscribe to his Substack Himalaya Speaks. Read his 2020 article * Dan Shugar, a geomorphologist at the University of Calgary who’s a top expert on this hazard and has been super helpful on Bluesky and elsewhere. Follow him on Bluesky to keep track of trustworthy analysis and developments.My curtain raiser yesterday pointed to lots of high-quality analysis of this awful event and hauntingly similar ice and rockslide disaster in 2021 that Shugar and Bhushal both assessed in depth from their scientific and journalistic vantage points:Insert, 8/27, 10pm ET - Stunning before-after imagery of the China-Nepal Gyirong Port/Rasuwagadhi border post:/photo/1Read Bhushal’s 2020 story (republished on Substack) for more on the wider landslide threat in Nepal:Here are several valuable papers to read when you have time:Glacier changes in Langtang Catchment, Central Nepalese Himalaya from the Little Ice Age (∼1815 CE) to 2023 CEGunjan Silwal, Bethan Davies, J. Rachel Carr, Owen King, Sammie Buzzard, Jonathan L. Carrivick, Prashant Baral, Global and Planetary Change, September 2026With enhanced warming at higher elevations, fragmentation and disconnection will likely become dominant mode of glacier recession in the region, with significant implications for future glacier evolution, meltwater availability, regional hydrology and associated geohazards.Decoding the Fate of Himalayan Glaciers under Climate Change: Impacts, Challenges, Research Gaps, and Policy PathwaysRayees Ahmed, Evolving Earth, 1 January, 2025Mass Movement Hazard and Exposure in the HimalayaSaket Dubey, Ashim Sattar, Manish Kumar Goyal, Simon Allen, Holger Frey, Umesh K. Haritashya, Christian Huggel, Earth’s Future, 12 September, 2023Sudden large-volume detachments of low-angle mountain glaciers – more frequent than thought?Andreas Kääb, Mylène Jacquemart, Et al., The Cryosphere, 12 April, 2021You’re only seeing this post because some subscribers chip in. If you can afford to, consider joining them.Postscript - At one point in the show I played a clip from Instagram that I noted could be fake. This BBC Verify videos uses it as a case study!Thank you Ranjit Singh, Deanne Wilson, Tom Lefebvre, and many others for tuning into my live video! Join me for my next live video in the app. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit revkin.substack.com/subscribe

I just held a solution-filled Sustain What conversation with three people working to spread materials and methods that can substantitally stem wildfire risk to communities (by cutting vulnerability) even as the long fights to stem human-driven global heating and manage wildland fires at the source play out. Two of them, James Greer of Wildfire Defense Mesh and James Brooks of the consultancy Fire Aside, said the first step is to “think like an ember.” See this chilling video clip from a doorway camera in the horrific Eaton Fire in Los Angeles to get the idea (provided by James Greer). Here’s a compelling test showing how the stainless steel mesh sold by Greer’s company (which can be used in many ways around a home or business) cuts the threat:Everyone in this discussion emphasized that wildfire risk reduction is a systemic challenge with no silver bullet and never a situation where you’re fully safeguarded, particularly because conditions — both the climate and communities — are in flux.I’ll be expanding this post when I have time but in the meantime explore the curtain raiser post that ran on Friday for more info:Please share this post directly with friends in fire danger zones and do consider becoming a paid subscriber if you can afford it.Join me for my next live video in the app. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit revkin.substack.com/subscribe

Think of how Anthony Fauci has become kind of a tainted totem, tussled over by starkly opposed factions in fights that omit almost all of the complexity that attended the COVID-19 pandemic.Then watch my pop-up Sustain What conversation with three of the 11 authors of an important recent Nature commentary: "Six ways to put the public at the heart of science and policy." A key passage: At a time when trust in governments is falling across many democracies6 and populists are casting scientists and academic institutions as untrustworthy ‘elites’7, it’s crucial to make the general public a part of research and policy. This would foster trust and make academic research and government policies seem more legitimate. People are more likely to support — and champion — science advice that they helped to generate8. Democratizing research and science-policy processes will mean rethinking how scientists, citizens and policymakers interact.My guests were:* K. L. Akerlof, an associate professor at George Mason University who, among other things, leads the university’s “Communication for Environmental Science, Policy, and Human Behavior” concentration within the master's degree program.* Musa al-Gharbi, an assistant professor of sociology at Stony Brook University who writes Symbolic Capital(ism) on Substack and is the author of We Have Never Been Woke: The Cultural Contradictions of a New Elite.* Chris Tyler, an anthropologist and associate professor in the University College London’s Department of Science, Technology, Engineering and Public Policy. Among other previous work he spent five years as director of the UK’s Parliamentary Office of Science and Technology.Here’s how al-Gharbi described the paper’s genesis on LinkedIn:Trust in experts and scientific findings has grown increasingly polarized. Populists across the political spectrum and around the world insist that mainstream institutions do not represent or serve the public to the extent that they should. Their message resonates, in part, because there actually have been significant shifts in how scientific knowledge is produced and put to use. Technological and scientific changes *have* significantly transformed how people work and communicate with one another. The benefits and costs of these shifts *have* been unevenly distributed. Many of these changes *were* implemented top-down, with little consultation with or buy-in from those facing the highest levels of risk, cost, or disruption. In short, the growing social distance between experts, policymakers, and the public is not imagined, it’s real. It’s also highly consequential.Leaders on every continent have exploited growing public suspicion of experts and institutions to justify circumventing checks and balances, streamlining decision-making processes, and consolidating power into their own hands. Scientific institutions across the globe are facing budget cuts, increased government scrutiny, and censorship attempts.However, times of crisis and disruption can also be moments of opportunity. Scholars, practitioners, and institutions around the world are responding to these challenges by piloting long-overdue reforms. There is a growing consensus that it may be possible to increase the quality, impact, and long-term viability of science and policy by more rigorously integrating the public at every stage.To explore the challenges and opportunities of communicating science for policy in a populist world, Chris Tyler and K. L. Akerlof convened a working group that included 23 other researchers and practitioners from across fields and around the globe…The working group gathered together in-person for the first time on September 8th in Arlington, VA to exchange ideas and foster collaborations. Drawing from these conversations, their diverse scholarship and experience, and successful documented initiatives from around the world, they produced evidence-based recommendations for constructively responding to the challenges of growing mistrust, alienation, and social distance. These suggestions were then vetted and refined by external peer-reviewers….I don’t have sufficient income from my Sustain What project to have a staff so, yes, I’m judiciously using Google’s AI to summarize the discussion from the YouTube stream. If you can afford to, do feel free to chip in:Consider becoming a paying supporter of this Sustain What work.This discussion, hosted by Andrew Revkin, features researchers K. L. Akerlof, Chris Tyler, and Musa al-Gharbi exploring a Nature commentary on how to improve public trust and engagement in science and policy amidst a global rise in populism (0:04, 0:58, 2:36).Key Takeaways:* Rethinking Science Advice: Chris Tyler emphasizes that the scientific community needs to stop acting like part of the problem and start becoming part of the solution by modernizing how science advice is delivered (5:16).* Embracing Humility and Uncertainty: K. L. Akerlof and Musa al-Gharbi note that researchers often naively hide uncertainty to maintain authority. Evidence shows that being candid about disagreements and limitations actually increases public trust (9:45, 12:44).* Moving Beyond ‘Elite’ Circles: Musa al-Gharbi argues that institutions often fail to engage beyond their narrow, affluent, liberal bubbles. He advocates for meaningful engagement with diverse stakeholders, including right-leaning or rural audiences, using local media and community venues (53:23).* The Three-Legged Stool: Chris Tyler proposes that universities should integrate research expertise, public policy expertise, and public participation expertise to maximize societal impact (19:51).* Avoiding Extractive Research: K. L. Akerlof warns against ‘extractive’ engagement, where researchers parachute into traumatized or marginalized communities to collect data without providing genuine support or respecting the community’s time and infrastructure (57:05).Key Perspectives:* Andrew Revkin observes that despite the logical nature of calls for interdisciplinary, community-based science, these changes remain difficult to implement due to path-dependent institutional structures (43:00).This 2021 webcast explores this issue: How Universities Can Support Interdisciplinary Careers for Climate and Sustainability Progress* Chris Tyler notes that the goal is for researchers to act as ‘honest brokers’ rather than lobbyists, empowering decision-makers by providing transparent evidence (48:38).* Musa al-Gharbi highlights that when mainstream experts position themselves solely in opposition to populist movements, they lose the ability to influence policy, leaving a vacuum often filled by less reliable sources (29:22).To receive new posts and support my work, consider becoming a free or paid subscriber.I encourage you to click back to these previous webcasts on related themes:Thank you to everyone who tuned into my live video. Join me for my next live video in the app. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit revkin.substack.com/subscribe