DiscoverLessWrong (30+ Karma)“A country of alien idiots in a datacenter: AI progress and public alarm” by Seth Herd
“A country of alien idiots in a datacenter: AI progress and public alarm” by Seth Herd

“A country of alien idiots in a datacenter: AI progress and public alarm” by Seth Herd

Update: 2025-11-07
Share

Description

Epistemic status: I'm pretty sure AI will alarm the public enough to change the alignment challenge substantially. I offer my mainline scenario as an intuition pump, but I expect it to be wrong in many ways, some important. Abstract arguments are in the Race Conditions and concluding sections.

Nora has a friend in her phone. Her mom complains about her new AI "colleagues." Things have gone much as expected in late 2025; transformative AGI isn't here yet, and LLM agents have gone from useless to merely incompetent.

Nora thinks her AI friend is fun. Her parents think it's healthy and educational. Their friends think it's dangerous and creepy, but their kids are sneaking sleazy AI boyfriends. All of them know people who fear losing their job to AI.

Humanity is meeting a new species, and most of us dislike and distrust it.

This could shift the playing field for alignment dramatically. Or takeover-capable AGI like Agent-4 from AI 2027 could be deployed before public fears impact policy and decisions.

Alarming incompetence

Public attitudes toward AI have transformed like they did for COVID between February and March of 2020.

The risks and opportunities seem much more immediate [...]

---

Outline:

(01:21 ) Alarming incompetence

(04:07 ) Race conditions

(06:39 ) Incompetent AI spreads alarm by default

(10:24 ) Resonances on the public stage

(13:00 ) Impacts on risk awareness, funding, and policy.

(14:51 ) Concluding thoughts and questions

---


First published:

November 7th, 2025



Source:

https://www.lesswrong.com/posts/qxmAqMAjxnhkzt6aF/a-country-of-alien-idiots-in-a-datacenter-ai-progress-and


---


Narrated by TYPE III AUDIO.

Comments 
loading
In Channel
loading
00:00
00:00
1.0x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

“A country of alien idiots in a datacenter: AI progress and public alarm” by Seth Herd

“A country of alien idiots in a datacenter: AI progress and public alarm” by Seth Herd