There is a common argument that AI development is dangerous that goes something like:

1. The “goal” of evolution is to make animals which replicate their genes as much as possible;
2. humans do not want to replicate their genes as much as possible;
3. we have some goal which we want AIs to accomplish, and we develop them in a similar way to the way evolution developed humans;
4. therefore, they will not share this goal, just as humans do not share the goal of evolution.

This argument sucks. It has serious, fundamental, and, to my thinking, irreparable flaws. This is not to say that its conclusions are incorrect; to some extent, I agree with the point which is typically being made. People should still avoid bad arguments.

Problems

Why an Analogy?

Consider the following argument:

1. Most wars have fewer than one million casualties.
2. Therefore, we should expect that the next war (operationalised in some way) which starts will end with fewer than one million casualties.

There are some problems with this. For instance, we might think that modern wars have more or fewer casualties than the reference class of all wars; we might think that some particular conflict is [...]

---

Outline:

(00:58 ) Problems

(01:01 ) Why an Analogy?

(03:10 ) Evolution Does Not Have Goals

(04:06 ) Mendel, Not Crick

(06:51 ) Humans Are Not Misaligned

(09:19 ) We Do Not Train AIs Using Evolution

(09:36 ) Evolution Does Not Produce Individual Brains

(11:42 ) ...And So On

(14:13 ) Further Reading

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

January 5th, 2026

Source:

https://www.lesswrong.com/posts/HhkjohHCmJzwWoBmz/the-evolution-argument-sucks

---

Narrated by TYPE III AUDIO.