In this episode, Divia Eden and Ronny Fernandez talk about the (strong) orthogonality thesis - that arbitrarily smart intelligences can be paired with arbitrary goals, without additional complication beyond that of specifying the goal - with light prompting from me. Topics they touch on include:

• Why aren't bees brilliant scientists?
• Can you efficiently make an AGI out of one part that predicts the future conditioned on some plans, and another that evaluates whether plans are good?
• If minds are made of smaller sub-agents with more primitive beliefs and desires, does that shape their terminal goals?
  • Also, how would that even work?
• Which is cooler: rockets, or butterflies?
• What processes would make AIs terminally value integrity?
• Why do beavers build dams?
• Would these questions be easier to answer if we made octopuses really smart?

Divia's twitter account: https://twitter.com/diviacaroline

Divia's podcast: https://mutualunderstanding.substack.com/

Ronny's twitter account: https://twitter.com/RatOrthodox

Arbital page for the orthogonality thesis: https://arbital.com/p/orthogonality/

Crystal Society: http://crystal.raelifin.com/

Video of a rescue beaver building a dam inside a human house: https://www.youtube.com/watch?v=-ImdlZtOU80

AIXI: https://en.wikipedia.org/wiki/AIXI

Kelly betting: https://en.wikipedia.org/wiki/Kelly_criterion