DiscoverMystery AI Hype Theater 3000Episode 28: LLMs Are Not Human Subjects, March 4 2024
Episode 28: LLMs Are Not Human Subjects, March 4 2024

Episode 28: LLMs Are Not Human Subjects, March 4 2024

Update: 2024-03-13
Share

Description

Alex and Emily put on their social scientist hats and take on the churn of research papers suggesting that LLMs could be used to replace human labor in social science research -- or even human subjects. Why these writings are essentially calls to fabricate data.

References:

PNAS: ChatGPT outperforms crowd workers for text-annotation tasks

Political Analysis: Out of One, Many: Using Language Models to Simulate Human Samples

Behavioral Research Methods: Can large language models help augment English psycholinguistic datasets?

Information Systems Journal: Editorial: The ethics of using generative AI for qualitative data analysis

Fresh AI Hell:

Advertising vs. reality, synthetic Willy Wonka edition

A news outlet used an LLM to generate a story...and it falsely quoted Emily

Trump supporters target Black voters with faked AI images


You can check out future livestreams at https://twitch.tv/DAIR_Institute.

Subscribe to our newsletter via Buttondown.

Follow us!

Emily

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Comments 
In Channel
loading
00:00
00:00
x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Episode 28: LLMs Are Not Human Subjects, March 4 2024

Episode 28: LLMs Are Not Human Subjects, March 4 2024

Emily M. Bender and Alex Hanna