Ilya Sutskever Calls Peak Data and the End of Pretraining

Ilya Sutskever Calls Peak Data and the End of Pretraining

Update: 2024-12-17
Share

Description

At a recent conference appearance, SSI founder (and former OpenAI leader) Ilya Sutskever claimed that we had reached peak data and that the era of pre-training as a scaling method had come to a close. NLW explores the implications. Plus, NotebookLM releases an enterprise edition.




Brought to you by:


Vanta - Simplify compliance - ⁠⁠⁠⁠⁠⁠⁠https://vanta.com/nlw


The AI Daily Brief helps you understand the most important news and discussions in AI.
Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614
Subscribe to the newsletter: https://aidailybrief.beehiiv.com/
Join our Discord: https://bit.ly/aibreakdown

Comments 
loading
00:00
00:00
1.0x

0.5x

0.8x

1.0x

1.25x

1.5x

2.0x

3.0x

Sleep Timer

Off

End of Episode

5 Minutes

10 Minutes

15 Minutes

30 Minutes

45 Minutes

60 Minutes

120 Minutes

Ilya Sutskever Calls Peak Data and the End of Pretraining

Ilya Sutskever Calls Peak Data and the End of Pretraining

Nathaniel Whittemore