OpenAI's Strawberry, LM self-talk, inference scaling laws, and spending more on inference
Description
Whether or not scaling works, we should spend more on inference.
This is AI generated audio with Python and 11Labs.
Source code: https://github.com/natolambert/interconnects-tools
Original post: https://www.interconnects.ai/p/openai-strawberry-and-inference-scaling-laws
00:00 OpenAI's Strawberry, LM self-talk, inference scaling laws, and spending more on inference
01:51 OpenAI's Strawberry
04:16 Self-talk in language models
07:45 Inference scaling laws
Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/strawberry/img_006.png
Fig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/strawberry/img_021.png
Fig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/strawberry/img_023.png
Fig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/strawberry/img_037.png
Get full access to Interconnects at www.interconnects.ai/subscribe
![[Article Voiceover] AI Safety's Crux: Culture vs. Capitalism](https://substackcdn.com/feed/podcast/48206/post/149851748/5dbaf463a3193f0320d3e30ae332a118.jpg "[Article Voiceover] AI Safety's Crux: Culture vs. Capitalism")
![[Article Voiceover] Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem](https://s3.castbox.fm/de/53/16/9e08bcb80bb30117167ed5823009231ff8_scaled_v1_400.jpg "[Article Voiceover] Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem")
![[Article Voiceover] Reverse engineering OpenAI's o1](https://s3.castbox.fm/c7/01/d3/7fa03400effff0109644d490f3c2fecf4f_scaled_v1_400.jpg "[Article Voiceover] Reverse engineering OpenAI's o1")
United States
