#190 - AI scaling struggles, OpenAI Agents, Super Weights
Description
Our 190th episode with a summary and discussion of last week's* big AI news!
*and sometimes last last week's
Hosted by Andrey Kurenkov and Jeremie Harris.
Note from Andrey: this one is coming out a bit later than planned, apologies! Next one will be coming out sooner.
Feel free to email us your questions and feedback at contact@lastweekinai.com and/or hello@gladstone.ai
Read out our text newsletter and comment on the podcast at https://lastweekin.ai/.
Sponsors:
- The Generator - An interdisciplinary AI lab empowering innovators from all fields to bring visionary ideas to life by harnessing the capabilities of artificial intelligence
In this episode:
* OpenAI's pitch for a $100 billion data center and AI strategy plan outlines infrastructure and regulatory needs, emphasizing AI's foundational role akin to electricity.
* Google's Gemini model challenges OpenAI's dominance, showing strong performance in chatbot arenas alongside generative AI advancements.
* DeepMind's AlphaFold3 gets open-sourced for academic use, while new chips from NVIDIA and Google show significant performance boosts.
* Anthropic and TSMC updates highlight strategic funding, regulation influences, and the complex dynamics of AI hardware and international policy.
If you would like to become a sponsor for the newsletter, podcast, or both, please fill out this form.
Timestamps + Links:
- (00:00:00 ) Intro / Banter
- (00:02:44 ) News Preview
- (00:03:34 ) Sponsor Break
- Tools & Apps
- (00:04:36 ) OpenAI, Google and Anthropic Are Struggling to Build More Advanced AI
- (00:16:22 ) OpenAI Nears Launch of AI Agent Tool to Automate Tasks for Users
- (00:19:14 ) Google drops new Gemini model and it goes straight to the top of the LLM leaderboard
- (00:19:14 ) Chinese AI startup takes aim at OpenAI's Sora with image-to-video tool launch
- (00:20:04 ) Introducing the Forge Reasoning API Beta and Nous Chat: An Evolution in LLM Inference
- Applications & Business
- (00:23:47 ) OpenAI Discusses AI Data Center That Could Cost $100 Billion
- (00:26:48 ) Elon Musk's massive AI data center gets unlocked — xAI gets approved for 150MW of power, enabling all 100,000 GPUs to run concurrently
- (00:29:34 ) Newest Google and Nvidia Chips Speed AI Training
- (00:34:45 ) Ex-OpenAI CTO Murati’s New Team Takes Shape
- (00:34:45 ) Amazon Discussing New Multibillion-Dollar Investment in Anthropic
- Projects & Open Source
- (00:37:52 ) Google DeepMind open-sources AlphaFold 3, ushering in a new era for drug discovery and molecular biology
- (00:41:29 ) Near plans to build world’s largest 1.4T parameter open-source AI model
- Research & Advancements
- (00:45:38 ) The Super Weight in Large Language Models
- (00:55:42 ) Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
- (01:03:47 ) Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
- (01:08:14 ) Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations
- Policy & Safety
- (01:11:14 ) The Code of Practice for general-purpose AI offers a unique opportunity for the EU
- (01:15:38 ) Three Sketches of ASL-4 Safety Case Components
- (01:23:05 ) U.S Department of Commerce finalizes $6.6 billion CHIPS Act funding for TSMC Fab 21 Arizona site , TSMC cannot make 2nm chips abroad now: MOEA
- (01:26:21 ) OpenAI to present plans for U.S. AI strategy and an alliance to compete with China
- (01:30:42 ) OpenAI loses another lead safety researcher, Lilian Weng
- (01:33:00 ) Outro