Pretrained

Richard and Pierce break down all the new AI web browser entrants with a particular focus on OpenAI's new Atlas, tradeoffs between vision models and text based dom parsing, potential security vulnerabilities, and more.

OpenAI releases their long awaited browser Atlas, Pytorch releases their distributed computation framework Monarch, the SALT reinforcement learning addition to GRPO, the HAL benchmark for agent evaluation, and trying to adapt the kv cache for text diffusion models.Further reading:https://openai.com/index/introducing-chatgpt-atlas/https://pytorch.org/blog/introducing-pytorch-monarch/https://arxiv.org/pdf/2510.20022https://arxiv.org/abs/2510.11977https://arxiv.org/abs/2510.14973

Richard and Pierce take the bull case on whether we're in an AI bubble. They cover circular financial deals, energy build outs, AI representing 92% of GDP growth in H1 2025, and a comparison with the hype in 2000s around meaningless dot-com companies.

Articles written by LLMs have stabilized at exactly 50% of the internet (at least - so far as our classifiers can discriminate), the price of embedding models, OpenAI announces a new job board and certification programs for applied AI, Amazon releases the public availability of Bedrock AgentCore, and how pre-training on low quality data affects the capability of post-training.Further reading:https://arxiv.org/abs/2510.13928https://openai.com/index/expanding-economic-opportunity-with-ai/https://www.tensoreconomics.com/p/why-are-embeddings-so-cheaphttps://graphite.io/five-percent/more-articles-are-now-created-by-ai-than-humans

OpenAI diversifies their chip suppliers through partnerships with AMD and Broadcom, Google starts a new AI Bug Bountry problem but only for computational security not for llm hallucinations, Nvidia ships their first prosumer computer, DeepMind has a new complexity theory proof solver, and Anthropic writes their own gibberish poison pill that works across model sizes.Further reading:https://openai.com/index/openai-amd-strategic-partnership/https://investor.nvidia.com/news/press-release-details/2024/NVIDIA-Announces-Financial-Results-for-Second-Quarter-Fiscal-2025/default.aspxhttps://bughunters.google.com/blog/6116887259840512/announcing-google-s-new-ai-vulnerability-reward-programhttps://marketplace.nvidia.com/en-us/developer/dgx-spark/https://arxiv.org/abs/2509.18057https://www.anthropic.com/research/small-samples-poison

You asked, we answered! Rich and Pierce do their first listener mailbag. Explaining RLHF, our current development stack, whether model competition is making things better for people using them, and more.

Breaking down California's recently passed SB 53 to legislate frontier model development, ISO standards in startups, and why this one passed where the older SB 1047 failed.

Building a modern AI app and architecting Sora II, first impressions of Sonnet 4.5, and the frontier labs go after n8n and Zapier.Further reading:https://openai.com/index/sora-2/https://openai.com/index/sora-is-here/https://www.lesswrong.com/posts/4yn8B8p2YiouxLABy/claude-sonnet-4-5-system-card-and-alignmenthttps://www-cdn.anthropic.com/872c653b2d0501d6ab44cf87f43e1dc4853e4d37.pdfhttps://www.testingcatalog.com/openai-prepares-to-release-agent-builder-during-devday-on-october-6/

Richard and Pierce respond to the Times podcast about the scarcity of junior engineering jobs. They talk through the academic difference between Computer Science vs. Engineering, AI as a new engineering primitive, talent arbitrage through intern programs, and more.https://www.nytimes.com/2025/09/29/podcasts/the-daily/big-tech-told-kids-to-code-the-jobs-didnt-follow.html

OpenAI & NVIDIA’s 10GW partnership, GDPVal as a new human curated benchmark dataset, Gemini Robotics-ER 1.5, and Apple's distillation of AlphaFold.Additional reading:https://nvidianews.nvidia.com/news/openai-and-nvidia-announce-strategic-partnership-to-deploy-10gw-of-nvidia-systemshttps://openai.com/index/gdpval/https://deepmind.google/discover/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/https://arxiv.org/pdf/2509.18480

Pierce and Richard recap Anthropic's Economic Index. Differences between country use of AI, autonomy versus augmentation, and the real business use cases that Anthropic is seeing so far.Further reading:https://www.anthropic.com/research/anthropic-economic-index-september-2025-report

The official Claude Code post-mortum, Deepseek R1 published in Nature, Meta unveils their smart glasses with built‑in display, the new apple pro, copyright law in the age of AI, and much more.Further reading:https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issueshttps://www.nature.com/articles/s41586-025-09422-z#MOESM1https://www.theverge.com/tech/779566/meta-ray-ban-display-hands-on-smart-glasses-price-battery-specshttps://www.reuters.com/legal/government/anthropics-15-billion-copyright-settlement-faces-judges-scrutiny-2025-09-09/

The Anthropic economic index report, a bug in claude's inference pipeline, OpenAI releases a flavor of GPT-5 just for coding, Microsoft's new inhouse LLM, and what really happens when you turn temperature to 0

Pierce, Richard, and Will join for the first in-person interview on Pretrained. For our video episode, check out:https://youtu.be/CInTOIgz-pAThey cover:- Will's history growing up in the UK- Getting an MBA and deciding what company to start- Trust building versus activating content- Building a personal brand for engineers and researchers- Entrepreneurship in Europe vs the US- & Much more

Kimi's serving architecture, mooncake to offload GPU memory to other chipsets, the ubiquity of vllm, and the growing standard LLM stack

Kimi K2 and Moonshot AI's history, avoiding loss spikes during training, the muon optimizer, and data parallelism

Gemini’s new image model, OpenAI is investing more in protein generation, Cohere’s SOTA generation model, and Anthropic working with DOE on nuclear securityFurther reading:https://blog.google/products/gemini/updated-image-editing-model/https://openai.com/index/accelerating-life-sciences-research-with-retro-biosciences/https://cohere.com/blog/command-a-translatehttps://red.anthropic.com/2025/nuclear-safeguards/https://www.picolm.io/demo-paper

Rich and Pierce speculate wildly (well, kind of wildly) on the internal architecture of Genie 3. They go into the history of variational auto encoders and diffusion models, 3D modeling with AI as an alternative to video game designers, a recap of the official Genie 1 paper, and possible applications of world models to the real world.Further reading:https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/https://arxiv.org/pdf/2402.15391

Richard visits his high school, LLMs in education, is AI a calculator or an oracle, and more

Elevenlabs launches a music generator, Claude Long-Term Memory, Reddit blocks the Internet Archive, NVIDIA’s Massively Multilingual Speech, and Self Questioning Language ModelsFurther reading:https://techcrunch.com/2025/08/05/elevenlabs-launches-an-ai-music-generator-which-it-claims-is-cleared-for-commercial-use/https://www.theverge.com/news/757538/reddit-internet-archive-wayback-machine-block-limithttps://www.theverge.com/news/757743/anthropic-claude-ai-search-past-chatshttps://arxiv.org/html/2508.03682v3https://arxiv.org/html/2508.05004v1https://blogs.nvidia.com/blog/speech-ai-dataset-models/