Deep Dive into Inference Optimization for LLMs with Philip Kiely

Update: 2024-11-05

Description

Today we have Philip Kiely from Baseten on the show. Baseten is a Series B startup focused on providing infrastructure for AI workloads.

We go deep on Inference Optimization. We cover choosing a model, discuss the hype around Compound AI, choosing an Inference Engine, Optimization Techniques like Quantization and Speculative Decoding all the way down to your GPU choice.

Comments

In Channel

2025-09-1642:33

Lessons from Building AI Agents with Rafal Wilinski

2025-08-1201:08:51

Building a High-Ownership Engineering Culture with Matt Watson

2025-08-0551:37

Building CI for the age of AI Agents with Aayush Shah

2025-07-2201:04:02

Valkey After the Fork: A Conversation with Madelyn Olson

2025-07-1601:21:14

Operational Excellence Is the Moat with Sam Lambert

2025-07-1501:06:12

Lessons from Transcribing and Indexing 3.5 Million Podcasts with Arvid Kahl

2025-07-0801:18:00

It's time to build Jarvis with Kent C. Dodds

2025-05-1301:21:17

Rewriting in Rust + Being a Learning Machine with AJ Stuyvenberg

2025-05-0601:21:36

Software Reliability Agents with Amal Kiran

2025-04-2951:07

From ORM to Infra: Prisma Postgres with Søren Bramer Schmidt

2025-04-2201:02:22

Fast Inference with Hassan El Mghari

2025-04-0853:06

Seattle Startups, AI’s Future & Big Acquisitions with Yujian Tang

2025-03-1301:02:54

Faster & Cheaper on PlanetScale Metal with Sam Lambert

2025-03-1101:19:43

Redis but Faster With Roman Gershman

2025-03-0401:00:51

Lessons from Building Tagged.com + AI-Driven Database Optimization with Johann Schleier-Smith

2024-12-1156:13

Building + Evolving Sentry's Architecture and Funding Open Source with David Cramer

2024-11-1201:13:06

Deep Dive into Inference Optimization for LLMs with Philip Kiely

2024-11-0501:04:05

Java and Building AI Applications with Kevin Dubois

2024-10-2956:58

SQLite, Turso, and the State of Databases with Glauber Costa

2024-10-2201:12:07

00:00

Deep Dive into Inference Optimization for LLMs with Philip Kiely

#box-pro-ellipsis-176153802026810{-webkit-line-clamp:2;}Deep Dive into Inference Optimization for LLMs with Philip Kiely

Deep Dive into Inference Optimization for LLMs with Philip Kiely

Software Huddle

Deep Dive into Inference Optimization for LLMs with Philip Kiely