In this episode, we’re thrilled to host Chip Huyen, author of AI Engineering: Building Applications with Foundation Models. Chip is a renowned AI engineer, entrepreneur, and educator whose insights have helped shape the field of applied AI. As a frequent speaker at ODSC, Chip has established herself as a crowd favorite, delivering deeply technical yet accessible talks that resonate with both practitioners and researchers.

Episode Summary:

Join us as we dive into the world of AI engineering with Chip Huyen. In this episode, Chip discusses the emergence of AI engineering as a discipline, the challenges of deploying foundation models, and the nuances of inference optimization. We explore strategies for RAG (Retrieval-Augmented Generation), fine-tuning, and managing private data with limited resources. Chip also shares her thoughts on user feedback, data synthesis, and the importance of system-level thinking in building robust AI applications. Whether you’re an AI practitioner or just curious about the future of applied AI, this conversation offers valuable takeaways.

Key Takeaways:

- AI Engineering vs. ML Engineering: The distinctions between these fields and why AI engineering is emerging as a critical discipline.

- Inference Optimization: Practical strategies to reduce latency and costs while improving model performance.

- RAG and Fine-Tuning: When to use Retrieval-Augmented Generation versus fine-tuning for domain-specific applications.

- Building with Limited Resources: Tips for training models on private data with minimal hardware.

- User Feedback and Iteration: How continuous feedback loops can refine AI applications.

- Data Synthesis: Using synthetic data effectively while avoiding pitfalls like model collapse.

- Chip’s Writing Journey: Insights into the hardest topics to tackle while writing her book.

- Future of AI in 2025: Chip’s perspective on what’s exciting and transformative in the year ahead.

Resource List:

- Chip Huyen’s Linkedin https://www.linkedin.com/in/chiphuyen

- Chip Huyen’s Book: AI Engineering: Building Applications with Foundation Models https://www.amazon.com/dp/1098166302

- Chip’s Website: huyenchip.com

- Chips’s Blogs: https://huyenchip.com/blog/

- Data Synthesis References: Tools like Snorkel AI and Faker for generating high-quality synthetic datasets.

- Top Skills 2025 Blog: https://opendatascience.com/ai-mastery-2025-skills-to-stay-ahead-in-the-next-wave/

- Lindy Effect: wikipedia.org/wiki/Lindy_effect

- An Empirical Study of LLaMA3 Quantization: https://arxiv.org/pdf/2404.14047

- Open Hands: https://github.com/All-Hands-AI/OpenHands

This episode was sponsored by:  

Ai+ Training ⁠https://aiplus.training/⁠ 

Home to hundreds of hours of on-demand, self-paced AI training, ODSC interviews, free webinars, and certifications in in-demand skills like LLMs and Agentic AI

And created in partnership with ODSC ⁠https://odsc.com/⁠ 

The Leading AI Builders Conference, featuring expert-led, hands-on workshops, training sessions, and talks on cutting-edge AI topics and tools, from data science and machine learning to generative AI to LLMOps

Never miss an episode, subscribe now!