The AI Concepts Podcast

This episode tackles the lever that turns powerful LLMs into something you can actually run: quantization. We explore what it means to store model weights with fewer bits, why that can cut memory in half at 8-bit and down to roughly a quarter at 4-bit, and the real tradeoff between compression and capability as rounding error accumulates across billions of parameters. We break down why large models survive this better than small ones, why 8-bit is often near lossless, why 4-bit can still be shockingly strong, and why going below that can make models fall apart. We compare the three practical paths you will see in the wild: GPTQ (layer-wise compression with error compensation), AWQ (protecting the most important weights), and GGUF (the local-friendly format that makes CPU and GPU splitting possible).

This episode addresses the real bottleneck after you build an LLM: fitting it into hardware that can actually run it. We explore why GPU memory is the scarce resource, how weights, KV cache, and activations compete for that space, and what that means in practice when prompts get long or concurrency spikes. We compare data center GPUs (high bandwidth HBM) versus local machines like the Mac Studio (huge unified memory but slower bandwidth) to show the core tradeoff between capacity and speed. By the end, you will understand how to choose hardware based on your goal, and why the next lever is quantization to shrink models enough to fit, with a closing reflection on perspective when something big feels like it will not fit.

This episode addresses how Reinforcement Learning from Human Feedback (RLHF) adds the final layer of alignment after supervised fine-tuning, shifting the training signal from “right vs wrong” to “better vs worse.” We explore how preference rankings create a reward signal (reward models plus PPO) and the newer shortcut (DPO) that learns preferences directly, then connect RLHF to safety through the Helpful, Honest, Harmless goal. We also unpack the “alignment tax,” the trade-off between being safe and being genuinely useful, and close by setting up the next module on running models at scale, starting with GPU memory limits, plus a personal reflection on starting later without being behind.

This episode addresses how we turn a raw base model into something that behaves like a real assistant using Supervised Fine-Tuning (SFT). We explore instruction and response training data, why SFT makes behaviors consistent beyond prompting, and the practical engineering choices that keep fine-tuning efficient and safe, including low learning rates and LoRA-style adapters. By the end, you will understand what SFT solves, and why the next layer (RLHF) is needed to add human preference and nuance.

This episode addresses the physical and mathematical limits of a model’s "short-term memory." We explore the context window and the engineering trade-offs required to process long documents. You will learn about the quadratic cost of attention where doubling the input length quadruples the computational work and why this creates a massive bottleneck for long-form reasoning. We also introduce the architectural tricks like Flash Attention that allow us to push these limits further. By the end, you will understand why context is the most expensive real estate in the generative stack.

This episode explores the foundational stage of creating an LLM known as the pre-training phase. We break down the Trillion Token Diet by explaining how models move from random weights to sophisticated world models through the simple objective of next token prediction. You will learn about the Chinchilla Scaling Laws or the mathematical relationship between model size and data volume. We also discuss why the industry shifted from building bigger brains to better fed ones. By the end, you will understand the transition from raw statistical probability to parametric memory.

Shay explains where a transformer actually stores knowledge: not in attention, but in the MLP (feed-forward) layer. The episode frames the transformer block as a two-step loop: attention moves information between tokens, then the MLP transforms each token’s representation independently to inject learned knowledge.

Shay breaks down the encoder vs decoder split in transformers: encoders (BERT) read the full text with bidirectional attention to understand meaning, while decoders (GPT) generate text one token at a time using causal attention. She ties the architecture to training (masked-word prediction vs next-token prediction), explains why decoder-only models dominate today (they can both interpret prompts and generate efficiently with KV caching), and previews the next episode on the MLP layer, where most learned knowledge lives.

Shay explains multi-head attention and positional encodings: how transformers run multiple parallel attention 'heads' that specialize, why we concatenate their outputs, and how positional encodings reintroduce word order into parallel processing. The episode uses clear analogies (lawyer, engineer, accountant), highlights GPU efficiency, and previews the next episode on encoder vs decoder architectures.

In this episode, Shay walks through the transformer's attention mechanism in plain terms: how token embeddings are projected into queries, keys, and values; how dot products measure similarity; why scaling and softmax produce stable weights; and how weighted sums create context-enriched token vectors. The episode previews multi-head attention (multiple perspectives in parallel) and ends with a short encouragement to take a small step toward your goals.

Shay breaks down the 2017 paper "Attention Is All You Need" and introduces the transformer: a non-recurrent architecture that uses self-attention to process entire sequences in parallel. The episode explains positional encoding, how self-attention creates context-aware token representations, the three key advantages over RNNs (parallelization, global receptive field, and precise signal mixing), the quadratic computational trade-off, and teases a follow-up episode that will dive into the math behind attention.

Shay breaks down why recurrent neural networks (RNNs) struggled with long-range dependencies in language: fixed-size hidden states and the vanishing gradient caused models to forget early context in long texts. He explains how LSTMs added gates (forget, input, output) to manage memory and improve short-term performance but remained serial, creating a training and scaling bottleneck that prevented using massive parallel compute. The episode frames this fundamental bottleneck in NLP and sets up the next episode on attention, ending with a brief reflection on persistence and steady effort.

This episode dives into the hidden layer where language stops being words and becomes numbers. We explore what tokens actually are, how tokenization breaks text into meaningful fragments, and why this design choice quietly shapes a model’s strengths, limits, and quirks. Once you understand tokens, you start seeing why language models sometimes feel brilliant and sometimes strangely blind.

This episode explores the hidden engine behind how language models move from knowing to creating. It reveals why generation happens step by step, why speed has hard limits, and why training and usage behave so differently. Once you see this mechanism, the way models write, reason, and sometimes stall will make immediate sense.

This episode is about the hidden space where generative models organize meaning. We move from raw data into a compressed representation that captures concepts rather than pixels or tokens, and we explore how models learn to navigate that space to create realistic outputs. Understanding this idea explains both the power of generative AI and why it sometimes fails in surprising ways.

Welcome to Episode One of The Generative Shift. This episode introduces the core change behind modern AI, the move from discriminative models that draw decision boundaries to generative models that learn the full structure of data. Instead of predicting labels using conditional probability, generative systems model the joint distribution itself, which allows them to create rather than classify. This shift reshapes the math, the architecture, and the compute requirements, moving from compression focused networks to expansion driven systems that grow structure from noise. It is harder and more expensive, but it is the foundation of everything that follows. In the next episode, we will explore where this expansion lives by stepping into latent space and understanding how models represent meaning itself.

Hello everyone, and welcome to The Generative AI Series. I’m Shay, and this introductory episode is about why this series exists and who it is for. Generative AI has exploded, but real understanding is still scattered. Between hype, shortcuts, and surface level strategy talk, it is hard to find a clear path from fundamentals to building systems that actually work. This series is for practitioners, builders, architects, and technical leaders who want to understand how these models work under the hood, why they succeed, and why they fail. We will go deep but stay accessible, moving step by step from the shift from classification to generation, through transformers, training, RAG, evaluation, and production realities. The goal is simple: build intuition, recognize failure modes early, and design solutions and strategies that work beyond demos, in the real world. Let’s get started. I’ll see you in Module One.

Welcome to the final episode of our Deep Learning series on the AI Concepts Podcast. In this episode, host Shay takes you on a journey through the world of autoencoders, a foundational AI model. Unlike traditional models that predict or label, autoencoders excel in understanding and reconstructing data by learning to compress information. Discover how this quiet revolution in AI powers features like image enhancement and noise-cancelling technology, and serves as a stepping stone towards generative AI. Whether you're an AI enthusiast or new to the field, this episode offers insightful perspectives on how machines learn structure and prepare for the future of AI.

Welcome to the AI Concepts Podcast, where we explore AI, one concept at a time. In this episode, host Shay delves into the transformative world of transformers in AI, focusing on how they have revolutionized language understanding and generation. Discover how transformers enable models like ChatGPT to respond thoughtfully and coherently, transforming inputs into conversational outputs with unprecedented accuracy. The discussion unveils the structure and function of transformers, highlighting their reliance on parallel processing and vast datasets. Tune in to unravel how transformers are not only reshaping AI but also the foundation of deep learning advances. Relax, sip your coffee, and let's explore AI together.

In this episode of the AI Concepts Podcast, host Shay delves into the transformation of deep learning architectures, highlighting the limitations of RNNs, LSTM, and GRU models when handling sequence processing and long-range dependencies. The breakthrough discussed is the attention mechanism, which allows models to dynamically focus on relevant parts of input, improving efficiency and contextual awareness. Shay unpacks the process where every word in a sequence is analyzed for its relevance using attention scores, and how this mechanism contributes to faster training, better scalability, and a more refined understanding in AI models. The episode explores how attention, specifically self-attention, has become a cornerstone for modern architectures like GPT, BERT, and others, offering insights into AI's ability to handle text, vision, and even multimodal inputs efficiently. Tune in to learn about the transformative role of attention in AI and prepare for a deeper dive into the upcoming discussion on the transformer architecture, which has revolutionized AI development by focusing solely on attention.