Code With Wilson

Running one token through a language model isn't one big GPU operation. It's hundreds to thousands of tiny ones, and each of those has to be launched by the CPU first. During decode, those little operations finish so fast the GPU starves waiting for the CPU to queue the next one. CUDA Graphs fix this by recording the whole repeating sequence once and replaying it with a single launch, so the GPU stays fed. Same math, same result, far less idle time.

The single idea the entire craft rests on is this: the model is a commodity, and the system around it is the job. A raw frontier model is a brilliant intern with amnesia, no hands, and no way to check its own work. The AI engineer gives it hands through tools, a memory across runs, a way to verify itself through evals, and a loop that keeps working until the task is done. That wrapper — not the model weights — is the thing you get paid to build. Learn to build it and you're employable.

Scaled dot-product attention is not a mysterious black box. It's four elementary operations — matrix multiplication, transpose, scaling by a constant, and softmax — plus three learned projections that produce queries, keys, and values. You can write it in about a hundred lines of plain C with no autograd and no libraries. Once you see it as loops instead of magic, you reason about cost, memory, and failure modes from first principles rather than treating the model as a sealed device.

An LLM engine like vLLM exists because serving a model well is its own systems problem, distinct from training it. Text generation is autoregressive and memory-bound, and the key-value cache is the real capacity limit on how many users fit on a GPU. vLLM answers three questions: which requests run next, how a batch executes, and where each request's cache lives. Its headline move is treating GPU memory like virtual memory, which slashes waste and unlocks bigger batches.

Most technical courses go bottom-up: math, then theory, then maybe you build something months later, by which point your motivation is gone. fast.ai flips it. You train a real, state-of-the-art model in lesson one, and only then peel back the layers to see how it learned. The engine that makes this possible for beginners is transfer learning, and that same "inherit capability instead of building it from scratch" move is the single most important pattern in modern AI engineering.