DiscoverThe AI Practitioner Podcast
The AI Practitioner Podcast
Claim Ownership

The AI Practitioner Podcast

Author: by Lina Faik

Subscribed: 3Played: 39
Share

Description

Real-world AI, explained simply — with code, use cases, and zero fluff.

aipractitioner.substack.com
25 Episodes
Reverse
Prefer reading instead? The full article is available here. The podcast is also available on Spotify and Apple Podcasts. Subscribe to keep up with the latest drops.What if AI agents could learn from every interaction without constantly rewriting their prompts? Each execution can reveal useful lessons, but these insights are often lost once the task ends.After exploring prompt optimization with TextGrad and GEPA, this episode introduces Agentic Context Engineering (ACE), which takes a different approach: continuously improving an evolving playbook of reusable knowledge rather than rewriting prompts.In this episode, we explore how ACE works, how to implement it in Python, and what a banking intent classification experiment reveals about its effectiveness and limitations.You’ll learn:* How does ACE enable continuous improvement? The roles of the evolving playbook and the Generator–Reflector–Curator architecture.* How can ACE be implemented? Building feedback loops, updating the playbook, and enabling offline and online adaptation.* How should the results be evaluated? Measuring classification accuracy, examining accumulated knowledge, and understanding the trade-offs of evolving context.👉 Enjoyed this episode? Subscribe to The AI Practitioner to get future articles and podcasts delivered straight to your inbox: aipractitioner.substack.com Get full access to The AI Practitioner at aipractitioner.substack.com/subscribe
Prefer reading instead? The full article is available here. The podcast is also available on Spotify and Apple Podcasts. Subscribe to keep up with the latest drops.What if AI agents could continuously improve their prompts as new challenges emerge in production? Even well-designed prompts can lose effectiveness as user needs evolve, new edge cases arise, and unexpected failures occur. Yet manually evaluating and refining prompts on an ongoing basis quickly becomes difficult to scale.In the previous episode, TextGrad introduced automated prompt optimization through textual gradients, using natural-language feedback to iteratively improve instructions. GEPA (Genetic-Pareto) takes a different approach: it combines reflection with evolutionary search to explore multiple prompt candidates and preserve promising alternatives through Pareto-based selection.In this episode, the focus is on how GEPA works, what a real banking intent classification experiment reveals about its effectiveness, and the trade-offs between accuracy, exploration, and prompt complexity.You’ll learn:* How does GEPA optimize prompts? The roles of reflection, mutation, and Pareto-based selection.* How can GEPA be implemented? Defining an agent, building feedback, and running evolutionary search.* How should the results be evaluated? What changes in the instructions, whether performance improves, and which trade-offs emerge.👉 Enjoyed this episode? Subscribe to The AI Practitioner to get future articles and podcasts delivered straight to your inbox: aipractitioner.substack.com Get full access to The AI Practitioner at aipractitioner.substack.com/subscribe
Prefer reading instead? The full article is available here. The podcast is also available on Spotify and Apple Podcasts. Subscribe to keep up with the latest drops.What if AI agents could improve their own prompts instead of relying on manual updates? In production, data evolves, models change, and new failures emerge. Keeping prompts effective requires continuous evaluation and refinement, but manually managing this process quickly becomes difficult to scale.TextGrad offers a different approach: it uses natural-language feedback, called textual gradients, to identify errors and automatically update prompts without modifying the underlying model.In this episode, the focus is on how TextGrad works, what a real banking classification experiment reveals about its effectiveness, and the challenges of evaluating prompt improvements when no correct answer exists.You’ll learn:* How can AI agents optimize their own prompts? Exploring how TextGrad turns errors into textual gradients and uses them to automatically update instructions.* Does automated prompt optimization actually improve performance? Examining experimental results, why some updates introduce regressions, and the trade-offs between accuracy, cost, and prompt complexity.* What happens when there is no correct answer? Understanding how LLM judges can guide optimization for open-ended tasks, why their feedback can be misleading, and what reliable evaluation requires.👉 Enjoyed this episode? Subscribe to The AI Practitioner to get future articles and podcasts delivered straight to your inbox: aipractitioner.substack.com Get full access to The AI Practitioner at aipractitioner.substack.com/subscribe
Prefer reading instead? The full article is available here. The podcast is also available on Spotify and Apple Podcasts. Subscribe to keep up with the latest drops.What if the key to better AI agents isn’t writing better prompts, but making prompts continuously improve?Even carefully designed prompts can fall short in production. Continuous prompt optimization uses real-world execution data, failures, and feedback to refine agent behavior over time, without changing the underlying model.In this episode, the focus is on how continuous prompt optimization works, why static prompts are no longer enough, and how automated feedback loops can improve agent reliability in production.You’ll learn:* Why optimizing text requires a different kind of gradient: How natural-language feedback can play a role similar to numerical gradients in neural network training.* How TextGrad and GEPA optimize prompts offline: How they turn failures into systematic prompt improvements, and the trade-offs between their approaches.* How ACE enables continuous improvement in production: Why repeatedly rewriting prompts can gradually erase useful knowledge, and how incremental updates help preserve it.👉 Enjoyed this episode? Subscribe to The AI Practitioner to get future articles and podcasts delivered straight to your inbox: aipractitioner.substack.com Get full access to The AI Practitioner at aipractitioner.substack.com/subscribe
Prefer reading instead? The full article is available here. The podcast is also available on Spotify and Apple Podcasts. Subscribe to keep up with the latest drops.What if improving an AI agent’s performance had less to do with choosing a better model and more to do with the system built around it?In one experiment, LangChain improved its coding agent’s benchmark score from 52.8% to 66.5% without changing the underlying model.The difference came from harness engineering: the tools, context management, execution logic, and feedback mechanisms that turn an LLM into a reliable agent.In this episode, the focus is on why harness engineering is gaining attention, how it works, and why it changes the way agent performance should be evaluated.You’ll learn:* Why are better prompts no longer enough? Understanding the challenges of long-running agents, from context degradation and lost progress to unreliable tool use and verification.* What actually makes an agent harness? Exploring the components behind autonomous agents, the approaches used by OpenAI and Anthropic, and the emerging engineering patterns.* Can the same model deliver radically different results? Examining real benchmark experiments, the impact of harness design, and why leaderboard scores may reveal less about model capabilities than they appear to.👉 Enjoyed this episode? Subscribe to The AI Practitioner to get future articles and podcasts delivered straight to your inbox: aipractitioner.substack.com Get full access to The AI Practitioner at aipractitioner.substack.com/subscribe
loading
Comments