Six Hard Lessons From Building With AI Agents
Description
In this episode of the Vernon Richard show, the hosts discuss their experiences with AI tools and agents, focusing on the challenges and lessons learned from using these technologies in coding and software engineering. They explore best practices for utilizing AI effectively, the importance of context in interactions with AI, and the future of AI agents in the workplace. The conversation highlights the balance between leveraging AI for efficiency while maintaining control and understanding of the underlying processes.
Links to stuff we mentioned during the pod:
- 09:16 - The LinkedIn post talking about Replit messing with someone's production code š³
- And the link to the thread of person who went through it
- The tool in question, Replit
- 13:01 - Rich's LinkedIn post with his tips
- 14:21 - GitHub Copilot
- 18:09 - VS Code
- 29:01 - Folks at different ends of the "AI Enthusiasm Spectrum"
- On the enthusiastic end
- Jason Arbon is on the positive side and is always creating something interesting like...
- On the unenthusiastic end
- Keith Klain has created a reading list to help get us up to speed...
- Keith's reading AI reading list
- You can see his full resources list here
- Maaike Brinkhof has a bunch of thought-provoking posts on the topic...
- Keith Klain has created a reading list to help get us up to speed...
- On the enthusiastic end
- 34:44 - Want to know what "conflabulation" means? Listen to Martin explain it on the Ghost in th code podcast (that's not a typo!)
- 37:24 - What is Context Engineering? Perplexity has answers!
- 46:38 - The legendary Lt. Geordi La Forge from Star Trek: The Next Generation.
- 51:48 - After recording, the very cool Paul Coles published his article The Subtle Art of Herding Cats: Why AI Agents Ignore Your Rules (Part 1 of 4, explaining the topic of Context Engineering. Itās brilliant!
- 59:04 - The promises of technology over the years...
- 60:50 - The always insightful Meredith Whittaker of Signal fame, where is the president and services on its board of directors, explains the privacy and security concerns with agentic technology.
Watch the clip, then go back and watch the whole thing!
00:00 - Intro
01:17 - Welcome
01:30 - TANGENT BEGINS... All kinds of egregious waffling follows. Skip to the actual content at 08:34
01:31 - Rich VS Tree Stump
01:57 - What on earth did Rich need the pulley for?
02:26 - Vern's nerdy confession and pulley confusion
02:52 - Does Rich live next door to Tony Stark?!
03:22 - What to do when you need a steel RSJ
03:35 - We admit defeat. 03:36 - Welcome to Rich's Garden Adventures Podcast!
07:25 - What has Vern been up to?
08:34 - We attempt to segue into the episode at last!
08:35 - TANGENT ENDS...
08:51 - Richās POC: using agents to help build AI tools
09:45 - The Replit disaster: vibe coding meets deleted production data 11:12 - Sociopathic assistants and the case for AI gaslighting 11:55 - Vernon wants his team experimenting with AI tools
12:50 - Rich explains the context for his latest AI adventures
13:18 - Richās bench project and āputting the engineering hat onāĀ
15:22 - Setting up the stack and staying in controlĀ
16:53 - A familiar story: things were going fine until they werenātĀ
17:00 - Ask vs Edit vs Agent mode in Copilot explainedĀ
19:06 - The innocent linting error that spiralled out of controlĀ
21:16 - Stuck in a loop: āI didnāt know what it was doing, but I let it keep goingāĀ
22:11 - The fateful click: āIām going to reset the DBāĀ
23:10 - The aftermath: no data, no damage⦠but very nearlyĀ
23:33 - Security wake-up call: agents are acting as youĀ
24:39 - You canāt fix what you donāt know it brokeĀ
25:52 - Can you interrupt an agent mid-task?Ā
27:14 - When agents get āare you sure?ā momentsĀ
28:15 - Tea breaks as a dev strategy: outsourcing work to agentsĀ
29:24 - Jason Aborn vs Keith & Maaike: where Rich sits on the AI enthusiasm spectrumĀ
30:41 - Tip1. The first of Richās 6 agent tips: commit after every interaction
32:12 - Why trusting the ākeep allā button is riskyĀ
34:01 - Writing your own commits vs letting the agent do itĀ
35:26 - When agents lose the plot: reset instead of fixingĀ
36:55 - āYouāre insane now, GPT. Iām giving you a break.āĀ
37:54 - Tip 2: Make the task as small as possibleĀ
39:59 - The middle ground between 'ask' and full agent delegationĀ
41:12 - Tip 3: Ask the agent to break the task down for youĀ
43:36 - The order matters: why you shouldnāt start with the form UIĀ
44:33 - Vernon compares it to shell command pipelinesĀ
45:09 - It can now open browsers and run Playwright tests (!)Ā
46:23 - Star Trek and the rise of the engineer-agent hybridĀ
47:57 - Tips 4ā6: Test often, review the code, use other modelsĀ
49:39 - Pattern drift and the importance of prompt templatesĀ
50:51 - Vernonās nemesis: m dashes, emojis, and being ignored by GPTĀ
51:48 - Context engineering vs prompt engineeringĀ
52:43 - When codebases get too big for agents to copeĀ
53:40 - Why agents sometimes act dumber than your IDEĀ
54:32 - The danger of outsourcing good practices to AIĀ
54:48 - Spoilers: Richās upcoming keynote at TestItĀ
55:01 - Agents donāt ask why ā they just keep goingĀ
56:42 - Goals vs loops: when failure isnāt part of the planĀ
58:32 - The question of efficiency: is training agents worth it?Ā
59:47 - Richās take: weāll buy agents like we buy SaaSĀ
61:08 ...























