High Agency: The Podcast for AI Builders

Sourcegraph have built the most popular open source AI coding tool in both the dev community and the Fortune 500. I sat down with Beyang Liu their CTO and cofounder to find out how they did it.We dive into the technical details of Cody's architecture, discussing how Sourcegraph handles the challenges of limited context windows in LLMs, why they don't use embeddings in their RAG system, and the importance of starting with the simplest approach before adding complexity.We also touch on the future of software engineering, open-source vs closed LLM models and what areas of AI are overhyped vs underhypedI hope you enjoy the conversation!Chapters- 00:00:00 - Introduction- 00:02:30 - What is Cody, and how does it help developers?- 00:04:15 - Challenges of building AI for large, legacy codebases- 00:07:30 - The importance of starting with the simplest approach- 00:11:00 - Sourcegraph's multi-layered context retrieval architecture using RAG- 00:15:30 - Adapting to the evolving landscape of LLMs and model selection- 00:19:00 - The future of software engineering in the age of AI- [00:23:00 - Advice for individuals navigating the AI wave- 00:26:00 - Predictions for the future of AI in software development- 00:30:00 - Is AI overhyped, underhyped, or both?- 00:33:00 - Exciting AI startups to watch--------------------------------------------------------------------------------------------------------------------------------------------------Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to  https://hubs.ly/Q02yV72D0

I recently sat down with Bryan Bischof, AI lead at Hex, to dive deep into how they evaluate LLMs to ship reliable AI agents. Hex has deployed AI assistants that can automatically generate SQL queries, transform data, and create visualizations based on natural language questions. While many teams struggle to get value from LLMs in production, Hex has cracked the code.In this episode, Bryan shares the hard-won lessons they've learned along the way. We discuss why most teams are approaching LLM evaluation wrong and how Hex's unique framework enabled them to ship with confidence. Bryan breaks down the key ingredients to Hex's success:- Choosing the right tools to constrain agent behavior- Using a reactive DAG to allow humans to course-correct agent plans- Building granular, user-centric evaluators instead of chasing one "god metric"- Gating releases on the metrics that matter, not just gaming a score- Constantly scrutinizing model inputs & outputs to uncover insightsFor show notes and a transcript go to:https://hubs.ly/Q02BdzVP0-----------------------------------------------------Humanloop is an Integrated Development Environment for Large Language Models. It enables product teams to develop LLM-based applications that are reliable and scalable. To find out more go to  https://hubs.ly/Q02yV72D0

50% of AI contracts at Ironclad’s largest customers are now automatically negotiated with the help of generative AI. Ironclad were one of the earliest adopters of LLMs, starting when the best model was still GPT-3.  There’s a lot of hype around AI agents without many successful examples but Ironclad had successfully deployed them in one of the most sensitive industries imaginable. In this episode Cai explains how they achieved this. Why they had to build their own visual programming language to make agents reliable and shares his advice for AI leaders starting to build products today.Where to find us: https://hubs.ly/Q02z2J6v0

Welcome to very first episode of the High Agency podcast! High Agency is a new podcast from Humanloop.Every week, I (Raza Habib) will interview leaders from companies, who have already succeeded with AI in production.  We'll share their stories, lessons and playbooks to help you build with LLMs more quickly and with confidence.To get notified of the first episodes with Cai Gogwilt or Ironclad, Bryan Bishof of Hex, Beyang Liu of Sourcegraph and Wade Foster of Zapier please subscribe on youtube, spotify or apple podcasts! (just search for High Agency podcast)----------------------------------------------------------------------------------------------------------------------------------------Humanloop is an Integrated Development Environment for Large Language Models.It enables product teams to develop LLM-based applications that are reliable and scalable.Principally, it lets you rigorously measure and improve LLM performance during development and in production. The evalutation tools are. combiined with a collaborative workspace where engineers, PMs and subject matter experts improve prompts, tools and agents together.By adopting Humanloop, teams save 6-8 engineering hours each week through better workflows and they feel confident that their AI is reliable.To find out more go to https://hubs.ly/Q02z2J6v0