Guests

• David Eason, Principal Product Manager at Trainline


• Billie Bradley, Product Manager, Travel Assistant at Trainline


• Matt Farrelly, Head of AI and Machine Learning at Trainline

Key Takeaways

• AI assistants need both scalable reasoning and deep domain context to be useful.


• Tool design and guardrails are as critical as prompt design in agent systems.


• LLM-as-judge evals make it possible to measure open-ended systems without massive labeling costs.


• Even legacy companies can move fast when they embrace experimentation and tight PM–engineering collaboration.

Chapters:
00:00  Introduction and Team Introductions
00:51  Overview of Trainline's Mission and History
02:30  AI Integration in Trainline's Services
05:08  Challenges and Solutions in AI Implementation
06:52  Building and Iterating the AI Travel Assistant
14:58  User Experience and Guardrails
22:26  Technical Challenges and Solutions
34:29  The Challenge for Product Managers in AI
34:55  Billy's Background in AI
35:42  The Rapid Evolution of AI Technology
37:14  Managing Information Overload
37:58  Collaboration Between Product Managers and Engineers
38:42  Trainline's Approach to Machine Learning
39:36  Scaling Up: From 450 to 700,000 Pages
40:21  Challenges in Data Retrieval and Processing
45:55  Evaluating AI Assistants
48:22  The Role of LLM as Judges
50:19  User Context Simulation for Real-Time Evaluation
01:06:56  Future Directions for Trainline's AI Assistant