In this special episode of the ODSC Ai X Podcast, host Sheamus McGovern dives into the real-world impact of GPT-5—from routing and hallucination issues to cost savings and open-weight models.

Joining him are two expert guests:

• Ivan Lee: Founder and CEO of Datasaur, who helps enterprises build private LLM stacks and has deep experience evaluating model upgrades.
• Nir Gazit: Co-founder and CEO of Traceloop, and co-creator of the OpenTelemetry Generative AI SIG, who brings insight into model routing, evaluation strategies, and observability tooling.

Together, they unpack what GPT-5 actually changed—and what teams should do next.

Key Topics Covered:

• Why GPT-5’s biggest shift is routing, not reasoning
• What casual vs. power users gained (or lost) with the rollout
• Hallucination benchmarks vs. real-world results
• Evaluation strategies using open-source tools like Phoenix and LangChain
• OpenAI’s OSS model release and its enterprise implications
• Why developers worry about black-box routing and lack of traceability
• How to migrate safely: pinning snapshots, running evals, shadow testing
• Whether GPT-5 gets us closer to AGI—or just better infrastructure
• What to expect from agent workflows, tool selection, and model specialization

Memorable Outtakes:

• Ivan Lee: “GPT-5 is an upgrade for 98% of users—but for the power users, the loss of model choice felt like control was taken away.”
• Nir Gazit: “Of course every new model crushes it on benchmarks—they’re optimizing for the benchmarks. That doesn’t mean it works for your use case.”
• Ivan Lee: “OpenAI’s OSS release might be the bigger story than GPT-5. Suddenly, enterprises are back at the table.”

References & Resources:

Guests

• Ivan Lee – CEO of Datasaur
• Website: https://www.datasaur.ai
• LinkedIn: https://www.linkedin.com/in/iylee/
• Nir Gazit – CEO of Traceloop
• Website: https://www.traceloop.com
• Blog: https://www.traceloop.com/blog
• LinkedIn: https://www.linkedin.com/in/nirga/

Resources Mentioned

• OpenAI GPT-5  https://openai.com/gpt-5
• OpenTelemetry Project: https://opentelemetry.io
• Traceloop OpenLLMetry: https://www.traceloop.com/openllmetry
• Phoenix (Arize AI open-source evals): https://github.com/Arize-ai/phoenix
• LangChain Evals: https://python.langchain.com/api_reference/langchain/evaluation.html
• GPT-OSS Open Weight Models by OpenAI: https://platform.openai.com/docs/models/gpt-oss
• Claude + Model Context Protocol (Anthropic): https://docs.anthropic.com/en/docs/tool-use
• ARC-AGI Leaderboard: https://arcprize.org/leaderboard

Sponsored by:

🔥 ODSC AI West 2025 – The Leading AI Training Conference

Join us in San Francisco from October 28th–30th for expert-led sessions on generative AI, LLMOps, and AI-driven automation.

Use the code podcast for 10% off any ticket.

Learn more: https://odsc.ai