TestGuild Automation Podcast

I asked thirteen people inside AI testing one question: can AI test your software for you? Founders, CTOs, project leads, and quality leaders, many of them selling AI testing tools right now. Not one said yes. This episode is my session from the Testear.la 2026 conference, built entirely from twelve months of interviews on this show with the people actually building this technology, including Applitools, Maestro, Bright Security, QA.tech, Cucumber, Ito, and the lead of the Selenium project. In it you'll learn why one AI testing vendor deliberately removed AI from the part where tests run, the real productivity number teams are measuring versus the number in the case studies, the four incompatible architectural bets vendors are making and how to tell which one your tool has chosen for you, why AI generated code is producing vulnerabilities far faster than it produces code, and the one architecture rule that stops an agent from editing the test that was supposed to catch it. You'll also get the list. Every task these thirteen people said they would never delegate, assembled from conversations that had no coordination between them. Plus two tests you can run on Monday, both free, that tell you whether you're ready to trust an agent with your suite. If you're a tester, automation engineer, or QA leader trying to figure out where you fit in all of this, this one is for you. The people building AI testing don't trust it to test on its own. That gap is your job. Exclusive Sponsor Quick note before we get into this one. Everything you're about to hear points at the same conclusion. The work is moving from writing tests to supervising the things that write them. That's a different skill set, and most of us are picking it up by accident. Automation Guild '27 is where you pick it up on purpose. Year eleven. February 8th through the 12th. Five days, live, with the best E2E automation practitioners working today, and you keep lifetime access to every session. The early bird list is open right now, and that's where the biggest discount goes. Head to testguild.com and look for Automation Guild '27 early bird, or grab the link in the show notes: https://testgld.link/yearlybird

Is AI going to replace software testers, or make them more valuable than they have ever been? Jonathon Wright says both, and in this episode he does not back down from either position. Jonathon Wright has spent thirty years in test automation, going all the way back to WinRunner. He now runs thirty AI agents across twelve screens, he sat with Gartner for their research on agentic platforms, and he says the last six months changed the job more than the previous three decades combined. We get into what he calls confidence engineering, why he has not opened a browser in five years, and what happened when he generated thirty seven thousand test scenarios from space agency specifications and put them in front of NASA. He also tells the story of a testing leader who taught his agents everything he knew about accessibility work and then got let go. A few minutes later he tells you your career has another fifteen good years in it. I pushed him on the contradiction. Listen for how he answers it. What you will learn in this episode: Why the flood of AI generated code makes verification the scarce skill, and testers the people who matter most What confidence engineering means and how to apply it to systems that never return the same answer twice Why browser automation was the easy problem, and what is actually hard now How intent driven testing lets you model a whole system before a single line of code exists Why getting the requirements right matters more with agents than it ever did without them The story of an agent taking twenty minutes to press one key, and what it tells you about the hype cycle Three concrete things to start doing Monday to stay valuable as your agent count climbs Whether you are an automation engineer, a manual tester, or a QA leader trying to work out what your team should do differently this quarter, this conversation will give you a clearer read on where the value is actually moving.

Everything you knew about performance testing changes when the system you're testing is non deterministic. In this episode of the TestGuild Automation Podcast, Joe Colantonio sits down with Kandasamy Selvaraj, Principal Architect and author of the free book Rethinking Performance Engineering for Agentic AI, to unpack what it really takes to move an AI agent from a working demo to an enterprise system handling millions of conversations per hour. Checkout his free book: https://leanpub.com/agentic-ai-performance Kandasamy shares the practical playbook he's built running agentic AI in production, including why the same request can take three seconds one run and eight seconds the next, how to use harnesses to bound tool calls, reasoning loops, and token budgets, and why your SRE dashboard can look perfectly healthy while your token costs quietly balloon to five times baseline. You'll learn how his team: Shifts performance gates left into every commit with JMeter Shifts right with synthetic monitors on blue green deployments Uses Langfuse and OpenTelemetry to spot context bloat before it hits production. You'll also hear how to: Slash AI costs with prompt caching Conversation capping Routing simple queries to cheaper models Why you should load test at the API layer before touching the UI, How to keep stubs honest with production sampled latency Why one misbehaving agent can starve every other agent sharing the same provider. If you're a tester, performance engineer, SRE, or architect building on LLMs, this conversation will change how you think about scale.

Most testers are stuck arguing about whether AI is coming for their jobs. Swati Seela thinks that's a distraction from a much more useful question: when we say "AI testing," what do we actually mean? What do you think? Take our TestGuild State of Test Automation Survey now: https://testgld.link/27data In this episode, Swati walks through the AI Testing Landscape, a practical framework she presented at CAST 2026 that breaks AI testing into five overlapping areas: AI assisted testing, AI evaluation and trust, AI development quality, testing products that contain AI functionality, and testing the AI models themselves. Each one has a different test intent, a different technique, and a different toolset, which is exactly why lumping them together leaves new testers with no idea where to start. Along the way Swati gets specific about the failure modes she keeps hitting in real work. Why AI opens almost every response by agreeing with you, and why that false sense of correctness is the modern version of a green dashboard hiding tests that stopped meaning anything years ago. Why her MCP test generator kept skipping the cataloging step, what that revealed about models trading completeness for speed, and how splitting one workflow into two got her reliability back. Why traditional Boolean assertions break down against probabilistic systems, and what your automation framework has to do instead. She also shares how she uses AI to learn hard material in the semiconductor world without losing the thread, what she absolutely will not accept without laying eyes on it herself, and her answer to whether a tester with a decade of experience should be worried right now. Her closing advice comes down to two things that have to travel together: testing fundamentals, and AI literacy. Don't hate it, don't love it, just use it.

Manual testing never died. IoT is the proof. In this episode, Joe Colantonio talks with Oleksii Cherkashyn, a QA team lead, automation engineer, and IoT testing specialist who built a complete test architecture from scratch across hardware, firmware, mobile, web, and API. IoT breaks the usual automation playbook. You are not testing a shopping cart. You are connecting a real microcontroller, flashing a sketch, validating that a physical command produced the right change in a dashboard widget, and then doing the same thing again through the REST API and the mobile app. No marketplace framework covers that, so Oleksii built his own. You'll learn: Why manual QA remains essential in IoT and which cases can never be automated How Oleksii built a custom Node.js library to simulate up to 50,000 device connections for performance testing without buying the hardware Why he chose WebDriverIO over Playwright and the mobile automation reason behind it How he uses WebDriverIO MCP and cloud coding agents every day to create and repair tests The hard truth about dependent test chains and where AI agents still hallucinate How to handle time based scenarios like sleep modes, daily triggers, and delayed notifications Static versus dynamic provisioning, and how to test OTA firmware updates What to do first if your company hands you an IoT device and you have never tested hardware If you work in test automation and you want a look at the layer of testing that AI is not coming for, this one is for you. Listen up, and check out the links below for everything mentioned in this episode.