In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss why enterprise generative AI projects often fail to reach production.

You’ll learn why a high percentage of enterprise generative AI projects reportedly fail to make it out of pilot, uncovering the real reasons beyond just the technology. You’ll discover how crucial human factors like change management, user experience, and executive sponsorship are for successful AI implementation. You’ll explore the untapped potential of generative AI in back-office operations and process optimization, revealing how to bridge the critical implementation gap. You’ll also gain insights into the changing landscape for consultants and agencies, understanding how a strong AI strategy will secure your competitive advantage. Watch now to transform your approach to AI adoption and drive real business results!

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!


• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn – 00:00
In this week’s In Ear Insights, the big headline everyone’s been talking about in the last week or two about generative AI is a study from MIT’s Nanda project that cited the big headline: 95% of enterprise generative AI projects never make it out of pilot. A lot of the commentary clearly shows that no one has actually read the study because the study is very good. It’s a very good study that walks through what the researchers are looking at and acknowledged the substantial limitations of the study, one of which was that it had a six-month observation period.

Katie, you and I have both worked in enterprise organizations and we have had and do have enterprise clients. Some people can’t even buy a coffee machine in six months, much less route a generative AI project.

Christopher S. Penn – 00:49
But what I wanted to talk about today was some of the study’s findings because they directly relate to AI strategy. So if you are not an AI ready strategist, we do have a course for that.

Katie Robbert – 01:05
We do. As someone, I’ve been deep in the weeds of building this AI ready strategist course, which will be available on September 2. It’s actually up for pre-sale right now. You go to trust insights AI/AI strategy course. I just finished uploading everything this morning so hopefully I used all the correct edits and not the ones with the outtakes of me threatening to murder people if I couldn’t get the video done.

Christopher S. Penn – 01:38
The bonus, actually, the director’s edition.

Katie Robbert – 01:45
Oh yeah, not to get too off track, but there was a couple of times I was going through, I’m like, oops, don’t want to use that video. But back to the point, so obviously I saw the headline last week as well. I think the version that I saw was positioned as “95% of AI pilot projects fail.” Period. And so of course, as someone who’s working on trying to help people overcome that, I was curious. When I opened the article and started reading, I’m like, “Oh, well, this is misleading,” because, to be more specific, it’s not that people can’t figure out how to integrate AI into their organization, which is the problem that I help solve.

Katie Robbert – 02:34
It’s that people building their own in-house tools are having a hard time getting them into production versus choosing a tool off the shelf and building process around it. That’s a very different headline. And to your point, Chris, the software development life cycle really varies and depends on the product that you’re building. So in an enterprise-sized company, the likelihood of them doing something start to finish in six months when it involves software is probably zero.

Christopher S. Penn – 03:09
Exactly. When you dig into the study, particularly why pilots fail, I thought this was a super useful chart because it turns out—huge surprise—the technology is mostly not the problem. One of the concerns—model quality—is a concern.

The rest of these have nothing to do with technology. The rest of these are challenging: Change management, lack of executive sponsorship, poor user experience, or unwillingness to adopt new tools. When we think about this chart, what first comes to mind is the 5 Ps, and 4 out of 5 are people.

Katie Robbert – 03:48
It’s true. One of the things that we built into the new AI strategy course is a 5P readiness assessment. Because your pilot, your proof of concept, your integration—whatever it is you’re doing—is going to fail if your people are not ready for it.

So you first need to assess whether or not people want to do this because that’s going to be the thing that keeps this from moving forward. One of the responses there was user experience. That’s still people.

If people don’t feel they can use the thing, they’re not going to use it. If it’s not immediately intuitive, they’re not going to use it. We make those snap judgments within milliseconds.

Katie Robbert – 04:39
We look at something and it’s either, “Okay, this is interesting,” or “Nope,” and then close it out. It is a technology problem, but that’s a symptom. The root is people.

Christopher S. Penn – 04:52
Exactly. In the rest of the paper, in section 6, when it talks about where the wins were for companies that were successful, I thought this was interesting.

Lead qualification, speed, customer retention. Sure, those are front office things, but the paper highlights that the back office is really where enterprises will win using generative AI. But no one’s investing it. People are putting all the investment up front in sales and marketing rather than in the back office. So the back office wins.

Business process optimization. Elimination: $2 million to $10 million annually in customer service and document processing—especially document processing is an easy win. Agency spend reduction: 30% decrease in external, creative, and content costs. And then risk checks for financial services by doing internal risk management.

Christopher S. Penn – 05:39
I thought this was super interesting, particularly for our many friends and colleagues who work at agencies, seeing that 30% decrease in agency spend is a big deal.

Katie Robbert – 05:51
It’s a huge deal. And this is, if we dig into this specific line item, this is where you’re going to get a lot of those people challenges because we’re saying 30% decrease in external creative and content costs. We’re talking about our designers and our writers, and those are the two roles that have felt the most pressure of generative AI in terms of, “Will it take my job?” Because generative AI can create images and it can write content. Can it do it well? That’s pretty subjective. But can it do it? The answer is yes.

Christopher S. Penn – 06:31
What I thought was interesting says these gains came without material workforce reduction. Tools accelerated work, but did not change team structures or budgets. Instead, ROI emerged from reduced external spend, limiting contracts, cutting agency fees, replacing expensive consultants with AI-powered internal capabilities. So that makes logical sense if you are spending X dollars on something, an agency that writes blog content for you. When we were back at our old PR agency, we had one firm that was spending $50,000 a month on having freelancers write content that when you and I reviewed, it was not that great. Machines would have done a better job properly prompted.

Katie Robbert – 07:14
What I find interesting is it’s saying that these gains came without material workforce reduction, but that’s not totally true because you did have to cut your agency fees, which is people actually doing the work, and replacing expensive consultants with AI-powered internal capabilities. So no, you didn’t cut workforce reduction at your own company, but you cut it at someone else’s.

Christopher S. Penn – 07:46
Exactly. So the red flag there for anyone who works in an agency environment or a consulting environment is how much risk are you at from AI taking your existing clients away from you? So you might not lose a client to another agency—you might lose a client to an internal AI project where if there isn’t a value add of human beings. If your agency is just cranking out templated press releases, yeah, you’re at risk. So I think one of the first things that I took away from this report is that every agency should be doing a very hard look at what value it provides and saying, “How easy is it for AI to replicate this?”

Christopher S. Penn – 08:35
And if you’re an agency and you’re like, “Oh, well, we can just have AI write our blog posts and hand it off to the client.” There’s nothing stopping the client from doing that