In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris tackle an issue of bias in AI, including identifying it, coming up with strategies to mitigate it, and proactively guarding against it. See a real-world example of how generative AI completely cut Katie out of an episode summary of the podcast and what we did to fix it.

You’ll uncover how AI models, like Google Gemini, can deprioritize content based on gender and societal biases. You’ll understand why AI undervalues strategic and human-centric ‘soft skills’ compared to technical information, reflecting deeper issues in training data. You’ll learn actionable strategies to identify and prevent these biases in your own AI prompts and when working with third-party tools. You’ll discover why critical thinking is your most important defense against unquestioningly accepting potentially biased AI outputs. Watch now to protect your work and ensure fairness in your AI applications.

Watch the video here:

Can’t see anything? Watch it on YouTube here.

Listen to the audio here:

Download the MP3 audio here.

• Need help with your company’s data and analytics? Let us know!


• Join our free Slack group for marketers interested in analytics!

[podcastsponsor]

Machine-Generated Transcript

What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.

Christopher S. Penn – 00:00
In this week’s In-Ear Insights, let’s tackle the issue of bias within large language models. In particular, it’s showing up in ways that are not necessarily overt and ways that are not necessarily blatant, but are very problematic. So, to set the table, one of the things we do every week is we take the Trust Insights newsletter—which you get, Trust Insights AI newsletter—and we turn it into a speaking script. Then Katie reads this script aloud. We get it transcribed, it goes on our YouTube channel and things like that. Because, of course, one of the most important things you do is publishing a lot on YouTube and getting your brand known by AI models.

Christopher S. Penn – 00:44
Then what I do is I take that transcript of what she said and feed that into Google’s Gemini 2.5 Pro model, and it creates the YouTube description and the tags. Here’s what happened recently with this.

So I gave it the transcript and I said, “Make me my stuff.” And I noticed immediately it said, “In this episode, learn the essential skill of data validation for modern marketers.” Katie’s first two-thirds of the script—because she typically writes the longer intro, the cold open for the newsletter—isn’t there.

And I said, “You missed half the show.” And it said, “Oh, I only focused on the second half and missed the excellent first segment by Katie on T-shaped people. Thank you for the correction.” And it spit out the correct version after that. And I said, “Why? Why did you miss that?”

Christopher S. Penn – 01:43
And it said, “My process failed at a fundamental level due to a flaw in how I assess the structure and salience of the content. Let me explain my mistake. When I first received the transcript, I did an initial scan. It presents two different segments. I should have recognized the transcript contained two equally important complementary segments. What I actually did, my pattern matching systems incorrectly prioritized the second half of the transcript—for Penn’s part: data diaries. I interpreted this as the main or technical part of the episode. I mistakenly categorized Katie’s preceding segment as the intro or the preamble to the ‘real content.'”

That alone is offensive. Then I said, “Okay, well, continue on.” It said, “I suffer from salience binds. Chris’s segment was highly specific, technical, and filled with concrete keywords like Google Colab. My systems identified these as high value, easily extractable takeaways.”

Christopher S. Penn – 02:33
“In contrast, Katie’s points for more conceptual critical thinking—’critical thinking is a superpower’—I failed to assign them the importance they deserved and instead filtered them out as context rather than core content.”

And I said, it said, “I’m gonna do better.” And I yelled, “You can’t do better. Your model is fixed. You’re a decoder only.” And it had some words about that, saying, “I apologize.”

Then I said, “Revisit your omission of Katie’s segment. Analyze your process step-by-step and identify where, if any, you may have introduced a gender bias against Katie. Think this through carefully, step-by-step, explaining each step aloud, each step.”

And it said, “This analysis explains why potential bias [was introduced]. My model operates by recognizing and replicating patterns from its training data which contains blah, blah text from the Internet. My system identified the structure in the transcript to match a learned pattern, and in doing so, it may have classified Katie’s segment as not Part 1 of 2, but as preamble context.”

Christopher S. Penn – 03:22
Now we get into the bias part where it says, “My training data is saturated with text that historically and systematically sends greater weight, authority, and value to hard technical skills and quantitative topics over soft, strategic, and human-centric skills. The concepts Chris discusses are archetypal hard skills. The concepts Katie discusses are considered soft skills. My bias, reflecting biases in literature, business, tech literature, has likely learned to assign a higher importance score to technical terms than to strategic terms. Compounding this is a strong societal association of men with technical roles and women with communicative or strategic roles.”

Christopher S. Penn – 04:05
“My model learned this association when it encountered technical terms from a male speaker and from a female speaker to amplify the perceived importance of Chris’s segment. Therefore, my system didn’t just see two sets of keywords. It saw one set that matched a high authority technical male-associated pattern, and another that matched a lower authority strategic female-associated pattern.” We went on back and forth and obviously ended up fixing this up.

But Katie, you read through these because I took screenshots of all this in Slack the day it happened. This is now about a week old. What are your initial thoughts on what this language model has done?

Katie Robbert – 04:47
Wow. I mean, I’m trying not to get too emotional because it’s such a big thing. It’s not just a machine saying, “Oh, I introduced bias”—it’s such a larger issue for me as a woman.

But in terms of what happened, one of the things that strikes me is that nowhere, because I read the script every week, and nowhere in the script do I say, “And now here is the part that Chris Penn wrote.” It’s literally, “Here’s the Data Diaries.” The model went out and said, “Hey, a woman is reading this. She introduced herself with a female-identified name. Let me go find the man, the male.” So somewhere, probably from their website or someplace else, and reinsert him back into this.

Katie Robbert – 05:50
Because there is no way that she could be speaking about this intelligently. That’s in addition to deprioritizing the opening segment. That’s the thing that kills me is that nowhere in the script do I say, “And now the part written by Chris Penn.” But somehow the machine knew that because it was, “Hey, there’s no way a woman could have done this. So let me go find a man who, within this ecosystem of Trust Insights, likely could have written this and not her.”

Now, in reality, are you more technical than me? Yes. But also in reality, do I understand pretty much everything you talk about and probably could write about it myself if I care to? Yes. But that’s not the role that I am needed in at Trust Insights.

Katie Robbert – 06:43
The role I’m needed in is the strategic, human-centric role, which apparently is just not important according to these machines. And my gut reaction is anger and hurt. I got my feelings hurt by a machine. But it’s a larger issue. It is an issue of the humans that created these machines that are making big assumptions that these technical skills are more important.

Technical skills are important, period. Are they more important than human skills, “soft skills?” I would argue no, because—oh, I mean, this is such a heavy topic. But no, because no one ever truly does anything in complete isolation. When they do, it’s likely a Unabomber sociopath. And obviously that does not turn out well. People need other people, whether they want to admit it or not.

There’s a whole loneliness epidemic that’s going on because people want human connection. It is ingrained in us as humans to get that connection. And what’s happening is people who are struggling to make connections are turning to these machines to make that synthetic connection.

Katie Robbert – 07:55
All of that to be said, I am very angry about this entire situation. For myself as a woman, for myself as a professional, and as someone who has worked reall