chatgpt-limitations.mp3: Audio automatically transcribed by Sonixchatgpt-limitations.mp3: this mp3 audio file was automatically transcribed by Sonix with the best speech-to-text algorithms. This transcript may contain errors. Speaker1: Welcome to Radical AI, a podcast about technology, power, society and what it means to be human in the age of Information. We are your hosts. Dylan and Jess were two PhD students with different backgrounds researching AI and technology ethics.Speaker2: And our last episode we covered some of the basics of chat. Gpt three. In this episode, we dig a bit deeper. We interview Dr. Emily M Bender and Dr. Casey Fassler about the limitations of chat. Gpt three We cover ethical considerations, bias and discrimination, and the importance of algorithmic literacy in the face of chat bots.Speaker1: Emily M Bender is a professor of linguistics and an adjunct professor in the School of Computer Science and the Information School at the University of Washington, where she's been on the faculty since 2003. Her research interests include multilingual grammar, engineering, computational semantics and the societal impacts of language technology. Emily was also recently nominated as a fellow of the American Association for the Advancement of Science.Speaker2: Casey FESSLER is an associate professor in information science at University of Colorado Boulder. She researches and teaches in the areas of technology, ethics, Internet law and policy and online communities. Also a public scholar, she is a frequent commentator and speaker on topics of technology, ethics and policy. And her research has been covered everywhere from The New York Times to Teen Vogue.Speaker1: And on a more personal note, we just wanted to shout out that we are so excited to share this interview with Emily and Casey with all of you, because we have known Emily and Casey for quite a while now. Emily, we first met when she was on the show, I think it was almost three years ago now. She was one of our our first community members, and she has been an avid supporter of the podcast and of this project basically from its beginning. And Casey is my one of my PhD advisors at the University of Colorado. So I've known her for quite a few years now too. And both of them come from totally different angles and areas of expertise. When it comes to chat bots and chat specifically, we have Casey, who has the law and policy background, and then we have Emily, who has the natural language processing background. And so we were just super excited to bring them together in conversation for this interview and we're really excited to share the outcome with all of you. We are on the line today with Casey Kessler and Emily Bender. Casey and Emily, welcome to the show and welcome back to the show.Speaker3: Super, super excited to be joining you for this.Speaker4: Thank you. This is Casey, and I'm very excited to be on the podcast for the first time.Speaker1: Yay! And today we are talking about a topic that has been receiving a lot of news attention as of late. Chat. We're going to begin with the basics here. So, Emily, our first question is for you. Could you briefly summarize how GPT works and what makes this technology so novel in the first place?Speaker3: Yeah. So. So what is chat GPT? Chat GPT at its core is what we call a large language model, so that the internals of it, the sort of guts are an enormous number of what's called parameters in this very large neural network that have been set through a training procedure where the system sees just piles and piles and piles of text and computer code. I think in the case of chat GPT. The text isn't just English and it's training task is predict what word comes next, predict what word comes next, and then compare that prediction to what was actually in the text. And the training algorithm then adjusts those parameters based on how right or wrong it was. And it keeps doing that over and over and over again, over just an absolutely enormous amount of text. That's the first pass. Once you've got that large language model, what they've done, I have to say it's not entirely known outside of open AI. They have not been open about this. But the blog post basically said we've done some further training and some of it seem to have to do with dialogue to make it be more like a dialogue system, because most of the text you can find on the web is not dialogue. And so there's some training about dialogue, and then there's this phase called reinforcement learning from human feedback, where human raters were given output of the system and asked to rate it as as good or bad or helpful or unhelpful or whatever their rating system was. And so now the system is not only trained to predict what the next word should be, but what the next word should be so that it would be well rated by humans, such as those who are doing the ratings.Speaker3: There's another layer in there which has to do with trying to suppress toxic or hateful or otherwise problematic output. And we learned from Billy Perrigo reporting in Time magazine that that was outsourced to Kenya to poorly paid workers and who had to do this very traumatic work of looking at these terrible outputs. So that that's sort of how it's built. So you also asked what's novel about this tech? And I would say not much. What's novel is the way it's been promoted. So we already had, well, internal to Google. There was already LAMDA, for example, that became big news when Blake Lemoine decided that Lambda was sentient and needed help, which is a very similar system as far as is known. There was already GPT three from Openai, which is kind of a predecessor. It didn't have this dialogue overlay, but it did have the ability to generate coherent seeming text that would be somewhat pleasing to humans. But what's new here with GPT is that open? And I set it up with this interface that allowed people all over the world to play with it. And so you went from a world where it was basically people in tech who were playing with this stuff to people from all walks of life, playing with it and being exposed to it for the first time. And I think that's the real novelty.Speaker2: And that novelty we've seen a lot of hot takes on in the media and beyond. And as our, I guess, social media star on this episode, Casey, the Tik Tok star, I know that you're well embedded into a lot of these communities, a lot of these conversations. And I'm mostly wondering, what are you hearing and what's your reaction to it?Speaker4: I think that a lot of people are very impressed by what can be by what can be accomplished with with chat or what it what it can do. And I think it's reasonable that they're impressed. It's it's impressive. But in in a way that like I feel like the shine is going to wear off kind of quickly because, you know, the first time that you ask chat to write fanfiction of Sherlock Holmes on the Star Trek Enterprise, you are incredibly impressed that it can do that at all. Right. But then as you as you keep going, you're like, well, it can do that, but it's not very good at it. And my friend, the fan fiction writer, could have written something much better. So I do think that there is some extent to which this sort of like, Oh, this feels like magic is going to wear away a little bit. But I do think that one of the challenges here is that everything that Emily just explained, which I think makes a lot of a lot of sense to a lot of our listeners, is still kind of challenging to wrap your head around, like what this actually means. Like, you know, people are saying like, oh, it's just fancy autocomplete and like kind of but also when you actually then see it do something, you're like, Well, that seems much more magical than than autocomplete.Speaker4: And I was actually reminded, like when everyone started talking about chat. Gpt, which, you know, is a chat bot, I was thinking back to Eliza so the, the, the chat bot that that Joseph Weizenbaum created many, many, many years ago and I think wrote an article about it in the 1960s and. It was you know, it was procedural. It was mimicking a psychoanalyst. And so the way that it worked was just a set of rules. Like if you say, you know, my mother makes me sad. Well, tell me more about your mother. And that makes a lot of sense, right? But even then, he pointed out that, like this was sufficient to really dazzle people at the time. But as soon as you explained how it worked, like as soon as you could use language understandable enough to explain, like the inner workings of it, the magic crumbled away. And so I think that's part of the challenge here, is that it actually is kind of difficult to explain how this works at a level that the magic crumbles away for people.Speaker3: Yeah, I think that that's spot on, Casey. Part of the reason for it, as far as I can tell, is the unfathomable size of these training data sets. So we just don't have lived experience with working with that much text in working memory or frankly at all. And so it's really hard to have an intuitive sense of just what you can get from distribution patterns over enormous collections of text. And so people can say, Yeah, yeah, yeah, I know it's fancy autocomplete or I know it's a stochastic parrot, but still, how did it do that? And especially it's very good at anything to do with mimicking the form of language. So, you know, write instructions for operating a VCR in the language of a Shakespearean sonnet. It will do that in a very impressive way. And I think that some of the magic there is that that's a difficult trick for humans. There are some humans who are very good at it, but it's also something that we find impressive because of what kinds of mastery it would require a human to have. But that doesn't mean that Chadwick is doing it in the same way. Ted has no understanding, no communicative intent, no sense of what the sort of social meaning or value of VCR instructions and Shakesp