Bryan and Adam and the Oxide Friends take on GPT and its implications for software engineering. Many aspiring programmers are concerned that the future of the profession is in jeopardy. Spoiler: the Oxide Friends see a bright future for human/GPT collaboration in software engineering.We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from March 20th, 2023.In addition to Bryan Cantrill and Adam Leventhal, speakers on MM DD included Josh Clulow, Keith Adams, Ashley Williams, and others. (Did we miss your name and/or get it wrong? Drop a PR!)Live chat from the show (lightly edited):
ahl: John Carmack's tweet
ahl: ...and the discussion
Wizord: https://twitter.com/balajis/status/1636797265317867520 (the $1M bet on BTC, I take)
dataphract: "prompt engineering" as in "social engineering" rather than "civil engineering"
Grevian: I was surprised at how challenging getting good prompts could be, even if I wouldn't quite label it engineering
MattCampbell: I tested ChatGPT in an area where I have domain expertise, and it got it very wrong.
TronDD: Also interesting https://www.youtube.com/watch?v=jPhJbKBuNnA
Wizord: the question is, when will it be in competition with people?
Wizord: copilot also can review code and find bugs if you ask it in a right way
ag_dubs: i suspect that a new job will be building tools that help make training sets better and i strongly suspect that will be a programming job. ai will need tools and data and content and there's just a whole bunch of jobs to build tools for AI instead of people
Wizord: re "reading manual and writing DTrace scripts" I think it's possible, if done with a large enough token window.
Wizord: (there are already examples of GPT debugging code, although trivial ones)
flaviusb: The chat here is really interesting to me, as it seems to miss the point of the thing. ChatGPT does not and can not ever 'actually work' - and whether it works is kind of irrelevant. Like, the Jaquard Looms and Numerical Control for machining did not 'work', but that didn't stop the roll out.
Columbus: Maybe it has read the dtrace manual 😉
JustinAzoff: I work with a "long tail" language, and chatgpt sure is good at generating code that LOOKS like it might work, but is usually completely wrong
clairegiordano: Some definite fans of DTrace on this show
ag_dubs: a thing i want to chat about is how GPT can affect the "pace" of software development
sudomateo: I also think it's a lot less than 100% of engineers that engage in code review.
Wizord: yes, I've had some good experience with using copilot for code review
ag_dubs: chatgpt is good at things that are already established... its not good at new things, or things that were just published
Wizord: very few people I know use it for the purpose of comments/docs. just pure codegen/boilerplayes
chadbrewbaker: "How would you write a process tree with dtrace?" (ChatGPT4)
printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND"");
printf(""%5d %5d %5d %s\n"", pid, ppid, uid, execname);
TronDD: That's interesting as expensive, specialized code analysis tools have been varying level of terrible for a long time
JustinAzoff: I did an experiment before where I asked it to write me some php to insert a record into a database. so of course it generated code with sql injection
chiefnoah: It's ability seems to scale with how many times someone has done the exact thing you're trying to do before
JustinAzoff: but then I asked if sql injection was bad, which it explained that it was. then I asked if the code it wrote me was vulnerable to sql injection. it then explained it was
Columbus: It misses empirical verification; forming a hypothesis, testing it, and learning from the result. There have been some attempts to implement this by feeding back e.g. command output into the prompt
JustinAzoff: so then the crazy part, I asked if sql injection was bad, why did it give me code that was vulnerable to sql injection. It the went on to say that the first thing it gave me was just for example purposes
JustinAzoff: so no wonder people get into "prompt engineering" since it's clear that you need to do things like ask for code that does something, and is secure, high performance, does not have buffer overflows or sql injection vulns
MattCampbell: In my test case ("Write a Win32 UI Automation provider in C++"), all it did was produce plausible-sounding crap
ag_dubs: pattern matching over very very large data sets
clairegiordano: Bryan just said this and I wanted to write it down, re GPT-3: "the degree that it changes its answers when you tell GPT-3 to think like someone else"
JustinAzoff: or even just, "do that, but better"
ag_dubs: i think a lot of the awe of gpt is recognizing how simple our own strategies are instead of how complex the AI's strategy is
chadbrewbaker: "How would Bryan Cantrill re-write this script?" (ChatGPT4)
#pragma D option quiet
printf(""%5s %5s %5s %s\n"", ""PID"", ""PPID"", ""UID"", ""COMMAND"");
self->indent = 0;
self->indent = 0;
this->curpid = pid;
this->curppid = ppid;
this->curuid = uid;
this->indent = self->indent;
/execname != ""dtrace""/
printf(""%*s"", this->indent * 2, """");
printf(""%5d %5d %5d %s\n"", this->curpid, this->curppid, this->curuid, execname);
/this->curpid == pid/
antranigv: that's pretty hit on!
chiefnoah: Yup. Most people are not doing things that haven't been done before. A significant portion of software is just building blocks of libraries
Wizord: intelligence is compression, in some sense.
dataphract: "critique the epoll API as though you are Bryan Cantrill"
ag_dubs: a brain would be much stranger!!
Wizord: the ability to reduce a large dataset to a coherent set of rules
antranigv: "Explain the issues of epoll, write as if it's a Bryan Cantrill talk"
JustinAzoff: I was even thinking if there's any sort of parallel between the type of saying like "never write your own crypto" to "use well established libraries, don't reinvent the wheel" to "don't write any code at all, use the AI to help you"
jbk: <@840382902163472385> futex syscall instead 🙂
Riking: the "attention heads"
flaviusb: Like, it doesn't know anything, any more than a text file with googly eyes stuck to it 'knows' anything.
Wizord: are you sure you want to make it self-aware as fast as possible?
dataphract: I don't know that we as people are capable of recognizing the point in time at which ML models become capable of "knowing", if such a time comes
Wizord: using AI to create more inequality is my #2 concern :\
flaviusb: There was a lot of hype when Rails was new and good code template generation tools were not commonly known around the lines of 'Rails is telepathic and a better programmer than you' - but it is a category error. Same with LLMs.
chadbrewbaker: As you get larger contexts, the information theory becomes more and more interesting https://arxiv.org/abs/2303.09752
ag_dubs: this convo has taken a fanatic turn...
flaviusb: And 'when will Rails become sentient' makes as much sense to ask as 'when will an LLM become sentient'.
clairegiordano: Here is Simon Willison's blog post: https://simonwillison.net/2023/Feb/15/bing/
dataphract: tsundere bing AI assistant
ag_dubs: kevin roose also asked it for its shadow self...
ag_dubs: it's not like it came outta nowhere 🤷
ag_dubs: i am more worried about the people who believe chatgpt is "thinking" then whether it really is
ag_dubs: like the worry is when people believe it is intelligent
Riking: Feels like it's pulling from the grand synthesis of romance novels
Wizord: the mere fact it gets this close is great. as limited as it is now.
perplexes: Your prompts locate you in 50,000 dimensional probability space, so, like your “for you” page on TikTok, it tells you wayyy more about the prompter than the model
Columbus: It’s a child prodigy that’s always on the verge of insanity.
ag_dubs: think about it like a tool!!!
Columbus: I think about it like a grad student.
perplexes: I’m curious about microsoft 365’s copilot ai grounding system and how well it works
ag_dubs: i think the most important thing will be ownership of training data. i think people will be less mesmerized when we make it easier for folks to create specialized models for their own content
ag_dubs: think about.. what is SEO in a chatgpt world- it's that training data
dataphract: this already happened with Replika, right?
flaviusb: Also when we make 'small ai' more possible so that you don't have as much centralisation and gatekeeping.
Wizord: re: "how do we make it more of a tool?"
this is an open question, and there's a tradeoff between how much of a "tool" AI is going to be and how autonomous it will be. also, at some point of "intelligence", a "mind" will "understand" that having more power will get you your goal faster.
ag_dubs: we need to empower more folks to build specialized/authoritative training data sets that can be deployed in a more timely fashion
perplexes: The alpaca.cpp direction is exciting, bringing models down to laptop size
flaviusb: I think of it as 'how phenomenologically transparent' it will a) seem to be and b) actually be.,
Me: What if I do `echo message | wall user` ?
GPT: If you run the command echo message | wall user, it will write the message ""message"" to the standard input of the wall command, which will then broadcast the