How GPT/ChatGPT Work - An Understandable Introduction to the Technology - Professor Harry Surden

agenda today I'm hopefully going to bring you from uh not so much just uh from scratch to knowledge about how GPT generative AI work and answer some of your questions in just a little bit of time uh but for those of you don't know me um Harry certain I'm the interim executive director of silk and Flatirons head of our Ai and law initiative and also just an ordinary old AI law professor as well as well as a bunch of you in my class so thanks so much for coming um what are we going to talk about today so today we've got a lot to cover we'll talk about what is generative artificial intelligence uh we'll do an introduction to chat GPT and modern language models then we'll figure out okay uh then we're gonna see how GPT Works uh we'll look at word vectors neural networks pre-training transfer learning Transformer and attention mechanism instruction fine tuning none of which both of you know right now but I'm going to explain this in really simple language so hopefully when we come out the other side non-technically oriented students will have a sense about how this amazing technology works uh and please hold all questions to the end um because we've got a lot to go through so let's start right out what is generative AI well uh kind of a more basic question is what is artificial intelligence so there's no agreed upon Definition of artificial intelligence lots of people will have their own but I find one useful definition is the following using computers to solve problems make predictions answer questions or make automated decisions or actions in the kinds of tasks that when humans do them are thought to engage intelligence higher order cognitive activities so I think something like driving you have to be thinking and using your visual Acuity or chess or problem solving uh these all engage higher order thought processes so when preparers do them and they do them quite a different way or when you're able to solve that kind of task we call it an artificial intelligence path so generative AI is a specific set of artificial intelligence that is focused on creating uh creative outputs that are generally associated with humans so these are things like art music text uh so for instance I used AI to create the logo for our conference I could never have drawn that myself um so let's just look at a couple of these examples generative art is really interesting there's something called mid-journey but there's some other companies out there there's one for an open AI called Dal e but you can basically type in anything you want and get anything so what about if I wanted to take Mona Lisa and uh show her in the style of Pixar the most famous painting in the world I I could never do that but with mid-journey I just do this there you go um or say I wanted a picture of a robot law professor teaching a law class again not something I could ever draw but with generative AI for mid-journey it's pretty easy to do in just a second so I encourage you all to try these really interesting tools you can pretty much generate anything you want high quality art it's quite amazing so that's generative Arts there's also gender of music you can create AI uh enhanced music this is a song I created so pretty impressive given that I don't really know much about music and there are lots of AI generated music fights out there for generating you can generate incredible music in any genre you can create uh voices that sound like particular artists it's opening up all sorts of New Frontiers but of all those the most important one in my view is generative text this is the real big one that's going to change our society so this is what we really need to focus on here so these are things such as chat gbt GPT models that can generate text so this is where I'm going to focus the most of our time because this is by far the biggest breakthrough of the three so let me dive in on that so uh what is chat GPT for GPT in large language models and why do I say those differently so uh many of us how many of us have played with chat GPT here okay so a lot of you have played with it so chat GPT is the chat based interface to an actual underlying technology called GPT which we're going to understand so that's not the only way you interface with it but that that's one way and GPT was originally developed by a company named openai and we'll talk about them a little bit more but it was General it was designed just to be an automated text generator so it could make stories or poems or articles or blog content in the style of people and that was kind of interesting so I had to write this uh funny poem about see the law using chat GPT and you know it's kind of a nice little poem um in Boulder we're a mountain Stand Tall lies a law school revered by the mall with a strong legal mind and a heartwarming kind sea loss at the standard for all right sounds pretty accurate so that's pretty impressive and that alone would have been amazing but we're gonna see it can generate many more uh amazing things uh but let's take a little bit of a overview of what it means uh so GPT those words uh those letters stand for generative pre-trained Transformer and we're going to break down each one of those words and hopefully you guys will understand what it means so we'll just start with generative so as I said earlier uh something like GPT is considered generative because it's generating texts as we will see it generates text one word at a time based upon all the other texts that's come before it and what you've past it so it's called generative because you can use AI for all sorts of other things besides generating text you know driving cars or predicting things so it's kind of saying this is our use of AI it's generating text pre-trained again we'll talk about this but this is one of the magic piece of GPT so pretend means they took basically a huge portion of the internet and they took tons of actual books most of the laws and they took this algorithm and kind of set it loose on vast quantities of text and it learned huge numbers of patterns about reasoning and how language works so that's the process of pre-training we'll see we'll see that's this very expensive process of getting the AI to learn the patterns of human thought and language um so that's the p and gbt and another word you might see out there are large language models or llm so something like GPT is known as a large language model because it uses lots of language vast quantities of language to during training to learn the different patterns of thought or language and then finally the tea part uh this is the Transformer we'll be talking about this too but basically this was an invention by Google in 2017 in a paper called attention is all you need it was a breakthrough in AI because it taught something like GPT to learn how to understand the context of the words that are being asked around it so previously these things couldn't really understand and I say that in quotes they couldn't understand what you were asking it and this was one of the big breakthroughs that allowed these models to be responsive and actually kind of understand um so and the Transformer is basically something uh a particular architecture or way of doing something called Deep learning so this is our high level overview and uh the next 15 to 20 minutes we're going to learn what all of these things mean and come out the other side and be AI experts um but just a little bit of History here so GPT was developed by uh what was originally a non-profit research standard that turned into a for-profit called open AI in the Bay Area and they built upon Google's Transformer architecture they took that and they made many huge improvements to get to the state of the arc where we are now uh they released gpt-1 2018. it was interesting um gbt2 in 2019 was kind of okay uh gpd3 in 2020 was the first one that could generate pretty lifelike text but it didn't really seem like useful AI it wasn't important like they're with chat gbt which uh you guys have used the original version came in November 2022 and that was a huge breakthrough as we'll talk to based upon two of these improvements the open AI made called instruction to find tuning and reinforcement learning from Human feedback again we'll learn what that means um but the there was a massive breakthrough and then gbt4 it's a chat gbt is known as GPT 3.5 just did it and now gbt4 was released just last month uh it's only available to people who pay for it or actually Bing Bing chat which you can get for free it uses a version Microsoft version of it it's a little bit Surly actually um but it's basically as capable it's incredibly Advanced so most of the things I'll be showing here today are gpt4 it was just released last month so to understand how good chat gbt and gbt is it's really important to highlight how bad the technology was or how not good it was just last year and I'm talking early 2022 so which is a huge leap from early 2022 to late 2020 or anything with shocked almost all AI researchers so when GPT 3 as I said came out in 2020 it was pretty good and then they made a couple of improvements up through uh early 2022 um but they weren't really that good other than generating kind of you know uh kind of fake Twitter posts or things of that nature kind of simulating human attacks so um luckily we can go back in time and see what these things were capable of back then so let me play that again uh there's something called the open AI playground where you can look at these old models not old like 2022 and play with them they're kind of Frozen in Time and Time Machine of the way the technology was just a year ago so if you ask it a common sense question and I've listed these Common Sense questions that I've been using over the years to test these machines to see if they can do reasoning and they would almost always fail and so here's one of them I would ask something you would ask a toddler uh how many legs does an apple have and a toddler would laugh and say an apple doesn't have legs and uh the old GPT confidently told us it was six and then I asked to write a motion for summary judgment not very good right uh this would not pass muster so many of us were you know thinking at the time you know maybe five ten years from now we'd be at the point where we need to answer a question like how many legs does an apple have or things like that but that's a long way off but we were wrong because in November 2022 um he came out where she had huge advancements and uh it wasn't just his ability to generate funny poems like I'm sure it was its ability to engage in reasoning and problem solving and that was totally unexpected among AI researchers we thought it would be able to generate tax we thought that was totally expected you didn't expect it to be about the reason and solve problems so here's a good example of a really hard problem I couldn't solve it you know something with like coins and things of that nature that you might find in a puzzle book and this thing can solve that no problem and that was totally shocking because it doesn't seem like a system that just predicts one word at a time to generate text could also do problem solving and reasoning this kind of shocked everybody um it was not at all expected it was an emergent property of this model um another thing that was super shocking to most researchers like myself is the ability to quote under understand what was going on and I say understand in quotes because we don't want to anthropomorphize these systems right these are still machines there's no evidence that they're actually thinking or they're sentient or anything like that they're just really good at responding uh they're good at simulating they're amazing at simulating and uh but we never had systems that could basically take anything you asked it and respond sensibly to it as if it understood so that was amazing that was that you can and if you to this day you can ask chatgpt just about anything about any topic and it will respond sensibly or in the past these things would not respond sensibly at all to arbitrary topics there was a huge breakthrough um and then uh researchers like myself thought wow back in November 2022 I wonder if this thing can do stuff with law and it turns out again a lot of stuff is we will see uh but here's an example of it writing a patent application gpt4 for a made-up substance called axolotlite um and wow you know I teach patent law it can generate a first draft patent that could definitely pass at least first muster at the patent office and no technology could do it that well on an arbitrary input um so that's amazing I've gotten it to draft contracts I mean just about any legal document you can say it can generate a first draft maybe not what you want to turn in to a judge but definitely as good as you know in my opinion uh like an okay first-year Associate's first draft which was shocking because you saw what this is what it in the early 2022 this is what it could generate and this is what it can generate now so here I am just uh you see the instruction there right in motion to dismiss I just copy of just a random complaint I found on the internet from a real case plugged it in there and there it's Off to the Races it is filing what you know filling in the law filling in the content it is amazing again you want to verify you wouldn't want to turn this in directly to the judge but nor in my experience what I want to turn in the first draft of an associate directly to the judge right uh I want to make sure that everything was okay and we'll see it does make mistakes and we'll talk about that but it is kind of amazing in one year it went from this to die I mean this is what is shocking I think everybody here's another example uh remember the Apple question how is gpt4 the latest version handling the Apple question well let's see um so if I zoom in how many legs does an apple have be sure to explain your answer thoroughly I mean it's incredible and then I decided to push its reasoning and state of the world reasoning abilities you know asking it about if it can follow the Apple so I mean this is amazing in my life so basically I asked it you know if a person had an apple and then another person went in another room and there was chopping sound what would you assume happened and it's like oh well the person in the other room is eating the apple and then I have to based upon what you know if the first person had intended to eat apple and hadn't given permission how would they feel about the other person it's like they'd probably feel annoyed or frustrated that is amazing it has a theory of mind again in quotes because we don't want to anthropomorphize it it's able to follow things from different rooms this is mind-blowing and I've been studying this for 20 years so I think it's no hesitation to say this is one of the biggest breakthroughs in artificial intelligence in the last 20 years this is a big deal um so we're gonna find out our books and you will come out the other side in the next 15 units understanding all of these things so let's just take a deep dive into how it works um Okay so uh the first and most important thing or not the first thing we need to understand is the idea of word vectors so these are also called word embeddings you'll see them somewhere but the general idea there's a huge breakthrough is the fact that you could take the meaning of words and encode them mathematically so no one really thought you could do that back before this happened in 2012. so this was invented by Google again one of the bigger breakthroughs and also Stanford came up with this uh so the basic idea and we're going to see how this works is that you can encode the semantic or the meaning of a word like dog cat bird or house or any any word really as a list of numbers so think of it like each row in a spreadsheet is a different word and then each column uh is a list of numbers called a vector that's just math for a list of numbers but these are like a thousand balance right and then each column represents a different aspect of the word so it doesn't quite work like this but if you see the word dog going across right there are all these different things that you could classify uh any word along so imagine if you had categories for anything you could put in any category right so a dog is a pet it's an animal but it's not a building and it's not a planet right and imagine any possible thing 12 000 of those that kind of describe any word out there right you can you could characterize that so the reason it doesn't quite work like this in the real world of just sort of explaining is that there aren't these clean categories like pet and dog but um for our purposes we'll just pretend it's like that because it's actually close enough for that so if you look at the dog here it's all the numbers are from negative one to one so a dog scores up point nine right it's really high it's it's a pen right um it's got fur so it's point eight uh it's not a building so it's negative point nine it is an animal etcetera Etc and then look at cat a cat is also a pet it's also got fur it's also not a building it's also an animal a bird is a pet but it doesn't have fur it's got feathers okay you got the point so how does it come up with this right so basically it reads tests and learns word in context so after so you were to release this on the internet somewhere out there somebody is written kind of similar sentences about cats and dogs right so someone wrote I took my blank to the van I took my cat to the vet I took my dog to the bed so it learns to associate oh uh cat and dog are kind of similar right uh my I like to pet my blank I like to pet my cat I like to pet my dog so if you do it turns out if you do this a billion times it starts to learning the mathematical similarities between words that are used together or think of like king and queen right the blank sat on the throne the king sat on the throne the queen sat on the throne the king is royalty the queen is royalty so if you break it down along all the dimensions you can think of they'll start to have mathematical similarity so this was a giant breakthrough and it turns out that it actually worked it was kind of shocking to me because I was like there's no way that's going to work how can you possibly encode the meaning of words in math but it turns out it really works so here's an exam of like you see if we have pets on one access this is like in math class if you remember the x-axis and the y-axis and animals in the other it'll kind of group The Animals together but nudge the pets closer so like birds kind of in the middle because some birds are pets and some are not almost all cats and dogs are pets but cows are not really considered pets right for the most part I mean it's kind of amazing that it uh it doesn't but it's all because of the way we talk about things and write it down right so you'd be like the cow is a farm animal whereas the cat is a pet but it both learns that they're animals but it kind of nudges them mathematically away so this is the key idea is that it's really mind-blowing I mean I hope your mind's a blowing because mine mines are blown that you could encode word meaning using numbers automatically by just releasing these algorithms on the internet and seeing what words are used together and uh you know kind of writing that down so these These are called word vectors or word embeddings and this is a crucial breakthrough that's allowed GPT that's number one number two neural networks and deep learning sound um neural networks is kind of an old technology believe it or not from it was actually invented in the 40s but it wasn't really 1940s right I guess it can be 20 40s it doesn't happen yet um but uh it wasn't really perfected really until 2012. so that's where an area of deep learning came called Deep learning came about which is basically taking these neural networks and scaling them up so uh on lots of data so what's what is a neural network basically it's very Loosely inspired by the human brain but it's not really the way the human brain works so it's like a rough analogy and it's just a way of learning patterns from data and encoding them so it's part of machine learning so I'll just show you like visually it's often drawn like this um and what you see here are what are known as the weights or parameters and what we'll see more how this works in a second but basically what these kind of lines represent are how important each connection is to the next connection so you kind of can think of it like a series of committees uh imagine the lowest level of committee here says oh the cat sat on the what and then a bunch of people in the committee vote and one person's like banana and another person's like uh you know don't I and then someone's like well I'm kind of the expert in this area I think it's Matt and like no no be quiet be quiet you don't know what you're talking about it's banana and then they voted off the chain and then they picked banana and they find out no that was wrong go back and figure out what you did wrong and then you're like well we should have listened to that third person actually saying mad right next time we'll listen to that person better so that's a little bit the way it works and the way we'll see uh this in action in a second but the general idea is that this is a super flexible technique for encoding patterns basically you can code any pattern it's been mathematically proven that any pattern can be encoded in a neural network so instead of just a couple of these imagine 175 billion of these okay so that's what GPT has not you know however many I have up here seven or whatever okay so now now we're starting to put stuff together okay so we take the neural network that we just learned which learns patterns from data we know about word vectors right which are taking words and representing them mathematically between what's around them now a lot comes GPT 3 GPT it says let's just run this whole thing on the whole internet and like two million books and see what happens right and no one has done that before because it was really hard to do it takes a lot of computational power and what they did was they basically praying they said to the neural network I want you to read every sentence and just try and predict the next word and then if you happen to predict the next word that's good we'll give you a thumbs up but if you pick the wrong word right we're going to go back and find out who told you to predict the wrong word and we're kind of devote that person and promote the person who was saying predict the right word right so that's called I don't want to use a jargon but that's the the learning aspect of it right you're constantly looking forward trying to predict see what you predicted and then uh you kind of demote the people who sent you down the wrong path and promote the people who sent you down the right path and it turns out if you do that a billion times you get uh this amazing problem solving machine so let's see an example here so if you see this little green weight here let's imagine that that weight the plus is saying uh that you take the cat's hat on the blank but send it through the thing and the top Arrow saying send it to the top path to Banana I think that's right and then the one on the bottom is like no it really should be mad but it's negative three so no one's really listening to it right so let's watch this little example here it goes through and then comes out with banana right and then uh it detects it it gave the wrong answer because it knows what it should be because it has the actual sentence the cat's hit on the mat it's just blanked it out right so then it Compares what it predicted to what it knows the text should be it's like oh we got it wrong we predict a banana let's go back and crack that let's demote that guy and promote that guy right and now next time we'll send it down a different path because we want to send it down the path of the one that was telling us Matt and now let's try it again with these new weights or parameters and then see what happens okay this time it's going down that path and then comes out and wow it came up with the right answer so that is what's known as training right you're training it it ran it's initially random and then doing that billions of times you kind of nudge the up and down 170 billion parameters and you teach it the English language and reasoning I know it's amazing um and then uh it just wouldn't have been possible though without uh the mathematics of it right and this is where the Transformer comes in so this was again Google's uh invention here and this takes advantage of the fact that we can represent words as vectors as numbers so what do you do you ever think you just said like what do you do if there's a word like rain or just two meanings Verge or the construction equipment what do you do with that how do you represent that mathematically like which is it is it one or the other so it turns out it's kind of smushed together as both and that's not really so helpful so what Google figured out was hey why don't we take what's being asked uh you know and the kind of sentence and see if we can kind of nudge it towards one meaning or the other based upon the context so here's a good example um we've got so here's the context unaware meaning of the word crane as a word Vector meaning that if we see bird on this access a machine the you know what you get just by reading the internet somewhere in the middle right so it doesn't really know what are you asking the bird but then Along Came the Transformer um and it said well why don't we look at what's being asked so look at this sentence here in the Serene Wetland a large-legged bird rated through the shallow water and it's like oh well let's you can mathematically figure out that we're asking for the bird version of it not the crane version by nudging it mathematically towards space so I'll show you how this worked like it literally nudges it mathematically that way like physically not quite physically but like that way so when you put it through the Transformer it nudges uh mathematically so we now have a version of crane which is closer to bird and that encodes the meaning the crane version of bird the the vector that comes out is like crane nudge towards bird and now it can like understand oh you're talking about the bird but imagine if instead you would ask this workers around the construction site operating many machines the tallest of which was that and it was like oh I see machine construction site I'm going to mathematically nudge it based upon that context uh we're supposed to nudge it that way let's pretend it nudged it that way um it was supposed to nudge it up there right towards machine so it's really amazing and the general idea is that uh these Transformers uh which gbt is this architecture that takes into context has 96 layers and the new one might have worn and as you're working your way through at the Split Second it nudges it in such a way along every word that it mathematically encodes the context of what's being passed plus what it said so far so GPT can look back at what it said so far and be like Oh I'm telling a story let's keep going with that and then it looks at what was asked and it mathematically encodes all that in 12 000 dimensions and that's how meaning is encoded like it's kind of it's shocking like I can't believe it um that it actually works nobody really thought it could and then uh two more three more things and then we'll answer questions um the other breakthrough was this idea that it's called transfer learning so open AI gets credit they said let's just unleash this thing on the whole internet and two million bucks and just scale it up and let's just see what happens maybe maybe if you make it large enough this General this thing can start doing other tasks we never trained it for so that was never the case with other AI systems before usually it was kind of a narrow task like driving a car or classifying cat images or you know opening detecting faces it was never like we can just do anything but they're like well let's just train it on everything let's convert the entire you know human output textual output compress it in a neural network of 175 billion parameters and there's try lots of things on it um and it turned out it worked you can take this General thing and then apply it to like almost any task poetry coding it was kind of shocking because that never worked but that's called transfer learning rather than creating a AI system that did a very narrow thing like driving a car which itself was kind of amazing it says like let's just make this General thing and see if it can do like anything um and it and it can uh okay so what's really fascinating I think this is important to understand the capability in 2022 in January 2022 was actually lurking in GPT three right that was the model that said how many legs does an apple have six right it seemed really dumb but the researchers at open EI were like we believe this capabilities in there we just need to pull it out by clever engineering so what they did was they did something called instruction fine tuning and they basically said we're going to take this General model that's been trained on the internet all that already knows language and we're going to give it thousands of examples of what a good question is and a good answer is that anybody could ask so you know uh write me a computer program to make a DOT bounce around a screen and then somebody programmed that up and gave it to gpt4 write me a press release uh what would that look like I'm sorry I gave it to gpg3 right whatever it write a write a limerick give it so they did 15 000 of these right and then it turned out it got really good by seeing these examples so that was one way of kind of nudging this kind of dumb model into chat TPT so that was one huge breakthrough no one had really thought to do that before but it required thousands of humans to do this to so open AI to spend you know probably millions of dollars and then they came up with kind of an even smarter idea which is kind of crazy so they said okay now that we've trained the model to be like pretty good at writing poems and stuff like that let's get a bunch of people in the room and have it produce five outputs like five poems and have the humans rank the best poem to the worst poem one to five right and then we'll take that score and we'll stick that in another AI system called a reward model so let's here's the best like you're solving a problem the first one did a bad job solving a problem the bottom three did okay and the fifth one did a good job stick that in the reward model so let's teach this other AI system what good output looks like rather than relying on humans so they trade up this other AI system and then this other AI system went and went crazy on gbd3 and produced chat GPT by going a billion times good output bad apple good outfit and that's where we are today that's where how we arrive at chat gbt being so good and so responsive and then they basically did a much better version of this with gpt4 which is the latest version okay so uh this thing is not perfect it definitely makes mistakes and these are called hallucinations I I do think there's a path forward for reducing that you definitely have to double check uh some of the outputs but this thing is amazing it's the biggest thing I the biggest breakthrough I've seen in my 20 years of studying this so as you can tell I mean for good if we're bad I think it's going to be disruptive for society and we need to get ahead of it I don't think it's going to be I don't subscribe to the AI is going to take over the world and that I think that's kind of nonsense but I think it is going to be disruptive for jobs and all sorts of ways but also good I think it's going to accelerate research and create new knowledge and things like that okay okay so let me pause there and just take questions because I've thrown a lot out you your heads are probably going to explode from all that stuff but hopefully you have a gist of at least how GPT works yeah

Channel: Harry Surden

Views: 8,712

Keywords:

Id: IMAhwv5dn8E

Length: 39min 2sec (2342 seconds)

Published: Sat Apr 22 2023