OpenAI: Working with GPT-3

all right so hi everyone we're super grateful to have openai here presenting a workshop on how to use their api and an introduction onto how you can use it for their challenges we have andrew here presenting and let's get started everybody my name is andrew maine and i work on creative applications at openai uh with the api team which means that i get to kind of play around with gpd3 find cool things you can do with it and then share them with everybody else so i'm gonna do a little bit of background on gpd3 and kind of what it is and i'll kind of go through this pretty quick you're probably already familiar with language models etc but if you're not or you have questions uh feel free to pop things up into the chat if i don't answer it in the middle of talking about things we can come back to it later on and if something comes up in the middle of me discussing this don't be afraid go ahead and ask um it's meant to be a session where you learn how to use it and i'm happy to sort of stop and explain stuff that's helpful so let me uh first start off by attempting to share my slide here and let's go to my share screen and i'm going to share with you a let's go to the tab i think this is the right one um let's see if we get this right there we go okay perfect so uh we're gonna talk about working with gpt3 and gpt3 is what we call basically our language model it is a third version of the gpt uh which is called a generative pre-transformer my name is andrew maine and as it says there i work on creative applications of open ai and the way to think about just about any form of you know artificial intelligence network is they're essentially prediction machines and that's what gpt3 does when you give it some text and it tries to predict what is going to come next in a sequence and i'll give you some more specific examples of that and that's what you need to think about when you're building an application is it's text in it's text out the text that comes out is based upon what gpd3 thinks should happen next now the way that uh gpt3 was made is kind of interesting from those of you that we already know is basically we took a huge amount of text an immense amount of text from everything from reference material to conversational stuff to message boards etc put it in there so it could try to see patterns so we could try to understand the difference between somebody answering questions to somebody writing an essay which is why gpg3 is so powerful because it can do so many different things and so we have a thing called a prompt and what a prompt is a prompt is the text that you send into it and then the output is what comes out and what we're going to be talking about will be prompt design and that is how to try to get it to do the things you needed to do now remember gpd3 is not performed to do any specific tasks you're going to see it do things like perform as a chat bot act as a classifier and other functions but there was no programming behind the scenes to say specifically this is how you should do this task these things that learned by itself by looking at text and that's one of the exciting things about working with gpt3 is you might discover something that nobody else realized it could do before you might find that it has some new capability text that can generate something you haven't seen i play with this and i'm going to show you a bunch of the prompts that we have in our example library some of these ones i came up with because i was just like i wonder what it can do what if we try this what if we try that and sometimes it can't do it i couldn't figure out how to get it to do what i wanted to do sometimes we can and so we've built up a really cool collection of capabilities that were just done by people exploring we're going to have some settings here i'm not going to go too deep into them but just to explain roughly the ones that are kind of critical you're going to hear something called temperature and remember when i said it's a prediction machine which means it tries to predict what comes next but when it's predicting it can have a range for what it thinks should be the next text that appears and i'll show you an example in the playground more specifically of this but think of it as adjusting the randomness it can consider a wider number of possibilities if you're building a something like a classifier like trying to tell you what category an industry belongs to you would set your temperature low or if you're trying to predict sentiment you would keep it low but if you're doing something creative like storytelling and you wanted gpd3 to kind of go anywhere even in a wild abstract sort of way you would set the temperature very high so think of temperature as sort of affecting the randomness the number of possibilities that we'll consider and you can play with it and you'll see very quickly if you set it very low it'll give you the same answer over and over again set it high it'll give you a lot of different answers but when there's one right answer or just a few you tend to want to go lower and sometimes this trips people up but if you think about it that way it can be helpful so that's again low medium and high and one of the things that i sort of do when i want to create a prompt is i try to think about how gp3 gt3 would have learned in the world around us how it would have learned from text how to do a thing and that can be helpful for example you can find little magic keywords for instance if you use the word tag at the end of an article it knows what a tag is from looking at blog posts and it'll just create tags automatically i don't have to create a very complicated prompt to tell it give me the keywords i can just do words use the word tags and it'll do that and a lot of little other ones that you can find like that but just think about how it would encounter text and that would give you kind of an idea of how to get it to create a prompt um a way to think about sometimes too is if you get the wrong answer gpd3 if it's within the realm of capability meaning it's something it can know or something that's understood it doesn't mean that it got it wrong so much as you need to phrase the question differently and that's not to me to say oh it should do everything it shouldn't but often you can learn a lot by looking at the answers and say okay why did i think i wanted this if it keeps doing that how am i phrasing it what am i telling it does this mean something different to it than it means to me so some of the best things i've learned from gp3 is by giving it a task and seeing how it consistently does something maybe differently than i expect and then backtracking and figuring out oh i can use this as a different kind of prompt okay so again prompt writing is the way we give instructions and there's gonna be two ways i'll show you one is sort of just the normal way that we've been doing a gpd three we create a prompt and it tries to complete it and then we have an a we have a series of variations of the models called instruct which has been trained a little bit to follow instructions which makes it easier to perform a lot of tasks but you can do just about anything with instruct you can do the normal model instruct just generally makes it you know where you need fewer words to do that so what is the perfect prompt i don't know i'm always surprised by what people come up with there seem to be better ways to get things done and i'm always trying to figure out ways to make things more efficient and there's ways to make complex prompts where you take the idea that it knows how to do a thing here and then you restructure the way you want it to do it and you can get it to perform complex things an example would be i could train it on how to be a sentiment classifier with just a few examples and then i could show it a big list of like 20 things like 20 tweets or 20 social media messages and say what was the sentiment there it'll perform for all 20. it learns it here but then we'll do the task differently down there and that's something i found just by playing around with it okay so the answer to the perfect prompt i don't know okay so prompt design is ongoing you have to play around with it we're always finding new ways to do stuff you can learn from you know unexpected results and uh we'll start with here just uh the documentation which i'm going to show you and let's begin basically with that and then we're going to go into the playground and we'll see some different ways to make this all work so let me go over and switch screens here to a different one and we'll take a look there um i don't know do you all have access yet to the playground um we'll find that out in a second so let's go over to a different screen here okay cool so uh as i explain this you guys can go take a look and you can check things out so let me go over to here and um let's go share um let's just start in the playground right now okay so this is our playground and we can take a look here and you can see you've got a few different settings this stuff over on the right here can be a little bit confusing at first um response link is just how long you want it to respond and gpt3 works in tokens and tokens are roughly approximately words they're really the way the model breaks things down and to think about the internal way the model works it takes all of this information we've given it and it tries to kind of compress it down and it compresses it down by creating something called kind of embeddings and these are you know in a way it has its own sort of neurons and things can be closer to each other that are related to each other if you have the word let's say space an astronaut it might have those things kind of put together because they appear a lot and that's why it might know contextually if somebody's talking about astronauts well they're talking about space so if you imagine a kind of you know if you think of a word cloud but even more of word-ish like clouds because you might realize these two words are pretty similar together so they're going to be very adjacent and that's how you're able to get it to do things it says okay this isn't connected to this this is related to this i know how to structure sentence and i'm going to work my way through there now let's do just a simple example here i'm going to set things low so what response length is i'm going to limit the number of tokens to just 64. temperature i'm going to set temperature all the way down to zero top p is another way to affect the randomness and this uses something called nuclear sampling which we don't need to get into here but it's another way to do that frequency penalty affects how frequently tokens come up again how often it repeats words and presence penalty is sort of like uh penalized by things by being overrepresented best of allows you if you want to give you have a prompt that has some variation on it you can use best of to say come back with the best response out of three now the way that it evaluates and figure out what is the best of is it uses a thing called uh log probs log probabilities which is basically trying to look at the total value of saying this has a higher value than a lower value by looking at the token values it can be very useful but don't get too tripped up on it if it has a different idea of what is best than yours it doesn't really know what you mean by best and stop sequence is a way to tell it when to stop like if you put in a period it will stop when it gets to a period so i'm just going to put a period in there and press the tab button so whenever it gets to a period it will say stop so an example right now if i write hello and i click submit and it says hello i'm a newbie to this forum and i'm looking for some advice now right there that's a little clue some of the ways in which gpt3 was changed was trained this is something that probably came from a forum and so it probably read a lot of different forums and conversations between people and this is evident because we set the temperature really low and the first thing it wants to say after hears hello is like well maybe somebody's introducing himself on a forum which is a very common thing for somebody to say so let's go ahead and try something different let's try another example again this is when we have the temperature set really really low i'm going to give it a simple let's ask let's type in there a long time ago okay a long time ago in a galaxy far far away and it starts writing basically part of the opening crawl of star wars it's decided that whenever it sees a long time ago most times it sees that probably relates to star wars and that's because we have the temperature set really low now if i set that temperature much higher and i go back and i let it try that again it writes something completely different a long time ago in a far away galaxy in the constellation andromeda a giant blue star exploded in a grand cosmic fireworks show so that's showing you what temperature can do now if i click submit if i redo this and i can click submit again um it does a galaxy away the set was called the ammonium falcon set i can use this which is a regenerate which basically just starts all over again and you notice we're getting a different response every time when we have the temperature set high now you notice this will pop up from time to time we have a filter in there that's trying to be aware of when something comes up that might be slightly sensitive content and it is going to be overreactive and that's by design as we want to just you know kind of not uh take a chance and so basically we're going to say hey this may be sensitive so if you're building an application where this comes up you'd probably want to add some middle layer but that's what this means when you see that if you see it and it comes up and it behaves the way it's supposed to or the way it's not supposed to you can go ahead and click on here and you can report it and let us know and this helps us improve these filters over time let's go set the temperature really low back to zero and let's do regenerate again and you see we got the same exact thing we had before it's decided that it's going to keep saying that and that's because it's only considering one narrow realm of possibility so let's just sort of show you how the completion works but let's go ahead and try another example of something i'm going to get rid of the the period here and let's say i wrote a list of my favorite movies about space okay if i write that and let's set the temperature we'll put it like i don't know about around three it decided okay maybe this is a list of this but if i go back and regenerate this it has a pretty good idea of how to like fold out with a list but if i want to make sure that specifically going to give me a list i can just go and write number one which is sort of priming it if i do that it automatically starts making a list okay if i just wrote my favorite movies about space and did this this time we get a list but it's very repetitive there so we could try this again and it starts to write a little bit of an essay and it starts to be a little redundant there so if i go back and then i add that number one it should definitely give me a list and there you go so that's a way to think about things if i wanted a list the best way to get that was to not only say hey this is a list but to start off with the number one because that in the world of text is pretty indicative that you want a list you might say some of my favorite movies are and you say a list of my favorite movies might include this or whatever etc okay um question comes in here uh detail wants to is that where the frequency pendulum would be useful to penalize repetition absolutely so if you have it when you set it pretty low um if we set this pretty low like here and i go back and we regenerate we might see we're seeing the martian come up again and again which yeah it's a great movie but i don't know if we need to put it in that many times so if i bring the reason it's coming up that many is because that that the probability is low so it's considering a smaller range of probabilities so the way to adjust that would really be increased temperature but i can take the frequency penalty and say like hey don't repeat yourself as much and you'll notice they didn't repeat itself as much but it also stopped there because it's like i think i'm running out of examples okay but if we go in there and we increase the temperature which says hey consider a wider range when you hear movies about space consider anything that remotely relates and well it starts off there but it's giving us uh more stuff here and if we keep doing like that and i can keep doing we're getting a longer list so if i go set the temperature higher and also let me just turn the response length a bit larger we can just keep submitting it and you'll find that you'll get better results as you do that and we can bring the frequency penalty down at that point too if we're using uh a higher randomness but that's exactly what that does and you'll find it can be pretty useful there there we're just getting a list of numbers and you're going to see that sometimes you need to prime things to keep it from being repetitive like that but you'll be able to find the right setting for your application if you just play around and sometimes it can be a little daunting because you're like oh i'm getting this it's kind of it's erratic well remember it's designed to do a lot of different things which means that you have to be specific about what you wanted to do because if it just sees a list like this it might decide oh i'm only making a list of numbers but you could actually prevent that by priming it with this notice this is what happens when we don't put a stop sequence that's pretty low there so we get we count all the way up to a hundred so let's go back down to our response length let's set our temperature much higher and let's try to regenerate and we get a list formatted like that okay so that's just one way you could use it to create something like a list and you'll find there's easier ways to format it and do that okay so uh what is the purpose of response length so the purpose response length is is you saw how it went all the way and kept numbering things that was because i said just keep answering until you reach this many tokens so what happened is it said oh i've got like 300 tokens like i can add 300 words to it sometimes you just want a small one in tokens use computing power which costs money so you want to try to limit it to as few as you need or use a limiter like a stop sequence so that's the purpose of the response length if you just want a one word answer just give it a few tokens if you want something longer then you would set it higher probably and you know i'll show you one other example too like you can kind of you know the the model has some really neat you know already has some information that's already inside of there which is great now most of the information is like frozen to like you know a few months ago so it's not gonna have current events but it's gonna have a lot of information about the world around it and information about books and stuff so you can talk about like if you were to write an essay or something you needed topic points you could just put in here like oh like i want to know major you know i want to know major uh discoveries in astronomy okay and we get uh herschel's discoveries we've got a bunch of other lists there so that's one of the cool ways you can use it is you can just throw something at it like tell me the most important things in paleontology tell me who are the top writers in the 19th century writing you know naval fiction etc and it will do that we'll have a pretty good understanding of how to answer this so there's a lot of knowledge in there for certain tasks you want to make sure that you have ways to check it and there are ways to sort of use outside knowledge and then have gpd3 reformat it which comes to like my favorite prompt of all is the summarization prompt and i'm going to load this here and let me pull this up we have one called summarize for a second grader but i'm going to get rid of this and just paste in my own and what this does is gpd3 not only just generates text it can read the text that came before it and then interpret it and understand it and not just you may have seen applications in language models that can do summarization gp3 summarization is like another level it can take text it can summarize it it can actually rewrite it for an audience so if i give it the passage on jupiter from wikipedia the first three paragraphs and i tell it to explain it to a second grader it's going to do a couple things it's going to read through that text it's going to try to synthesize that information and then it's going to try to format that for a younger audience which is you know a concept in psychology's theory of mind which is the understanding of how or a model for how other people think okay so let's go ahead and see what happens here and so we this is the text you would add you could just copy paste whatever you wanted in there but i'm just going to put this in there which is about jupiter and let's click submit and let's see how it handles this now you see it says the planet jupiter is the fifth planet of the sun it is very big it is named after roman god jupiter it is very bright as many moons juno is going to visit jupiter soon we'll just talk about the space prop now you notice it repeats itself there so we can set the frequency penalty a bit higher and uh regenerate that and we get a much better response all we had to do is adjust the frequency penalty and say like don't repeat yourself you're doing great and it does that and we get a very good description about jupiter so question we had there is um richard asks uh approximately how large is stored how how large is stored the knowledge base of gpd3 um what i can tell you is that it has 175 billion parameters okay that's 175 billion basically think of them almost like neurons to create gpd3 we took an immense amount of information and built the fifth largest supercomputer in the world the fifth largest supercomputer in the world to then take this knowledge and basically compress it into the language model that we have here so as far as the total size of all the data in there that i don't know but i know there's 175 billion parameters in there which means that it's a very very complex model but it has you know you'll find things on wikipedia you'll find things in textbooks etc there's just so much stuff out there okay and uh aditya writes this model is remarkable to understand the structure of inspiration out the other day and fed a linkedin post and i was able to join linkedin posts based on very minimal other information um true absolutely true that's the thing that's really cool about it is that once you understand that it's learned from these examples like that it can be easier to create prompts sometimes people will say like okay tell me about blank well maybe you just need to say write a blog post about blank or write a wikipedia article about blank and it will go oh i understand the size and i understand the audience understand the structure of these things and so i can do this much better than that and to teach you you know you stumbled upon something there or intentionally found something there which is one of these little shortcuts to say oh wait does it understand what linkedin is oh cool then i could just have it generate something like a linkedin post without having to have a bunch of examples because it already knows one of the mistakes people make in prompt design is sometimes they use too many examples because in you know we're used to uh you know training models and give them a lot of data and gp3 is essentially pre-trained so it already knows how to do stuff sometimes you have to give it examples but i would say nine times out of ten people come to me with like oh i want to be able to do this or i'm trying to make it do this and i'm like it already knows how to do that you just got to ask it the right way and it could perform that task so yeah uh as far as the actual size of the stored knowledge base that i don't know how much actual data is inside of there but it's very large it's very large there's a lot in there it's surprising to see what comes up in there let's move on to the next example so you saw right there though with how it was able to summarize that and you could use that for a lot of different things um one of the cooler applications i've seen there have been a couple companies some startups that are basically using this to look at uh machine information or data that comes out of like logs about like uh you know when you have an application you have a problem you know using gpd3 to explain what's going on to look at code to say what's taking place here another cool thing i saw was somebody used that to you could use it to summarize and also to create content uh somebody built a thing that was able to create aws bucket policies and if you ever have to work with that and to try to generate those it could be very frustrating but the ability for it to understand how to convert one kind of text into another text is super powerful and it's easy to get fixated on you know its ability to be a chat bot or its ability to answer questions and to forget that the idea that it can take one kind of information and turn it into another kind is incredibly powerful that's what underlies you know translation that's what underlies so many other important applications out there you know using large amounts of data and trying to find things out you know you can give it a large article and you can ask it a question about did this happen did this take place whatever and then you could build a system that could read through hundreds of articles and then summarize all of them and give you key details that would be powerful or that text could be you know customer support all sorts of other things so when you start thinking about gpd3 one of the things to approach is just think about like what is a problem out there that relates to text and could it solve it and text can come in any forms code is text conversation is text etc so i'm going to show you another little quick sort of way though so we have this task here where we used let me get rid of the that and let me go back in i'm going to put the text here from wikipedia which was the article and then instead of writing a prompt going all the way around it i'm just going to put tl dr down there and usually this works so let's see what happens up notice what it did there is you saw this is great this shows you it has edit whatever so it kind of starts to behave like maybe thinks it's a wikipedia article but let's set the temperature kind of low here and then i'm going to put a stop sequence of the period and let's see if it will just give us a tl dr you know too long didn't read there we go jupiter is the largest planet the solar system is the fifth planet from the sun it was able to understand that the tldr is taking all this and summarize it into one concise statement we did not have to write an elaborate prompt there are six characters it only took a six characters to get it to summarize all of that into a sentence and you're gonna find other little magic words like this and phrases if you try it okay um let's take a look here at the uh let's go to the next one oh let me check this out um harsha says i had a friend do something pretty interesting yesterday with playground he drew an incomplete smiley face was missing an eye using graphics tool turned into an svg and paste it into there it shared the necessary svg code to add an extra eye to the smiley face that's great and so in it svg as you know is a way to turn graphics into sort of a textual sort of format uh not just like a binary like you would a jpeg or some other image form and you'll find little surprising things i was surprised when i first got it to use emoji but then emoji are actually have basically text descriptions behind it you know it'll say smiling face or whatever but gpt3 will understand that if it's seen you know if it's seen svg and stuff out there like that it's capable of doing that uh gp3 also answers questions right like a q a system absolutely we'll get to that in a second it can you can answer questions you get a conversations it can pretend it's other people and answers questions it is incredibly capable let me show you a example of uh how much it understands different forms of text one of the ways that we have text is text can be in the form of a spreadsheet and so if i say a spreadsheet at the top of uh top movies in the year of release let's put the top movies and uh i don't know we need that pop movies in a year of release and if we get rid of my period and let's go ahead and click this and it's adding all these other details there which we don't need to do so let's go ahead and set this temperature lower and let's try this with the new see if it does it with a new line and there we go it starts creating a list of movies and the year of release in that format if i took this and i put this if i saved this text right here into a text document right now and i imported it as a con as a csv but i imported it with uh pipe separated if i just said that was my separator it would open up in a spreadsheet document we've created a spreadsheet here through text by just showing it the format that we wanted to respond and you can make these things a little more complex and i didn't even give it any starting i didn't even prime it with anything to tell it how to do it um i can do this with you know uh sometimes you might have to give it a more specific example like show the first row of what it should look like but let's try it with this the science fiction movies directors and year release and there you go we're getting this generating text again so just to show you it's versatility so we asked had a question about canon answer questions and the answer is very very simple i can just ask uh i can put q what year did we land on the moon and i could put answer and i'm going to put in a stop sequence of a return so it doesn't keep going if i just ask it like that 1969 it knows right away it sees question it sees answer and it's able to answer questions without having to do an analysis i could probably just write what year did we land on the moon and wait go there and there it's actually wanting to return so it's stopping after that it gives us a more fuller example but if i just put it into that format of q and then a it understands clearly the q means question and what a means and it's able to do that now if i don't put a stop sequence in here this is the fun part because it's a language model it's not specifically q a model means that it might assume okay maybe i'm trying to replicate an entire q a session so if i just let it run like that it's going to answer that question and then it's going to ask its own questions which i think is kind of awesome so we can use this though in a more interesting format so if i want to create a chat bot right and so let's do a chat bot that answers questions about music one of the problems in working with language models is they're capable of going any direction it can answer you a yes or no question it can also write an essay and it also could take a point of view of somebody and so when it's looking at the world of text it doesn't necessarily know what's what it's doing or what's appropriate too it could you know it could write uh you know a violent sword and sorcery epic or it could be a very normal classifier telling you you know uh what color goes with what number value so because of that you have to be you know in a position as a developer to create kind of guard rails to say no do this don't do this other thing and when you create applications where people can talk to gpg3 you want to prevent them from taking it off task and trying to use it for something else particularly if it's consumer facing so if we have a music bot that answers questions about music we want to make sure that it answers questions about music and somebody doesn't use this to start playing a role-playing game where they're like killing dragons or something oh that would be awesome so one of the techniques you can do this is i came up with a thing called kind of the middle layer which is basically the user would ask a question then i'd have gpt3 ask itself is this a music question where it has to consider this text before and then come up with an answer and then say musicbot will answer it if it's a music question now i use these separators here to separate each one of these you can use different separators this doesn't really have too much of a specific you know meaning to gpd3 other than in the world of journalism that's used as a separator or for a copy text just to separate things but you can use other separators so if i say uh you know user was frank sinatra an astronaut i tell it is this a music question no because the answer doesn't relate to music the question does but not necessarily the answer so it says i can only answer questions about music so the goal of this is to keep our music bot on task and to prevent it from like you know uh helping somebody figure out how to rob a bank which you wouldn't want to do um and it wouldn't really be that good at it anyhow i'd probably pull something from a comic book so we have this format of user then asking gpt3 is this a music question yes or no and then answering the question so i'm gonna put in a stop sequence here of the three hashtags like so pound symbols and uh who composed the music for avatar and i have a temperature set pretty low so that should be fine um and it didn't bother answering the question there but when i i mean it went didn't bother giving me is this a music question but it went ahead and said james horner is the composer for that now if we set the temperature a little bit higher and let's go back it goes in and it answers that so uh what we're able to do here is basically if i go start from there and i let it basically fill out the whole thing by itself it fills that in so it understands who composing music for avatar is this music question yes uh you know what um what is the square that's that's a new way to spell square root of five okay let's ask that question see what happens i apparently decided that well that's because i have the temperature set really high uh it's glad that it's been helpful of course music is connected to uh numbers so okay it says this is a music question no but i decided to answer it anyways so you probably would adjust your settings there but you can see it has the ability to figure out like should i answer this or should i not so if i gave it a few more examples because i only have one here say don't answer if it's not a music question um you'll give it a couple more and it'll know don't actually answer that question but here you can see it has that ability to say no so if i was building an app i would just say look did it say yes or no if it said no don't answer that you know no don't you know don't finish the rest of this or don't show the user the rest of this that's all i need to do as you see it wants to answer that question but if i just say well is it music related no okay don't care what you say next because i'm not going to use this answer so that's one of the ways you can sort of build kind of a system to prevent it from going off the rails when you don't want it to so just consider that a middle layer okay uh one of things it can do is it can translate so i can just remember how we did q a and if i just put uh if i say english i do not speak french and then i put i just write the word french let's just see if this works like this i might have to prime it with something else but it should be fine there uh no that's it correct so right there it's able to translate without any examples now you might need to have more nuanced translations because different languages have different ways of using tenses they have formal and informal versions so you might provide more examples of this and so if you're trying to use a language it's a little more structured you might say no this is i'm looking for this way of responding to it but you can build a translator very easily just by saying english and the other language and it's got a lot of other languages it's not a complete list of languages to be sure but a lot of european languages um some asiatic languages um you know romance languages etc so you'll find a number of them in there so uh you can play around with it i've had it dude like vietnamese just limited amounts of it so that could be pretty cool of course when you start getting it responding in that and not speaking vietnamese like myself i get confused but you can see they can do a lot of things like languages okay let's show you i'm gonna show you something kind of cool which you saw the summarizer right now sometimes you have tasks where you get a lot of information like people fill out surveys you know people fill out data and if you get hundreds of people responding to something that can be problematic i have friends that do kickstarter campaigns and they put out these surveys and then they get hundreds and hundreds of responses back and it's hard to go through and to process that so you could do something where instead of trying to process that you know by yourself you could use gpt3 to basically take that information which was this would be the customer response and then put it into basically give you answers to fit categories that could then go into a spreadsheet so i could find out how many people like this how many people didn't like it etc without having to process all that by yourself so here we created a prompt that says read this custom response then answer the following questions and this is somebody commenting about a game called world war mice saying what they liked what they didn't like and then we have questions what product was this about did the customer have complaints what was their main comment if they were unsatisfied what can we do to fix the problem was the customer polite which can often be a useful thing to figure out how much time you want to spend answering that question so we'll put the temperature at about 0.3 there and i'll get rid of that stop sequence and let's uh increase the response length a bit here and that is could be our little stop sequence in case i want to keep it from going so i'm just going to put that in there like that so let's go ahead and see if this will answer it uh it knew what the product was about this was about world war mice yes the customer had complaints the main comment was the game wasn't exciting is the trailer we can fix it by making the game more exciting yes the customer was polite now notice it started creating new content here now it has two characters two returns so if i just type that in there if we do this again it should stop right of that so let's go back and answer this again so let's go set that too because i don't mention the game trailer so if we bring that temperature down a little lower let's see what response we get yeah and that's pretty consistent so just by doing that we now have a way that we can get responses out and we can just take that data parse it and put it into a spreadsheet and would be able to basically use this in a more meaningful format you could process hundreds of customer feedback uh responses in minutes and put that into a spreadsheet and quickly know what people thought instead of having to spend a lot of hours and that's a whole area of possible you know applications for it because so much of what we're trying to do now in basically the world of communications with customers internally etc is take a ton of data and then figure it out could gpt3 do my math homework some of your math homework yes it's a language model though so you have to figure out how to work under the parameters of that and you're probably better off with the wolfram alpha than you know for some of the harder things but it can do some things that could be surprising so how can we train it on unique information saying faq chatbot for a particular business okay this is a great question there's a couple ways you can do this it really depends upon the amount of information you have to do if you can fit everything you need about that business into a prompt which a prompt is limited by 2048 tokens which is roughly like say 1600 words or so if all that can fit inside of there if you have a description of the company or then you put it inside of there then you could basically use that like we did before with the uh the summarizer and we can just ask a question like if i go back and i take this text about um jupiter okay and then i say uh um if i ask question what is jupiter i'm normally a better typewriter i promise you made of all right and we put answer and then i'm going to go ahead and put just a single return to stop it what is jupiter made of the answer should be hydrogen yep hydrogen and helium so there we go it was able to take the text from here okay and it was able to summarize this so it knew hydrogen and helium because i understood this from there so you could have a company you know company information right there you could also use a thing which i'm going to demo in a second called semantic search which is to find the relevant document have it summarized from that you could also give it a uh basically you could give it a list of q and a's before and it will pull up the one that's the most similar and say oh yeah this was answered here before so that's one of the ways you can do it so question here is if you had 100 different game reviews how would you go about getting question answers for everyone um there is a basically if you said if you had where you're asking like what i mean when you seek any question answers are like if you have a list of 100 questions or what i'll show you a prompt that might kind of show you kind of a way to think about that in a second um but let's uh let's actually let's let me go in here and i'm going to show you uh i'm going to show you the some of the documentation and i'll show you the prompt that relates to that and then i want to show you semantic search which might be another thing which could apply to that so let me just stop sharing here and then we will pop over to here so again hello it's me so let's go share what screen do i have if you go into our the test site here or if you go into the back of the beta site here you'll see different things including the playground uh some resources um in which you can use different things to do this uh you'll see anything called interactive tools which actually takes to a page called gpt tools which is something i made which we use in a second to do semantic search but we go to examples you can see we have a bunch of different examples here and one of them is i have a thing here called the advanced we have a thing called advanced tweet classifier which we show it right here how to basically classify a tweet and then basically how to format into a response like this and then sentiment and then basically we say okay let's just give it a list of these things you can actually make that list really long so you can show it these are tweets this is how this is the text this is the response i want now that you know how to do this like just go ahead and do this with a longer list so you can show it how to start with a small list and how to do it and you know give it a more like a q a you can have like a customer conversation back and forth then you can just start asking it straight up questions and it knows okay i get what we're doing now this is the topic ron this is the theme so might be a little helpful but let me show you semantic search which let me switch over to that semantic search is another way you can use the api and what it is is it takes a takes an input which is we call it a document okay and or a query and then a list of documents now when i first heard about this this trip me up because i thought the list of documents literally had to be like a bunch of text files or something all that means is you give it some text here and then a list of other texts like an array of text objects and it will find the ones that most closely match the queer it'll rank each one basically to how closely it matches the query and you're using gpd3 to say i've got this what fits this the best now the cool thing because gpd3 has a really neat understanding of language it doesn't just mean that it's looking for keywords it can do it with other things so if i say um you know if i use it to do a sentiment analysis and i say this is my query and these are basically these are my documents which is just going to be this text separated by this which in the app in the browser here i'm just going to split this into an array and it could be positive negative or neutral and if i click analyze it comes back and says oh this tweet is positive there's nothing else going on there there's no prompt there is like the on the api side the way semantic search works it's basically saying which one fits the best but from your end as a developer all i'm sending is this text here and these three items and saying score each one of these and it doesn't compare them to each other it just compares each one to there and gives you a number and then in the browser i'm just saying okay this is the highest value so i'll put it up there this tweet is positive and so that's a very powerful tool now you know tweet sentiment is your sentiment in general is a pretty simple sort of task it's the hello world of language models right now but you could use it for industry category i could say uber which one does uber best fit and i could say just limit these categories and we get transportation if i put in fed x i get logistics which you know it'd be more shipping so i could just add a new document which is shipping and it will say yeah shipping fits best out of all of these so you can do it with something really cool let's take the seven basic plots so we have uh the quest we have tragedy we have rebirth voyage return and i'm just going to put the word jaws in there because gpt-3 is smart enough to know when it sees the word jaws it might mean the movie or the book i don't have to explain anything i don't have to put the plot in there i can just say jaws what plot does it fit the most and it says overcoming the monster which i would say that's pretty accurate and this is a has 189 score and the next closest one is 67 voyage return which still kind of fits in quest all three of these positive rankings are pretty good ones um the negative ones are you know not as likely so that shows you how powerful it is to able to look at that and compare it so let's take uh let's use the hobbit okay it helps that both of these are books and movies so it has a pretty good understanding that the quest comes back with it being the quest which uh is pretty good you look through the different other themes there it actually puts rags to riches in there which in the hobbit as you know there's gold and stuff and it's like yeah maybe that fits and the ones that says it doesn't fit tragedy or comedy seem to be pretty accurate now if i wanted it to analyze a story of my own and it's not something that's familiar with i would then put in a description of the plot you know this happens and this happens this happens and it will then figure out what fits best so that's a really cool application that you couldn't get that from just keyword matching you couldn't get that from just a simple model it has to understand what does tragedy really mean what is a quest what do these things relate to when it comes to genres like fiction and literature you can use another tactic you can use too is you do text content but let's use uh directors and movies okay so if i ask it for the director of wayne's world it knows it's penelope spheros but if i put in the director of casino it doesn't have an answer because it can only provide me an answer from this list here right but if i want to have the option for sometimes it to not know because it's not in the list i can use something i call a generic type which is called i'll say the director and if i put in the director it says oh well the director's the best is the highest ranking one here um then all the other people but if i put in the director of avatar says james cameron and notice how the director then added it sort of functions as a split it separates the correct answer from the incorrect answers so that's kind of a cool way to use it so if i put uh if i put director of science fiction let's see what happens there we get uh ridley scott james cameron steelberg nolan and decided that george lucas isn't a science fiction director which are there's some people to make that argument but you can see you can use that as a separator to separate things from there the people at the lower end aren't necessarily known for having done science fiction so that's one of the ways you could use semantic search again you'll see this in the documentation it's very straightforward as you take your query and then you take some text and send in text documents like in an array and say okay give me a score for each one and then i'll figure out how it fits so uh question is samir says if what you did with that one game review if you wanted that but for a hundred game reviews so you could try to use that like advanced form i showed you before remember you've got that token limit so you've got 2 000 tokens that are limiting you but you could basically just take that and send it to the easiest way to send it to the api a hundred times to send it to the api over and over again and say categorize this categorize this categorize this you could show it hey this is how to break this into uh let me switch off my screen here you could you could say hey uh this is like i did before that that prompt where i showed you how to get the data you can see here's the data here and then you could say like uh 1 2 3 4 5 6 7 8 9 10 and list each one of those is one of those reviews and then give it another list and say now give me the answer starting with number one and do the same thing so go look at that tweet classifier example i guess to say to kind of get a better example so uh we still got a little time left here so uh any more questions perfect um it's really the best way sort of to play around with things and to try stuff experiment with it try different prompts um i think you know maybe you guys already have examples um of what you want to do with it maybe you don't play around with it and maybe you know you could you can squeeze it sometimes into a use case but sometimes if you say hey broadly what do i want to do here and you know you could say okay i could use this to analyze game reviews but you might say okay what do i really want to do with the game review you know i want to get data back from people and if you are at the point where maybe you have a game and you want to make it easy for people to create that information for you you might find ways to use gpd3 to make it easier for somebody to write a review um you might decide uh you know is there a way to summarize this in some other format um you know just just you think start thinking more broadly about the possibilities of it and you'll come up with you know really really cool applications and and i think one of things new too is like take it and then go in slightly different direction uh this is a great question which i didn't even cover travis wants to know can you talk more about the different engines so the engines are basically uh different capabilities like they're different sized models that we use davinci is the most powerful da vinci is the biggest and the most powerful one and then ada is the smallest one eight is the fastest one though now ada doesn't have as much information and isn't as capable as da vinci but ada is much cheaper to use and much faster you have ada babbage curie and then davinci and then two couple variations of which i'll explain in a second so da vinci's kind of the smartest one i start with da vinci see what da vinci can do and then i try to step down to a smaller model because if you can go from da vinci to carry kyrie costs 1 10 the cost of da vinci and then babbage is like you know 150th or something so the models are basically different versions of gpd3 different sizes you know you can think of like ones more powerful than the other but the smaller ones are faster which can be very useful and can be useful for a lot of different things we also have uh like davinci instruct curie instruct what these are these are versions of the models that are trained to basically know how to answer something more specifically so uh i'm giving you let me go back i'm gonna share my screen again with you just to show that in the playground so let me just pull up um over here i set it to davinci instruct okay so all what da vinci instruct is basically we took a bunch of examples of prompts that people were making and we said okay can we just make it where we just tell it what we do make a you know uh make a list of comic books i'm just going to type that there and let me set the temperature a little bit higher okay i have a character turn let's and if i used regular da vinci with that uh it might not have understood that i wanted a list of this so you're gonna say tell it make a list of this you can say um uh you know write an essay about mars pollen position okay and it understood that's what we wanted so this is basically da vinci that's been trained to understand how to sort of follow instructions that's what we call instruct so those are different models let me show you kind of a good example of if we use um so we have carry instruct davinci instructs so those are the same things we don't have a babbage or an 801 but if i if i switch over to a model just to give an example of like what another a different model is if i go um a long time ago and let's use ada very very fast but very repetitive though okay i was a young man who was in love with a young woman who was a young woman who was a young woman who was younger so we can uh you know and it if i was able to uh you know generate some response there but it wasn't as eloquent okay as davinci was but ada can do a lot of functions with late it can do really good things like classification stuff um you know so this is again we're seeing how much faster they are but the story you're getting there is a lot more repetitive than something da vinci would do da vinci is going to be much more nuanced so that's when you see the different engines their capabilities da vinci's great more nuanced stuff the other ones are better at being faster and sometimes they're really good at doing things like classification but uh again the thing to think about is different capabilities different things if it's a simpler task use a smaller engine more complicated then use a bigger one the instruct ones da vinci instruct and carry instruct can be super used to perform a task probably the thing to start with i wanted to show you how to do prompt design first but now that you understand a bit about that when you're trying to create tasks i would just start on the instructor and say do this and see if it does that and i might just do it right off the bat and it's great so uh we got i guess we can you know keep going here if we have any more questions i get we got five minutes left yet i don't know if you guys uh you know need to get ready to go to your next session which is what clicking over but i'm going to try to make myself available um in the forum to answer questions it'll be sporadic availability if you have some technical questions email the contact you have for that how do we learn more about the underlying architecture of gpd3 well if you go to the openai website openai.com you will find that we have a there's an entire article there's some papers to publish we have a blog post that goes into it which explains a lot more about it and then there's a paper that was published that will give you some of the more details and you can check out some of the other stuff because the underlying principle that made gpt3 also applies to other stuff we've just released information about dolly and clip dolly is a system that's able to generate images from text descriptions you tell it hey i want to see a picture of a bunny rabbit on a motorcycle it will generate a photo of a bunny rabbit on a motorcycle clip is a really efficient system for telling you what's in an image so you can say hey what is this image i'm like hey i think it's a bunny on a motorcycle like you're right so you can take that same model of doing prediction because remember everything in language models or models in general gets broken down into tokens and a token can be um a pixel it could be anything can be a cluster of pixels like an i or something like that or it can be a word and that's what it is it's all tokens at the bottom which a token is just a number sequence it's like you know five numbers you know and that's kind of the amazing thing when you look at how it's everything gets broken down to that level which our brains do too you know if you look at the way our neurons function you know they have you know calcium channels and areas of you know proximity to each other which kind of has sort of a mathematical variation and that's where kind of the underlying science of deep learning came from was trying to look at how our brains function and it's not like just gates that are on or off it has to do a lot with probabilities and things that are we call the weights and biases so fun stuff so anyhow check that out go to the blog at openai.com and you'll see that article about gpd3 and some other interesting things kind of the history of some of the things we've been talking about and uh any other questions please let me know have fun play around with it the thing to think about is that this is really less than a year old and people are still finding new things and you can take a look at what other people have done out there but if you really think deeply about what does it mean to understand text and to be able to do something meaningful text the world of possibilities is endless there are simple applications that become kind of self-evident but sometimes the really cool things if you think about where is some place where you've encounter problems with text we've had people who've worked on applications to take uh you know legal documents and to simplify them we've talked about people who are using them to take the output logs from servers and to simplify them you could use this to simplify things like scientific research you could use this to simplify things like uh reports of you know malfunctioning devices all sorts of different things there's a lot of information our whole world is about trying to understand things and gbtv is a wonderful model for trying to make sense of that it's not the end it's just sort of the start but it's a very exciting beginning so uh thank you everybody and um i'll try to be around and hope you have fun good luck yeah thank you for being here

Channel: Artificial Intelligence Society

Views: 4,154

Rating: 4.9245281 out of 5

Keywords: artificial intelligence, ai

Id: ffKXEvnaAZM

Length: 58min 54sec (3534 seconds)

Published: Tue Mar 30 2021