How to Fine-tune GPT-3 to Get Better Results and Save Cost | NLP | Python | #gpt3 #openai

hi everyone welcome to my channel today's video is about gpt3 and we're going to see how do we find tune a jpt3 and you can see we have already done i know uh four videos on gpt3 including you know how to write a gt3 prompt how do we integrate that gpt3 prompt into a python code and even done some other videos including gpd3 embedding or doing some sentiment analysis using a gpt3 so today we're going to focus on a fine tuning we're going to see why do we need a fine tuning a gpd3 and then we're going to see you know what are the different advantage do we have with gpt3 uh fine tuning and then finally what are those steps and we will fine tune agbt3 model that's what we're going to do so first of all why do we need a fine tuning so if you're if you have seen my earlier videos when we build an application with gpt3 mostly what we do we give a gpt 3 an instruction that what exactly do we want to achieve and then we come we give some sample example maybe two three or four examples to gpt3 to understand what we exactly mean by this statement and then finally we give a test example and then we want to produce the output now take this example this example is about you know intent classification or intent detection so you will have an intent detection kind of thing let's say you have a customer support where people are going to write some query and they want to know about certain thing if it is bank your customer query could be related to the maybe they have their way you know looking for a loan or they may have a you know query related to the credit card or their balance check or any other issue right so when that query comes you want some way to identify you know what is the intent of the customer so that you can forward that query to the appropriate department so if it is related to let's say loan then you can directly forward to the loan department if it is related to let's say credit card you can pass on to the credit card so you always need such a system where you can automatically you know identify the intent of the customer who is raising the query and this particular example what you see on the screen is related to i think i took this particular data set from the kaggle which is the airline travel information system where i have the link here so here the travel customers or the people who you know fly there is some query and we want to identify whether the query is related to the flight time is it related to some abbreviation they wanted to know or is it related to some ground service and there are some other intents are there so usually how do we write we simply write something like you know classify the below text into one of this intent this this and this intent right something like this and then we give an example if this is the text then it is actually a flight type then you add some separator so that you can separate those examples what we are giving and then you add a second example and its intent third example an intent and the next we assume that now gpt3 has learned because we have given instruction we also given three examples here that what do we mean by and this is the time that we can test gpt3 whether it is able to produce so let's say we give a text and we want to see whether gpt3 can identify you know intent for this particular sentence and we can submit a request here and you see it did identify that this is related to the flight name this is the awesome thing about gpt33 that just giving two three examples it understand you know what we are looking for maybe we can give some different example here let me go here put a different example and let me submit that request again and let's see what it identifies and it say it is related to the ground service okay maybe actually this has to be the plane or from terminal it is nothing but the internal airport uh travel kind of thing so it is able to do pretty good but you know what happened most of the time the most of the time you don't have just this two three intents maybe you have 20 plus intents or the kind of problem statement that you are dealing with just giving this three to five examples you will see gpt3 is not able to produce a good results for you then you have two options you can put more examples here maybe instead of three examples you might put 30 50 or 100 examples here and see whether gpt3 can understand but this prompt has some limit first of all the how many you know token you can put here i think it is currently 4 000 limits so you definitely get limited by how much you can put here that is the first thing the second thing related to the cost every time you make a request to gpt3 when i you know put a cursor here and i submitted request dptv33 send everything gpt3 sent my sample example plus my test example to predict this particular thing now imagine instead of these three examples you have a 30 50 or 100 examples here it is going to be big cost for you because just to predict that one single example you will be sending those hundreds of example so that is the another reason the cost and the the other reason was the uh the amount of and they they even say that the first of all that we can be going to save our tokens once we fine-tune the model because once you find your what you will do you don't need to send everything this because you are already using the model which is finding for you so you simply pass this text so you're going to save your cost because the cost of the gpt3 is depend on what you are sending and what output you are creating so if you're going to send to multiple example it is going to cost you right those things they even claim that the fine-tuned model has a lower latency you get the faster response right that that is the something that you have then what are the steps involved to you know do a gpd refine doing the first thing we need to prepare and upload our training data we need to prepare the data the way gpt3 or opening i expect once we prepare and uploaded data then we can train our new fine-tuned model and finally we can use that particular model and we're going to see these steps i'm going to show you how do we do that particular thing and again we require an open eye library that we require anyway whenever we want to programmatically access the open ai if you haven't seen the earlier video then i would encourage you to go and check those video here we discuss about you know overall gpd3 what exactly this gpt3 prompt mean and how do we write those particular prompt and then here we discuss how do we integrate that prompt in a python code so at least these two videos you should be familiar before we go to this third video which is nothing but the fine tuning the gpt3 so let's do one thing so first of all i need one dataset kind of thing so as i mentioned i took this dataset from the kaggle which is related to the airline information system something and so i loaded the data set here and it has two column one is the intent and this is the query that user use we can call it as a text and we can call it this as intent so that's what i'm going to give that data frame name and let's see how many unique unity intents we have right so we see there are 22 intents are available total in this all airline information system and you could see some of these intents are actually separated by hash which is something like you know airlines and then this is related to some flight number some kind of intent right so first thing i want to do i want to remove this hash and make it underscore so that i want this to be a single big you know intent that's what i'm going to do so it will create this uh bigger number again this this uh this thing we don't want this atis which is that you know this is the airline information system we just want this simple you know flight or it is a ground service or something like that so that is i'm going to do i'm going to replace i'm going to take this column you know intent column and i'm going to replace wherever i see this particular value ati is underscore with nothing so that i just get rid of this particular thing now i have something clean flight flight time airfare and kind of stuff right and we will see you know uh for each indian how many examples we have so we save a flight which is more of like a generic intent everything about the flight is we have more than 3000 examples but related to the airfare we have some 400 and something for ground service we have 255 there are very few for the others you could see but for a demo purpose what i am going to use i am going to choose only few of these intents to show you the demo so i am going to pick up the flight ground service airfare only few of those intent to show you the demo and then i am going to filter all the rows from this data frame where intent is in our labels our interested labels and that's what we're going to do and so we got now final data frame with having only the flight airfare ground service abbreviation okay uh don't worry uh i will show you what exactly this abbreviation mean let me say that i could put a control c yeah so the abbreviation is means like uh the query could be you know someone is asking what is this ap 57 so this is like uh someone is asking about an abbreviation ground service is like you know it's like for example anything related to the ground service maybe that luggage handling or how do you reach to airport uh sorry inside the plane from that particular terminal right this all query is related to the what i say uh related to maybe the ground service so anyway i am going to give link to this database data set so that you can go and you know learn more about this particular dataset what are the different things it has right i will more focus on the methodology that how do we do it once you have your own data set okay now what i will do this i think phi intent that we're going to choose and this intent has a let's say 54 i'm going to choose 40 examples from this each intent 200 examples are enough actually the openings suggest that you should have 100 examples for each of these but for demo purpose i'm just going to choose only 40. if you give instead of 40 if you hundred for each of them definitely the quality will be better but what i'm going to do i'm going to take this data frame i'm going to group by this each of intent and once i group them i am going to sample a 40 example from each of this group okay and that's what i'm going to take as a sample this is what that panda statement does here now you see each of this class i have a 40 examples and these 40 examples i actually want to fine tune gpt31 nothing but we simply narrow down our you know data set so that it becomes easy for me to play around and you can do as per you know what you wanted to do next thing i think our dataset has first intent and then text but i will make it you know first text and then enter because on the basis of text let's say we want to predict the intent so i'm just you know altering the column arrangement so i have now text again the basic sanity i just want to make sure there is no extra space from left or right side now here comes the part where what should be the format of our data set that we want to prepare for the uh jupiter so if you go and read about you know uh some guidelines jupiter we will be having so if you want to find a gpt3 you should have each example is like a json object so this will be a json l file which is nothing but json uh what is useful form i think json newline or something format name it just you know um json object notation this is the same thing anyway this json l this particular format is nothing but you know i think we had a link here right yeah json lines that l for only the uh lines because we want each json object on each line that's what so that it is yet separated by the um slash in that particular thing so that is something jpt3 is expecting that you should have each json and they are separated by this slash slash n that's why you call it as a json line format and you don't see any array here right so it's not like array of this json object no you simply have a json object on each line that's it that's what this format mean so in that each example you have a two keys one is the prompt and they this is the completion so again here prompt mean this is our input and the completion means what is our expected output this is our completion and then if you read there are some guidelines best practices that when you write a prompt make sure you end your prompt with some you know a unique character combination if it will act like a separator that will help jupiter that okay a prompt is ending even you can add same some things to the completion thing so let me see whether uh i could show you an example what i actually mean by that do i have here or maybe anyway we can see now yeah so let's say this is like prompt so this is the example prompt but they are ending with this particular thing so this will act like a separator and this will be present on every example so that it knows that this is where your prompt is ending so you can have such a you know separator for your prompt that we're going to add so you know this is what the basic requirement is needed now let's go back and think of like how do we convert this particular text and this intent into that format right which format the json format but before we do format what we will do as the best practice is suggested we're going to append some you know a character combination to mark that this is the end of our particular text so i choose this thing which is you know two slash n slash n and then i choose intent now this intent need not to be there you can even write something like you know triple hash hash something like this what is criteria here this thing should appear on every example since it is a data frame column it is going to be appeared on every example another thing this whatever separator you are using this should not come inside anywhere inside this text right so what if i use only the slash and separator then there it will cause project because slashing can occur here also right so then that will mislead the gpt3 we want whatever you are using make sure this has no chance to occur here inside so i'm not i'm damn sure that this will not be occurring any of the text here inside right so that is what this will mark as a separator and i am also choosing a separator for my intent that every intent i am going to mark as a space and end to indicate that this is the end again you can choose whatever word keyword you want to use but i'm i'm sure none of my abbreviation has this word in so this will act as a separator for all of them and it is not going to conflict with the what we have right and and this is kind of a best practice so see what happened this is my query maybe if i do this thing it is easy to see this is my query and then i added this in i think there is a slack you are not able to see maybe uh maybe this is oh then this cause no double issue maybe i need to run now this whole thing again okay let me see from where do i need to run this thing maybe i need to run this thing from where did i yeah i created here right let me run again this thing right because i twice run that particular thing and it appointed yeah you could see each of them are ending with this particular value and even our intent is ending this thing value this is for best practices another thing is that your completion should start with the space that is also one of the uh you know suggested if you see it is starting with the space completion should start with the space and you can play around it i did play around lot of different thing i even try to put a instruction directly inside that particular and then give our takes and so you can experiment with it there is no you know these are just best practices to follow minimum minimum you should have at least this text and this intent and this is the thing that is going to make your model better right so now once we have some separator for each of this line let's generate the data that we want right here you could see how does our example look like so this is the query user has given and then we add our separator here whenever the gpd3 came across this particular thing slap double slash n here here to slash that this is it means that the prompt has finished the text has finished similarly whenever it will encounter the space and end it will come to know now this particular intent has finished and maybe now next example will start so this is just for the sake of putting those separators and again you can put anything instead of this intent you could put simply you know triple hash or phi hash anything that you think it will not occur here maybe it's becoming repetitive now anyway so we have this thing again we want those column names to be the prompt and completion we just saw here the prompt and the completion so let's put them prompt and completion and this is the statement actually that going to convert our you know a data frame into those json lines so if you search this two json function of pandas you will see it can convert a pandas data frame into the json format called 2json format and it has an important you know thing called what is that a parameter which is the orient this particular thing now depend on what you are using for the orient it generate the record format so currently we use the thing called records we need to orient as a record so that each row will come as a dictionary kind of structure you have a key and the value and this is what we required we require a prompt and its value the completion at its value and there are some other ways you can see sometimes you might want to have a data where you just have a maybe you know a index and then column value or maybe something like this column and then index and value there are many variations you know you could see but we stick to this because this is what gives us what we want taking each row as a record which will become a key value pair and that's what we're going to do maybe we can see that thing once we created so let's look at the same position it will be here somewhere okay and again this is json lines and these lines is equal to true is what actually doing that particular thing putting so let's see this not not this one this one this is our example now because each row has become now this dictionary and here this is our prompt first column its value here second column that is what i mean by key and the value key and the value and again you could see our prompt everything has ended with this thing so this is consistent across every example so whenever gp3 came across this it knows that the prompt has finished and similarly this thing for everything we are ending and you don't see any array symbol here right because it's not an array it just a json object on each line that is the format expected okay now let's go back to the notebook again now we have a data in the format we watch now we can pass the data to gpt3 because if you look at our steps here we need to prepare and upload now we're going to we have prepared it we're going to upload now data i'm just installing the open air here let me clear this output right and then open air once you open your install you know open as tools to prepare our data which is actually fine tunes dot prepay and the name of the file so our name of the file was the intent json sample.json so let's run this statement so it it actually gets uploaded to now it is asking certain things to us what it is asking it has found certain duplicate in everything right and it is recommending that we should remove so let's listen and let's remove them it is now asking whether you know do we want to split our data into the training and validation let's say yes and now it's want to write that to a new file that is also fine okay so that data preparation has finished maybe we if we refresh we can see now we got two file the intent sample has become now prepaid train and prepared validation right now we have prepared data both training and validation we can fine tune our model to do that thing the first thing we want to make sure that we have open api you know open air api key and if you have watched earlier video you know how to get this particular key so i'm just going to say that thing as an environment variable and then we're going to use the next command so on top you use fine tunes dot prepare data now we're going to use the create which is the creating the fine tune model and we have to give two things one is the hyphen t which is our training file created here on this step hyphen v validation file that we created here and then which base model so we have a darwin c this is the model that we're going to use because this is the most powerful model they have and then we're going to submit the request so once you run this thing so now it is saying that uh what file i'm uploading seems to be they have that particular file already there it could be because i have run it multiple times so what i'm going to do it is saying if i want to you know still upload it again then i have to just press enter and that's what we're going to do and it is going to same thing for me for my validation file and i'm going to press enter to anyway upload my file again now what is this thing okay it is showing that the fine-tuned cost is you know this particular and it has skewed my job there and it might take some time now because it's going to take maybe 10-15 minutes to get this job run on open ai server so i will pause video here and once it finishes then i will start recording again okay now you can see the fine tuning has finished one thing i want to tell you so when we ran this particular sale you see the stream got interrupted we get some response from them right as a progress and when it get interrupted you will also get a command that you can again run so it is fine if you are running in notebook and your notebook got disconnected or you know the laptop got shut down because process is running on open ai server and not on our machine so we can always ask for you know what is the progress there so even you can run this follow-up command to get the progress on your particular fine tuning and that's what we did i ran this payment and now i could see the progress that it has completed four epochs this is the default box it does and we got our new model which is this a darwin c model with some you know uh some unique name to it so that we can identify now we can use this model to make a request to jpt3 and that we can try using this particular model so let's do one thing now what we will do let me comment this particular thing so that let's say we have this is the prompt that we want to send so we want to ultimately want to make a prediction on something like this right so we have this thing you know do we have a london flight on monday something like this this is the take that we want to predict the intent but again do you remember we have put a separator after each of that prompt and we need to pass that same separator here also so we will put that you know uh twice our slashing in 10 and again those last lesson that is one thing while passing the prompt make sure you pass the same way the way you find you and then let me copy our model name here which i'm going to put in open here request so let me replace this old name which i have with the new one that we got right again if you see i am also putting is a stopping criteria which is nothing but the separator that we have put for our label if you could remember let me go here right so since it is generating model sometimes it generates something extra but that we don't want also so let's say in that case this might happen that it generate that for this thing intend is abbreviation it will generate end and it will still keep generating something extra also so we can tell that you need to stop when you encounter n because that is the end for our intent so you can also parse this particular extra step like a stop when it should stop generating the input and we know that if it came across this one while generating it means it has already generated our label because our label ends with this particular thing so let's do this thing let's make a request and this is the same request that we have used earlier right so this is the way we make a request to our open ear api and this is not just fine tuning it could be any model so if it is not fine-tuning you only have this model name davinci that's it nothing this thing this is all so this is nothing but taking the model name so if you want to see from where do we get this if you come to our playground here and if you click on view code section you have different language you can select python one and you see what you see there in the notebook it is something taken from here only so by default model name will be this takes damage this particular model default model we have but now here we have our own fine tune model that's what we're going to do and everything remains you know same nothing changes so let's make a request and see whether it is able to predict now it is saying flight okay it is related to the flight the one of the intent that we have now let's see whether it predict this particular thing even i don't know let's see whether this intent where we are actually asking some abbreviation here it does predict it is an abbreviation now this is something another prompt example which is related to some ground service and pc and it is saying the ground service so it is able to do without now giving this all these examples what we do where was our playground this is the example we simply pass in our test example why because model knows this example already we have fine-tuned it right this is what we did so this is how you will uh fine tune your model now in this case the particular example that what we saw we are getting already good result without even fine tuning it but if you consider all those 25 or 22 intent and they're all sort of different examples just writing in a prompt might not be a sufficient and you might have a use case where you literally need to fine-tune it and this is the steps to fine-tune uh the particular model let's see what happen if you remove this thing let's come in this thing and you know we don't require this parameter now and let's run this thing you see we got some extra keywords coming in so that is the reason we put a stopwatch so whenever if we come across the single word end it means it needs to stop generating okay because we already generated and why do we decided we want to keep end here because this is the separator we have used for our label and since it is generating label then we can use that separator that this is going to occur after our label and then we don't want to generate anything because that is what we wanted so this is something you can read more about it actually what is this you know stopping sequence i might not have you know justified in that one minute or something but you can always go read about the prompt the prompt parameter there are other parameters what we have right so this is what we do what we did we simply uh take our data set you know we do some basic cleaning because we have to think text and the intent we make sure we have a proper separator for our prompt which is our query and then our completion which is our intent so we had a separator for them and we make sure we generate data in a format what is expected by the gpt3 which is nothing but this particular format json line format where we have each line is our one example and our each example has two part one is the prompt and it's a completion both of them are ending this thing so i hope you found this video useful if you haven't subscribed to my channel do subscribe and i would urge you watch this earlier videos where i talk more detail about gpt3 right so this was not an introductory video because i assume that you know already jpt3 you oh you know already what is gpt3 ap and all these things right and then we should go so i'm i'm not spending any time discussing this because i have already discussed that thing you know earlier these two videos so please go watch this video let me into the comment that you know what kind of videos you are expecting from me or do you have any doubt related to gpt3 videos that what i have done thank you

Channel: Pradip Nichite

Views: 21,976

Keywords: gpt-3, fine tuning gpt-3, fine-tuning gpt-3, finetune, gpt-3 fine-tuning, gpt-3 explained, fine-tune, aws gpt-3, benefits of gpt-3, #gpt3, open ai gpt-3, access to gpt-3, openai api gpt-3, ask gpt-3 a question, fine tune gpt3, fine tuning gpt 3, fine tuning gpt3, gpt 3 fine tuning, fine tuning gpt 3 code, finetuning, fine tuning, krish naik data science, krish naik machine learning, 1littlecoder

Id: muxtjezs3BQ

Length: 31min 28sec (1888 seconds)

Published: Mon Sep 12 2022