Fine-tune GPT-3.5-TURBO - A Crash Course

and in this video you'll have your own fully fine-tuned model using open AIS API I'm going to show you how in seven steps first you gotta ask yourself should I do fine tuning only if prompts aren't working for you or if your prompts are really long ideally fine tuning is a second Resort when does fine tuning work it can work reasonably well if you want to get the model to write in a certain style or tone or if you want the model to give structured answers for example an array with a name and an address where it won't work well is for memorizing data there it's better to try again and put the information in a prompt or to use a vector embedding database which I'll hope to cover in another video so here's what you're going to need to fine tune you'll need a CSV file with at least one column of data to train on and second of all you'll need an open API key I'll put the instructions below from open AI that you can follow along by yourself or I've provided a script and guidance for purpose that will help you to automate preparing these prompts and running analysis on your fine tuning let's get started the first step is to prepare a CSV file here I've got a CSV with one column and in each row I have the content of one of my blogs you need to have at least 10 examples I recommend having at least 15 because you want to split at least some off for a training set ideally you have about 100. now as you can see I just have one column here I'm going to open this up train.csv just so you can inspect it I recommend doing this just to check that all of the data is correct here you can see the content of one of the blogs the next step is adding a second column for the prompt now you may already have a second column for the prompt if you have a question answer style data set the question will be the prompt column and the answer will be the completion column but if you have data like what I have where it's just a blog or it's a tweet then you won't have a question that leads to that blog or tweet as an answer but you can use a little trick called rewinder and you can ask an llm to generate a prompt that would provide a completion of that type let's try that out here I am in visual studio and I have again train.csv and I'm going to run python rewinder.py take a quick look at that script essentially the script will send each of the rows each of my blogs to open Ai and ask it what would be the question that would lead to a Blog of that type respond immediately and only with a brief prompt that would be appropriate for generating the following content I've allowed that to run for just about a minute and now we can open up again train.csv and we'll see here that now in addition to the completion column there's a prompt for example discuss the potential impact of China's rise in Innovation and the changing landscape of global r d and the answer then will be my blog our next step is to prepare the messages in a format to send to openai API to do that we're going to run another short script python repair messages Dot py the script is going to take the CSV file and convert it into a format for openai it's going to ask me to provide a system message I recommend putting in a system message just because it'll help during inference although you will need to include the system message also when you're operating with this fine tune model I'm going to say for the system message you are Ronan McGovern write a blog on the following topic do you wish to split out a test data set this is going to take 20 of the training set and use it as a test set to evaluate the performance of the fine tuning so I'm going to say yes how all of the messages are structured I have a system message I have a user message which is the prompt and then I have the assistant message which is my blog you can see that the very same format is used here for the test data set validation right before we submit to open AI let's check everything is in order so we don't run into any bugs again I'm looking at train.csv and I'm going to run python validate Dot py th this is going to look through both the test and the training data set it's going to check for errors it'll show you the distribution so looks like it'll give us data on the number of of tokens in each it's also going to estimate the cost so it's saying that for the training data set here there's 22 000 tokens and I'll be charged for four epochs so it's going to tune four times with that data which is 88 000 tokens and at the prevailing costs of fine tuning that's going to be about 70 cents as a very rough estimate now that we've passed validation we're going to try fine-tuning let's get started I'm going to run python fine tune Dot py so give me a selection of options and the first one is what I want which is to create a fine-tuning job this automatically is going to upload train jsonl and then upload test jsonl and once they're uploaded it's going to start off a fine tuning job on openai okay the script will automatically wait if it's not ready to submit it will keep on retrying every 10 seconds and here you can see our job has been submitted there's going to be four epochs of training as I previously previously said it's going to be training the GPT 3.5 turbo model uh if you wish you can rerun that fine-tune script and we can retrieve um a list of fine-tuning jobs I'm going to show 10 jobs and I'll just scroll up to the top to find the latest one so here we have this fine tuning job and you can see the status of this is running it's going to take anywhere between a few minutes to even it could be an hour depending on if there's a lot of backup at openai for fine-tuning the model once your fine tuning job is done you'll get an email from open Ai and you'll start the visualization here's what the email looks like you'll see here at the fine tuning job number and also the model number so I'm just going to grab the fine tuning job number here copy paste back over in the terminal we can again run python fine tune.py this time selecting download and save a result file so it's asking here for the file ID to download now I don't have that file ID so I'm going to press Ctrl C to exit run fine tune again I'm going to do number three retrieve the state of a fine tuning job and put in that fine-tuning job number I got from the email sorry I'll have to do that again number three and put in the fine tuning job number that will give me the result file so I've got the result file from this fine tune and now I can run again python fine tune this time download and save result file and I'm going to enter the result file ID and that has now created a result file that is present right here foreign the next step is to graph those results simply I'll run python graph.py and I want to enter in the result file again and here we have a graph of the training and the validation loss so in blue you can see the training loss the slope of that is the green line so you can see that we are reducing the training loss and then the validation loss is in Orange and that is also improving but at a much slower rate now what you're looking for here ideally is that you would like the validation loss and to be going down and ideally you wouldn't like the training last to be going down a lot faster than validation so I think this shows you and by the way this is with my full set of blogs I've done a data set of over 200 blogs in this case to get some more significance but it highlights how fine-tuning is really quite difficult you can see some improvement in the model perhaps representative of capturing my personal style but obviously the content in different blogs is going to be different and so it's going to be hard to train say on the topic of Korea if it's only seen data with respect to my views on maybe Ireland and Russia for the very last and possibly most important step let's try out manual inspection of the data for this again we have a script that we can run and the script is going to do a few things let's just run it first with python eval results Dot py I need to try that again python evaluation.pui and I need to enter the fine-tuned model ID so I'm going to copy that over from here in the email and here's what the evaluation script is doing it's grabbing a random row or a random system prompt and user prompt from my test data set and it's sending that to open AI to get a response it's getting a response from GPT turbo without fine tuning then it's getting a response with fine tuning fine-tune model response and lastly it's showing the correct response which is the actual response so basically you should be able to see what the raw GPT would do where to find tunings having any effect whether it's making it sound a little bit more like what I am in this case you should be using the quality of the fine-tuned model response to determine whether your fine tuning has worked or not if it doesn't work very well you might try adding more data points if possible or maybe making more effort to clean up your data set so that it's tidy or you might find that it makes more sense to go back to Simply prompting the model that's it for the open AI fine tune and crash course I've put the links below to the open AI API documentation I've also put a link to purchase access to the repo that will run all of these scripts let me know any questions on this in the comments cheers

Channel: Trelis Research

Views: 1,834

Keywords: openai fine-tuning, gpt 3.5 fine-tuning, fine tune gpt, fine-tune gpt-35-turbo, fine-tune openai, openai fine-tuning tutorial, fine-tuning demo, gpt fine-tuning tutorial, ai fine-tuning tutorial, how to fine tune gpt, gpt-3 fine-tuning tutorial, python fine-tuning tutorial, python fine-tuning, fine tuning course, openai fine-tuning python, openai fine-tuning api, openai fine-tuning vs embedding

Id: NocZi9hyrDg

Length: 11min 29sec (689 seconds)

Published: Thu Aug 31 2023