Fine-Tuning OpenAI

foreign [Applause] hello this is Richard Walker  from lucidae and welcome to this seventh video   in the series on gpt3 and Transformers in this  video we're going to continue our discussion   on fine-tuning Transformer neural networks  fine-tuning allows you to create your own   bespoke models on the most up-to-date content in  the last video we discussed why you would want   to use fine tuning in both this episode and the  next we'll discuss how you fine-tune in this first   video we'll look at the basic methods that you  can use for prompt Engineering in the next video   we'll take this to another level and show you how  you can build much more complex data pipelines and   get your prompts from audio and video sources  such as podcasts videos and conference calls   in both these next two explainers we're going  to dive into some code written in Python   but this is not a coding tutorial there are way  better YouTube channels out there for that we'll   just be using some python to cement the concepts  and show how straightforward fine tuning is   if you know some python already then that's great  I hope these concepts are useful if you don't know   any python then I'm willing to bet there are a  bunch of people in your firm that do so if you   need to manage projects involving fine-tuning AI  models then I hope you get something from this too   so in this video we'll discuss a couple of easy  methods for creating prompts and completions and   we'll also look at some simple text manipulation  so that you can build automated pipelines for   fine tuning it should be pointed out that  the most straightforward method of creating   prompts and completions is just to manually  populate an Excel spreadsheet with two columns   one will be called prompts and the other called  completions now I'm not going to recommend that   you do this I'm going to recommend that you look  at some of the more automated and scalable ways   to write pipelines this approach will work and you  absolutely can fine tune a model this way the end   model won't behave any differently if you've  trained it manually or if you've used voice   recognition from podcasts more on that in the  next video but manually updating a spreadsheet   won't scale and the reason you're interested in  AI in the first place is because of the efficiency   multipliers that you get so I'm recommending  that you do not do this or you only do this as an   absolute Last Resort by the way you will do it and  if I'm being completely honest I've done it too   so with that shameful admission out of the  way let's look at something more systematic   it may have come to your attention that the  internet is full of content a huge amount of   text sound and video in this tutorial we'll  look at text and save audio and video for our   next episode I'll share some tools with you  that you can use to manipulate text as well   as tools to retrieve relevant passages of text  from the internet in our last video we discussed   a couple of reasons why we would want to find you  firstly we'd like our content to be as up to date   as possible so that it's relevant secondly we'd  like to make our NLP model less of a generalist   and more specialized in our specific field such as  capital markets accordingly we'll want to train it   on recent and relevant articles by fine-tuning  it on relevant prompts and completions during   training we're getting the model to update its  query key and value matrices in its attention   heads these updated qk and V matrices will mean  that during inference the model will be able to   generate output that is both more relevant and up  to date so by fine tuning Our Transformer with a   prompt of an article title and a completion of  the article text we're updating the attention   heads in the Transformer likewise if we  use an individual sentence as a prompt   in the next five or ten sentences that follow  as our completion we're providing the grammar   with updated q k and V matrices that are more  representative of recent developments in our field   when we fine-tune a Transformer we're teaching  it new stuff when we teach it new stuff it learns   the manifestation of that learning are updated  weights a query key and value matrices if you're   unfamiliar with the concept of attention query key  and value matrices please click on the link in the   description or the pop-up on your screen there are  two specific problems that we must solve firstly   we want to be able to grab relevant text to  train and model from the internet this text will   likely be in the form of Articles and reports  that we will use to build our prompts and our   completions rather than have these articles pop  up nicely formatted in our browser we'd instead   like just the text of these articles to pop up in  an app that we write so that we can generate some   prompts and completions from this text secondly  once we've got this text in our app we need to   do some manipulation on it so that we can break it  up into prompts and completions by manipulations I   mean things like ripping out titles and section  headings which are good candidates for prompts   likewise paragraphs of text are great candidates  for completions similarly we'd like to be able   to break up paragraphs into component sentences  so that we can use a particular sentence as a   prompt and the text from the next say five or  ten sentences that follow it as its completion   this means we'd like some code in our app that  will grab text and chunk it up into things like   headings and sentences let's tackle the second of  these problems first let's start with some simple   text in this example we have a title and a sample  paragraph of text what we want to do is feed this   text into some function that will chunk it up into  individual sentences we can then use some of the   chunks as prompts let's say every third sentence  will be a prompt and then we'll want to use the   remaining chunks of text between the prompts as  our completions so let's grab the first sentence   here highlighted in purple as our first prompt  gpt3 can be fine-tuned and then the next two   sentences in yellow are the completion text that  follows we'll then repeat this with the fourth   sentence and the two sentences that follow that  then finally the seventh sentence and the single   sentence that follows that this gives us quite a  high resolution prompt and completion sample on   chosen text we can add to this with a lower  resolution prompt and completion by then taking   the title text as our prompt and the entire eight  sentence paragraph that follows as our completion   this will give us a total of four prompts  and four completions from this sample of text   so we'll need to write some code to do this please  look away if python code makes you feel squeamish   firstly here's a function in Python that will  take a string of text as input and break it   down into its component sentences and return  a list of all of the sentences in the text   the python magic here is this regular expression  this checks for a full stop a question mark or an   exclamation mark followed by one or more spaces  it uses this delimiter to split the string into   a list of all the sentences and returns  this list as the output of the function   now that's a great start but we need more than  just a list of strings we want to be able to take   the nth sentence and use it as our prompt and  then use all the intervening sentences as its   completion until we hit the next prompt we might  like to vary how frequently we'll generate prompts   will it be every three sentences in the example  we've just seen or might we want it to generate   less frequent prompts for longer completion say  a prompt every 5 or 10 sentences to do that we   should pass a variable into the function that will  determine how frequently we generate our prompts   so here is the function now it's doing a little  more so it looks a little bit more intimidating   to the non-pythonista but let's break it  down once the text is split into sentences   we've seen how to do that the function groups  them into prompts and completions a prompt is   simply the first sentence of each group while  the completion is the remaining sentences in   the group the function creates two lists one  for prompts and it does this by slicing up   the sentences and taking the nth sentence in  the list and then another list for completions   which it does by looping through the intervening  sentences and appending them to a completions list   finally the function creates a dictionary which in  Python is a set of name value pairs that Maps each   prompt to its corresponding completion and uses  something called the pandas library to convert   this dictionary into something called a data frame  think of a data frame as a tabular data structure   in Python similar to an Excel spreadsheet the  resulting data frame contains two columns one   for prompts and one for completions making it  easy to feed the data into a language model for   fine tuning or other applications if we run  this code on a dummy text string containing   50 sentences we can see that it's taken every  fifth sentence as a prompt and the intervening   sentences as completions now at this point we're  pretty much done frankly this code will work well   for most purposes but to make it just a little bit  more robust We'll add some simple error checking   as well as some methods to Output the results  of this function to a Json file a CSV file and   an Excel spreadsheet we can bundle all of this up  in a simple class to initialize an object of this   class you can pass in the string to be split and  a number that represents how often we'll create a   prompt say every fifth or tenth sentence we have  our familiar function to split our sentence in   this version we've made it a little bit more  robust with some simple error checking finally   we've added three functions to the end that will  create firstly an Excel file secondly a CSV file   and finally a Json string from our pandas data  frame we could then write a short script to create   such an object as inputs we pass in a string and  the number five to indicate we would like every   fifth sentence to be a prompt when we run this  code we can see the generated spreadsheet that   contains our prompts and completions created  from the text we supplied so we've solved the   problem of splitting up our text the next thing to  solve is to be able to get text from the internet   do that we're going to use a popular  python package called Beautiful soup   when you view a web page in a browser you make  a request to a web server and it sends back   some HTML this contains all of the content and  formatting so that your browser can display it   beautiful soup allows you to grab the individual  components of the HTML and instead of displaying   them in a browser they're returned to your Python  program so here's an example of how it works make   sure that a website you visit and use beautiful  soup on doesn't mind having its content scraped   a website called quotes to scrape is a pretty  safe bet here my browser has rendered the content   here's a simple python script using beautiful soup  that will visit this site and pull back the HTML   it then uses beautiful soup to display the  data from our program when we run it we can   see the first Einstein quote now let's modify the  script just a little so that we can see more of   the content of the page in both cases we're able  to get the content we need back as a text string   now recall we can use this string as an input into  our previously described splitter class so finally   let's look at the Wikipedia page for beautiful  soup scraping this page might be a little bit   like a chef using their own recipe book to cook  dinner or a hairdresser cutting their own hair but   hey sometimes you just have to embrace the meta  and let the beauty of beautiful soup do its thing   in this video we've continued our discussion on  fine-tuning Transformer neural networks we spoke   about why fine tuning is important and how it  can be used to create bespoke models we discussed   basic methods for prompt engineering and simple  text manipulations to build automated pipelines   for fine tuning the most straightforward method  is to manually populate an Excel spreadsheet with   two columns one called prompts and the other  called completions but this is not scalable   please only consider this as a last resort do  as I say not as I do the internet is full of   content and we can use tools to manipulate text  and retrieve relevant passages from the internet   we identify two specific problems that need to  be solved one grabbing relevant text from the   internet and two chunking it up into prompts  and completions we saw how to solve the second   Problem by building a splitter class in Python to  solve the first problem we introduced beautiful   soup this popular python Library allows users to  extract HTML components of a web page instead of   displaying them in a browser we saw how to use  beautiful soup to extract text from a website   such as quotes to scrape or the Wikipedia page for  beautiful soup finally we emphasize the importance   of fine-tuning AI models to achieve efficiency  multipliers and scalability this is Richard   Walker from lucidate thanks for watching please  join me in the next episode where we will look at   more sophisticated ways of building prompts and  completions using news feed apis audio and video [Applause]

Channel: Lucidate

Views: 9,492

Keywords: ai, chatgpt, gpt3, gpt-3, fine-tuning, attention is all you need, attention, transformers, prompt, completion, prompt and completion, prompt engineering, machine learning, artificial intelligence, fine tuning, fine tuning gpt3, beautiful soup, web scraping, how to fine tune gpt3, how to use chatgpt, foundation models, transformer neural networks, neural networks, neural network

Id: uFiI5fK-7B4

Length: 16min 28sec (988 seconds)

Published: Sun Feb 19 2023