Build a Text Generator with OpenAI GPT and Python

in this video we are going to create a text generator with the openai model which is going to create um texts on its own just given a title so for example here i've given a title of the jedi and it came up with some really funny texts like uh jabba the heart's jabber had an affair with princess leia's second cousin jabba palpatine he gave her a secret but she refused to reveal it in the end he was forced to kill his way to his former master and even through he'd got the new apprentice he never got this wish so as you can see this is completely generated with our model using machine learning using the openai gpt2 and so we are going to create this entire app here uh using flask and we are going to download the model train it in this case we are going to train it on the star wars text here but you could train it on whatever you would like and we're going to test it with certain titles so if that sounds interesting to you let's get started so this is how our development road is going to look like first we are going to create the gui we are going to create our flask application set up our templates our index.html file that you have just seen and basically set up the application how it should work and where it's going to pull its data from then we're going to download openai train the model we're going to feed it some text and we're going to configure the code so it fits in our application so that we can actually run it in our browser here after that we're going to test it fix a few bugs and tweak it and see how we could improve it further so let's get started okay let's get started i have opened here a blank new project in visual studio code and now we can start creating our application so first of all we are going to create a flask gap that is going to serve our frontend that we have just seen and underneath that we are going to have a python script that is actually going to calculate the values or generate the text that we need so let us start with that let us start by creating a new folder and we are going to call that folder text generator since we are going to create a flask cap we need to give it some type of name in which it's going to live so in this case it's going to live under text generator now let's create a new file and call it generator.pipe inside of the generator.pi uh we are going to actually have our classes and have the modules and things that are actually going to generate our data so first of all we are going to start with creating our flask app so in order for a flask app to exist we need to have an init file in this init file we are going to initialize our application okay we need to move it inside of that we are going to initialize our application we are going to do all the important imports that we need and our application is going to live from there because when we are starting a flask application we started by typing in flask run and so the init file is going to be the initializer of our application um first of all let us install a few requirements that we need so now open your anaconda prompt i am running here on windows so i'm using anaconda or open your terminal depending on the environment you're running in so let us install a few requirements that you're going to need so pip install flask and we are also going to need python dot environment okay i have already installed those packages so it's going to say requirements already satisfied now after you have installed those requirements you can head to the init.pi file and let us import those so let's import from flask let us import flask now we can define a create app function is actually going to create our app and now we can pass certain settings to the app so we can pass a config file so we are going to have a completely separate file that is going to contain our configurations called settings.pi now we can create our app so we are going to call flask pass a few parameters so name which represents the name of our file that is actually running so for example let's say this is a global variable that we have access to in python so let's say that we run python generator.pi the name is going to be generator.pi so this is what we are passing here as a first argument now we are also going to pass a static url path um this is important when we are creating a front end in flask we are using a templating engine and for example let's say we have some static files like images and videos or something like that that we need to show in our application we can't just use any path for example let's say images is inside of the images folder we can't just access it like that we need to specify a static path within flask so in this case i'm going to have a temp folder and this is where our files will live so we also need to call a static folder which is going to be temp so far so good and now we can configure our app so for that call app.config [Music] config now from file actually it's from pi file and now we're going to pass our config file which is going to contain our config settings here now we can register a blueprint in order to have a we are going to create the routes the python file which is going to contain our routes that we are going to hit and there we are going to configure our blueprint so a blueprint is just going to make the entire setup easier especially when creating larger apps since it's going to give some level of organization to our entire application you will see this shortly let's call app register blueprint and our blueprint we are going to call it generator so we need to import that let us do that return app so we are going to create a new routes with pi file again move it to our text generator here and we are going to import this from routes import generator so this is a way how we can refer to our routes file here and from there we are going to import our generator we are going to register to blueprint and then we are going to return the app that we have configured here so as a next step let's go to routes and configure the routes there we need to import a few modules from flask so from flask import blueprint that we have just mentioned render template template request and redirect and from uh our generator we are going to import our ai so we are going to create a class here called ai we're going to instantiate an object from it and this is what we are going to import here so we can comment this line out for now but i'm going to leave it here so if you don't forget it later uh now we are going to create our generator blueprint that we have imported inside of init here so generator is going to be equal blueprint generator now we need to pass it the name and we again need to pass the name of the file here so if you don't entirely understand the concept of a blueprint uh here is the documentation so flask uses a concept of blueprints for making application components and supporting common patterns within an application or across applications so blueprints can greatly simplify how large applications work and provide a central means for flask extensions to register operations on applications so basically what this is it's going to make our life easier because we can use this type of structure to create the roads to create our templates and to basically just create a flask application so what i mean by that is for example we can do the following right now when we want to create a route we can specify an add generator specify a route and now we can give it the route that we need for example we are going to have uh slash which is going to be our index now we define the function behind it which function is going to be executed when we hit this route and now we are going to return a template so we could return something like a string and something like blah blah and when we hit that route then we are going to see this blah blah string on the page but we want to return some type of html there so let's use render template which we have imported here and inside of render template we are going to give our html file that we want to render in order for this to work we need to generate a new folder here called templates and while we'll add it we can also generate a new folder called tmp for our static files that we have specified here so this blueprint and flask setup that we have configured here is going to look inside of templates for the files that we are going to return here so it is important that the folder is named templates and that we have those html files inside of that so we are going to have an html so index.html file here and the render template is going to search inside of templates and return the correct file that we require and there is also since we are using a templating engine here we can exchange data between our html here and our route here for example if we pass a query string or if we pass some parameters we can pass those parameters like this parameter is equal to something some value that we have provided and then we can access this inside of our html file as parameter and we can do the reverse as well we can pass data from here which we can extract inside of our application here like the following for example let's say we have a variable called parameter we could access it like request for example uh form let's say we have passed something through a form and then we could access it like parameter here so we will see this um in a few moments how to actually do that but for now we are going to return our index file here now let us create a new route so i'm going to copy this over here and now we're going to have an analyze route which is going to be heated and when we actually submit our title that the title is going to be submitted then the ai is going to generate some response and return it back so this is going we also need to define a method here inside of methods and this is going to be a post request now we again uh define our method our function here we can call this however we want but in order to keep it more consistent i'm naming it like the extension here so now it would be the time to actually generate a bit of the html so that it's easier to understand what we're actually doing here so navigate to index.html in order to make it looked a bit nicer i'm going to use bootstrip here so you can navigate to the bootstrip page and copy this link here to import the css files so you could of course make it much nicer and import javascript files or create your own templates or create the styles you would like but since i don't want to focus too much on the front-end side of things i'm just going to use bootstrap and create a simple form here so what i'm going to do now is actually paste in my template paste in my html here so we don't lose too much time on that and explain you how the html works actually so let me paste my html in here so what i have done here is i created two div divs one for the left and one from the right side in the right side i'm just going to have the picture of wally and i have created some styles here for example this is going to be flex direction row i've given it the width and the height of 500 and giving some background color then here this is the important part i have created a form again i have provided some stylings here which you can do as you would like then it's important to provide the method as post and then give it an action so the action is going to redirect us to a certain url so it's going to be url for generator analyze so it's going to get us to this route here after we get to that route you're going to pass in some data for example we have an input here again i have specified a type a style a class a placeholder all not that important but this is important i have provided a name and the name is title so in order to access this inside of here okay this should be a call we are going to be able to access um to access the data in here and work with it so let us do that we are going to extract the title like request form and now the name is title so we are going to extract the title then we are going to generate some text with our ai we are going to call some function on our ai and we could call it generate text so this is not yet created but i'm just writing how it should look like and we're going to pass in the title as a parameter here and after that we're going to return render template and again we are going to return index.html but we are going to pass in some data in this style in this one we are going to pass text as text as the response that our ai generates and that text is used here as you can see we have those two curly braces so this means we can write our templating functions inside of that and we're going to access the text inside of here so those are basically the two important factors that you need to consider everything else is just plain css and html so let us save those files our route is finished we have created a temp file i just need to put my picture in there of wally we have created our templates now it is time to actually create our model and drain it before we can get into creating the model uh let us just finish a few little things before we forget about them and have errors later so let's create our settings.pi file for that we're going to import os and we're going to give it an admin for now we are probably not going to need this but let us do that still so you can see how this works so we're going to access from our os and we're on get and now uh we are going to create a new file called dot n and we are going to access um our dot n files in here uh our settings the pi file so we are going to have an admin username we are going to have the same for a password let us create a password and have an admin password here great so what we would do now is have those variables actually and username would be an admin an admin password would be password for example so i have just shown you how to work with those environment variables so that um everything is safe and everything is um done in a professional way but we're probably not going to need this here but still we can leave it and move the environment file to our source here and not inside of the text generator and one more thing we are going to need is for our flask app we are going to need a flask environment this one is definitely necessary what we are going to provide here is give our flask app which is going to be the folder within our flask app so this is text generator [Music] and you need to provide that information here so if this was named differently this would need to be named adequately and we also need to pass a pass flask n which is going to be equal to development and you also need to pass that variable as well so far so good we could actually start creating our model now which is actually the most interesting part uh and one more thing i have also uh created a wally image here which is a png image and moved it to our temp folder so we are going to use the open api gpt2 so in order to access this go to the following link paste in this link so i'm on github.com and as shepard gpt2 and go there so the difference between this one and our normal gpt2 repository from openai is that there are some helper libraries here uh like in code and um train and so on so that are going to help us um train our model with some data that we provide to it and then we're going to adopt the code a little bit in order to be able to run it with our application so you can just download this i'm going to download it as a zip here after it's downloaded you can extract the files i'm going to delete this one okay and now we're going to move this to our desired folder so what they have done here is i have just extracted this to gpt2 fine tuning and i have just pasted in the same source location that our text generator is at so now in order to work a little bit easier with our application we're going to do the following move the train to pi inside of source move in code also to source and this uh train slash horowitz pi also move it to the source folder now we are going to need to install all the requirements from our text here so open anaconda go to to fine tuning and type in pip install minus r requirements dot text so this is going to install all the requirements all the packages from our text file i have already installed the requirements so that's going to be fine and one thing that is not included in the requirements but that is extremely important is the is tensorflow so you are going to need to install pip install tensor flow but it's important to install a specific version of tensorflow and that's going to be 1.8 so um the package is a little bit outdated like not too much but there are a few functions that have changed so while i was testing it i either had to change those inside of our train file so for example if you go to our train file there are certain variables that are outdated with tensorflow which you could either update yourself or you could use a certain version of tensorflow that still supports all of that so i think it's better to use the certain version of tensorflow since it's going to reduce a lot of errors that happen throughout of that so really important to install pip install tensorflow 1.8 if you have another version it's going to delete the other version and install 1.8 so you can just hit enter requirements already satisfied so it's not going to change anything with my code here uh the next thing is to download our model so we can again go to actually the download model is inside of our source here and not inside of this source so just type in python [Music] download model.pi and now we need to specify which model we want to download so in this case we want to download the 117 million parameters model um there are i think three or four models uh the biggest one is i think the 1.6 billion parameters i'm not 100 sure but i think that's the biggest model but i would recommend to go with the smallest model uh since it's going to take a lot of time uh to process the larger models and you're going to need a better hardware now since i have a pretty mediocre cpu here i'm also going to go with 170 million parameters model since it's just going to be faster the results i have run the larger models in the cloud and the results are better there are you can really notice that the results are better but the process is completely the same and if you want to go with the larger models just download the larger ones and follow exactly what we are doing here so what we are going to do now is move uh our model that we have just downloaded here set the first source folder to keep everything consistent so the models then code model sample and train everything should be inside of our source folder now and the docker files are really not important the gift ignore files are also not the four important you can ignore those you could even delete those ones so what we need to do now is create a new file and call it uh training dot txt the training.txt file is going to contain all our data that we are going to train our app on so it's going to for example here we want to focus on the star wars universe and we want to specialize our model according to star wars so i have found something interesting where you could use all the data from which is this website here the entire star wars story finally explained so what i have done here is i have just copied all of that of course without the images and just paste that inside of the training the text file so the url of this is on www.looper.com entire star wars story finally explained so you could just google this and find the same article so when you paste this it's just going to be like that a bunch of text 737 lines of star wars you could of course use anything you would like this is just the example i went with so now run the following command python encode pi you're going to provide the training to txt file and we're going to output a training dot npc file so which represents an encoded model which the training script will use so in code oh okay my pad we need to move to the source folder because the encoded pi file is inside of our source folder so let's let's run it again uh you should you should see something like this this is going just to say that um the version that we are using of tensorflow is outdated but you can completely ignore that and you should see something like this reading writing files and after that you should have an npc file which should have some size so if it's zero kilobytes then something will draw and in this case things that can go wrong is if you have special characters uh something like this it could happen that it can't encode it because it's using utf-8 and certain characters that just can create problems but it's going to tell you on exactly which line it happened so you could just remove it but with this text this didn't happen now it's actually time to train our model in order to do that we're going to run python train dot pi and we're going to give it a data set so the data set is going to be our training dot npc file there are a few things before running this that need to be configured so in order to do that let's go to the train.pi file and as you can see um it's going to train for indefinitely until we interrupt it with the keyboard press and after that it's going to save our progress so this is something that from time to time crashes so it doesn't save it or doesn't work as it should so in order to limit that we are going to inside of our train.pi file on the line two five eight two five nine we are going to create a new if statement and we are going to say if our counter is less than a certain max number of iterations we are going to declare that variable so what you need to do now is you need to indent all of that like that so if the counter is smaller the counter is going to increase i'm using the counter that already exists in here and then we are going to have an else statement where we are going to save so we could also print something like print finished and then call to save on that and here we are still going to leave that but we're going to uh have this um have this if here because it can happen that this won't work on certain machines certain versions um it could happen that it crashes so we just need to declare this variable here which we could do here probably or it would be maybe cleaner to declare it somewhere like here and we're going to provide our max number of iterations to be for example 50. uh the higher the better what i found out is everything over 120 or so doesn't get you any new gains but around 50 should be nice so when we return here we should be able to run our train with pi file so again we can ignore those here okay as you can see um it's training now and it's going to train for the number of iterations that we have specified here after it's finished we should see the text finished here and our model should be saved inside of checkpoints here so inside of checkpoints run one our model should be saved so i'm going to leave this running and after it's finished we're going to continue so after everything is finished we can open our checkpoint here we have run one and those are the files that have been created so you should see a checkpoint counter events model data and so on so what we need to do now is the following here uh we have our model so what we need to do now is copy it and paste it again so we could just use this one but since we want to uh leave it there you're going to do it like that so we have copied the model and we can rename it into something like trained so you could name it whatever you would like but i'm going to call it 170m trained so and as you can see the model contains already those checkpoints and data indexes meta files and so on so what we need to do now is go again back to source go to our checkpoint or run one and just take everything that is inside of that and copy it to our new model so again go to models and go to trained and paste it inside of that and replace the files in our destination folder so don't worry about other files you could delete them or just leave them in in there it won't really matter the important thing is that our checkpoint uh let's open it inside of our models here the important thing is that our checkpoint is going to refer to this model 50 here so as you can see here our checkpoint refers to for example model the checkpoint and here it refers to model 50 the checkpoint so you could rename this as you would like to adapt it to some naming conventions you would like but this is the simplest way to go about this so after everything has been trained and finished we can actually start our model and see if it's going to work so in order to do that we could use there are certain um certain files that we can run one of the best one is the interactive conditional sample of pi which is located in the source folder here and as you can see the model name this is the default model that it's going to use but when running it we can provide it a different model and use that one so let's just run it in our terminal to take a test so run the following python inter active conditional samples dot pi we are going to top k set to 40 and now set our model name to the model that we have created so this is going to be 117 m trained and um give it some maximum length that we can produce so let's say 150 words so let's run that so here is our model prompt we could type in something like the jedi and wait for the output [Music] as you see the output takes a little bit and the larger the model the longer it's going to take um the film central idea would be to it would be to be able to infiltrate the jungle group via hidden network of spies in the desert where the members would work out various secrets secret corporations who are secretly plotting and directing their own actions so yeah you can you can see it um gives really interesting results so i'm not really sure if there is something about django uh okay yeah there's definitely something about django i'm not uh an expert in star wars sorry for that i just watched the movies once but yeah as you can see the responses can be really really interesting and fun and even useful so now uh since we have pretty much created our model here we have created our flask application it's time to combine those and to display our results inside of our flask application so let's do that so what we need to do now is go to our generator.pipe and think of a way how to get our interactive so what was it called interactive conditional sample how to get that code inside of our generator.pipe so first of all let us copy the entire file here since you have noticed it's working over the prompt we are typing in commands in the prompt and getting our results here written to the console so we need to change this in a way that our results are actually uh returned uh to return through our definition uh through a through a function to our analyze here inside of generate text so a few things we need to do is move our um move our model sample and encoder inside of here inside of our text generator so let us do that we are moving those because our interactive sample as you can see imports a model a sample and our encoder so we need to move those files so our sample model and encoder which is here so let us copy that and let's paste it inside of here so we have our sample model and encoder.pipe great there are a few things that we need to change for example here we are importing model which is that file here since when running with flask there are certain things we need to change there because uh when we run our flask application everything is initialized from this init.pi file and every everything is treated like a module which is inside of this text generator so in order to import this model we need to change the import here to from text generator import model so inside of the sample.pipe change this import to from text generator import model and this is pretty much everything we need to change in our sample file so the next thing we need to do uh is adopt our generator so let us start here we are going to so let's first change our so let's first change our imports here to from text generator import model let's do the same from text generator import sample and from text generator import encoder the next thing is you're going to define a class here so we're going to call this ai since this is going to be inside of a class we will need to indent this like this so it's indented and we're also going to remove this one since this has to run no matter if it's main or not i'm also going to remove this one here in order to have more space um now we are going to rename this into generate generate text remember the function that we are calling here generate text this is the function that we have created here and since this function is not going to take um any parameters besides the text we're going to change this into first of all we're going to provide a self variable here since we're in a class and we're going to provide a variable called input text let us remove this and also remove those commas from here we will also define which model we are using so this is going to be our 117 m trained this is going to be 40 and our length is going to be 150 words so we can pretty much leave this as it is okay and this is also going to be a problem when running uh with our flask application uh we will need to do the following we first will need to find our current path which is going to be os path directory directory name from our current file so this is going to give us um our current uh location here where our generator.pi is plus um we will need to go into models plus we will need to go into our model name which is this here so we are going to provide our current path here and just add a plus to hp params.json so uh we have just replaced we have done nothing uh fancy here we have just replaced um this join with our current path plus our models folder plus model name which is a 117m trained plus hp per amps so this is just going to go into models 170 nm trained and it's going to find for example this hp params.json so which reminds me that we actually need to copy those models the model folder inside of our text generator here paste so now we have the models 100 uh 117m and 117 am trained and inside of that there are our params so do this everywhere um where we have this joint thing for example here we want to go only models and model name so in this case we could just go with current path here so this should be fine the the thing we need to change now is this while loop here we're not going to run uh endless loop here we are going to change that so first of all let us define a variable uh for our class which is going to be self response which is going to be an empty uh string at the beginning so what we are going to do now is remove this here and remove that indent so we are also not going to have any road text here since we are going to get the text through input text so we can completely remove those parts so we are still going to get our context tokens so we are going to tokenize our input here from our input text we are still going to leave this as it is and this is actually our response here so this is this is what we get so we are going to remove all those print statements here and just give ourself response is going to be equal to that text and then we are just going to return our result here just should be here i think return [Music] self response so i think this is no it should be here okay great so this should uh return our response here we have just adopted this uh piece of code a little bit to work in our case now we need to initiate it with a i is equal ai and then we could access [Music] this through a i [Music] generate generate text here and we could provided something some input like the jedi and then this would be equal to a variable and text and we could print out this text there let's actually see if this is going to work so in order to check this let us exit this here [Music] return to our source uh now navigate to text generator and um we could actually run our python generated pi now okay we forgot to add colon here let's check out now return outside of our function [Music] so this should be inside of here [Music] okay so this is going to make a problem this is going to run only in flask like that so if you're going to try it to run like here then we would need to change our import but since we have already configured our imports we are going to run it with flask since we are pretty much finished with the code here uh we can oh there is one more thing to do uh we need to uncomment this here so we can import it now we could actually uh go and run this so for that type in flask run and navigate to the root folder so don't go into the text generator but go into the root folder and run flask run from there so where we have our environment files and now open the browser go to localhost 5000 and see if this is going to work so okay we made a mistake there that is easily fixed let us refresh okay this error is a bit strange the problem is that this is actually the wrong sample file this is the sample file from our gpt2 fine tuning here should open this sample file here so from text [Music] generator import model so i hope we have uh haven't made the same mistake again somewhere set of our generator seems also fine let's try to refresh okay um it says argument must be a string or bytes not tuple this is because we have this comma here we haven't deleted the comma but this is all good so you can see what errors can happen okay this should be the last error i think we need to fix uh this is inside uh i think inside of our encoder so let's go to our encore encoder.pipe and this is the problem we have fixed earlier with the path the join so we know we need to do the following we need to have this current path here let's create our current path which is models model name and then we just need to add current path to that so let's do the same here [Music] current path remove this one and let's check out now okay um we are missing still a slash so we have our models model name and we would still need a slash here so have we made the same mistake over here as well model name hp params [Music] so yeah i hope that should fix it now okay um it seems to be working now we can enter our text let's try something like the uh sith lord press enter to submit to our file now we need to wait a little bit until openai generates a response and let's check out if this is going to work out great uh we got a response here um of the troll thumler the king of the rich and the most ambitious of the draw thumler is in charge of all skyrim's most powerful warriors as head of one of the atari's most editorial holdings okay skyrim it went for some reason to skyrim um still interesting but as you can see it's working now uh we are receiving a response here from openai it takes a bit of time to process in sphere using the 117 million parameters model it's even the fastest one so with using the more complex one it can take even longer so let's for example try the jedi here and click on submit so as you can see it when we are clicking submit it goes to the analyze that url and after we receive a response it's going again to return us back the index.html file pass the text and the text is going to be displayed in here okay uh we got a response here and jabba the hutch jabba had an affair with princess leia's second cousin jabba palpatine he gave her a secret but she refused to reveal it in the end he was forced to kill his fae to his former master and even through he'd got a new apprentice he never got his wish so that's that's really amazing and interesting um so you can play around with this um you could train it on different sets you can go with the larger model and let me know what results you get let me know if you have any errors or need help so i hope you enjoyed this one and you learned a lot on the way and you got familiar with uh the open ai framework so thank you so much for watching i hope you enjoyed it and you learned a lot on the way see you in the next video [Music] you

Channel: NovelTech Media

Views: 10,517

Rating: 4.970696 out of 5

Keywords: text generator tutorial, text generator gpt, text generator python, open ai gpt, openai gpt, openai gpt2, gpt2 tutorial, build a ai text generator, build a text generator with openai, build a text generator with python, python tutorial, python machine learning, build an app with python

Id: WRVx8vGDKf4

Length: 53min 46sec (3226 seconds)

Published: Sun Sep 20 2020