ChatGPT Plugins: Build Your Own in Python!

open AI have just announced the release of custom plugins for chat gbt now what does that mean exactly well we've come over to charge upt we can see on the top here we have our model we can go on to here and click on plugins and you can see now that we have this extra little drop down for plugins and there is this line chain dots plugin now that's the plugin that I'm going to show you how to build in this video but I just want to show you the other plugins as well okay so you come to this plug install and there are basically a ton of these different plugins these are the featured plugins and yeah there's quite a few there but then you can also see at the bottom you have develop your own plugin install the unverified plugin and so on so you can actually develop your own now what do they actually do well if you've ever heard of Agents then this is a very similar concept if not no worries I I haven't really spoken about it before so we'll just do a quick 101 on What an agent is so an agent in the term of large language models it's kind of like a tool right so imagine as a person right I have a complex math problem in front of me I'm I'm not just going to rely solely on my brain which would kind of be the equivalent of the large language model in this case I am probably going to get out a calculator or my phone and I'm going to type in the calculation all right I'm using a tool agents are kind of like the same thing but for large language models and they can't do very similar functions so you can get calculator agents so large launch models are typically not that good at performing complex calculations so what you can do is tell what the large language model if you need to do a calculation I want you to use this calculator tool and to use this calculator tool you need to write some text which is going to be like three plus four to the power of 4.1 and we will take that text that you wrote we will give it to the calculator and it will respond back to you with the the calculated answer and then the large language model can relay that back to the user okay so it's literally a large language model using a tool that is what these plugins in chat GPT seem to be right I don't know exactly how it's working because I I've no idea it's open AI as far as I'm aware there isn't any documentation on how this works at least not yet maybe there will be some coming soon I'm not sure but nonetheless it seems to be working very similar to an agent so what can we do I'm going to show you the Lang chain dots plugin so let me first disable it and I'm going to say I'm going to default to my typical example of how do I use a lamp chain in line chain okay and we can we're going to have some more complicated questions later but I just want to start with this one because it's relatively simple but you don't need to understand it anyway okay now you can see here as of my knowledge cutoff date September 2021 line chain is not a specific software library or package I'm aware of I'm not familiar with the term LM chain so on and so on basically is telling you I have no idea what you're talking about okay fine but because line chain is a new library I I'm not surprised by that whatsoever so let's start a new chat let's enable the line chain dots plugin and let's ask the same question again so what is the LM chain in line chain let's see what happens now okay so it says using Lang chain docs okay Lang chain is a python library that facilitates the development of application using large language models how cool is that using mm's isolations may not be sufficient for creating powerful applications so and so on introduces concept of chains chain alarms together so on and so on right so it's kind of describing what llms are what these chains are in line chain and so on right and then it also gives you some links to documentation so I can try and click this all right cool all right so I I get direct straight to that and we can also lens documentation okay straight away we get these sort of things and if I go into the next page from the previous one we actually get this aquarium with the LM chain right so that's pretty cool right now how how did we do that well we did that with this plugin and what I want to show you is how to actually build that and Implement that yourself so let's let's jump straight into that so the first thing we're going to start with is getting a template application or template plugin application now for that we're going to be going to the open AI chat GPT retrieval plugin repo which you can see here now from there what we want to do is go over and Fork this repo so that you have your own version of this in your own GitHub from there once you have that you navigate to your repository then you click on code you want to have clone here you copy this and then you want to head over to a terminal window and what we're going to do is just git clone the repo that we just saw okay so get Clan chat GPT retrieval plugin once you've done that we should see that we have chat EBT retriever plugin we navigate that okay so here we can see the directory essentially all this plugin is is going to be a duck container where we are deploying an API that we will allow GPT to interact with so let's go through and try and understand what it is actually doing okay so I've opened up the the directory here I want to mainly focus on what we have inside the server here okay so we have the upset and the query now what these do is this here so we have our let's say we have our API it's going to be here okay so this is the plugin that we're working with at the moment right on this side let's say this side is us we have the upset and we have query okay those go into the API there are two endpoints that we're going to be interacting with okay so here we have the API that we're going to be creating okay it has this up certain this query behind the API there is going to be a vector database okay this is a Pinecone component we're also going to be taking our line chain dots so Lang chain dots and we're going to be giving our large language model access to those line chain dots via this whole Asian thing that we have going on here right so there's a few components here then we also chat GPT down here and let's just figure out how they all go together all right langjin Ducks Pinecone chat gbt API what we are going to do is we're going to take all of our lane chain dots we're going to run them through a python notebook here we're going to download them and we're going to throw them into this upset endpoint here what that's going to do is it's going to come out on the other side it's going to go to an open AI embedding model so we can how do we how do we write that I'm just going to put AI embed right and that is essentially converting the text in those line chain dots in to meaningful numerical representations of that information okay and they all get stored within our Pine convective database here right so this pinecon Vector database is now a potential source of information for chat gbt down here okay but we still we still haven't made that link yet there's no link between GPT and that that database that's where the query part comes in so given a particular query like the earlier example is what is the LM chain in line chain we'd ask we'd actually charge gbt chat GPT is going to come up here it's going to think about it and it's going to be like I have no idea what that means but I have some instructions that tell me if someone asks about line chain I should send that query over to this thing over here this query endpoint okay so then our API gets that question what is the LM chain in line chain is going to bring it over here it's actually also going to take it into that same AI embedding model and it's going to get to what we call a query vector okay so this query Vector here this query rated then goes to Pinecone and Pinecone says okay I've seen sip things that are similar to this before okay in in the line chain dots that we embedded before there are some documents that seem kind of similar they mentioned the LM chain I mentioned line chain chains and all these sort of things I'm going to return those to you okay so it returns those documents we're going to what we're going to do is return five of those at a time okay so we then return nose back so through the API we return those through the requests and they go back to chat GPT chat GPT now has information from panic and it has information from the line chain dots so it has its query and it also has this extra information from here up here which we're going to call the context and now it can actually answer a question okay so then it gives that back to the user the query in context and just finishes like completes the answer based on that new context that it has so it now can actually answer questions that is what we're building here right this this plug-in this API here is what enables all of this what enables the interaction between chat GPT and the outside world so the first thing that we need to do is make this API accessible we need to deploy this API so let's jump straight into that so we are going to deploy it I mean obviously it depends on on your own setup but for the safe Simplicity what we're going to do here is use digitalocean so dish lotion uh we can you have to sign up if you haven't used it for otherwise log in once you have signed up you should be able to make your ways if something looks like this you you might have to click on the side or click through a few pages in order to get to this but what we can do is everyone to come to here and you can see an app that I've already created here that is the app that I showed you before we're going to just recreate a new one okay okay and we can come down to here and what we want to do is actually create a web app so I'm going to click on create up here apps we're going to create a resource from source code we're going to be using GitHub this you will probably need to authenticate so to allow digitalization to access your guild account and then in your repo here you're going to need to select the repo that you fought from open AI so I've just selected chat cheap C retriever plugin Branch main The Source directory is just a top level directory and we can Auto deploy as also anytime we make change to the to the main branch it will just be redeployed that's useful tab okay so we have dolphin app I think that's just kind of like the the app which will contain the the plugin uh yeah I think that all looks good right and then we have environment variables so this is where we do need to make some we need to add some things right so I'll explain these as we go through them so there are two specific environment variables that are we we need to use these for the app in general okay and these are the bearer token and the open AI API key okay both of those are required by the app and then there are a few more environment variables that we need to specify that we're using the using Pinecone as our retrieval component which I will explain a little more later on and up to give us access to that so we our data store that is going to be pinecan we're going to need a Pinecone API key we're going to need a pine cone environment and we're also going to need a Pinecone index so Pinecone index you we can name us whatever we want I'm again I'm going to explain a little more detail very soon so I'm just going to call mine chain dot plugin right and then everything or not not quite everything so the bearer token this is a secret token that is going to be used by our plugin in order to authenticate incoming requests right so we're going to include the bearer token within the headers of our API requests open our API key that is so we can encode everything with an open AI embedding model the date it's always just saying that we'd like to use Pinecone and again I'll explain that in a little more detail this will give us access to Pinecone this will specify the Pinecone environment that we need to use and this will specify the panko index if none of that makes sense it's fine I'm going to explain it as we as we go through that for now all I'll do is just show you how where you need to find each one of these items so for the bearer token as far as I know you can just pay anything you want in here but the sort of standard practice that a lot of people go with is to use a Json web tokens I think it's called so JWT yeah Json web tokens and you just come down to here and you would put in something um so you know I'm not super familiar with this so I don't know what the best practice is to do here all I'm going to do really is kind of make something up so I understand to put in like something like okay my name that's gonna be James Spriggs or something and I'm gonna put in some other random things right and I'm just going to see what I get for the bearer token here there is probably a best practice doing that but I I personally don't know it now the other thing is your openai API key you need to get that from platform openingi.com you go to your account in the top right and then you can click view API keys and you just get your API key from there other things the Pinecone API key for that you need to head on over to apple.pinecone.io obviously you may need to create an account but everything is free so you shouldn't need to pay anything at least not if you're just creating a single index here and you just come over to here so API keys on the left you have your environment here which we need so let's just copy that now put that there and then we also have and here are our API key so we will just copy that and then paste that in there as well and then once you've added all of those you would just press save okay and you should have to see this environment variables there so we click next okay or it's good you can change the region depending on what is you where you are what you're wanting to do and oh yeah cool all right and then they'll come up with this monthly app course I believe this is predicted rather than a specific or that I could be wrong there but one thing that you should be aware of here is that you will hopefully get some free credits so you can also use those so you shouldn't need to pay anything but don't quote me on that so I'm going to create that resource okay and then that will take some time uh to deploy okay so our deployment is is live now so what we can do is begin adding data so we refer back to our our earlier drawing here you can see we have the the first literally the first set is line chain dots through this python code to the upset of the API so that it goes to Pinecone that's what we now need to do okay so we head over to co-lab and there is this notebook here there'll be a link to this notebook so you can follow along but we're not going to run through the whole thing we are first just going to install any of the prerequisite libraries right now that is just this and there'll be another one a little bit later okay and there's this whole preparing data stage I don't want to go through all of this I've actually included this in another video for which there will be a link uh somewhere in the top video right now or you can just kind of follow along with the notebook if you like to but there's a lot going on there so let's skip ahead and what we're going to do is directly load the data set so the data set that we've built from hugging face today assets so to do that we need to pip install data sets we then load data set like so okay and then we need to reformat this into the format that is required by the API component okay so what does that format look like it is somewhere in this notebook here okay so we're going to have these three Fields ID tapes and metadata metadata is going to contain like additional information that we might need the text is going to contain obviously the text and then we have the UniQue Ideas here all right so we need to create that format let's have a look at what we have at the moment okay so we have ID text and source so it's very similar we have ID and textbook now we need metadata so if we just have a look at one of those items maybe it'll make it clearer what we need to do so ID that's fine these are unique ideas the text that's also fine problem is the source at the end here so we need to put this we basically need to just create a metadata dictionary and put this Source inside that so that is actually what we do here we just rename Source URL run that okay and now we have our the correct format that we need for our API okay so we have ID and text like we did before but now we have metadata and then in there we have the URL and the link okay cool now we're ready to move on to that indexing part right so because we're going to be interacting with our API we do now need to include the bearer token so this is basically how we authenticate that we are allowed to interact with this API so that Bearer token that you created earlier you need to add it in to here okay so you just put it replace this Bearer token here with your bear token or if you want to say as an environment variable you can use this part here okay I I've said that for mine so then I'm gonna move on to the headers and the authorization here and bear bear token right so this is we're going to add this to our request so that it knows that way it authorized to actually use the API and then here we need to add in our endpoint URL so this is for the previous one that I created now I need to update this to the new one which is just here right so this dolphin app we can open it so if I open this we're going to come through to this page here where it just says detail not found that's fine that's kind of expected so copy that web page and we just paste it into there okay cool so what are we doing here let me let me walk you through that so here we're setting up a where basically what we could do if I want to kind of explain this in the simplest way possible is we could actually just remove all of this pretend it isn't here and we could just put requests dot post like that right and that would post everything so we'd be going through all of our documents in batches of 100 and we'll be sending them to our API right the reason that we have this other part it's extra little bit here is every now and again what we might see is we might see an error when we send our request okay and we don't want to just cancel everything if we see one error one time because usually it's just like a brief error and then you send a request again and it's going to be fine that's what we're setting up here we're saying we want to retry the the request a total of five times and what we want to do so we don't overload so we're not just sending request after requests super quickly we also had this back off Factor so it's basically it's going to wait like a tenth of a second then it's going to wait a fifth of a second then a second and then a few seconds something like that and then the the status codes that we will do this for all these so these are like internal server errors right that's all we're doing here and then we're setting that up so that our post requests use this retry strategy that's all that extra code is doing there okay so we can go ahead and run that and this is just going to send all of our documents I'm going to put them in the API uh what we've done here tqdm so I just need to import that so from tqdm.auto import tqdm okay so this is you understand all those documents to our API our API is going to send those to the open AI embedding model so that is text embedding order 002 and then it's going to get those embeddings those embeddings are going to be then stored into the pycon vector database that we can then refer to at a later point when we're making our queries through this chat GPT plugin okay so I'm going to get a high latency here because I think my colab environment is probably running somewhere in the US and I I kind of set this up to be in Singapore which wasn't very intelligent on my part so I'm gonna go and wait for that and we'll come back when it's ready okay so that has just finished I what I am doing is switching to I think I went to San Francisco in the end hey yep here so I just created a new app switch the location and obviously run and much quicker so we've got like two minutes there in set okay that will also mean hopefully I would expect that this will mean it will run faster with chat GPT as well Okay cool so with that we're ready to move on to querying so this is basically what we are going to be implementing inside chat gbt okay so here are a few examples we have our queries we're going to have a query what is the Lem chain in Lime chain how do I use pine can in line chain what is the difference between Knowledge Graph memory and buffer memory for conversational memory all right so if we run that we should hopefully see that we get a 200 response cool and then we can print out and we can see you know what is actually being returned here so see what is it a LM chain in line chain and we get like these documents from those web pages okay so we have these are increasing orders going complexity a lens and prompts we have chains and then another one is is talking about chains again how do I use pine cone in line chain uh we we actually get this really short it almost it almost feels sarcastic it's just like from line chain breaks towards in Port panko yeah that that is true um we would hope for a little bit more information but it wipes so Pinecone contents and then here it says this place goes how to use Pinecone ecosystem within line chain so that should hopefully goes a little more information and then we'll come down to here what's the difference between Knowledge Graph memory and buffer memory conversational memory then we get conversation memory graph memory contents and talks about the knowledge graph memory and then I'm hoping at some point it might talk about buffer memory and and so on and so on so that all looks pretty good now how do we integrate this into chat gbt so we'll just jump straight in chat GPT and we're going to try and add it as is without any modification so first thing I'm going to do is just I'm going to uninstall this one so I'm going to go to my unverified plugins I'm going to click uninstall okay and to create a ram plugin we need to go to develop your own plugin you say my manifest is ready let's just it isn't but let's say it is okay please provide the main view website where the AI plugin Json file is hosted okay so for that we just need to copy our web address here and we place it in here okay it's a find manifest file and we're going to see okay there's some errors in our manifest what does this say manifest API URL and that your app url.com okay it says it is not under the root domain Lobster app so and so on okay and we can see all this so okay let's try and figure out what is actually going on there okay so this here is uh we saw the main.pi before all right this is our directory for app we go to this well known and we can see that we have this Ai plugin.json and in here we have all this information about our app right so you can see here we have that default web address here we need to fix that so I'm just going to copy this and I'm going to put it in here and there are also a few other things that we need to update so we have the name for the model and name for the human Okay so name for the model is a name of this app that the the model the Jack gbt will see so we're going to change this to Lang chain dots DB database and the name for the human will just be long chain dots right and then the description for the model now this is important it is it is prompt engineering right this is the description that the model is going to use when deciding if to use this tool or not okay so we need to specify when the model should use this tool and when should it use this tool well we should say use this tool to get up-to-date information about the line chain python Library okay now that is pretty explicit we're saying when to use it all it's sometimes these tools can get used too much without a large language models okay so I I also just want to be super careful and say do not use this tool if the user did not ask about line chain okay and that's it that's all I want to put in there as we'll see later on this isn't the only information that chat gbt gets about this tool and and we'll explain that soon so the description for the human what should this be I want to say basically the same as what I said to the model but a bit shorter and nicer so I want to say up to date information about the line chain python Library that's it okay so and that should all be good so we're going to save that and I'm just gonna come to the the get repo again get status okay I'm gonna add that they said the change we just made and I'm just going to commit it to the rebound okay so now if we just go over to dot well-known AI plugin Json we can actually see the and the Manifest file that we just spoke about okay and we can see that at the moment it hasn't been updated that's because it takes a little bit of time for this change to propagate through everything and we can see here that it is building the service again because I just pushed a new deployment right so that is just going to take a moment to finish so we'll just skip ahead to that okay so the deployment's just gone live so if we come to here refresh we should see those changes in there okay looks good now let's try again with this so we're going to try and refetch the Manifest okay the Manifest has been validated now looks good we can see you have long chain dogs up-to-date information about the line chain python Library we have the open air logo there we can we can actually change that if we want to by just switching the logo that we can see here but for this demo it's fine so click done okay we click over here we go to plug install we say install a unverified plugin and this is where we need to take out URL again we paste that into here click find plugin they're going to give up give us this little warning and then we also need to enter your HTTP access token below now this is going to be our Bearer token that we we created earlier so we need to get that Bearer token yet again okay and then we can install the plugin okay great so that is now that's not activated so we can go ahead and we can say what is the llm chain in line chain and we're going to see this okay could not pause open a API spec for plugin okay so we're seeing this kind of weird error now as far as I can tell this is just an issue with the current version of the open API spec that is currently being used so we're just going to go in and fix that okay so we have open API dot yaml here we go into here first okay straight away we can see okay the URL isn't correct right but there are as far as I can tell there are a few other issues with this so I'm just going to go in and we're going to fix those okay so I've just copy and pasted this new open API dot yaml in this one works for me there'll be a link to this in the video in the top right now in case you find that the previous one also doesn't work for you you can just use this one so there are a few things that we should change here like the URL the description and so on okay so we're going to say this API lets you search through the Lang chain python Library docs documentation we should call it retrieval plugin now I don't think we should call it or is it Lang chain dots API Maybe and for the URL the URL should be our the one that we used before so we actually have it in the AI plugin so we're just going to take this and we're going to put it here Okay cool so I've tested this a little bit and it seems that chat gbt is actually also reading this openapi.yaml and it seems to be using that in order to inform how it's creating the the queries to this if you're having problems with the query you should update it so that it knows what sort of format it should be using okay so there's one thing that I would like to update here which is this description okay it's quite long at the moment so I'm just going to shorten it a little so I'm saying this is the description that is kind of telling you how to use a query request and we just say this is an array of search query objects each containing a natural language approach during query and an optional metadata filter if you if you'd like to include that and it describes how to use the filters and so on right okay that should be all we need so I'm going to make those changes so I could say it gets add and this is just update the open API yaml foreign okay so that is going to be building again we'll wait for it to deploy and we can also check over here for the the yaml file so it's open API yeah okay we can see that we still have the the old version there so this oops wait a moment it will update soon okay once it's deployed we can refresh this page everything will be there now and okay so let's use uh what is the LM chain in Lang chain we're going to solve this one here we need to make sure we're using the plugins make sure our plugin is active over here and search okay cool okay this is more specifically talking about chains in general so we can we can probably modify this to try and return more items but for now we'll leave it there and let's just try and ask how do I use pine cone in link chain see what this comes up with inline training huge pine cones are better to solve tests such as semantic search or example selection that's kind of cool they got both of those to use paint and line chain you need to install Pinecone python SDK and import it from here so pep install power inclined from this use this okay it looks pretty good we can also click here for the python documentation which also takes us through a little bit more but I want to ask one more question so new chat and I'm going to say what is the difference between Knowledge Graph memory and buffer memory for conversational memory in light chain we should just specify okay let's see this is a hard question I I'm not really expecting too much here but we can see okay different types of memory two which are conversation Knowledge Graph memory and conversation buffer memory okay let me let it generate and then I'll come back to it okay conversational Knowledge Graph memory users Knowledge Graph to recreate memory you know it's actually a pretty good summary for example views says say Hi Sam and then ask who is Sam the user can use the knowledge graph to remember that time as a friend okay that's kind of cool conversation on both memory keeps track of a sequence of chat messages okay that seems pretty good in summary conversation Knowledge Graph memory uses a knowledge of structure and organized information from the conversation while conversation buffer memory maintains a sequence of chat messages for short-term memory both times memory used to provide Concepts aware responses based on previous interactions with the user I think that's all actually pretty accurate and I'm kind of surprised it got that to that degree that that isn't that isn't too bad at all okay so that's it for this video exploring the new plugins for chat GPT there's quite a lot to cover there to be honest uh it's not it's not it's not super straightforward process because obviously there's a lot of different components to this that we need to consider but that being said given what you can do with this I think it's actually not as complicated as you you might expect so I think particularly as people get more use to chat gbt plugins like it's brand new so there's obviously going to be some things that need ironing out and and best practices that need figuring out and so on but I think once people get a lot more used to it this sort of thing will become a lot more natural especially for people that can develop in these things all the time so anyway that's it for this video I hope all of this been interesting and useful so thank you very much for watching and I will see you again in the next one bye

Channel: James Briggs

Views: 83,533

Keywords: python, machine learning, artificial intelligence, natural language processing, bert, nlp, Huggingface, semantic search, similarity search, vector similarity search, vector search, chatgpt-retrieval-plugin, chatgpt, chatgpt plugin, chatgpt python, chatgpt plus, james briggs, openai, openai chatgpt, openai chatbot tutorial, chat gpt tutorial, massive upgrade to chatgpt, ai chat, ai news, chat gpt ai, gpt 3 chatbot python, gpt 4, gpt 4 chat, chatgpt 4, openai plugins, gpt 4 demo

Id: hpePPqKxNq8

Length: 41min 6sec (2466 seconds)

Published: Fri Mar 24 2023