Prompt Templates for GPT 3.5 and other LLMs - LangChain #2

today we're taking another look at the Lang chain library and we're going to be focusing on what are called prompt templates which are a very core component of the library and this mirrors the usefulness of prompts for large language models in general now although prompts maybe don't seem as interesting as the models themselves they're actually a very critical component particularly in today's world of large science models the reason I say that is in the past when we consider different tasks within language we all use different models for those so name entity recognition or question answering there were different models trained for each one of those purposes now the separation between these tasks has over time decreased with the introduction and transform models it became the case that you would pre-train a single big language model like Bert and then you would form transfer learning in order to just change a couple of layers at the end of the network in order to adapt it to different tasks and with the more recent adoption of large language models the separation between different use cases has decreased even more so now rather than actually using different layers at the end of of the same model like we did with transform models or just using completely different models as we may have done before using transform models we now actually use the same model for completely different tasks now things like named entity recognition question answering summarization even translation are all done by the same models the only thing that actually changes now is the prompt the input that we feed into the model we literally just say can you do this or can you do something else right that is all we're changing now so the models themselves no longer need changing it's actually just the inputs to those models that change in order for us to modify the task that we are performing so with large language models it turns out that prompts are the most important thing for us to learn the Live Science models themselves have already been trained sure we can fine tune them sure we can add an external knowledge base sure we can do all these other things but in the end one of the key components for us to learn when we're using large lines models is how to do prompts correctly the line train Library recognizes the importance of prompts and they have built an entire object class or a few object classes for them in this video that's what we're going to talk about we're going to talk about prompt templates and a few of the other things that are kind of parallel to prompt templates so we're going to get started with just taking a look at this simple prompt here and we're going to break it down so at the top here we have the instructions of the prompt here we have context or external information here a query and here is what we call a output indicator now each one of these component units serves a purpose but actually when we look at maybe what we would usually put into a large loans model or what a user would put into a large loans model it's only this little bit here okay this little bit which libraries the model providers offer large models that's all we're actually expecting our users to put in there so that would be you know our query and what we're actually doing here is considering that that's the only thing our user is going to be inputting we're actually providing all this other information in order to kind of guide the large lines model to answer the question in the way that we think our user would like the question to be answered in this case we're doing what I would call factual q a which is what you can see here answer the question based on the context below so this based on this information here if it can't be answered I want you to say I don't know okay that's what I would call factual q a so we basically don't want to answer the question if we can't find the information behind that answer and the reason we might want to do that is because low drainage models have this very strong tendency to make things up and make it seem super convincing so it can be good to do this sort of thing in order to avoid that now let's go to our code there are a few things that we need to install here so pip install line chain and openai of course you can do this with other Frameworks as well it doesn't have to be open AI you can use cohere you can use hugging face it's completely up to you but for what we're doing here the open AI model is is very good so here's our problem this is exactly the same as what I showed you before so I'm going to run this and you can see I just can't explain what I just explained there instructions context question output indicator so using line chain uh first thing we want to do is actually initialize a model so I'm gonna go open AI API key here so I've already created this variable this just contains my API key which you can get from if you click on this link here there will be a link to this notebook either at the top of the screen now or in the video description otherwise what you can do is head over to this web address here it's a platform.openai.com account API keys that may change in the future but for now that is where you would go so we initialize our model I'm going to run this and then what we can do is we're going to just make a generation from the Pronto we just created so the The Prompt up here right this is just going to it's going to extract a few things okay looks good the only problem is like we wouldn't typically want to write all this within our prompt in this format right so like facade is here is the user's query so the user should be inputting whatever is here and as well as that we have these contacts here this would actually come in as an external source of information so we also wouldn't hard code that into our code either so line chain has something called prompt templates which will help us handle this now for now I'm just going to keep the contacts in there but what I'm going to do is replace the the queried users query with this just it just um kind of looks like an F string in Python but it's not an S string otherwise you know we would have this up here in fact it is actually just a plain string but it will be interpreted Like That by The Prompt template so what we need to do here is just replace the where we would expect the query to go with query and we need to make sure that within the input variables of this prompt template object which we have imported here we need to make sure that this here aligns with our F string type thing here and then after that we just have our template which is obviously just this here and that will create our prompt template now if we would like we can just insert a query here okay so you can see what I'm doing we have promptemplate dot format then we have this query which is just going to be the question that we had before right and we can run this and print it and we can see that we now have the same text but now we have our query in there instead and we can change this like what is a large language model or something what is a live zones model right we could put that in there and it would change our query here now in this case we don't actually have an external knowledge based setup so the context doesn't change that's fine this is just an example we don't need to worry about that right now so what I'm going to do is take the the first example where I had prompt template and we have the actual question and we're going to feed it into this open AI object here this here is actually a large language model that we just initialize and if we run that we should get this here okay so we basically within you know these few lines of code we kind of just replicate what we did up here but a lot more dynamically so let's come back down and what I'm going to do is show you why we would actually use this because honestly right now it seems kind of pointless for example we could just put this as an F string and write some little little code around it it wouldn't be that hard so what is the point I'm actually using this well one it's just nice it's easy it's clean but two this isn't the only thing it does so if we come down we can also do something like this so This is called a few shot prompt templates now this G-Shock prompt template object is ideal for doing something that we would call few shot learning for our large language models and what few shot learning refers to is the idea of feeding in a few examples into a already trained model and essentially training it on those few examples so that it can then actually perform well on a slightly different domain now the approach to few shot learning can vary in the more traditional sense it would be that you're feeling in a few items to the model and training on those few items as you usually would train a ml model in this case we're actually feeding these examples into the bundle via the prompt but this actually it seems weird but it makes sense because with large language models there are two primary sources of knowledge those are the parametric knowledge and the source knowledge the parametric knowledge is the knowledge that the large zones model has learned during training installed within the model's weights so something like who is the first man on the moon the model is going to be able to answer Neil Armstrong because it's already learned that information during training and it's managed to store that information within the model weights the other type of knowledge Source knowledge is different that is where you're actually feeding the knowledge into the model at inference time via the model input I via the prompt so considering all of this the idea behind line Chain's few shots from template object is to provide future learning via the source knowledge via the prompt and to do this we just add a few examples to the prompts that the model will then read as it's reading everything else so you remember earlier on we had the instructions context query and output indicator in this case it would be like we have instructions examples query and output indicator now let's take a look at where we Mount to use this now in this prompt I'm saying the following is a conversation with a AI system it is typically sarcastic and witty producing creative and amusing responses to the questions here are some examples actually we're not doing that yet so let's remove that so this is all we have right so we have the instruction and then we have what we use this query and then we have the Apple indicator we set the temperature here to one so that just increases the randomness of the output I.E it will make it more creative and then we can run this right and we get the meaning of life is whatever you make of it I mean to me it's not sarcastic it's not witty or creative it's not funny so it's not already doing what I want it to do so what we can do here is do few shot learnings so this is the same I've just added here are some examples onto the internet and then I'm just adding a couple of examples so kind of like sarcastic responses to our users questions how are you I can't complain what time is it it's time to get a watch and then I'm going to ask the same question again at the end and then we'll see what the model outputs and it's not perfect but we are more likely to get kind of like a less serious answer by putting in these less serious responses now we can probably fine tune this like we can say the assistant is always sarcastic and witty here are some examples like we can cut this bit out and that might might help us produce more precise answers and here we get quite a sarcastic answer of you need to ask someone who's actually living it which I think is quite good try a few more okay somewhere between 42 and a double cheeseburger it's good 42 again 42 again and so on so we're getting pretty good answers I think we should have gone with this prompt from the start now we come down here what we can do is just show you how these few shot prompt templates work so import future prompt templates and we create these examples the examples each one of them is going to have a query and an answer okay so you can see that here this here would be our query and this would be our answer okay so we initialize that and then we pray what is called a example template same thing as before it looks like an S string but it actually isn't or it's not yet so we use the example template and what we do is we actually create a new prompt template based on this example template okay so we're creating like a example prompt so it's going to take in the query and an answer this time then we need to break apart our previous prompt into smaller components okay so I'm going to here are a few samples we're going to use the same one as we used before so this is just the instruction and then the suffix here is essentially well actually we have two things we have the query itself that the user is going to put in and then we have the output indicator then we go ahead and actually initialize our fuse shop prop template we have our examples which is this list up here also one thing we should note here is that these for every single example needs to line up to this okay we have our example prompt which we have initialized here we have the prefix suffix input variables right this is not the same as what we have coming into here because this is actually just a query from the user so it needs to satisfy this part here and then we have this example separator so sample separator is just what it's going to use to separate each one of those samples within the prompt that we're building so let's run this and we're going to say what is the meaning of life again and we'll just print this out so we can see so the following excerpts so on and so on this is the same as before and you can see that was separated each one of these are two new lines we say you know we have all those examples that we fed in through our list okay and then to generate with this we do the same again okay so we have our future prompt template we use format query let's run this okay run again it doesn't like whatever I've done to the prompts so let me come here are some examples so let's change this to some examples I don't think that should make a big difference I'll just change the separator a little bit as well okay and then we get our joke answers 42 again Okay so we'll get a few good responses again it's not perfect but it's just an example now what I actually want to show you now is why would we also use this over just feeding things in with an F string well it's also a little bit more logic that we can use so in a lot of cases naturally as with typical machine learning models it's better to feed in more examples for training than less examples and we should try and do that as well with what we are doing here whether it's with feeding the examples into our prompt so what I've done here is created a a lot of these kind of examples and we're just going to okay yeah we can just run these now we're going to want to feed in as many of these examples as possible but at the same time we might want to limit the number of examples we're actually feeding into it so there are a few reasons for this one we don't want to create excessive text that are separating the instructions and the query itself sometimes that can be distracting for the model and on the other hand we can actually add in so many examples that we exceed the maximum context window that the model allows so that's basically the number of tokens from your query or from your prompt and from your generation you add those there together and that creates your context window every model including the Open Air Model we're using here has a maximum context window and we can't exceed that otherwise we're going to throw an error so we definitely don't want to go over that limit and another thing I want to consider is that we don't want to use too many tokens because it costs money to run this so we might also want to limit the number of examples we're bringing through because of that and we might want to limit number of examples based on how long the user's query is so if the user just has like a small three word query we can include more of our examples the user is like kind of writing us a little bit of a poem then we might want to limit the number of examples we're bringing through and that is where we will use something like this so there are a few of these what we call a sample selectors the most basic of those is called the length base example selector with a lamp based example selector we would feed in our list of examples we would feed in our example prompt that we created earlier and then we'd also select the maximum level what we're doing here anyway the default setting is super simple all we're doing is splitting based on new line characters or white space so for example with this text here in this first bit we have eight words and then here we have another six words so we can split based on new lines and spaces and we will get this okay and here is a number of words that we have there that is all that this is doing so when we set max length that's web setting the match length for the number of separate tokens based on white space and new lines so from here we're going to initialize the what I'm going to call Dynamic prompt template now this is just a dynamic version of a few shot prompt template so in here before we just put in the examples okay so we had examples equals examples that's just saying feeding all the examples every time this time we've already fed in our list of examples to the example selector up here so we can actually use this example selector to select from those examples a certain number of them based on whatever prompt uh this few shot from template will receive later on so let's run this and actually I need to run up here as well so run this and what we're going to do is just quite a small prompt here so this would be four tokens run this and we can see there are a few examples here so we have four examples in total before we get to our final part here right and then if we wanted to run that we again just pass it through open AI right and we get this kind of sarcastic jokey answer now let's try and ask a longer question so this is what I mean when I'm saying occasionally maybe someone is going to write you a poem when they're querying something so we have they can just rambling on right it's much longer so what happens if we query with this okay uh we can see straight away that we actually get just one example being put through so because this is a much longer question we're not including as many examples and of course we can modify this as to what makes sense for us so so we can increase the max length here and we'll just re-run everything there so we have the the problem template we recreate it and then run that again with the same long question okay here and we can see that we're actually now including five various samples because we've just doubled the number of example words that are allowed through now this is just a small example of what we can do from templates for example if we wanted to use a different example selectors we can do so I showed you the very simple length base example selector here but we can do what I think is better things with this as well so we can actually base the samples that we include on similarity so we embed our examples as vector embeddings and then we calculate similarity between them in order to when we're asking a question always try to include relevant examples rather than just kind of filling it with examples that are maybe not so relevant to the current query and then there are a few other ones as well this one is very new the engram overlap example selector and we're going to cover all of these at some point in a future video but for now that's it for this video you know as you've seen we've just gone through the basics of prom templates and a few short prompt templates with a very simple example selector and for a lot of use cases that's probably all you're going to need so with that in mind I'm going to leave it there for this video so thank you very much for watching I hope this has been useful and interesting and I will see you again in the next one bye

Channel: James Briggs

Views: 35,401

Keywords: python, machine learning, artificial intelligence, natural language processing, nlp, Huggingface, semantic search, similarity search, vector similarity search, vector search, langchain, langchain ai, langchain in python, gpt 3 open source, gpt 3.5, gpt 3, gpt 4, openai tutorial, prompt engineering, prompt engineering gpt 3, llm course, large language model, llm, gpt index, gpt 3 chatbot, langchain prompt, gpt 3 tutorial, gpt 3 tutorial python, gpt 3.5 python, gpt 3 explained

Id: RflBcK0oDH0

Length: 22min 57sec (1377 seconds)

Published: Wed Feb 08 2023