Working with Prompts in Spring AI - Effectively Communicating with LLMs

in today's tutorial we're going to continue our exploration into the project known as spring AI this is the project helping Java developers build intelligent applications with artificial intelligence now if you miss my introduction I'll go ahead and leave a link to that in the description below as well as a card somewhere up here in that introduction we talked about some of the terms that we throw around in in artificial intelligence space as well as getting started with spring Ai and open AI now spring AI is just an abstraction on the top of a lot of these llms so you can use whatever llm that you want to work with but as you get into building these applications you start to realize that you have to deal with other challenges things like prompting output parsing embeddings Vector databases and more today I want to take a look at prompts and to do so we're going to dive through a couple slides so I can talk about the problem and then we'll jump into our editor and start building a spring a application with prompts today so here we are in my slide deck I want to talk about this first prompts serve as the foundation for the language based inputs that guide an AI model to produce specific outputs for those of you familiar with chat gbt a prompt might seem merely like some text you put into a dialogue box and that's sent to the AI API however it encompasses much more than that in many AI models the text for the prompt is not just a simple string so that's important to remember now there's a word that's thrown around or a term that's thrown around called prompt engineering and I cringe a little bit when I hear it but I'm going to kind of put my thoughts aside on that for a second because it is a very important subject when we talk about building AI applications for me this is just learning how to effectively communicate with the AI system this is the input that is directing the AI model to produce the specific outputs that we're looking for so if you don't get the output or something that you're kind of hoping it to produce you can always adjust that by changing the prompt that we send to the AI U model outputs are greatly influenced by The Prompt style and wording so again being able to effectively communicate with the AI is very very important and probably the most important thing we have to do here when it comes to prompt engineering there are some techniques and effective prompts uh that you can share in the community the really nice thing is some of the examples that we're going to look through today are things that we craft ourselves out in there in the ecosystem there are entire uh collections of prompts that you can go ahead and copy and paste into your application somebody who's gone through the trouble of crafting a specific prompt to get a desired output so go ahead and take a look out there for those there are two links here uh the open AI guidelines and the mini course for chat GPT engineering that I will take a look at in a second these are really nice resources to kind of start learning how to effectively communicate with an AI now prompts and spring so prompt management relies on a text template engine we'll see that we when we get into prompt templates there's a way to craft a string and then have some type of dynamic variables or templates that we can go ahead and uh input later you can think of this as analogous to the view in Spring MBC right when we are working in a controller we may have a model we may put in uh customer first name is Dan customer last name is Vega and we go ahead and send that model down to the view in this case our prompt is really the view to the AI so here's an example of the prompt class again we'll see this in a live coding example but we have our prompt class here and a prompt has a few different Constructors you'll notice that uh it doesn't just pass in a string it actually creates a new user message so we'll talk about user messages in a second there are different roles when it comes to creating prompts you will also see some of these Constructors take a message and again user message is a specific type of message if we go in and look at the implementations of this message there are a few things like an assistant message a chat message a system message and a user message some of these boil down to roles what are the different roles I'm not going to read all these to you you can kind of go through them but a user role represents the user Us's input what is the user asking uh what is our question what is their command this role is fundamental as it forms the basis of the ai's response now on top of that we could have something called a system role this really guides the ai's behavior and response style setting parameters or rules for how an AI interprets and replies to the inputs it's akin to providing instructions to the AI before initiating that conversation that the user might have with the AI we'll look at an example of this and why this really putting all these pieces to the puzzle really helps us get our desired output finally we have some prompt templates here uh this is an example of a prompt template so we have a string message so we have a message in there we have um some curly brackets with genre in there and genre is what we will kind of replace with whatever is passed into this particular method so we'll see an example of that today so that that's my kind of Spiel on prompts and again I I hear a lot of people say Well Spring AI is just an abstra you know abstraction on top of calling the llms and I want to make sure we understand that that's not just the case yes it is a rest API call to a lot of these llms but there are a lot of challenges that you're going to run into with kind of crafting these prompts and then getting the desired output to parse to some type of object in your system and more challenges so what I'm trying to do with these videos is really kind of introduce you to Spring Ai and the different components that you're working with so with that let's head over to start. spring.io create a new application and work with prompts all right so before we get in there I just wanted to mention these two links that I shared in the slides uh this is open ai's prompt engineering guidelines this is a really good read just again on effectively communicating how to write clear instructions how to provide reference text splitting complex tasks into simpler tasks again if all you've ever done is type into chat jpt a string and ask it for something these are really good tools to kind of hone in on that craft because it is very important there's also a really good free short course uh from chat gbt prompt engineering for developers this is with Andrew n g and an employee at um open aai this is just a really good course on going through learning how to effectively write these prompts it is important and it's something that I think we should all work on so those are two really good resources for starting with that with that uh let's create a new project I'm going to use Maven I'm going to use spring boot 3.2.4 let's go ahead and call this dev. Dan Vega we'll call this prompts and we'll use Java 21 today and then I'm just going to go in and pick spring web and I'm going to pick the AI llm that I'm working with in this case case it's open AI so I'll go on down and select open AI I will generate this and open this up in my favorite ID intellig you can open up in whatever ID or text editor you're most productive in with that let's write some code we'll call this the simple prompt controller and in this I will go ahead and Mark this as a rest controller now again if you didn't get a chance to watch the intro maybe you can dive into that I'll kind of quickly run over some of this stuff but the first thing that we need is a chat client uh there happens to be only one implementation of this right now because we're using open AI so if we look at that uh we can see there's an open AI check client and this is what's going to help us communicate with open AI so I'm going to create a get mapping and we'll call this um we'll just use this at the root that is fine and then we'll go ahead and say this is going to return a string we'll call this simple and if you were with us in the last video we took a look at chat client chat client basically has two methods in there one called call that takes in a string and one called call that takes in a prompt we used just the simple string last time but you'll see underneath the hood it actually creates a prompt for that and it creates a special prompt using a user message so now that we know about different roles like user and system that kind of might make a little more sense we'll dive into that as well you'll see at the end of this that creates a generation from there you can return the output and the content to get a string back so we're going to use this one we're going to pass in a prompt and we're going to see how we can get the string back so we're going to say return chat client. call and now we need to create a prompt so we're going to say new prompt and then the uh contents is just going to be the string message so uh tell me a dad joke right so that is going to again return a not a string so what we need to do is we need to get the result we need to get the output and we need to get the content and that will return a string so this is just a little bit more uh the next level of just passing in a string so if we go ahead and run our application oh actually we have not done this I thought we were continuing from the last one so I am going to need to enter a couple of things in here so spring. a. openapi open AI - API key and I want to go ahead and set the model that I'm working with so I'll say open AI uh chat. options. model we'll say we're working with gp4 now I don't want to just paste in my key here so I'm going to go ahead and say open uh I think I had this as open AI API key okay and with that in place I can go ahead and run my example hopefully everything works it looks like it started so I can head over to a terminal and I'm just going to use a command line uh program called HTTP iie uh curl for us humans and I'm going to curl uh 880 and if everything works we should call it to open Ai and get a dad joke why don't scientists trust atams because they make up everything so we have a joke there um that worked fine now that is not the interesting bits I just wanted to kind of Next Level uh use that chat client. call that takes in a prompt so now we now that we understand um what that prompt is and actually if we go back to it I just want to take a look at this this class we looked at it in the slides but if we're here let's go ahead and look at it again uh it takes in a string which creates a user message so this is the role of user message that we're sending to the AI uh you can pass in message directly you can pass in a list of messages um there are also other Constructors here as well so we'll go ahead and take a look at some examples of those all right in this next example I'm going to create a new class called YouTube and again this is going to be a rest controller and inside of this rest controller we are going to have a git mapping of Slash poopular you let's just call this YouTube so let's do this let's set a request Quest mapping here of Slash youube and this will be popular all right so I'm going to create a new method here that is going to return a string we'll call this find popular YouTubers YouTubers uh by genre so uh we have by genre we're going to take in a request parameter and this request parameter is going to be um genre so we'll say genre and we'll assign this to string genre and I just also want to set a default value in case we don't get one we'll say by Tech so we want some tech YouTubers and then we can go ahead and create our message so I'm going to go ahead and uh paste in this message just so we don't have to watch me type it this is the one we saw in the slides before so I have a message here list 10 of the most popular YouTubers in John ra along with our current subscriber counts if you don't know the answer just say I don't know this is good we're instructing the AI to not hallucinate don't make something up uh if you don't have an answer just let us know so now we can create a prompt template from this so that we can basically replace this with the actual genre so to do that we're going to use prompt template so prompt template we'll call this prompt template and we can say new prompt template and we can pass in our message from there I can create a new prompt template I can say create and what I'm going to do here is I'm basically let me just get a VAR of that yes that looks good I'm going to create a map and I'm going to supply key value pairs I'm going to say hey for genre I want you to replace it with genre that I'm getting as an argument to this method so now we've replaced that the string template if you will and now we have a prompt so now with a prompt we can call our check clients so we've already seen this at oh we don't have one yet we need one of those Dan so let's do that again private final check client check client we'll get this through Constructor injection and now we can go ahead and use this so we can say chat client. call we've already seen how to pass a prompt so we're going to pass in that prompt we just created and just as we did before we want to return a string so we'll say get result get output and just get me the content because that's all I care about in this case now when we start working in real applications we're probably going to want to uh parse this output into a object and that's where output parsing comes in which we will cover in a future video so now that I have this now I should be able to restart my application we're going to hit the endpoint /youtube poopular it's going to have a default genre of uh Tech so let's say 80youtube poopular and let's see if we didn't screw anything up we should get a list back uh this will give us 10 of the most popular YouTubers in the tech space along with their subscriber counts now remember this is up to a certain point right so these aren't going to be entirely accurate because the lm's training data goes up to in this case September of 2021 so you see uh I'm sorry I can't provide real time da data my abilities are based on pre-existing data up to October of 2021 so here are some uh popular YouTubers in the tech space along with their subscriber counts so that is really cool now again this is a list of strings at some point we might want to turn this into an object like maybe we have a record that includes the YouTuber's name and their subscriber count uh as fields uh so we'll get into that in a later episode but this is a really good start uh and I want to take a look at one more uh kind of example of this and we'll go back to the idea of jokes I want to create a new class here I'm going to call this the dad joke controller and I want to say this is a rest controller and what we are doing here we'll need one more of those uh chat clients so I'll say chat client chat client again through Constructor injection and I have a git mapping of Slash dad jokes right so again we want to get a string from this we'll say string jokes and in here we're going to do a little bit something different the first thing I'm going to do is create a user message so we've been creating just a string message um I can say VAR um user is equal to new user message let's see tell me a joke about cats and now that we have that we can create a new prompt from that so we'll pass in our user message that will give us back a prompt and we know that we can say return check client. call we pass in our prompt and we get the result we get the output and we get the content so let's go ahead and save that Rerun this application and let's go back over to the terminal and we're going to run that again and why don't cats play poker in the jungle because there's too many Cheetah I like that one that's a good one but uh not very specific to being a dad joke right um or at least in this context so what I want to do is add a new system message to basically instruct the AI that this is the type of application that you are so I can say system system is equal to new system message and I can say your uh your primary function is to tell Dad jokes right uh if someone asks you for any other type of joke please tell them you only know dad jokes right so now we have this system message and this user message so remember if we look back at prompt one of the things that I can take is a list of messages so instead of just sending in a user now we can say list. of I want you to pass in let's just say system and user right so now what I want to tell my user message to do is tell me a serious joke about the UN Universe all right so let's go ahead and try and rerun that application oh messed up somewhere okay rerun that looks like that worked let's go ahead and rerun this and it says I'm sorry I can only share dad jokes here's one related to the universe why didn't the sun go to college because it already had a million degrees that is really good and that is a dad joke so I like that um we can kind of instruct the system of what we're actually working with here instead of letting the user kind of run wild and just input anything we are saying this is specifically what this system is meant to do cool so that was a nice example I got one more for you I'm going to go back to our YouTube example and one thing that we did here is we kind of hardcoded this string in here right but what if we wanted this in many places we wouldn't want to have to repeat ourselves so let's go into resources and I'm going to create a folder let's say a new directory called prompts and inside of prompts I am going to create a new file and I'm going to call this one youtube. St for string template uh you'll see intellig pi this up I have an extension for this again this is something the overall AI Community uses so you can go out and find prompt templates like this and share them and then go ahead and just bring these into your applications so I have that same one with the same placeholder for genre but this time I'm not going to include that right here so I'll go ahead and get rid of this and now I can kind of externalize this right I want to go ahead and pull this from that file so I can say private resource uh YouTube prompt resource and we can go ahead and ask for this so oops so I'm going to say that we are going to get this using the at Value annotation and we are going to say this is coming from the class path under prompts and youtube. St so with that that there now we need this message uh we can just go ahead and pass this in so YouTube prompt resource and now this one should work exactly the same I'm just going to run it make sure it runs uh it will work exactly the same so I hope you found that valuable we took a look at what prompts are why they're important to uh AI applications some resources for really kind of honing this craft which is developing all these different types of prompts we have user prompts and uh system prompts and assistant prompts these are different rules that we can pass to the AI to really refine what we want to get from the output and as I said the next kind of piece of this puzzle is really taking what we get and not just using a string but uh we saw in the YouTube example we probably get 10 uh lists back of the popular YouTubers we want to turn that into an object we can actually work with so in the next one we will work with the output parsers so hey I hope you found value in this friends if you did do do me a big favor leave me a thumbs up subscribe to the channel and as always happy [Music] coding

Channel: Dan Vega

Views: 5,727

Keywords: dan vega

Id: ACpLp2KXqgE

Length: 22min 44sec (1364 seconds)

Published: Thu Apr 04 2024