🆕 Bedrock Agents and Knowledge bases from a developer perspective with Demo!

hello world and happy New Year first video of the year and today I want to bring you two really cool launches from last rent we are going to talk about bedrock asan and knowledge based you might think that these are not very cessy and I hope that by the end of this video you are proven wrongly but I think these launches are really great for developers if you're are a developer and you want to get into AI you will really love agents and knowledge bases so in this video we are learning what are these two launches what like they mean from the perspective of a developer and we are going to get to build our first Asian so let's get started Bedrock agents has been around for a while so they were announcing preview in the middle of the year but in ring band they were made generally available so everybody can play with them and and try them so what is an asent if you never heard about agents in the AI context or in the Shen AI generative AI word basically the idea is that agents can manage and perform task by making API calls to different pets they extend the foundational models by understanding the user request break down complex task into multiple steps carry on the conversation with the user collect information that they need to perform the task and then fulfill the request that the user wants it's like a chat bot that we have been using for many years very maybe rudimentary in the past where you are going to your airline I don't know WhatsApp or Facebook or Instagram and you start chatting hey I have a problem with this with my reservation and then the board will say you which reservation and then you put the reservation number and then it will ask you to follow sens steps in order to help you to solve the problem but the idea here is that the agents can understand better natural language and also from the back end size it's that you don't need to program so much in the back end so I started this channel like six years ago and one of my first series was to work with Amazon legs Amazon legs was this chatbot kind of generator where you could use lamam to make this user request and then you have to program all the specific intents and all the API connections between legs and Lambda and also when you wanted to have these prompts you have to be very specific of what you ask so everything was very programmatic with Asians it's like the next level of that you as a developer do less and the asent works way better so let's see that in a demo in a moment but asan in Bedrock allows you to automate a lot of tasks so that's idea stays the same so you can do claim processing bookings I don't know shop whatever you can do with a chat bot or more so you can ask about this asan how much a flight will cost in Christmas from monio to Helsinki and the boat will get you an answer and if it doesn't have for example if it needs a return date it will ask you is it a return flight or one way whatever how agents works for this work you need to tell the agent what foundational model to use so that's the first step you can decide from all the foundational models that are available in Bedrock then you need to provide the basic instructions of what your agent should do I don't know book a flight or check reservations then you need to give access to some extra data to perform the task the foundational models are train on the internet but not in your own corporate data so if you want your model to be able to access I don't know your customer database or your reservations database you need to provide that access and then finally you need to specifies what are the actions that they can do so if they want to book a flight you need to connect that agent to the booking flight API and basically then you are done after the agent is configured then it will do all the prompt engineering and orchestration of the requested task so this is very mindblowing going and for me it was not what I expected I never worked with Asians before so when I started working with this it was like what it's doing all the orchestration for me and it does and it helps you to to build the prompts and translate all the prompts into API calls to perform those tasks and then it converts the API responses into something that is friendly for the user and we will see this in the demo in in a second so how this works when a request comes into the agent will use the foundational model to generate a sequence of steps so that's something that the agent will do it will orchestrate all the steps that they need to do gather all the information needed for example if you want to book a flight it will ask you to where what kind of flight direct over or if it's one way return whatever it needs dates and then it will get all the apis that it has available in order to work and then it will know when to call them and with what parameters then when it has all the information gather it will execute the plan and call all the apis with all the information that it collected and then it will return the response to the user in the chat so you will see at the end of this demo how does Bedrock all the heavy lifting for you while we build our agent but first let's chat a little bit about another important launch that we have this rein bent and it's very important as building an asent that is the knowledge base and you might be wondering what knowledge bases are they also have been around in preview for some months but they were made available in the conference a knowledge base is basically a way to connect foundational models in Bedrock to your company data using retrieval augmented generation or rag so basically these knowledge bases will do all the rag processes for you so when a model needs to access a specific information from your organization to generate very accurate and context aware responses without the need to retrain the model as well because we don't want to retrain these very expensive foundational models then they can fetch this information from a knowledge base and the knowledge base basically helps the model as well because it will provide like the source attribution from where it got the data so this will help to improve the transparency of your model and minimize the hallucinations of your model so then you know that whatever the model is returning has a fact behind it and it's coming from this references the cool thing of knowledge bases in Bedrock is that it gives you a fully managed rag experience in the demo that is coming next you will see how simple is is to do that and basically just need to put data in a threee or in somewhere point it the location to your data then you can select the EMB models to convert that data into vector embeddings that those will be the ones used by the foundation models to figure out what uh will happen and then Bedrock will create the vector store and store all that data in there so if you never heard what is a vector and B is basically a way of storing Text data in a numerical representation within your documents the idea is to capture this semantic or contextual meaning of data so it's a cat a fail line tiger lion they all might be grouped together like you see in the screen here in this scraft that you see in the screen you can see that there is different groups and you can see for example the light blue is like kitchen vanity sink bathroom toilet bath W so these are I don't know watery things then you have the purple one charger battery tool so some tools and all these things are grouped together so when the vector database Vector store is looking looking for something it will find things around this context and this semantic so it will be easier so now let's go to the demo because I think you have all the knowledge that you need in order to move on in this demo we are going to build the simplest Asian in the planet CU I want to do it fast and I want to show you a little bit of the magic but let me know if you want to build more complex things or if you want to do it as infrastructure as code I will be more than happy to do it but this boat what it does is a simple weather chatbot in where you ask the weather in some particular City and the uh boot Returns the weather in some kind of friendly format this asent uses a knowledge Bas where we are going to store all the latitudes and longitudes of all cities in the world and then it has a Lambda function to perform the um fetching of the weather of that particular City so let's go to the console and create the uh knowledge base first I will go here to my AWS account and go to bedrock just remember to be in a region where betro is enabled and here you can create a new knowledge base I already have one and I will use this one because it takes a while to load but let's fake it a little bit and create it from scratch so you can give here a name a description it will create the roles for you so so that's nice you can tag it so here you can Define where your data is so I will point it to exactly the same place as I have already uploaded my data we'll look at that in a second then here you can choose the embotics mods there is only one available in my account the Tian embeddings G1 and then it will create a new store that's recommended that makes an open search S as Vector store and then you press next you review everything and and you create the knowledge base and this what it does it creates the knowledge base for you I have it here because it takes a while to make that open search Vector store and here we have it the latitude longitude c word Vector space and it has some description and the RO that it was created automatically here is the file that I have and basically here we can test the knowledge so we can select select a model to work with let's pick this one and let's say give me the L and long for there see if it works so good what is the L and long for hink key good so here we can click and it will show us the raw data so if we open the S3 uh file is a CSV file where we have basically um list of uh cities with latitudes and longitudes it's just so simple so now we have our knowledge base and in order to create the knowledge base with that list of longitude and latitude I took it from this uh page I can show it to you called Simple Maps I will leave you the link in the description box where you can download for free free list of cities with the latitude and longitude so I use that one so it looks something like this and this is very useful for my particular demo and this is what I uploaded the when you download this you get a CVS file and this is what I uploaded to a so if you want to try this demo get the link from here now that we have a knowledge base in place we need to code the Lambda function the Lambda function will be the one called in the weather third party service that that I'm using in this case I'm using open weather map that offers a free tier to get the weather for a particular City so basically this is the site open weather map I will leave you the the link in the description box so you can register if you want and and try it out but it's a very simple API that you can access and do whatever you want to do it has some limitations if it's in the Freer but it works very well the thing with the weather API that it takes as input parameters not the name of the city itself but the latitude and longitude of that City that's why we have to have that knowledge base with this list and the agent will need to figure that out so that's the magic part there is no integration in our code so now you might be wondering how all these things get together let's see so we have the user making a request to the agent and the agent will be using the foundational model to communicate then we'll have an open API definition for our function where we are going to specify all the input parameters and what each of the apis do in natural language and in this way then the patient will know when to call it this function so now we can go to the code and see this in action so the function I already have deployed it and everything because I don't want to waste time but I can show you the code and you can also get the code from GitHub so the first bit is the template Jamel so this is built with some here we have our function it's a normal function with some code Noe 20 very little memory and then the other important thing is this one we need to give permissions to betrock agent to access the the function so we need to give this kind of resource policy so the function itself we have the code here index JS it doesn't do much than calling the open API but I have to say that I think the most important critical part of documentation that you need to save from this video is the documentation from the input and outputs that the Lambda function will give when it's connected to bedrock agent and needs to return to the Bedrock agent in order to work so this is a Lambda input event for Bedrock agent and basically this is the event object that the Lambda function will have and basically from here you need to dig out all the information that you need in order to work with this um agent and then this is the response that Lambda needs to give to the agent and it needs to give this particular format in order for the agent to know what to do so I will leave this link as well in the description box so you can uh bookmark it if you're working with agents so now we are getting this this event object with all that things that I show you I'm going to get the API path from that event and here we could have multiple apis in the same function so we could have multiple functions or we could have multiple apis in the same Handler in my case I only have one so I have the weather one and as I told you this weather API for calling the open Weather takes the latitude and longitude so we are just grabbing those from the event object then I'm calling the get weather and basically that is calling the we uh open Weather API with the right parameters and an API key and then I just returning this data back I'm returning this data and then I'm building the response in the format that that Bedrock agent needs so this is a Fluff of just building things so basically my business logic is that and it's calling an external HTTP thing but so with this in place the other important bit that I want to show you is the open API definition and this one I think is quite hard to write there is the whole open API specification is open there and the good thing is that the asent will tell you when something is missing and when something is required so the first first thing we need to do is to have an info then we have the path for the different apis I already said that we only have one and in this case it's a get weather and here we have the description and this is very important because the acient will read this get the weather for specific latitude logitude represent a city then we have the parameters these are the input parameters in our case is latitude and longitude and here we need this description in place in order for the acient to know no then you need to have the schema and things like that and then the responses again the word description for giv City and then you pass what is the type in this case is a Json that I'm returning back so now we deploy the function and we have the knowledge base the next thing we need to do is to upload the open API definition Json that I show you to S free and then we will tell uh the agent word that is so that's something I already have done so now we have everything so let's go to create the asent so let's go back to Amazon bedrock and in here let's do asent you can see that I have an asent already they are ready but we can create a new one so let's call it here asent two weather can give a description asent that gives their weather for a city and then we can tell them if the agent can ask for more information or not so I will say yes it will I will tell them to create the permissions for me because I don't want to do that and then the idle sessions that good the timeout 30 minutes and it could be even shorter but we don't care about that then we need to select the uh model and in this case we have anthropic and then close to and here we can say something like this you're an agent designed to help customers to get the weather details for for a city the customers will provide you with a city name and you need to translate that into in latitude and longitude return the results in a user friendly manner good the next thing we want to do is to do the action groups and here we can Define as many as we want we can add and these are the calls to Lambda so we can do uh Lambda one weather I get the weather here we can select our Lambda now I have deploy and the API schema that we uploaded already here it is and then we do next the next step is to select the knowledge base this is the one that we created before and we can also give instructions to the acient in order uh to what to do in that knowledge base so I will give something like this search for the latitude and longitude for the city that the user has given you in the prompt I'm not an expert on this but this works and now we have the asent details that we can review the model and the structures the action calls the knowledge base and then we can create the agent this takes a little bit and then we can test the agent when this is ready so now we have our agent in place and you can see let it pop up this test alas working draft so if you go here you can see your working draft so now let's test this and in order to test this we have this test thing here that we can tell give me the weather for Monte video and this takes a little while to think and we can go to the Lambda function to the logs and we can see if that is getting invoked let's see should we can should see that the function has run here we have it we can see the event input all this long thing with the parameters we can see the call to the open API and we can see the response and everything so if we go to the chat bot it says the word mon video is condition standstone with rain moderate rain temperature 20° C feels 20° humidity blah blah blah one of my favorite things and at least where I learned quite a lot from Bedrock is when I click these show Tres and here you can really see how the asent reason so this is a pre-processing state and you can open it here and you can see all these prompts that the model is doing and deciding so you have not written any of these you can overwrite them but it's just really cool to see the rationale and see here here based on the fride the functions that are all these that are here the input give me the weather from mon appears to be a request for weather information from the CD mon video to fulfill this request the function calling agent will need to first use the get Amazon knowledge based search function to find the latitude and longitude form of the video it will then pass this coordinates to the weather agent function to retrieve the actual weather function since the sequence of functions could possibly retress the requested information and believe this input falls into category D that is from the list of these things then the next step is the orchestrational knowledge base so you can see here that what is going on let's see where the asent replies this is all generated and orchestrated for you so this is really cool and you can see here is was calling the knowledge base and it's doing some searching and blah blah blah then he it's is doing trying to find that here is the user question question what is the latitude and longitude fromo question is refer for a search result and you can really see what is going on so let's see if he gets the answer eventually because he's passing so much information I have received the latitude and logitude for mon from the knowledge base I will not call the weather so this is part of the orchestration and this should be calling the weather and making the response human readable you can see all the steps and if you have some pre-processing you will be seeing it here so now you have your agent and what you can do is you can deploy it if you want you can create an elas and and deploy it and then you can use it in your web applications so that's the video for me today it's a long video with a lot of information but I would love to know if you want me to talk more about agents how to build them do more complex things build as this as infrastructure as code because you can do it if you want to see how to integrate this in our client application let me know we can build that together I'm always happy to experiment on this things and uh here you can find the playlist with all the announcements from rent and this year and you can find here a link about the functions and Bedrock so if you want to orchestrate your generative AI in a more like traditional way go and check that one think this is a great way step functions and bedro for the asynchronous part of orchestrations when you want to do uh generative AI in more as synchronous flows this asent is really cool for the synchronous flow to talk to a human a chat B so there are different use cases and I see you in the next episode of [Music] w

Channel: FooBar Serverless

Views: 11,595

Keywords: foobar, reinvent23, bedrock, genai, bedrock agents, bedrock knowlege bases

Id: P9n8BE693go

Length: 24min 52sec (1492 seconds)

Published: Thu Jan 11 2024