Text Summarization with AWS Amazon Bedrock - Cohere FM, AWS Lambda, API Gateway [HANDS-ON]

hi my name is Rah and I work as an a solution AR and UD instructor a Bedrock has been in general availability for a few weeks so I thought I'll create a video which will showcase an endtoend use case in the manufacturing industry and we're going to build that end to end use case usings AP Gateway Lambda ands bedro where we'll use a coher model if you build this and follow along it's going to cost you a few cents this video is part of my AWS UD course which you can check from the link below now let's go ahead and see what is the use case we're going to build in this section we're going to implement a common manufacturing industry use case where we'll do the text summarization using the AWS Bedrock Foundation model for faster issue resolution to improve the productivity of technicians so if you look at any large equipment manufacturer such as manufacturers for turbines or windmills you'll see the that these windmills will be deployed in remote locations and if there is ever a failure then you will have a technician who will be at this remote site we'll take some pictures and create a detailed issue log then these images of the breakdown and the detailed issue log would be accessed by a technician sitting at a remote office through maybe a custom application that's been built now the challenge is that once this technician looks at these images and detailed issue logs sometimes a detailed issue lock can be too detailed and time consuming for this technician to look through so now the question is how can generative AI help here one possibility to improve the productivity of technician is to send these detailed issue logs to AWS bedrock and Bedrock invokes one of the foundation models and then summarizes the key issue lock and sends it back to the technician then the technician can look at this summary of of issue log rather than the entire detailed issue log to help make decisions faster now let's take a look at the architecture that we're going to build as part of this use case so here we have the user which is a technician in our case sitting in some remote office so you'll invoke a rest API and then send a prompt along with this API request and this prompt in our case will be a detailed issue LW so once this rest API has been invoked it is going to trigger a Lambda function and then this prompt is going to be passed as an event to AWS Lambda function and then this Lambda function is going to invoke the AWS Bedrock API and query a foundation model that we have selected so in our case for this use case I have selected the coher foundation model which will will be using for text summarization now how we can select the right foundational model will depend on a lot of parameters such as modality whether it's for image generation or for text or summarization the cost of the foundation model there are multiple factors maybe I'll create a separate video to describe what are the factors that you could take into account for selecting the right foundation model now there is a little bit of limitation to this architecture in terms of size of the input prompt that you can use for summarization so you can have the prompt only up to certain characters the reason being that if the prompt is too long then we have to introduce another element called as Lang chain but for now we are just going to stick with this architecture and we're going to provide a relatively short prompt and when I say prompt it means the user input now the steps that we're going to follow to create these use cases first we're going to going to write the AWS Lambda function then we're going to create the rest API using the AWS API Gateway but before we start writing the Lambda function there are couple of prerequisites one we have to enable the foundation model in our AWS account at the moment AWS requires you to enable all these Foundation models by going to the AWS Bedrock service so in future if this is provided by default or you have already enabled it then you can skip this step so now let me go to the ad console and show you how to enable these Foundation models from the console okay now the first thing we're going to do is how to get access to these Bedrock models so I'm on the AWS console I will search for bedrock and here you can see Amazon B now I can go on the left side menu and here you need to scroll down and go to model access I have already enabled model access on my AWS account but if you go here and see that you don't have access then you just need to click on this edit button and then select all the models you want access to and then do a save change and AWS should be able to provide you access within 15 to 30 minutes so now if you go to this overview section it will show you all the models that databas Bedrock supports at this moment so if I just click on this so here you see you have the model from AI 21 Labs so if you have a use case where you require multilingual support So then this is a great model and then you have the general purpose one from Amazon where you have the Titan Titan embeddings and Titan text G1 it's still in preview so you won't have access to that then you have anthropic then you have cohar which is the one which we're going to use for the usec and then you have the stability AI so if you need to generate images Etc then you use this stability AI get also provides you with a playground where you can just try it out from the console so if you go to this playground chat and from here you can select the model say if I select AI 21 and then you just select the model and then you can just play around from the console with this model so I can just say what is my name and hopefully I should get a response and you can see here it gives me a response hello I'm an open assistant how can I help you similarly if you want to generate an image so you click on image and then you have the stability Ai and I can say gener create an image of a dog and I just need to fix this spell and if I just do a run so you can see it's generated an image of a dog so now let's go ahead and build our use case end to end so start by writing the Lambda function for this use case so I've created a step by-step guide but before we start coding for the application we need to meet some prerequisites okay now let's do that so the first thing we are going to do is we're going to create an AWS Lambda function so I'm on the AWS console so I'll search for Lambda so I'll click on the create function let's call it demo betro so run time I'm going to select python 3.11 which which is the latest version and then just going to click on create function okay now we can see our blank Lambda function has been created now if I can go back to the pp you can see as per the boto 3 documentation the minimum version for boto 3 should be 1. 28.63 to be able to use bedrock and the current boto3 version for Lambda that it shows is 1.27 I little bit of Google and realize that this is the command that you can use to check the boto3 version for your aw Lambda function so I'm just going to copy this and go back to my Lambda function so here the first thing I'm going to do is I'm going to import boto 3 import boto 3 and then I'm just going to use the Sprint function which will give me the version of the boto 3 that is being used in LDA function so just click on the deploy and now let me just test it so I need to create a test event so I'll just say test and just do a save and now let's see what's the version of B3 in our current Lambda function so you can see the version here it is showing is 1. 27.1 and the minimum version that we require is 1. 28.63 so we need to upgrade this B3 version to minimum of 1. 28.63 and here's a link I found through which you you can update the boto 3 version using the AWS Lambda layers so let me go to this link you can also just do Google and find this article so what I'll do is I'll go to the bottom here you can see manually create the Lambda layer that uses latest b23 version so what you need to do is you just need to run these commands and it will upgrade the version of your B3 so let's do that so I'll run these commands through the cloud shell so I'll just open that okay now my cloud shell has launched okay now let's go to the documentation and you can see you have to run all these commands that have been highlighted here so you can replace the boto 3- my layer and you can see here we have boto 3- my layer with any name that you want so I already installed all these commands when I was trying out the use case earlier so I'm going to use a slightly different name maybe my layer 9 but if you have not installed this before you can use the same commands as it is so let's do that and I will just change my layer with N9 9 okay so I go back and run this mkdr [Music] command okay now let's do this pip install so this takes a bit of time and now let's do this CD poo 3- my layer and I have to change my layer to my layer 9 and Sal add a nine you don't need to add this if you have not installed this before okay now let's SC this one okay I will change the name okay it will take some time to install and then finally we have to publish the layer and so I just go to copy this command and I just need to make the change here at add nine and I need to add one here as well and I think that's pretty much it so now you can see it's created this layer Arn and layer versin AR okay I'm going to copy this layer Arn from here and let me just do a duplicate and now I'm on this LDA function that we had created earlier so I'll just click on this layers add a layer and then I'll just specify an Arn and I'm just going to paste this Arn here do a verify and add now you can see this layer has been added and you can see a one besides the layers now if I just go ahead and test this Lambda function and see what is a version of boto 3 that's been installed so if I just run test now you can see the B version that we have is 1. 28.6 5 and as for the documentation it should be minimum 1.28.20 or more so in terms of the prerequisites I think we are good and we can go ahead and start coding for this function now let's go ahead and create the Lambda function so I've written some broad steps so that you can easily write this Lambda function so the first one that we have to do is we have to import the boto 3 and create the client connection with the Bedrock so let me just copy it from here okay let's just paste it somewhere here so that it's easier for you to write this code um we have already imported the boto 3 next one let me just comment it out because it's no longer [Music] required this was only to check the boto 3 version now we know we have the right boto 3 version okay now we have to create the client connection with the Bedrock service so what we can do is we can just go ahead and search Bedrock 3 around time let me just click on this documentation and you can see uh this is the documentation for how to make an API request to the Bedrock service so the first thing we have to do is import boto 3 so that we have already done the next one we have to create a connection with the Bedrock runtime service so I'll just copy this and let's write it here okay now we have done that now let's look at the next step so the next step that we have to do is we have to create a request syntax get the details from console and body should be Json object okay this is just a guidance for you on how to write the Lambda function so let me just go here and I think we get rid of this too implementation okay so this is the second now in the documentation we'll check what is the right request syntax to make this API call so if I go here so you have to use invoke model method so just go here so you can read a little bit about this invoke model but this is the me method we have to use so this is how the request syntax should look like so you have the variable response and then you have the client. invoke model and these are the attributes that need to be added so let me just copy this from here and just go here let me just paste it here now let's see what all this means so if you go here so first one is accept and you can see it's a string and the default values application Json so that's what we're going to keep so I'm just going to copy it from here and let me just place it here I mean it's not a mandatory field as you can see only these two are required the rest of them are optional but we'll just keep it as it is okay so we have the accept now let's look at the content type again it's not mandatory it's is string again the default values application SL Json so we'll just use it it's not actually required as I said it's not a mandatory field but just use it anyways so next the model ID which is a string and a required field so to understand what this model ID really is let's go to the Bedrock service through the console so I'll just search for Bedrock the console okay I'm on the Bedrock service so I click on the left panel and let me just click on text and now let me just select the model category so as we discussed earlier we are using Cal and then I'll just select the model from here and as we discussed earlier our main use case is summarization so let me just say hello and let me just run this and it gave me a response but now what we can also do is click on this view API request and you can see see it gives the API request attributes and we can get the model ID from here so basically what it tells the API request is which llm model that we are going to use so if we going to use AI 21 then the model ID would be something like AI 21 but since we are using coh here so that's what we're going to use so I'm just going to copy this and as you saw in the documentation it's a string so I will just change it here okay now we have populated most of the fields the only one that is remaining is the body so before that let me just move this a little to the right so that our syntax is correct now on how to populate this body field now I can go to the console and copy this body attributes from here what I've seen is there's a lot of syntax T error when we do that so what I'll do is I will go to the documentation and populate this field through the documentation so you can see here in the body it's required and its bytes are seable file like object okay and it varies based on each llm model that you're going to use so there is more details on the format and content from this particular URL so let me just go here and show you so this is for all the LM so I will go to the cave now here you can see they have given some data points on how to actually make that request so you can see here there are three key attributes that you need to populate the first one is a prompt so the prompt is obviously the most important one it's the prompt that the user is going to provide and based on that the llm is going to provide you a response the second one is the temperature so what temperatur does is it's one of the most important parameters for Randomness and diversity and its default value is 0.9 it can be anything between 0 and 5 so what does this temperature do so if you put temperature as 0 that means that there is going to be very little Randomness okay let me give an example say you are asking chatbot Hi how are you and it's going to give you a response I'm doing good today so if if you put the temperature as zero then every time you ask it the same question it's going to give you a same response now say if you increase the temperature to its maximum value it's going to give you a randomiz response to the prompt that you have provided so if you provide the same prompt again Hi how are you this time it's going to give you a different answer so it controls the randomness of response the third key thing thing that you need to look at is maximum tokens so maximum token determines the length of the request and the response so you can see the default is 20 and minimum is 1 and maximum is 496 but you also need to be aware the max tokens that you define is going to cost you more but if you provide much lesser value of this Max token then your request and response is going to be truncated the other key thing you need to know is that the body field should be a Json object okay so what I'm going to do is I'm going to just copy this from here and going to replace the body values with this Json object okay let me just move this back so I'm only going to use three key attributes prompt temperature and Max tokens okay the rest of them I'm going to remove so I'm just going to do a little bit of formatting so now let's TR add some values so what I'll do is I'll just copy these in a notepad and then modify that because it's difficult okay so we have the prompt string temperature we discussed that we're going to keep the values randomized so we'll just keep the default value 0.9 so it's a float so it's a 0.9 and max number of tokens so you can see the default token length is 20 and when I just make a request you will see that request and response both are truncated if they go beyond certain length and there's a direct correlation between maximum tokens and the length of your request or the number of characters in your request and response so we'll keep it as default as 20 but later we'll modify uh when we try out our application so see and and now we have to Define The Prompt so we'll hard code The Prompt for now and try try out our function once we're satisfied that our Lambda function is working okay then we're going to pass a prompt through a test event and try it out and then finally once we integrated with the API Gateway then we're going to pass the prompt from the API Gateway which would be stored as a variable in this Json object and then get a response so let's say hello Okay so I'll copy this from here and just paste it now as we saw in the documentation this body needs to be a Json object okay now let's convert it into a Json object so we'll say Json dos and put back here so let me just copy it from here and modify this I think I just need to put a comma here okay so I think this looks good so what we can do is we can try it out and let me just [Music] say this response okay and let's see what the response structure is going to look like so it will be a streaming body and then there will be some content type okay so maybe uh because it's a dictionary you can see the response structure is dictionary so I can just add dictionary element here and say body okay now let's go ahead and try this out let me just do a Diplo okay now let's just go ahead and test this out okay so it's giving me an error it's saying access denied exception so what we need to do is we need to create an IM R and attach it to this Lambda function so let's do that uh um so I'll go to the configuration and I will also increase the timeout limit so let me make it 1 minute see and let's look at the permissions let me open this particular IM roll so what I noticed during the testing is that there's no particular role that I can attach to the Lambda function which will give it access to the Bedrock service so if I do an add permission attach policy and if I search Bedrock here so I don't get anything so what I'm going to do is I'm going to give it admin access I mean I know it's not a great permission to provide to any service but for now it should work so I just do add permission okay now let's go back to our code and let's go and test it out okay so let's look at the functional locks so you can see the functional locks here so it's giving me a bore. response streaming body and that's what I think we were expecting now let's go ahead and convert it into a string so let me just take look at my steps now we have to convert the streaming body into bytes and then byes to Strings so let me just do that okay okay so we'll [Music] say response underscore let's call it uh maybe right and just a variable random variable name that I have provided should be equal to response and uh so it will be a dictionary so sponsored body do and finally we'll convert it into a string so we say response string and this will be Json Dot and it will be Json do not and we'll say response right okay I think this looks good so the only thing we have to change is that the body here once it returns it should return response point or response tring okay so this is the response string here that we should get now let me just do a deploy and let's just test it out and let's see if we get a response from the Bedrock service awesome so I think we have got a response so you can see the prompt that we had provided was hello and the response that we have got is hi am a digital assistant how can I assist you today okay so we had hardcoded the prompt here and maybe I can change the prompt say how is the we in P blue and now let me just deploy and let me just pest it out okay so we got a respon the weather in Bangalore today sunny with high of 34° C and low of 18° c and this is the prop so now we're getting a response from the coher llm model but we have hardcoded the response so ideally we should be getting a prompt from an event that's passed on to the Lambda function in the end user scenario that request is going to come as part of the API request but before we create the rest API let's pass that prompt in form of a event through the test event in the Lambda function now before we do that let's go ahead and look at the next step in our guide so this is the final step so this is a fourth step I will paste it here because this is where it should be okay now let's look at what are the steps that we should follow to have that prompt coming as part of a request event rather than hardcoding the value okay so the first thing that we are going to do is uh print the event so that we know the event is correct or not so I will say print and event and this the event that we're going to get here okay so the next thing we're going to do is we're going to store the input in the variable so we'll get the details of the prompt and as part of this event and let's store that somewhere so that we can replace it here as a variable okay so let's say user _ promp which will be equal to and we'll say event and let's call The Prompt as prompt so I'll say and this is a name we'll have to give when we are passing the test event okay now we have the variable user prompt here so let's go ahead and replace that prompt value in this body request that we're going to send so we'll say user _ prompt so now we are not hardcoding the prompt that the coher llm model is going to get from Lambda function but it is going to be a variable that is going to be passed as a part of the event so now I'll just do a deploy now let's go ahead and create a test event so I'll go here and say create a new event so let's call it as promp test and um here we have to change the key value pair so let's call this as WR that's a key that we had provided and let's say how is the weather in London okay so this a format just Json just to check this correct or not okay I do save and now let's just go ahead and test this event [Music] okay awesome so it says the weather in London can be changeable throughout the year the average temperature is in January which is the coldest and you can see how is the weather in London and you can see in the functional logs The Prompt how is the weather in London that we had passed okay now we written the Lambda function now we able to pass the prompt as an event to the Lambda function and then get a response back from the coher llm as part of the Bedrock service now let's take a look at the architecture so we have already written this Lambda function we have selected the model from the Bedrock service so only thing we have to do is create the rest API and then through the rest API we're going to pass this prompt and invoke this Lambda function and get a response back so now let's do that now let's go ahead and create the last piece on the puzzle which is creating the rest APA from the aist APA Gateway so here are the broad steps so we're going to create a rest API from the API Gateway then we're going to create a resource demo bedra and then we going to create a method get let me head over to the console so I'm on the console let me search for AP Gateway so I'm on the API Gateway console you can see AP Gateway is introducing a new console so I'll just do a create API and then I'll just go to rest API do a build and I'm going to name it as demo bed and we just go to create API now it's created this API so next we're going to create a resource now let's just call it as demo text summarize a so I just do create resource and then and then we can just go here and click on create method and I'll say it and then we have to provide the Lambda function that we're going to use so I'm just going to go here and copy this demo Bedrock this Lambda function is going to be triggered every time someone invokes the rest API so say saying invalid OT bedro okay so here it is so I just do a create method so you can see now on the left we have the resource demo summarization and then we have the get method so we need to make some changes so if I go back to the template so first thing we have to do is go to the method request and in the URL query string parameters add a input parameter so let's go to the method request here and then URL query string parameters because we're going to add the prompt as a URL query string parameter so let me just do edit here and I'll go here add quy string and let's call it as PR because if you see here um we have the the event value is PR so just go here and I will say required now do a save next we're going to enable the request validation so I just do an edit and here request validator I'm going to say validate query string parameter and header so without the quer string parameter you will not be able to make an API request so I'll do a save okay next we need to go to AP Gateway integration request and and make some changes so I'll just go to integration request and then just scroll down and look at this mapping template so click on create template and in the content type we need to provide application SL Json so that's what we're going to do and then in the template body we we have to update this so I'll just tell you what it means so what it means is that the prompt user is passing as an API request is converted to event and passed onto the Lambda function this template body is written in a templating language called as velocity language templating so that's what we are doing here so we just do a create template okay so now this change has been applied okay now let's go ahead and deploy this API to our stage so say deploy API new stage let's call it as D deploy okay now let's go ahead and test this AP so I'll go to resources and do a get and go to this test so maybe I will give it a prompt saying which is the best foundation now model in AWS bedro so we'll just start with something simple and see what is the response that we get okay so we have got a response let me move this to a little left and you can see the response body so we have status code 200 and here it says there are several Foundation models in itless Bedrock that are designed to provide a secure and compliant foundation for ad deployment and it gives me some more details so you can see I think the answer seems to be fairly reasonable and if I see the prompt here is the prompt which is the best foundation model in AWS bedro now let's go ahead and check if it can summarize something meaningful so I don't have any meaningful issue log data that I can request the Bedrock to summarize for me but I found something interesting so here is an article I just read a little bit of Google best football players in the world so maybe what I'll do is I will just copy some of the text here and see the response that we're going to get okay let me go to AP Gateway and let me change the prompt to what we just saw now let me just test it out so here is the respon it says it is difficult to say who's the best football player in the world as it is a matter of personal opinion some of the top players in the world include these three so it seems fairly accurate and if we see what was the prompt uh who's the best football player in the world the question will be asked the answer will be never unanimous and then you can see it gives some more details but looks like it's giving a fairly reasonable answer in summarizing what was there in this passage so hope you found this use case and end to end demo useful thank you

Channel: AWS GenerativeAI and Amazon Bedrock with Rahul

Views: 10,759

Keywords: aws bedrock, aws bedrock tutorial, aws bedrock ai, aws bedrock workshop, aws bedrock setup, aws lambda, aws api gateway, aws generative ai, aws bedrock mini project, aws bedrock real world implementation, aws bedrock usecase, aws bedrock hands-on, aws bedrock step by step, bedrock end to end demo, aws bedrock chatbot, aws GenAI, apigateway lambda, aws serverless, genAI, amazon bedrock, amazon bedrock tutorial, aws amazon bedrock demo, aws lambda api gateway, aws cohere

Id: HI80O8fizPg

Length: 39min 53sec (2393 seconds)

Published: Tue Oct 24 2023