Level Up your chatbot game with AWS Generative AI LLM | AWS OnAir S05

hey Sarah hello ladies wonderful show going on you are just spiking up my energy great job I'm highly caffeinated so I would hope it's exuding Works yep that how's going it's going well um hello everyone love the audience I've seen so much of interaction really enourage and then love it keep on going myself Sarah Mir I'm a partner Solutions architect at AWS and today I wanted to talk about something interesting I know you have some some sort of heard it but something new about it so uh shall I ladies let's do yeah let's get going so tell us about it sure would like to directly um address this question kind of hypothetically uh to my audience is like do you want to take your Chad Bot game to the next level well then I would say buckle up because AWS gvi LM is here to sprinkle some magic into your conversations well I'm pretty sure you all have heard about the term Chad bot yes I know but but imagine a Chad bot on steroids yep that's what we will be talking about oh I think we're gonna see it yeah I'm intrigued first I think we're gonna get to see it oh yes absolutely see it in a very detailed format so but first things first I would like to get some things away but because I know the topic is about deploying multimodal and multi rag power chatbot using aw cdk I know all these big buzz words llms in there so I would like to ask our audience what do they understand by the term rag no not not that rag ex trust me not that rag you thought you all thought about this rag stands and work on three mechanisms starting with retrieve analyze generate easy to understand right so basically imagine this rack technology which we are trying to embed into these chatbot is like your chatbot superhero outfit with your superpowers it helps the chatbot pull up some information which is the retrieve part understand it which is the analyze part and then spit out some smart responses which is the generate part so think of r as as a secret sauce that makes your Chad bot super awesome got it so how does rag is rag something that's like a button you press to add it in is Brag uh line of code that you you like pull something from AWS how do you how do you add rag to your it's the mechanism it's certainly the mechanism you embedded into it it's already ready to deploy you just go through uh this Rag and then you go through all these uh open-ended open search um um and mechanisms to add through rack to retrieve information from various uh search engines and and yeah it's very easy to deploy which we will be seeing very shortly so um next thing I would like to also talk about llms because I know I've been getting feedback that the acronyms right we we all get them right so lot of acronyms y y so I wanted to make it so clear for our lovely audience today llms stands for large language models so imagine these LMS like the uh I would say brainiacs of your chatbot World they are like the big brains behind the chatbot I know rag is the mechanism but the big brains the Nerds of your Chad barar the Geniuses are these LMS packed with tons of knowledges and language skills they are the ones who understand what you are saying process it and come up with those clever responses for rag to throw it out there from various different models language models let's say so next time when you you hear about Rag and llms just think of them as a as a dynamic duo uh powering your chat bot incredible abilities retrieving analyzing generating and using their big brains to chat with you like a pro does that that helps Brak it down oh yeah like you're having some big brain thoughts I'm curious I mean like we're talking about these amazing multimodal chat bots so how could I not uh I think it would be really helpful if if you can start to show it to us and I think that will help people start to understand how this works how can they can start to actually be able to use it in their own application sure sure let me let me quickly um talk about or or let's say you know what let's let's let's share our screen yep there is my architecture diagram okay so this is what our chatbot would be build one just wanted to give you all a a glimpse of the architecture what is going in the back end let me talk about that that these are built on our great uh Services you are seeing here uh this you there will be a UI which you will be seeing soon and these are fully fledged user interface repository includes AWS constructs which to deploy a fully fledged built with react hosted on Amazon S3 as you see on the screen protected with Amazon Cognito authentication of course and plus it's backed up with some serious text like app sync uh for real-time interactions with your chatbot also designed by the system provided by AWS Cloudscape design system so there you have it and let's dive further into our actually I've got one question here if you can go back to the architecture di because I I get this question all the time whenever we post any of these cloud formation templates that allow someone to be able to just deploy something really quickly um is they say oh but wait a minute I don't use Cognito as my identifi I am not using Dynamo DB how do I can I customize it for my bu own use case for now this is the model which is ready to deploy and it it will be so much easier if if this V service is our leverage because it's pre-built and the and the user doesn't have to do much yeah so I it will be beneficial to leverage these services this is how quick is it and which is which I'm about to show right now as next perfect yeah and it totally makes sense especially those who've got a brand new use case I work with startups so this is the kind of thing like you're a startup and you're starting your own chatot as a service compy it's like don't build it from scratch like use this instead oh absolutely absolutely and I'm pretty sure audience is going to love this idea how quick and easy and convenient is it you don't even have to be a coding expert uh anyone can go ahead and deploy it quick within minutes trust me that's exting and these are these are the git GitHub links which will be shared with you all definitely by our experts in the chat so feel free to leverage them this is this is the one uh Julian I was talking about how just a glimpse I wanted to show it to our audience before I jump into the actual super lovely UI is these are the basic configs where you are trying to implement within yes and no option and then your open search engine options like Aurora or other workspaces to add in so how quickly just enable it yes which region just the name the role you all all those models llms you just add it in there and there you go boom that's it it's just going back and back oh wait wait sorry I got that was it went little I have a question on that there was in that uh it was there is an an option for which model was it an option for which large language model to use yes there is an option for which large language models not only that which the foundation models so choose and also one great point to mention that it it has several uh ways you can you can deploy it it I have done it with Cloud9 AWS Cloud9 you can do it locally with console you can do another wave through sag maker there are several ways which which you is more favorable for you all to do it so this is but the the configuration is detailed here all the magic configs which which works their magic in the back end are are mentioned in all these uh cdk files here for you all to leverage awesome so we do have a question in the chat and I know you're about to show us this but as you're showing it to us so when you're pulling those those different models in how how do you decide which one of also where are they I guess two questions one how do you initially get started like if you're starting from complete scratch your a startup and you don't know where to figure out which one is best for you or even which ones to compare how do you figure that out you're basically um let me guess you are um more or or the question is more emphasized on the functionalities of the model because let's say an easy example like some models are for text generation some Foundation models for image generation right so the user wants to Define or get clarified on the functionality what according to their use case right yep yep in this documents the the document which will be shared with you all the models involved uh are are definitely defined that way for this purpose but not only that there's so much of a do a great documentation for Jumpstart uh playground where I have done the enablement previously which also defines in a two to three line of every Foundation model what it is good for oh great is that um is there a website the native Us website that says that or is that an inter is that a training that you did that's no no it's it's it should be openly shared from the jumpstart perspective and Bedrock perspective Amazon yeah okay so maybe we can get that get that link put in the chats that way people can take a look if they're interested in our on liners about different options on liners about it those are pretty I find them very convenient for my for my partners my customers to show them it's easily definable and then it's a huge playground out of all the Foundation models we support and you can easily go and just read okay that that will do my thing so you can just pick it up great okay let's move on shall we let's do it okay before I start I wanted to give a higher level overview what we will be talking or or showing uh you the magic about is uh uh with with this AWS generative EI llm you can easily deploy multiple Mo models and dive into the world of rag options and now we all know what rag is right guess what it comes with a super Sleek UI which you're seeing right now and that will make you feel like you're chatting with a futuristic AI buddy so let's talk about what fun things you can do with your futuristic AI buddy here first off you can experiment with multimodel modes I am about to show that second would be uh our multi- session chat and Then followed up with some multiple rag options with workspaces and debugging tool so let's let's start with that great playground this is our chatbot and this is our rag retrieval augmented generation oh so the first one I would like to show the playground and this is the most interesting one I know uh I'm still a kid I would say um I have some U history here so this is an interesting one what this does is that it can analyze the images and pictures that means you can throw in some pictures ask questions about them watch the chat bot do its thing describe those images and even generate text based on them how cool is that so what I did just to fool or or give more challenges to a chat bot I did not pick this picture from uh the web this is my very own personal my own cooking show picture and my own ingredients Where I Was preparing a dish and look it came out exactly what I I made it so I I was pretty impressed and these are the outputs from different uh Foundation models I I think this is from Amazon Titan this is from this is from Claude which it gave me that based on these ingredients shown in the image appears that you're making for a chicken Tika Masala that's an Indian popular dish and I'm pretty sure we all are aware of so this has given me all these ingredients how cool is that that is just seeing in the picture that scanned everything for me that's very cool yeah there's two questions in the chat that kind of in the same theme so I'm going to combine them into one question um so as context for the the folks listening all of our pricing you can see it on the pricing pages so and then there's an AWS pricing calculator so you can go in and put in numbers based on your usage to get an estimate of like how much your costs are going to be but I'm curious like for this whole solution is there anything on the actual GitHub repo that helps people to understand what the their cost would like potentially be oh from the cost perspective yes I'm pretty sure uh we can send them some link to discuss because I know there is A3 involved there is uh UI and profy involved there is Lambda in the back end and other services Cognito so they will be separately calculated all to begin into one so yeah yeah I will I will encourage my my my experts to send some links regarding that but just to let the audience know this is this is a combination of multiple Services integrated together how they are working in the back end and then that's how it's going to come out to be yep yeah perfect that's helpful it's always good to know what you're going to see on the bill before the bill comes yeah yes absolutely right so they should have a great understanding and then we always have lot of good free trials going on so I I'm pretty sure they all can leverage a lot of those services from that aspect so talking about yeah coming back to this I would like to show a live demo so how about we pick another pick from one of my collection let's see maybe this one um this is how you go ahead and add it so I had this Foundation model chosen here I added an image now since the image is added right you can see here here let me let me move this so that it's more clear now I'm going to see what can I prepare from this and and and and notice that I'm not even saying uh from this image I'm not even saying from these list of ingredients I'm not saying anything that it's pretty vague right and see what what it does this is also a very personal image and I know I have given it out the hint is about the soup but let's see what what our our uh Foundation model is out to be is killing me right [Music] now why okay based on the ingredients you in the mid it appears you have components make a very hearty vegetable soup or stew party wow yeah yeah this is the whole whole winter three four months it's very very frequent in our house so and uh I love it that it picked up everything and uh yep yep it's giving us possibly turkey or chicken could be browned and added to the pot so it's giving pretty much all the details and adding every single ingredient which is shown in here right so this is this is this was about the images and doesn't have to be food the reason I put chose food is so that our spouses your girlfriends boyfriends don't have any excuse or our teenagers don't have any excuse that okay you don't know what to do there you go we we got a solution for you just just throw throw this over to them and they can find a recipe whatever is left in the kitchen or fridge so I hope you all are enjoying it next one is I would like to talk about is um if if you all ever wanted to test multiple models at once well with the multiple session chat you can totally do that now sending the same questions to a bunch of different models and and seeing how each response based on their own smarts knowledge back and nerd brains and everything so let's move on to that multiat playground any questions I think we're getting a lot of support and a little bit of curiosity on how quickly customers are able to adopt this is this something that you're seeing is very that is one thing I can assure 100% it is so quick and and all the magic andig is uh uh provided in the cdk and repository and if someone wants to go and look at the details it will show what each config does here so yes it's it's fairly quick awesome let's see so what I'm going to do I let's see uh for a fun uh fact um let's pick Amazon Titan text light here and I would pick clot clot version 2 so for both of these different Foundation models um let's let's let's let's see what I can get a help like right um since we are in gen gen use case for me for for financial services industry generative AI use case sorry let's see what these both uh Foundation models come out to be I know cloud is more chatty and while we're waiting for all that chat the answer to come out there was a really good call out uh in the chat so the eight of us on air posted a link to Party Rock if you haven't checked that out definitely uh check it out it's a really fun way to get play around with geni you can create your app I have seen uh my my peers doing uh the party rock thing they have created an app that who would advise what would advise them better playlist better movies to watch and I mean the the the the sky is the limit there with it's awesome it is as as this one which we are showing so coming back I I asked these two models to write me a a use case generative a perspective from the technology and the industry should be financial industry and this is what it is talking about both of them I'm afraid I don't have enough context okay for for full Genera VI use case but it did give give out some things here automated customer service interaction which is employee or customer experience is boosting and here they are uh spanning up more onto the banking I think industry and they're spreading it out in that way so this is this is what you can do and there there the limit is endless I mean you you can even uh ask from the retail industry perspective write uh an advertisement for this this a phone charger or something like that and let's see what these different uh Foundation models simultaneously in real time do do do that for you and then you can choose it later on which one is better right oh awesome could you just show the drop down again of all the different options when it comes to the models that you can choose from just that way anyone absolutely absolutely let me let me do that let's go back yeah it's cool that you can be able to like choose from not just two add more models and like G's Point there's just so many options I hear a lot of people right now they're trying to really figure out based on their use case what's the right model to be able to use uh for their application so it's a it's great if you're someone that's wants to inter implement this yeah and wanted to clarify this for our audience that as you see the name Bedrock here these are all the models which you have access or requested access through Amazon Bedrock uh interface the UI when you go into the Management console you go and access manage uh management for these Foundation models and you click for allowing it and then you get access those are the ones uh I have access and I'm pretty sure you all do would do as well so this is where it's leveraging from the backend in architecture we saw how Bedrock is embedded and that's how we are getting all these access for all of them to leverage here right we've got a good question that I think is relates to a lot of the audience here we've got a lot of people who use AWS so um tidom wants to know how do we integrate this into an existing application that's already hosted in AWS yes uh a very good question if if uh there is an um maybe I think they mean to say that if there is a like a Wiki page or something or an application already running and they are are are you looking to integrate this like a regular chatbot right but but more efficiently there is there is a way I think for that and in this in these CD case we we can embed it same way very similarly because it is still a chatbot but on steroids right so we can embed it the same way with our backend uh uh coding of how you add it from from our Python scripts um it is somewhere here so yeah you have to dig in find it um but but you can surely do that because it is still a chat bot right so yep you you sure would be able to do that really cool yep so talking about those last two which are more dependent on uh on Rag and uh options with workspaces you can create your own workspace I have done for my own my knowledge space and would like to show something and then I'll create one for you all there so when you create your workspace these are the aspects or the formats of information you can upload and then you will let the rag to analyze it which these workspaces what they do they they can unlock the full potential of rag options it is like a playground or for your CH bot where you can test different settings embedding models configurations to see what works best for you for your scenario your leadership your company whatever you're looking for or maybe school homework right who knows right so these are uh where where I have uploaded a file in a PDF format text yes um yep U me being a a quantum physics and a space buff fanatic I wanted to talk about what are black holes since if any of us or my myself get into the chance to go into the space out of space and into a spaceship we should know what black holes are and and try to stay away from very important got to be prepared gotta be prepared so Q&A what is rag evaluation just for fun just for many examples but this list can go on and listly websites I put doc. AWS amazon.com and RSS feeds these this is I think recently added it was not uh previously added so RSS feeds I'm if not if anybody's not familiar this is a formatted text document that contains all important information about your show or something you're trying to host it's more more on the social media aspect it's hosted on a server usually has a public URL so I created my own URL for the fun sake my analysis and this the these creating RSS feeds from social network is even simpler just enter a URL like we did here and what you want to talk about so I wanted to see the feed back for this Amazon docs website into my analysis and and and analyze it later with the the rag what what came out to be more solid and positive and negative tone or connotations so talking about this let me show you all and in fact not show let's create a knowledge base for you all and then we will go back to our semantic search so how we do it first we need to create no not not adding I would like to create a a workspace so here is where you go go ahead and select these open search engines rora serverless Amazon open service serverless and of course Kendra I haven't chosen it yet that's why it's not showing active but while when we are doing the configuration right here at this point it will ask you that would you like to add the Kendra Enterprise Edition or or block it or create it you can choose yes or true and it you you would be able to show it after the see it after the deployment as pretty active Okay very yeah wait can you pause on this for a second so you're so this is related to a question that's actually in the chat so right now you're showing us in this Vector engine the actual data sources that are currently supported yes okay absolutely yeah these areed okay perfect more coming do by the way okay perfect so this this goes aligned with a man Raider jsp's question um and say and that want say my company has been building our own chatbot would be able to use the training data we've gathered to feed it to this roed out chatbot um and so that's where what we're seeing here of it would have to be in one of these supported uh data sources at this time that is correct that's that's a very good idea I mean I think this is a perfect scenario for them you can you can add your open search uh engine here and addit it deploy it and also after like if if you create your like your knowledge base or something like that and you can uh add your PDF files or whatever you wanted to analyze in very different uh formats here files text Q&A websites feeds everything and then we will show another aspect of debugging tools and yes that's that's a good scenario they can do that cool very cool and I think we have a few more questions that we might want to hit I know we're running a little tight on time I think so we want to make sure we get these someone ask if you have image recognition we you did have you know a model processing images do you consider that image recognition do you consider that more do you consider that it is it is image recognition also altogether but there is a particular model of the topic besides this also that Claud uh the very latest version has come up with recent uh image recognition in a way that if you just wanted to upload a a PDF or a scan document it will make the whole notes for you I mean that is so cool usually we we will embed some text and then try to summarize it right but scan documents pictures PDF that is hard but you can just put a scan documents of a PDF or a taken picture it will get the very proper awesome notes for you Amazon CL Claud uh uh we latest version V2 V3 would do that certainly for you in Amazon Bedrock that's awesome because that also hits on our other question about textract so thank you there was this is this can do even more and I think one of the big takeaways there is that as all the models you know improve and add more functionality that gets delivered to the customers as well that are using this absolutely it's all in the coming V are getting so many previews and gas are soon to be announced so certainly there there's more to come but the scad model like I said before if you have your Bedrock access you would be able to pull it right here in this UI of our gen gen chatbot and you would be able to leverage it in any any aspect definitely that is a a strong option here for our audience to uh leverage and with that let me show you some of uh little bit more ntegrity and uh debugging tools uh of course very beneficial because usually this is our UI right and it's not very common to have the debugging which is already very boring kind of a structure logs you have to read manually kill your eyes so the debugging is also present here and that is what uh is very awesome and I would like to show let me go into some uh in fact you know what first of all let's let's do some semantic search you all have seen my knowledge base where I have uploaded a certain uh PDF so let me select that and let me let me throw out a question that um what is uh black hole of course I'm biased I will go ahead and select anything to do without a space and there you go so this was better into my knowledge Bas and this is how you can analyze your docs now yes I know there are too many things Vector search we will definitely talk about it a black hole is an astronomical object all that details what are the source type and what is the ranking we will talk about this Vector search keyword search and the these are different uh um responses they're getting from different uh models and the ranking shows which is more precise and more accurate to the actual question asked here and this is how you you will find out that which is the best content to pick for your uh uh documents or for your uh pieces whatever you writing for right very cool and let's talk about the embeddings and um let me choose large one or maybe Bedrock Titan okay now I will load some sample data so this is how you this is just a sample data it this can be all your data to analyze so this is just one liner for the sake of our ease today and generate some sample data let's see if it does there you go I know don't don't get turned off by this uh uh too much of numbers there this is these are just the way of uh the chat bot or or the AI analyzing what we are saying so what is a vector vector is like ever wondered how music apps suggest songs to you or shopping apps suggest products that perfectly matches your taste to understand how you have to dive deep into these world of vector databases where data isn't just stored in tables and rows and words like that but it is more mapped as geometric points in space and also embedding Works simultaneously with Vector like a partnership aspect but it's a numerical representation what you see here on the screen and let me describe this so what you see is on and the essence of the process is to convert an object what we have sent as as a normal human text such as image or text yes into a vector that encapsulates that semantic content uh somehow while while discarding irrelevant detail as much as possible so an embedding takes a piece of content like a word sentence image and map it into multi-dimensional Vector space which we see here so they have certain certain ratings for it which we see here Co coign similarity uclean distance which which means it measured the straight line distance between two vectors in a space I know I'm getting too technical sorry about that stop me right away ladies if it is too too boring it's great I think our audience is loving it so by yeah I'm great great then so I'll keep going on so it ranges from zero to uh infinity so we zero if you see something starting with zero it it represents identical vectors which is more of a match larger values represent increasingly dissimilar Vector so this is what it is showing us for this sample data what we have just pulled and this is the vector value and then you ukian distance is what is what we just described but there's a cosine distance what is uh this is the similarity measure calculates uh of the angle between two separate vectors in a space it ranges from negative one to one a positive one where one represents identical zero represents a little bit orthogonal just two way on the graph and negative 1 represents completely diametrically opposed so that means um zero or or or less is better option for it simultaneously they work hand in hand here we are seeing uh these Vector values and here we are seeing embeddings where it's showing the rank and this is how our numerical values are placed in the back end whatever we saying this sample data is is translated into this form and I think yeah pre uh our our Fanatics for for technical aspects I'm pretty sure they are very well aware of this aspect so this is another thing you can analyze for your D if you wanted to analyze a document or or compare between multiple PDFs right you can throw it in there upload it uh pull up sample data and analyze it which one has great results for your sake it's so cool um I mean we're running out of time but I'm sure a lot of people are going to want to rewatch this especially this part because they're like I want to really understand what's going on so fortunately you can watch all the reruns on the AWS on air twitch Channel or the AWS twitch Channel you can go and watch this entire thing uh and rewind this part there other aspects of it you can do the ranks to see which which response has the highest rank rating I just wanted to quickly throw that out there but yeah yeah there are some thank you so much Sarah this was awesome yeah this was really helpful and I think our our audience loved loved seeing it and loved hearing all the very specific details it wasn't just an overview you know this video can be used used as come back just to mention these use cases are endless and I'm I'm pretty sure if anybody's understanding it well this can fit into any use cases out there in the industry so thank you so much thank you perfect thank you bye-bye

Channel: AWS Events

Views: 978

Keywords: AWS, Amazon Web Services, AWS Cloud, Amazon Cloud, AWS Events, AWS OnAir, AWS On Air, S05, Gigi Boehringer, Jillian Forde

Id: 5HyJvIxCc84

Length: 33min 48sec (2028 seconds)

Published: Tue Apr 02 2024