Amazon Bedrock Tutorial – Model Access, Playgrounds, APIs and Fine-Tuning | AWS Generative AI

hi everyone with Amazon Bedrock you can do all the things you've come to expect with a generative AI offering like generating images of a cute kitten of course very nice or having a chat with the AI here we'll choose the AI 21 Labs model and asking about the meaning of life you can also create your own custom models starting with a base model and then fine-tuning it on your own data and of course you can also interact with the models through code using the apis in this video we're going to get into all of these things but first a quick look at the theory and how Bedrock can ultimately be used to create AI applications back here in AWS console you want to make sure that you're in one of the regions where this is currently generally available if you select your region on the top right here according to the official documentation it's it's in three regions although I'm seeing five regions highlighted here but officially it's Us East 1 North Virginia US West 2 Oregon and AP Northeast 1 Tokyo so make sure you're in one of those regions I expect that will change over time but that's the current General availability and then if you come over to the overview tab here on the left navigation you'll see that you've got these six foundational models so you can pick and choose one or more that's best suited for what you need this is different than something like open AI where you're limited to only their GPT models here you can kind of mix and match up on the build and test tab you'll see that you can also F tune or build your own custom models using your own data and ultimately what you really want from all of this is to build your own AI applications there's a pretty cool demo out here at AI stylist. AWS player.com this is Amazon's demo I did not build this but if you try the free demo I think it's a great illustration of how it's pulling everything together behind the scenes so let's start exploring and get some styling advice here if we start now here in the prompt it says that we're traveling to New York we need an outfit to wear in the office and then we can generate my look so what's happening behind the scenes we've got the customer prompt over here is all of the AI magic that's happening you'll see that we're using Claude one of the base models this is used for Content creation and complex reasoning we're also using stable diffusion for image generation it's going to generate some images of outfits that we can wear we're also using Amazon Titan and then claw again and then we're also pulling private data so we've got a product catalog fashion trends we've got order history customer reviews and so on so we're using the base models augmenting all of that with our own private data to generate some outfits so if we view your looks it's selected two outfits for us we've got business formal and business casual so a nice illustration of ultimately how you could use this in the real world and then just a quick geology lesson for you if you're wondering why it's called Amazon Bedrock I couldn't find an official answer for this but if you look at the image of Bedrock here it's these rocks here so a very solid reliable foundation for building your AI applications now let's go out to the console and see how to get started with all of this but first things first what will all of this cost pricing for Bedrock varies widely depending on your model but it all starts with tokens out on the pricing page here you'll see there's a breakdown for each model per 1,000 tokens for estimation purposes Amazon recommends using six characters as equal to one one token so it will vary depending on which model you're using but you can see it here for this demo and the examples that I've done I spent about 10 cents so it's not going to break the bank but just know that there is a price to all of this and it'll show up on your regular AWS bill there's also different pricing models here you'll see on demand that's where you're just paying for what you use that's what I'm going to be using if you have a larger workload that needs guaranteed throughput you can use the provisioned throughput model you can choose between 1 month or six-month commitment terms and then for fine-tuning or your training it on your own data and this can get very pricey so I would recommend staying away from this one unless you really know what you're doing all right with pricing out of the way let's head back over to the console and talk about how to get access to the models so we already talked about using a region where Bedrock is generally available and you can just navigate there and it will magically work work but you don't automatically get access to the models so to do that you'll want to come down to model access here on the lower left I have already requested access to a lot of these but if this is the first time you're here most of these will say available to request or not available and so on so what you want to do is just come into manage model access and then you'll want to click on anything that you want access to you'll notice that some things are unavailable because they're in preview or whatnot and then for some things like Claud this will require you to submit a use case before you can access it so this one's not just open to the world but for some of these others you really just have to click the check box to get access once you've selected everything you want then you'll just say save changes if you want to review some of the differences of these you can always just come into base models over here I'll open this in a new tab and here you can view more details about all of the different models what they're used for Max tokens and so on so figure out what makes sense for you you don't have to select all of them obviously but select the ones that you want save changes it can take up to 72 hours to get your access granted in my experience it was actually pretty immediate that might vary depending on how many people are requesting access but eventually you should get access to most of the things that you want and then you're often running once you've got that access you can come into the playgrounds over here on the left we'll just start with chat the playgrounds are a great way just to play with things get familiar with the models and how they work this is similar to open AI playground if you've done any work with that but from here you can select your model so for chat the two models that I have access to are AI 21 labs and then meta I don't have any custom models at the moment but if I did those would show up here but we'll just go back to AI 21 labs for this one I actually had two different models available so we'll just go with the Jurassic to Mid here currently using on demand capacity I could switch though I will just leave that and then down here for chat you can give it a Persona so these are the instructions or sort of background information so we'll say you are a travel agent specializing in adventure travel just any kind of background or context that you want the model to take into account as it's giving you responses and then let's say I'm planning a drip I want some suggestions if I want to hike in kayak somewhere with moderate climate and then ask for suggestions and here on the chat playground this is really meant to have multi-turn conversations you can go back and forth it looks like this is actually getting cut off so there are some configurations you can make here these top two will control Randomness and diversity the length l so it looks like I'm getting cut off at 200 for this demo I'm just going to leave it there that's fine but just know that this controls the tokens basically or how much you're charged so don't go wild with that if you don't need it then some other things that you can configure here as well and update as you go along but that's what chat does text is pretty similar except you're not able to give this a Persona or instructions I do have a few models available for chat though let's go with Co here and the one model there I could also configure some parameters for this and we'll say something like write a poem about working with AWS and run and there's our poem very nice and for all of these you can view the API request which will come in handy in just a few minutes when we get into code but this is basically what you could use to pass in in your code and then finally the image playground this will generate images similar to dolly or mid Journey if you're familiar with those here I only have one model available for images from stability Ai and let's see what we can get for a robot sitting at a laptop drinking coffee I've got some configuration I can do over here on the right for each of these you can click on info and get more detail about what the parameter is but let's see what we get and there's our robot all right so those are the basics of working in the UI with the playground but I'm guessing a lot of you are interested in using the apis and writing code so let's do that I'll just get you started but I will put a link to the API documentation in the description for the video there are lots of ways that you can work with the apis including the CLI the sdks or even a sagemaker notebook if you work with sagemaker at all for this demo I'll just quickly spin up a Cloud9 environment which is aws's in browser IDE just makes things super simple so I'll come back to the console open up Cloud9 if you have a VSS code or another IDE set up locally that should work just fine as well I'll just quickly create the environment you can follow along if you want go with all the defaults here the t2 micro free tier eligible and create this will take a minute or so to spin up so I'll pause the video and be right back and success if you are using Cloud9 just a reminder to delete this when we're done I'll walk you through how to do that but for now let's open this up and we're just going to do a really simple python example of submitting a prompt and getting back an answer so over here I will create a new file from template this will be a python file and I'm going to paste in some code I will put this code in the description as well if you want to use it then I'll hit contrl s to save I'll call this bedrock.la and let me increase the size here there we go now for this you will need to install the AWS SDK for python it's called Bodo 3 to do that you can just type pip install Bodo 3 and there we go so walking through the code we're importing Bodo 3 and Json then we need to create the client for interacting with bedrock here make sure that you're using a region where this is available so again the ones that I mentioned in the beginning uswest 2 is one of them and then moving down we've got this invoke model call if we look back at the API reference here under Bedrock runtime we've got invoke model and you'll see the request parameters that we need here and so encode we're basically making an input object with all of those here I'm using the meta llama 2 model content type Json accept everything and then this is the important bit here basically our prompt I need an idea for an app to build on Amazon Bedrock we've got some parameters once again you can grab all of these from the playground so back here that was text once you enter your prompt you can just say view API request and then you can get those values bring them back here to code update that and you should be good to go here we actually invoke the model get a response back and then we're just going to print it out to make sure everything is working in the real world you'd probably send this back to the UI to your user or something like that but to run this save the file we just say Python and then the name of the file bedrock and we should get a response back okay and here's what the model is saying I've been brainstorming for a while but I'm having trouble coming up with something oh dear but it is giving us some ideas all right so those are the apis pretty easy to get started with those if you did set up a Cloud9 environment and let's shut that down so you don't forget and have a surprise bill just come back to Cloud9 select your environment and then delete and delete and then the last thing I want to show you is the custom models so I'll come back to the Bedrock UI over here zoom out a little bit and come into custom models here on the lefthand navigation and here's where you can take a base model fine-tune it using your own data to get started on this you'll say customize model and create fine tune job now this is where things can get really really expensive so I would recommend that you don't do this unless you really know what you're doing I'm actually not going to do a full fine tune job here but I'll just show you how to get started so you want to select the source model here I've only got the options of the Titan models from Amazon I'll just go with the light version you want to give your finetune model a name give your training Joba name Additionally you can give it tags you can update the virtual product cloud our VPC settings the network basically and then here's where you point it to the data that you want to train on this will need to live in an S3 bucket and then you can also give it a validation data set as well also in S3 depending on your model there will be different hyperparameters that you can tune here or you can just leave the defaults for output data this is where the training job output data will live so you'll want to select an S3 bucket for that bedrock will need permissions to write to S3 on your behalf so you'll need to set up a service role with those permissions and then basically find tune model this is going to start your training job this could take a while depending on your base model and how much data you're passing in they'll be a way to monitor you can analyze the results and then eventually use that fine-tune model so that's how to get started again I'm not going to do it and I would recommend that you do not unless you really know what you're doing and you have the budget to support it all right so those are the basics of Amazon Bedrock how to get access to the models how to use the playgrounds getting started with the apis and with custom models if you found that helpful give me a big thumbs up on the video and also consider subscribing for more content like this and thank you so much for watching

Channel: Tiny Technical Tutorials

Views: 9,441

Keywords: technical tutorials, technical training, technology, aws generative ai, aws genai, amazon generative ai, amazon bedrock, aws bedrock, ai on aws, bedrock tutorial

Id: 32D7NJK9QIk

Length: 15min 46sec (946 seconds)

Published: Sun Nov 19 2023