Introduction to Generative AI Studio

welcome to the introduction to the generative AI Studio course in this video you learn what generative AI studio is and describe its options for use you also demo the generative AI Studios language tool yourself what is generative AI it is a type of artificial intelligence that generates content for you what kind of content well the generative content can be multimodal including text images audio and video when given a prompt or a request generative AI can help you achieve various tasks such as document summarization information extraction code generation marketing campaign creation virtual assistance and call center bot and these are just a few examples how does AI generate new content it learns from a massive amount of existing content this includes text audio and video the process of learning from existing content is called training which results in the creation of a foundation model in llm or large language model which Powers chat Bots like Bard is a typical example of a foundation model the foundation model can then be used to generate content and solve General problems such as content extraction and document summarization it can also be trained further with new data sets in your field to solve specific problems such as financial model generation and Healthcare Consulting this results in the creation of a new model that is tailored to your specific needs how can you use the foundation model to power your applications and how can you further train or tune the foundation model to solve a problem in your specific field Google Cloud provides several easy to use tools that help you use generative AI in your projects with or without an AI and machine learning background one such tool is vertex AI vertex AI is an end-to-end ml development platform on Google Cloud that helps you build deploy and manage machine learning models with vertex AI if you are an app developer or data scientist and want to build an application you can use generative AI Studio to quickly prototype and customize generative AI models with no code or low code if you are a data scientist or ml developer who wants to build and automate a generative AI model you can start from model Garden model Garden lets you discover and interact with Google's foundation and third-party open source models and has built-in ml Ops tools to automate the ml pipeline in this course you focus on generative AI Studio generative AI Studio supports language vision and speech the list grows as you are learning this course for language you can design a prompt to perform tasks in tune language models for vision you can generate an image based on a prompt and further edit the image for speech you can generate text from speech or vice versa let's focus on what you can do with language and generative AI Studio specifically you can design prompts for tasks relevant to your business use case including code generation create conversations by specifying the context that instructs how the model should respond and tune a model so it is better equipped for your use case which allows you to then deploy to an endpoint to get predictions or tested in prompt design let's walk through these three features in detail first is prompt design to get started experimenting with large language models or llms click on new prompt in the world of generative AI a prompt is just a fancy name for the input text that you feed to your model you can feed your desired input text like questions and instructions to the model the model will then provide a response based on how you structured your prompt therefore the answers you get depend on the questions you ask the process of figuring out and designing the best input text to get the desired response back from the model is called prompt design which often involves a lot of experimentation let's start with a free form prompt one way to design a prompt is to Simply tell the model what you want in other words provide an instruction for example generate a list of items I need for a camping trip to Joshua Tree National Park we send this text to the model and you can see the model outputs a useful list of items we don't want to camp without this approach of writing a single command so that the llm can adopt a certain behavior is called zero shot prompting generally there are three methods that you can use to shape the model's response in a way that you desire zero shot prompting is a method where the llm is given no additional data on the specific task that is being asked to perform instead it is only given a prompt that describes the task for example if you want the llm to answer a question you just prompt what is prompt design One-Shot prompting is a method where the llm is given a single example of the task that is being asked to perform for example if you want the llm to write a poem you might provide a single example poem and few shot prompting is a method where the llm is given a small number of examples of the task that it is being asked to perform for example if you want the llm to write a news article you might give it a few news articles to read you can use the structured mode to design the fuse shot prompting by providing a context and examples for the model to learn from the structured prompt contains a few different components first we have the context which instructs how the model should respond you can specify words the model can or cannot use topics to focus on are avoid or a particular response format and the context applies each time you send a request to the model let's say we want to use an llm to answer questions based on some background text in this case a passage that describes changes in rainforest vegetation in the Amazon we can paste in the background text as the context then we add some examples of questions that could be answered from this passage like what does lgm stand for or what did the analysis from the sentiment deposits indicate we'll need to add in the corresponding answers to these questions to demonstrate how we want the model to respond then we can test out the prompt we've designed by sending a new question as input and there you go you've prototyped a q a system based on background text in just a few minutes please note a few best practices around prompt design be concise be specific and well-defined ask one task at a time turn generative tasks into classification tasks for example instead of asking what programming language to learn ask if python Java or C is a better fit for a beginner in programming an improve response quality by including examples adding instructions in a few examples tends to yield good results however there's no one best way to write a prompt you may need to experiment with different structures formats and examples to see what works best for your use case for more information about prompt design please check text prompt design in the reading list so if you've designed a prompt that you think is working pretty well you can save it and return to it later your saved prompt will be visible in the prompt Gallery which is a curated collection of sample prompts that show how generative AI models can work for a variety of use cases finally in addition to testing different prompts and prompt structures there are a few model parameters you can experiment with to try and improve the quality of responses first there are different models you can choose from each model is tuned to perform well on specific tasks you can also specify the temperature top p and top k these parameters all adjust the randomness of responses by controlling how the output tokens are selected when you send a prompt to the model it produces an array of probabilities over the words that could come next and from this array we need some strategy to decide what to return a simple strategy might be to select the most likely word at every time step but this method can result in uninteresting and sometimes repetitive answers on the contrary if you randomly sample over the distribution returned by the model you might get some unlikely responses by controlling the degree of Randomness you can get more unexpected and some might say creative responses back to the model parameters temperature is a number used to tune the degree of Randomness low temperature means to select the words that are highly possible and more predictable in this case those are flowers and the other words that are located at the beginning of the list this setting is generally better for tasks like q a and summarization where you expect a more predictable answer with less variation high temperature means to select the words that have low possibility and are more unusual in this case those are bugs and the other words that are located at the end of the list this setting is good if you want to generate more creative or unexpected content in addition to adjusting the temp aperture top K lets the model randomly return a word from the top K number of words in terms of possibility for example top 2 means you get a random word from the top two possible words including flowers and trees this approach allows the other high scoring word a chance of being selected however if the probability distribution of the words is highly skewed and you have one word that is very likely and everything else is very unlikely this approach can result in some strange responses the difficulty of selecting the best top K value leads to another popular approach that dynamically sets the size of the short list of words top P allows the model to randomly return a word from the top P probability of words with top P you choose from a set of words with the sum of the likelihoods not exceeding P for example P of 0.75 means you sample from a set of words that have accumulative probability greater than 0.75 in this case it includes three words flowers trees and herbs this way the size of the set of words can dynamically increase and decrease according to the probability distribution of the next word in the list in some generative AI Studio provides a few model parameters for you to play with such as the model temperature top K and top P note that you are not required to adjust them constantly especially top K and top p now let's look at the second feature which creates conversations first you need to specify the conversation context context instructs how the model should respond for example specifying words the model can or cannot use topics to focus on or avoid or response format context applies each time you send a request to the model for a simple example you can define a scenario and tell the AI how to respond to help desk queries your name is Roy you are a support technician of an I.T Department you only respond with have you tried turning it off and on again to any queries you can tune the parameters on the right the same as you do when designing the prompt to see how it works you can type my computer is slow in the chat box and press enter the AI responds have you tried turning it off and on again exactly as you told the AI to do the cool thing is that Google provides the apis and sdks to help you build your own application you can simply click view code first you need to download the vertex AI sdks that fit your programming language like Python and curl SDK stands for software design kits they implement the functions and do the job for you you use them like you call libraries from the code you then follow the sample code and the API and insert the code into your application now let's look at the third feature tune a language model if you've been prototyping with large language models you might be wondering if there's a way you can improve the quality of responses Beyond just prompt design so let's learn how to tune a large language model and how to launch a tuning job from generative AI Studio as a quick recap The Prompt is your text input that you pass to the model your prompt might look like an instruction and maybe you add some examples then you send this text to the model so that it adopts the behavior that you want prompt design allows for fast experimentation and customization and because you're not writing any complicated code you don't need to be an ml expert to get started but producing prompts can be tricky small changes in wording or word order can affect the model results in ways that aren't totally predictable and you can't really fit all that many examples into a prompt even when you do discover a good prompt for your use case you might notice the quality of model responses isn't totally consistent one thing we can do to alleviate these issues is to tune the model so what's tuning well one version you might be familiar with is fine tuning in this scenario we take a model that was pre-trained on a generic data set we make a copy of this model then using those learned weights as a starting point we retrain the model on a new domain-specific data set this technique has been pretty effective for lots of different use cases but when we try to fine-tune llms we run into some challenges llms are well as the name suggests large so updating every weight can take a long training job compound all of that computation with the hassle and cost of now having to serve this giant model and as a result fine-tune a large language model might not be the best option for you but there is an Innovative approach to tuning called parameter efficient tuning this is a super exciting research area that aims to reduce the challenges of fine-tuning llms by only training a subset of parameters these parameters might be a subset of the existing model parameters or they could be an entirely new set of parameters for example maybe you add on some additional layers to the model or an extra embedding to The Prompt if you want to learn more about parameter efficient tuning and some of the different methods a summary paper is included in the reading list of this course but if you just want to get to building then let's move to generative AI studio and see how to start a tuning job from the language section of generative AI Studio select tuning to create a tune model we provide a name then point to the local or cloud storage location of your training data parameter efficient tuning is ideally suited for scenarios where you have modest amounts of training data say hundreds or maybe thousands of training examples your training data should be structured as a supervised training data set in a text to text format each record or Row in the data will contain the input text in other words The Prompt which is followed by the expected output of the model this means that the model can be tuned for a task that can be modeled as a text to text problem after specifying the path to your data set you can start the tuning job and monitor the status in the Google Cloud console when the tuning job completes you'll see the tuned model in the vertex AI model registry and you can deploy it to an endpoint for serving or you can test it in the generative AI Studio in this course you learned what generative AI is and the tools provided by Google Cloud to empower your project with generative AI capabilities specifically you focused on generative AI Studio where you can use gen AI in your application by quickly prototyping and customizing generative AI models you learn that generative AI Studio supports three options language vision and speech you then walked through the three major features in language design and test prompt create conversations and tune models this was a short lesson introducing generative AI Studio on vertex AI for more information about natural language processing and different types of language models like decoder encoder Transformer and llm please check the course titled natural language processing on Google Cloud listed in the reading list now it's time to play with generative AI studio in a Hands-On lab where you design and test prompts in both free form and structured modes create conversations and explore the prompt Gallery by the end of this lab you will be able to use the capabilities of generative AI Studio that we've discussed in this course have fun exploring

Channel: Google Cloud Tech

Views: 48,583

Keywords: generativeAI, genAI, GoogleCloudLearning

Id: -7nf5EJ2Fsc

Length: 16min 7sec (967 seconds)

Published: Wed Jun 28 2023