Introduction to Vertex AI Studio

[Music] welcome to the vertex AI Studio course vertex AI studio is a primary tool for a cloud developer to access Google's Cutting Edge generative AI or Genai models it facilitates the testing tuning augmenting and deployment of these models enabling the creation of gen powered applications this course teaches you the knowhow of vertex AI Studio it starts from explaining the Gen workflow to introducing the major features of vertex AI Studio including Gemini multimodal prompt design and model tuning to enhance your learning experience a Hands-On Lab at the end of the course provides an opportunity to practice the skills you have acquired generative AI is transforming how we interact with technology so what is generative AI simply put it is a type of artificial intelligence that generates content for you what kind of content well the generated content can be multimodal including text images audio and video when given a prompt or a request gen aai can help you achieve various tasks such as document summarization information extraction code generation marketing campaign creation virtual assistance and call center bot and these are just a few examples how how does AI generate new content it learns from a massive amount of existing content this includes text image and video the process of learning from existing content is called training which results in the creation of foundation models a foundation model is usually a large model in terms of the significant number of parameters massive size of training data and high requirements of computational power an llm or large language model like Palm which stands for pathways language model is a typical example of a foundation model other Foundation models trained by Google include Gemini for multimodal processing Codi for code generation and imagine for image processing note the list may change as Foundation models Advance also note that Gemini May replace some of these models as it is capable of processing data in multiple modalities the pre-trained foundation model can be used to generate content and solve General problems such as content extraction and document summarization it can also be further trained or tuned with new data sets in your field to solve specific problems such as financial model generation and Healthcare Consulting this results in the creation of a new model that is tailored to your specific needs how can you use the foundation model to power your applications and how can you further train or tune the foundation model to solve a problem in your specific field vertex AI is a comprehensive machine learning platform offered by Google Cloud it supports endtoend ml processes including model creation deployment and management vertex AI provides two primary capabilities predictive Ai and generative AI in predictive AI you can build ml models for forecasting in generative AI you can use and tune gen models to produce content so how can you access the Gen models with vertex AI let's walk through the workflow input prompt via the vertex AI Studio UI input a prompt a natural language request to gen AI models responsible Ai and safety measures the prompt undergos responsible Ai and safety checks configurable through the UI or code Foundation models the screened prompt proceeds to Foundation models like gemini multimodal or other gen AI models like imagine and Codi based on your choices model customization optionally customize gen AI models to fit your data and use cases by further tuning them results grounding geni models return results that undergo grounding and citation checks to prevent hallucinations final response the final response appears on the vertex AI Studio UI after a final check through responsible Ai and safety measures in essence vertex AI Studio provides an intuitive interface and enables you to build gen applications in a low code or even no code environment where you can rapidly test and prototype models tune and customize models using your own data augment them with real world up-to-date information and deploy models efficiently in production environments with autogenerated code vertex AI Studio facilitates multimodal language vision and speech related tasks as you progress through this course the range of supported tasks May expand with multimodal capabilities data can be processed across various modalities like images videos and text this versatility enables the execution of multimodal tasks such as extracting text from an image for language you can design a prompt to perform tasks and tune language models for vision you can generate an image based on a prompt and further edit the image for speech you can generate texts from speech or vice versa in the upcoming lessons you will explore the capabilities and applications of multimodal and language let's first explore Gemini multimodal Google's most capable and general model yet so what is a mult M modal model it's a large Foundation model that is capable of processing information from multiple modalities including text image and video the generated content can also be in multiple modalities for example you can send the model a photo of a plate of cookies and ask it to give you a recipe for those cookies Gemini a Google trained multimodal model available on vertex AI Studio provides input processing capabilities for text images and video currently its output is limited to text however this might change as you progress through the course how can Gemini help you with your business use cases Gemini excels at a diverse range of multimodal use cases here are some notable examples description and captioning Gemini can identify objects in images and videos providing detailed or concise descriptions as needed information extraction it can read text from images and videos extracting important information for further processing information analysis it can analyze the information it extracts from images and videos based on specific prompts for instance it can classify expenses on a receipt information seeking Gemini can answer questions or generate Q&A based on the information it extracts from images and videos content creation it can create stories or advertisements using images and videos as inspiration data conversion it can convert text respon respes into various formats such as HTML and Json can you think of a real world use case to apply Gemini multimodal in light of these exciting advancements how can developers engage with Gemini and create applications that leverage multimodal capabilities there are three primary approaches each essentially achieving the same objective using a user interface UI with the Google Cloud console this no code solution is ideal for exploring and testing testing prompts using predefined sdks with notebooks like collab and workbench which are seamlessly integrated within the vertex AI platform utilizing Gemini apis in conjunction with commandline tools like curl regardless of which method to access the Gemini multimodal you start with a prompt so what is a prompt in the world of generative AI a prompt is a natural language request submitted to a model in order to receive a response you feed the desired input text like questions and instructions to the model the model will then provide a response based on how you structured your prompt therefore the answers you get depend on the questions you ask let's look at the anatomy of a prompt which includes one or more of the following components input context and examples an input represents your request for a response from the model it can take various forms a question that the model can answer question input a task that the model can perform task input an entity that the model can operate on entity input partial input that the model can complete or continue completion input for instance what should I do when my computer freezes here what should I do when my computer freezes is a question that you expect the model to provide an answer for context can serve multiple purposes specify instructions to guide the model's Behavior provide information for the model to use or reference in generating a response when you need to supply information or limit responses within the scope of your prompt include contextual information for instance you could assume the role of an IT help desk and consistently advise users to restart their computers regardless of the nature of their inquiries examples are pairs of inputs and outputs that demonstrate the desired response format to the model incorporating examples in the prompt is an effective technique for tailoring the response format for instance you can enter input output pairs in the following manner lost internet connection reset couldn't find the network printer restart subsequently you can type in the prompt what should I do when my computer runs slowly the model will likely suggest that you reset the computer context and examples are extensively utilized when training or tuning generative AI modes to behave as you desired the process of figuring out designing the best input text to get the desired response back from the model is called prompt design which often involves a lot of experimentation you'll learn more about prompt and prompt design later in this course let's explore the Gemini multimodal feature within vertex AI Studio navigate to the vertex AI Studio overview page and click on multimodal powered by Gemini try it now you'll see three sections a prompt field at the top a response field at the bottom and a configuration panel on the right click insert media and upload an image for example let's use a departure board image from an airport enter your first prompt read the text from the image before clicking submit let's look at the configuration on the right you can choose from a list of models the default model is usually the most recent Cutting Edge model which is currently Gemini Pro Vision the temperature setting controls the degree of Randomness in the response with zero being the most expected answer and one being the most creative the safety settings allow you to adjust the likelihood of receiving a response that could contain harmful content content is blocked based on the probability that it's harmful for example for hate speech you can choose from block few block some and block most adjust these settings based on your use case and click save in the next lesson you'll learn more about advanced settings such as top K and top p once you've completed the configuration it's time to get the response click submit and wait a moment here's the result if the result is not easy to read you can further adjust your prompt to read the text from the image and put it into two columns time and destination does the result look better you can also be more adventurous and do some analysis for example you can change the prompt to calculate the percentage of the flights to different continents and put them into two columns percentage and continent here is the result if you desire to further develop the application and make the process productional lied you can click the code located on the top right corner there you'll find the code describing the prompt and the settings in the user interface alternatively you can retrieve curl which serves as the API to call in a commandline interface Additionally you have the option to open a notebook with the sdk's code of your preferred programming languages such as python these automated generated coding and the integrated development environment significantly simplify the production process hopefully this provides a straightforward guide on how to utilize Gemini multimodal with vertex AI Studio at the end of this course you practice with prompts and settings in a Hands-On practice you explore Gemini multimodal in the previous lesson now it's time to focus on the language capabilities offered by vertex AI Studio let's explore what you can do with the language models design prompts for tasks relevant to your business use case including code generation create conversations by specifying the context that instructs how the model should respond and tune a pre-trained model to get better performance for a specific task or use case you'll walk through prompt design and model tuning in detail let's first delve into the Art and Science of prompt design to get started experimenting with the Gen models click on new prompt let's start with a free form prompt one way to design a prompt is to Simply tell the model what you want in other words provide an instruction for example generate a list of items I need for a camping trip to Joshua Tree National Park we send this text to the model and you can see that the model outputs a useful list of items we don't want to camp without this approach of writing a single command so that the model can adopt a certain behavior is called zero shot prompting generally there are three methods that you can use to shape the model's response in a way that you desire zero shot prompting is a method where the model is given a prompt that describes the task without additional examples for example if you want the llm to answer a question you just prompt what is prompt design oneshot prompting is a method where the llm is given a single example of the task that it is being asked to perform for example if you want the llm to write a poem you might provide a single example poem and F shot prompting is a method where the model is given a small number of examples of the task that it is being asked to perform for example if you want the model to write a news article you might give it a few news articles to read you can use the structured mode to design the few shot prompting by providing a context and additional examples for the model to learn from the structured prompt contains a few different components first we have the context which instructs how the model should respond you can specify words the model can or cannot use topics to focus on or avoid or a particular response format and the context applies each time you send a request to the model let's say we want to use the model to answer questions based on some background text in this case a passage that describes changes in rainforest vegetation in the Amazon we can paste in the background text as the context then we add some examples of questions that could be answered from this passage like what does lgm stand for or what did the analysis from the sediment deposits indicate we'll need to add in the corresponding answers to these questions to demonstrate how we want the model to respond then we can test out the prompt we've designed by sending a new question as input and there you go you've prototyped a Q&A system based on background text in just a few minutes please note a few best practices around prompt design be concise be specific and clearly defined ask one task at a time turn generative tasks into classification tasks for example instead of asking what programming language to learn ask if python Java or C is a better fit for a beginner in programming and improve response quality by including examples adding instructions and a few examples tend to yield good results however there's currently no one best way to write a prompt you may need to experiment with different structures formats and examples to see what works best for your use case for more information about prompt design please check text prompt design in the reading list so if you designed a prompt that you think is working pretty well you can save it and return to it later your saved prompt will be visible in the prompt Gallery which is a curated collection of sample prompts that show how generative AI models can work for a variety of use cases finally in addition to testing different prompts and prompt structures there are a few model parameters you can experiment with to try to improve the quality of responses first there are different models you can choose from each model is tuned to perform well on specific tasks you can also specify the temperature top p and top K these parameters all adjust the randomness of responses by controlling how the output tokens are selected when you send a prompt to a model it produces an array of probabilities over the words that could come next and from this array you need some strategy to decide what it should return a simple strategy might be to select the most likely word at every time step but this method can result in uninteresting and sometimes repetitive answers on the contrary if you randomly sample over the distribution returned by the model you might get some unlikely responses by controlling the degree of Randomness you can get more unexpected and some might say creative responses back to the model parameters temperature is a number used to tune the degree of randomness low temperature means to narrow the range of possible words to those that have high possibilities and are more typical in this case those are flowers and the other words that are located at the beginning of the list this setting is generally better for tasks like questions and answers in summarization where you expect a more typical answer with less variability high temperature means to extend the range of possible words to include those that have low possibility and are more unusual in this case those are bugs in other words that are located at the end of the list this setting is good if you want to generate more creative or unexpected content in addition to adjusting the temperature top K lets the model randomly return a word from the top K number of words in terms of possibility for example top two means you get a random word from the top two possible words including flowers and trees this approach allows the other high-scoring word a chance of being selected however if the probability distribution of the word is highly skewed and you have one word that is very likely and everything else is very unlikely this approach can result in some strange responses the difficulty of selecting the best top K value leads to another popular approach that dynamically sets the size of the short list of words top P allows the model to return a random word from the smallest subset with the sum of the likelihoods that exceeds or equals to P for instance P of 0.75 means you sample from a set of words that have a cumulative probability greater than 0.75 in this case it includes three words flowers trees and herbs this way the size of the set of words can dynamically increase and decrease according to the probability distribution of the next word on the list in some vertex AI Studio provides a few model parameters for you to play with such as the model type temperature top K and top P note that you are not required to adjust them constantly especially top K and top P having explored prompt design a fundamental aspect of interacting with geni models let's progress to a more advanced topic model tuning if you've been prototyping with the generative AI models like an llm you might be wondering if there's a way you can improve the quality of responses Beyond just prompt design let's learn how to tune customize a gen a model and how to launch a tuning job from vertex AI Studio you have different choices to customize and tune a gen model from less technical like prompt design to more technical like distilling you're already familiar with prompt design which lets you tune a gen AI model using natural language without ml background The Prompt which serves as the input text is designed to elicit a desired outcome from the model to enhance the model's performance you can provide provide context and examples to guide its responses prompt design does not alter the parameters of the pre-trained model instead it improves the model's ability to respond appropriately by teaching it how to react one benefit of prompt design is that it enables rapid experimentation and customization another benefit is that it doesn't require specialized machine learning knowledge or coding skills making it accessible to a wider range of users but producing prompts can be tricky small changes in wording or word order can affect the model results in ways that aren't totally predictable and you can't really fit all that many examples into a prompt even when you do discover a good prompt for your use case you might notice the quality of model responses isn't totally consistent one way to address the issues is to tune the models using your own data however fine-tuning the entire model can be impractical due to the high computational resources cost and time required as the name large suggests llms have a vast number of parameters making it computationally demanding to update every weight instead parameter efficient tuning can be employed this involves making smaller changes to the model such as training a subset of parameters or adding additional layers and embeddings this approach has gained significant attention in the research Community as it aims to reduce the challenges of fine-tuning large language models by only training a subset of parameters for example you can have adapter tuning which is supervised tuning lets you use as few as 100 examples to improve model performance reinforcement tuning which is unsupervised reinforcement learning with human feedback if you want to learn more about parameter efficient tuning and some of the different methods please check out the summary paper included in the reading list distillation a more technical tuning technique enables training smaller task specific models with less training data lower serving costs and latency than the original model this technique is exclusive to Google Cloud through vertex AI you can access the newest techniques from Google research available with distilling step- bystep this technology transfers Knowledge from a larger model to a smaller model to optimize performance latency and cost the rationale is to use a larger teacher model to train smaller student models to perform specific tasks better and with improved reasoning capabilities the training and Distilling process uses labeled examples and rationals generated by the teacher model to fine-tune the student model rationals are like asking the model to explain why examples are labeled the way they are similar to how you learn better by understanding the reason behind an answer this type of teaching makes the student model more accurate and robust now let's move to vertex AI studio and see how to start a tuning job please note the UI may change when the product progresses from the language section of vertex AI Studio select tuning and distill for model details you can choose from either supervised tuning which is the adapter tuning as mentioned earlier or unsupervised tuning which is reinforcement tuning give the tuned model a name choose the base model the region and the output directory parameter efficient tuning is ideally suited for scenarios where you have modest amounts of training data can be as low as 100 training examples tuning data set is where you specify the location of your training data set please note it needs to be uploaded or existing on cloud storage your training data should be structured as a supervised training data set in a Json file each record or row contains a pair of text Data the input text which is the prompt and the output text which is the expected response from the model this structure allows the model to learn and adapt to your desired Behavior you can then start the tuning job and monitor the status in the Google Cloud console when the tuning job completes you'll see the tuned model in the vertex AI model registry and you can deploy it to an endpoint for serving or further test it in vertex AI studio in this course we've covered a wealth of valuable information to assist you in embarking on your own gen AI project let's recap the key points vertex AI Studio serves as the primary interface for accessing geni models allowing you to test tune augment and deploy them you began with the concept of gen AI the Gen AI workflow on vertex Ai and the significant capabilities provided by vertex AI Studio including multimodal language vision and speech you then focus on Gemini multimodal which is Google's most capable and general model yet you learn different ways to inter with Gen AI models and walked through a demo to use Gemini multimodel with vertex AI Studio next you explored prompt design you learned different types of prompts zero shot one shot and few shot and the configuration of model parameters like temperature top K and top P finally you explored model tuning an advanced topic you learned different ways to customize and tune a gen AI model from less technical like prompt design to more technical like distilling you were also showed a demo to start a tuning job with vertex AI studio all right then with this new knowledge in hand it's time for some Hands-On practice with vertex AI studio in the lab you'll have the opportunity to analyze images with Gemini multimodal explore multimodal capabilities design prompts with free form and structured mode and generate conversations by the end of this lab you'll be well equipped to perform various tasks using Gemini multimodal with vertex AI Studio please note that this course is a brief introduction of vertex AI Studio the tool and product used to access generative AI gen AI models for a more indepth understanding of language models and transformative Technologies such as decoder encoder Transformer and llm please refer to the course titled natural language processing on Google Cloud which is included in the reading list we hope you enjoyed this course be sure to check out other Google Cloud courses for continued learning thank you for watching please feel free to check out our other videos for more topics like this one

Channel: Google Cloud

Views: 27,186

Keywords: generativeai, cloudcomputing, googlecloudlearning

Id: KWarqNq195M

Length: 27min 50sec (1670 seconds)

Published: Mon Apr 08 2024