Overview of Azure OpenAI Service

hello good morning good afternoon good evening wherever you're joining us from around the world uh thank you for joining another Microsoft reactor live stream event my name is Michael Watson I'm the regional lead for the Microsoft reactor program in the Asia Pacific and it's my absolute pleasure to welcome you again to join us today I'll be joined by Dave Glover and Dave is a principal Cloud Advocate based in Sydney and works with a number of developer communities to support their growth on the Microsoft products and services Dave is a passionate software developer with interest in AI embedded platforms Cloud systems iot you name it Dave has had some experience with it in his over 30 years at Microsoft and today we have a great opportunity here from Dave around the overview of azure open a Services which as we all know is a very Hot Topic at the moment before we kick off I just want to cover off the Microsoft reactor code of conduct I just want everyone to acknowledge that this is a an event that we want to encourage all to participate in so there is q a so please ensure that you put your q a in the comments and we'll Endeavor to get to those as we can but please remember that we are being joined by a number of people so be respectful of everyone and please do not make any inappropriate comments without further Ado I will now hand over to Dave and this should be a fantastic session so as I said please uh please share your questions um and we will now pass to Dave so Dave hello Dave hey thanks Michael it's all yours thank you thank you very much um 30 years makes me feel um quite old actually uh anyway so I'm a I'm a cloud developer Advocate I'm based in Sydney Australia and as Mike said I've been involved in a lot of areas over the years and um really kind of round like the internet age and the mobile age and things like a whole lot of inflection points that we've seen across the technology Arena over the last 30 years 40 years and I would say that we're kind of reaching another one of these inflection points uh with AI so this is kind of what I guess what you're tuning in and why there's a lot of interest um out there around AI at the moment um so this session here is a overview of azure open AI service and I hope I can answer your questions now there is an opportunity to ask questions I will try and keep a bit of a question um I'll try and keep a bit of an eye on the um the questions and if it seems appropriate I will answer kind of in stream otherwise I'll answer questions at the end now also do feel free to um connect with me on LinkedIn and it seems to be like the preferred place for most people nowadays and I'm David Plummer and you can find me on LinkedIn there's glove boxes there's my Alias and on Twitter at DD Glover but as I said Twitter seems to be coming LinkedIn seems to be coming to place uh for folks to congregate um so really not a day goes past without some sort of AI headline capturing our attention and the Really these models and the AI and open AI has really captured people's attention if you think about um this technology we saw with open AI we saw within the first week of being generally available there was something like one million new people signing up onto that service and in two months there was a hundred million people signed up and using chat GPT and I've been AI so there's been I think across history or certainly the history of Technology um it's one of the fastest rising apps are in technology um so not only has it been an appetite from consumers but there's also an appetite for businesses who are looking at ways in which they can automate uh business processes or collaborate with AI um for to for to improve productivity and customer services so as I mentioned before we really are at an inflection point and I would say it's as big as what we saw with things like the printing press the Industrial Revolution the Advent of the microprocessor the internet we've got mobile technology all of those areas we saw a huge shift in the way in which we live our lives or which we communicate way we use Technologies the way we run our businesses and I really do firmly believe that what we're seeing now is AI is kind of making that breakthrough from more Niche cases to where folk and businesses are looking to adopt these Technologies within their day-to-day lives and running their businesses and really if you are running a business then then really should be thinking about okay obviously you've got today's business uh to be running but you need to be thinking about well how can I improve this process or reduce my costs or improve productivity um by using um AI now check gbt and openai did not come out of a vacuum and the these things did not not happen overnight so you can really go back into the back to the 1950s when people started when obviously technology and Computing started to become a bit more available though obviously very limited Hardware capabilities very limited software but back then this kind of concept of artificial intelligence was really the field of computer science that seeks to create intelligent machines that replicate or exceed human intelligence and that was kind of what the the thinking was back then and of course very limited capability but we did see the kind of the the Advent of things like expert systems or decision support systems where they were kind of mimicking the behavior of maybe human processes probably very very much rule-based but certainly very much back then the ideas around what could we do around heavy machines kind of emulate intelligence I'm kind of following that really is a subset of that we talk about much more around machine learning now machine learning is much more about what we call algorithm approximation so you've got a whole lot of data and let's take an example of properties and house prices you think about a house house prices and you might have the number of rooms the number of bathrooms a number of car parking spaces and all these types of things and then you might say well historically that house for that set of features would sell for this amount of money and what the way machine learning worked would look at all these data points and I'm just picking house prices as an example you would look at all that and then and then you would approximate an algorithm to say okay we'll get an algorithm that approximates this and if I go and put in this number of bedrooms and this number of car parks and Etc then I can generate a prediction for the the house price and that was very much around the idea of Machining machine learning and again that's been applied for a whole range of things insurance policies would be a great example um What premium are you going to make at a charge on for this person based on these Health factors their risk factors their lifestyle whatever it might be to come up with a premium so you can kind of see that machine learning has been used in a wide range of areas across business and of course our lives um for quite some time now this kind of got even more interesting in the 2000s with something we call Deep learning now um this is very much about building mathematical models that um that simulate the structure and the function of the human brain so I think a lot of work was done around how does the human brain work and what can we do with mathematical models uh to simulate that and to simulate intelligence and that is where we got this whole area of deep learning and you'll see this again in your day-to-day I'm almost afraid to say it because I think every device that I'm wearing or in my room is going to pipe up but you can say hello Google or whatever whatever device it is and that's deep learning and it's being able to understand wake up words and also be able to understand text uh and bring understand speech and return that as back as text but you can see that as handwriting handwriting recognition um real-time speech to text um transcribing and those types of things so again we're seeing those a lot in our day-to-day but these are very fundamental Technologies uh which already have really moved forward um the capabilities of AI and then we're at where we're at now so I guess with 2021 um we started to hear Whispers of things like generative Ai and this is the theme that's really captured people's attention and generative AI who kind of got the whole chat GPT thing and that was where that's where people's first kind of introduction to generative AI generative AI let's create new written a new visual auditory experiences and things like that so you you've no doubt played around with um you type in something and it comes back with with a prompt or maybe you've tried out Bing and Dali and you've created some interesting art piece artwork things like that um so that's kind of where we've got to today with generative Ai and we've got a number of models out there which are very capable and I always like to think about when I think about generative AI there's actually two things going to going on you're what's doing your typing in text and that text is what's called a prompt and you've got AI that can understand the context or the semantic understanding or the intent of that question and then turn back um some text to you or an image to you that kind of makes sense uh within the context that you've given uh the the AI models okay so the next thing I'm going to talk about is um open AI the organization and Microsoft the organization because they are two distinct entities and that's really important to understand so open AI they have a vision statement around ensuring artificial general intelligence benefits humanity and Microsoft's Empower every person on the organization the planet to achieve more Etc now what Microsoft and openai are doing is that we are collaborating to build Advanced AI models so open AI have got um the focus around their organization is building these models and the focus of Microsoft in this in this context is around building the super computer infrastructure for the folk and the open AI world to be able to build models on that infrastructure because as you can imagine you need a huge amount of compute capability and access to a huge amount of data to be able to build these models so that's the relationship so we've got it's a collaborative relationship Advanced AI Microsoft is building up a super computer platforms for these models to be built on and for those models to be executed on because as you can imagine when you're using these models there's a lot of processing power that is going on to predict make the predictions as to what the next word in a sentence might be or things like that so if you have an important thing about this is that there is the openai.com and in there you'll see a number of models you'll see check GPT and gpt4 and various models those same models are running at azure so you'll see inside Azure open AI service you'll see chant gbt gpt4 you'll see Dali and things like that and the encodex models so there are two distinct organizations you can run the models either on the openar.com or within the kind of security and infrastructure and Enterprise Promises of azure and Azure open AI so that's kind of where that how that gets together and these are the models that were predominantly that that we're talking about so the first kind of models that kind of capture people's attention were what are called conversational models uh we've had GPT 3.5 we have text and now what we get into is more conversational models um chat conversation and I'll show you examples of these but these these kind of come under the realm of check GPT and gpt4 are what kind of more conversational based and we've got codex around generating code and we've got Dali there for generating images so as I mentioned there's the models that have been built by openai are running in Azure open AI service so you have access to all the same models they're large what are called large language models they're pre-trained what are called foundational AI models of which that you can also go and customize them and tune them should you need to for your data though often we encourage people to look at prompting rather than tuning but that sort of might get back to that later on Okay so we've got um various models over here so we've got what we call generative text models and they have varying capabilities and varying price models and things like that so GPT 3.5 I think it has about four models that support it and we've got gpt4 which is currently in preview we've got chat GPT which you've probably had an opportunity to play around with so those are what are called generative text models then we've got codex so codex is around as a specialized model and that's for generating um code and you might have seen this and you might have seen various uh reports done where codex is almost magical if you're a software developer and you're using it for things like copilot or GitHub Copilot um it's almost almost magical because it kind of almost second guesses what you're trying to do and it's a huge productivity boost and we've also got things like Dali which are generative image models so you can go and describe something I'm going to show you an example um I often use generative image models I use it for some I've used it for some technical campaigns and and where I've wanted some interesting imagery work uh for for work that I've been doing but I also use it for my kind of personal life I'm thanking somebody I will generate a I don't know a superhero of a cook or something like that so I kind of use it in multiple ways personally okay so I've mentioned the word prompt a number of times and that's pretty key in what we call in this whole generative um AI world and you can see some different types of prompts in fact I'm just going to go for the the um the build so you can see over here on the left hand side we've got a prompt and a prompt is something as simple as write a tagline for an ice cream shop and the response back from the GPT 3.5 mod in this case would be we serve up Smiles with every scoop now it's not always going to be that because you can change What's called the temperatures you can change the creativity of these models and these models will bring back their the really important thing about these These are statistical models that are making predictions about the next word and they're statistical and by that nature it will vary um so that's called it that's what's called a um a completion um and then the next one over here we've got what's called a chat completion so this is more chat orientated so the the example here would be hey look I'm I'm having trouble getting my Xbox to turn on and the responsible and the response could be there are a few things you can try to troubleshoot strategy so troubleshoot the issues and you get some information and that's great and then the prompt would be thanks that worked what games would you recommend for my 14 year old but you can kind of see it's a conversational thing I'm going to get to this a moment um the other thing we will talk about is is what we talked about codex and this in this example here um this is a um a prompt I want to generate a SQL query so it's for a database back end that might be supporting a system and you might go and describe what the table looks like and you say in this case you um create a SQL query for all the customers in Texas named Jane and this would come back and say this is the query you need and again huge product productivity boost and I've already mentioned um darling okay so where does Azure AI um how does how does everything fit together um when you think about Microsoft and and the and the products and services that you're using uh day to day how do all these things with AI and various office and Microsoft 365 and Dynamics are different together so what you'll notice so what you would almost certainly would have read about is that Microsoft is in the process of infusing AI into pretty much everything we're doing so if you're using word or Excel or PowerPoint you'll see that AI is going to become more prevalent in those Solutions on those in those applications and if you think about it a lot of the and if you think about I'm going to um and a Word document and you need to go and create a a job description now a lot of that's boilerplate so if you think about you'll be able to say you better say tell officer you better tell word um I want to create a job description and then chat gbt will go off or gpt4 whatever model supporting it and they'll come up and say oh here's a template and we all know it's much easier to start with a template to say okay well this is basically the description of this document okay that looks about right um I maybe I've set a job description for a software developer uh with c-sharp skills and python skills and then it will go off and generate this and you're saying well okay that's not quite right that doesn't quite fit that but you've got to you've got a great starting point and that is the beauty of using seeing the AI again infusing its way all the way through all these applications um Excel you might be an absolute Wiz in Excel and you might be able to create formulas and functions and things like that I'm personally not um and I go into X and I kind of know what I want to be able to do but I don't necessarily know how to do it and the ability to better say hey look I want to be able to describe um describe uh what I want to create a pivot table or create these charts and things like that it would be easy it's great to be able to say look this is what I want to do and have Excel go and generate that for you um so that's the kind of thing um you can do now the question was can you I just want to say can I share PowerPoints yes I will I'll figure out a way to do this but we will be able to share the Powerpoints um so we've got the same thing for Dynamics you'll find it's AI is being infused into Power bi powerapps automate Etc now we've also got other what we call our other services so we've got what we call Applied AI Services now out of that the most popular ones and ones you might have heard of are things like cognitive search um hugely popular inside organizations where you want about an index like all your PDF documents all your PowerPoints or all your Word documents or a whole range of that and you then you want about equip and better find them and we have this called cognitive search and it supports something called semantic search or what I call more intent based search so provides a very powerful way of which you can go and index and then and then index for intent um so it makes it very powerful when you go and search for documents you might have with inside an organization another one that's very popular is things like form recognizer now we all invariably deal with forms um and each country has dealt with this differently but certainly in Australia we've all had to fill in forms every time we went and had a covert jab so forms are still part of our day-to-day existence but we have ai in form recognizer which is there to be able to automate extracting content from forms and then passing that through into some back-end business processes business process so again really powerful scenarios or solutions for really day-to-day problems um and then we've got what we call customizable models so we've got models for vision being able to understand though what's in the context of an image and and things like that we've got speech we have fantastic models for speech to text and things like that language that we understand things like sentiment uh and then we've got things like Azure open AI service and that kind of fits in this world of cognitive services and at the bottom layer that we have something called Azure machine learning and that's ready for a data scientist and machine learning Engineers um who are going to go out there and build models um So within the organization maybe you want to go and build a machine learning model to fit some sort of date you've got some data set you want to go and build an algorithm um that matches to that data set so those are kind of services so we're going to cut the top two layers are kind of more for business and the bottom three layers of War for data scientists and developers machine learning experts now we've talked all about this fantastic possibilities of of AI and and um um generative Ai and things like that um what I want to talk about next is really kind of what we call responsible AI now I think we have all seen within the Press there's always articles about saying that the the dangers of AI and things like that and of course there have been lots of examples where AI where introduction of AI has gone wrong and that's been something very much recognized so uh back I think it was back in 2016 I've got some notes Here 2016. Microsoft introduced a bot called T now some of you might remember it and it didn't take long for people to start getting that model to spew out hate speech and that's obviously not what we want um there are also examples where we've seen this within law enforcement where they've taken historic models of behavior and crime statistics and things like that and then use those to go and train models to predict future crimes now the the challenge or that of course is that they often had bias in them so they had human buyers typically there might be a certain part of the population that the police enforcement thought were more likely to commit crimes so those are so those crimes of people committing those crimes were more likely to find a way into data sets and then of course if you go and use those data sets uh to train up models then all you're doing is reinforcing human bias so we don't want that with an AI so we're doing the very best we can to ensure that data sets that are being used to train up these models are fall under this category of what we call responsible AI that the data sets and the ways in which these models are trained are fair they're inclusive so they include across the population we should be able to look at the accountability how is this model generated what are the factors we used what are the capabilities of this model or more correctly what is this model not good at doing because it's really easy to assume that hey this model can do anything um fairness of the data does this represent all the across the cross-section of community or does it does it have bias in there and again we've all seen bias and every every part of our Lives be it crime or be it um going to the bank or um trying to get a job or whatever it might be um we all see this bias and what trying to do is make sure that those biases are not reflected in the AI models that we're going to be using day-to-day day-to-day so what we have is we have Microsoft's responsible AI principles and there are six principles and I encourage you to go search for responsible AI principles and there is a link down the bottom there if you want to take a screenshot of that and um that your link after a document now it's all very well having these principles but it's also very useful to have a set of guidelines and policies and tools to help us um to help us Implement these responsible AI principles So within Azure you'll find particularly Azure machine learning you will find in there a whole set of tooling that make it easier for you to understand the data sets that you're using when you're training these models and for example it would tell you saying oh this section of your data is not representative you know this section here it represents 30 of your data but it's not particularly representative so you get some Fantastic Tools in there to actually look at your data and understand what you can do so I would encourage you to go to that link at the bottom and it would take you through a process and introduce you to more about those tools and how they can be used so the next I want to talk about is that we talked about responsible AI next thing about is you hear this time and time again Microsoft runs on trust this only works if you trust what Microsoft's doing what open hours doing responsible AI the data that's been used Etc now the next I want to talk about is a question that often comes up it's okay I've got my data and my Azure subscription and I'm using it maybe to fine tune a model does that mean that Microsoft or open AI are going to be using my data because I don't want Microsoft or open AI to be using my data and what I want to to confirm or really emphasize is that your data is your data it is not being used train up any models so super important because you might have sensitive data what if you do not want that data to be used for training so you want to emphasize this is mainly about emphasizing that your data when you're using your data for these models fine-tuning whatever it might be um it is not being used um to train foundational AI models and the other thing that I kind of touched on this before is we've got open a as an organization and we've got Microsoft where you can go and run these models and the important differentiation behind that is that when you're running them side Azure you're getting all the security and Enterprise Promises of azure so you're getting things like encrypted storage encrypted Keys you're getting private Networks you're getting all the security controls that you're used to that enterprises need and demand when they're running these these um AI models so hopefully they understand so when we're not using your model so your data so your data is safe not been used for foundational models and you're you're dating your Communications your security your keys all these things here are also protectors part of the day-to-day what you expect from from azure already so open AR model considerations so is it so the question you need to ask yourself well is open AI something that we need inside my startup or my organizational or I want to experiment with um so the general kind of question is like do I need a general general purpose model that can handle model tasks um things like translation entity recognition sentiment analysis things like that um I need to generate human-like context while preserving data data privacy so things like summarization content writing paraphrasing things like that I need rapid prototyping quick time to Market models a little note training Etc so we're going to go through if those are the kinds of things you're looking to do then open AI is quite possibly the thing that you should be looking at um if you're trying to do things like I want to go and do um speech to text well no you'd go and use a model that's much more geared around that we have models and and that AI that can do that for you but you might end up with text at the end of that process and that text might be something that you might then go and feed into open AI so what I'm trying to say is open AI is not the only kid on the Block there are other scenarios like Vision speech AI language models decision models that together you might use either individually but you might use them in unison with something like open AI service so hopefully that makes sense those are kind of scenarios and but very often you're going to be using other models uh to kind of augment uh what you might want to do with open AI services and to kind of give you a bit of a bit of a kind of top capabilities um and this one this scenario is really around call center Analytics and you might want to go and generate uh automatic um you might have a bot and you might have that bot go and generate an automatic response back to a customer or you might want to customize the UI based on on the customer's questions and things like that again I'm call center um where you just think about the huge volume of voice that's coming into a call center being able to convert that into text and then be able to pass that model into open Ai and say hey look can you summarize that for me or can you bring out the Salient top points for that and then pop that information and your customer response CRM system so those are the kind of capabilities for summarization can be incredibly powerful um to say look here's a here's a couple of pages of text bring out the Salient points of that code generation I've already mentioned that before and semantic search being able to understand the intent of a question and that's really the power of this we talk about semantic search um what I like to call intent-based search which is understanding the intent of that question not necessarily the way you've worded it but what does this question really mean and then mapping that down into the data that's sitting behind uh these systems to be able to support that and again end-to-end course centers classification sentiment analysis anti-extraction these are the kinds of things that you can do with open AI and I've got an example here so this would be you think about I mentioned before about forms Okay so we've got a whole documents a set of instead of documents that we've got inside a data store somewhere we could go and use something like form recognizer to extract the text of that we grab all the the the text we go and index that with a cognitive search and then we create an index behind that and then the end user comes along and puts a question on there saying okay look I've got a question we send that question across and then what we we go into the index the search index and we grab relevant content from this case cognitive search not the only way you can do this but this is you could do this with cognitive search and we we kind of augment that data that query with data that's relevant to the data that's inside your organization and what we call this is we call this grounding with your data so you'll see there's a lot of documentation what does grounding mean well it means I've got a fairly generic question a question comes in for example um I've got an end user wanting to know more information about insurance policies and particularly around what cover for for eyewear okay that's great so the end user said tell me more about your insurance policies for eyewear and then what I would do is I'd have the back-end documents that support the policies Etc inside the insurance company and I would use that data to ground that query so I ground that query with some data that's relevant to the business and then I would get the data and I'd add that data to that prompt and then I'd send that prompt which has got the original question on it but prop but plus some additional data that I got from my organizations back in data stores I'm passing that into open AI I've created this kind of like a super prompt pass it through open Ai and then I've got the response and then I've got a response that's useful back to the end user that's got data that's relevant to your organization the systems that they're supporting that query so that's exactly the example of how that works um course under analytics I've kind of touched on this already a huge amount of voice data do speech to text store that data away index that text you could go and use queries you could populate the CRM or you could have a user generated query tell me about the status of this customer where they're at what was that what were the main issues in that Etc so incredibly powerful to be able to go and use open AI to go summarize these uh speech to text and then go and store it away in a CRM system or look at what sentiment it is or the what what are the the issues that customers are facing um so comparison to models can talk a bit about here around chat gbt so if you're going to start the place to start is really around chat TBT should be your first point of call it's also the most economical um next Port of Call you should be looking at GPT version 4 which is currently improved and sorry in preview um it has significantly improved capabilities but kind of think about check gbt uh it's your starting point it's the most economic model figure out if that's going to work and then move on to gpt4 um with what you want to do and then um look at other models so maybe that's not doomed but there are other so for example at the moment GPT 3.5 is also deployed into Data Centers in Western Europe that might be critical for you you might have some use case specific uh examples already I'm just going to take a quick look at some of the questions so just give me a moment um probably going to come back to some of these it might they might go a bit further down okay so what we've got we've got customer mic customizing models in fact I think someone's got yeah so I'm going to talk a bit more about what we call grounding so we've got we've got these models we've got your internal data that you've indexed um yeah but your index your data sources and then you're going to be using your prompts so you'll come along I'm going to type my fairly generic question that I'm going to use your data within this within the scope of your application within your organization and then you're going to pass it through into the open AR models which are going to respond back with with a with text that's relevant to that question which is grounded in the data within your organization and that's kind of how that works and to talk a bit about some of the key terms that you'll see within this so we've got in this case here this has got an email so we've got an extract from an email with an address and we've got um The Prompt so this case here this is um a text so text input um and then we've got um what's called a prompt The Prompt in this case here is extract the mailing address from this email and you'll see that this is the body of the email and you'll see through here that we've got the email address also we've got the um the mailing address and you can see down here this is what's called in red this is what's called a completion and again I'm going to show you an example of this in a moment now the other thing you'll see is tokens and again what's a token the first time I saw about this well what nurse a token now a token is a word or typically part of a word so for example um thought might be broken down into number of tokens maybe it's into syllables or something like that there is a tokenization process and there are tokenization libraries and you can kind of see what's happening behind it but tokens are just basically words or partial words or maybe a sequence of numbers or things like that and it just makes it easier for the model to be to scale better effectively when you're using tokens so that's what a token is this a word or a part of a word that gets fit into a model and you remember what's happening is that these tokens have been fed into a model and then your prediction has been made based on these models is what is the most likely next likely token to return back in a sequence so that's what's kind of going on okay so what I'm going to do now is just going to go for a very quick demo and give you a bit of a sense of how some of these things work so the first thing I'm going to do is I'm going to go into um um what's called yeah so what's called the um the um Azure AI Studio plat and inside that there was something called the playground now there are two places here you can most that you mostly started with so the first thing I want to point out actually is I have deployed models open AI models into my subscription so the inside the playground this playground is associated with the my subscription and the models that I've deployed and you'll see that I've got a number of models that I've deployed I've deployed four models and the one I'm going to be using is this DaVinci model version three and I'm going to do a completion so the first thing I'm going to do is I'm just going to paste in happy birthday to you now happy birthday to you is going to generate um the most likely statistical response is going to be happy birthday to you happy birthday to you happy birthday Dan Etc so that's just remember there's a statistical model running the background and it's making a prediction and the most likely prediction for that is going to be happy birthday to you now just say and this is a bit of an aside but we'll just give you some of the idea of the power of these models I might say translate to French and then generate that and it'll come back this is not the way you'd sing this in French but this is the French translation for happy birthday to you um so you can see there's some quite stroke quite capable models that are quite capable capabilities in there um I can also look at view the code if I go and look at view the code you can see I've got some uh sample code in here which will get you up and running using the open AI models and you'll see there's C sharp curl Etc and Json so we basically help you to go and get started building applications behind this now what I'm going to do is I'm just going to delete that out and we'll pop back to the slide and the next thing I want to talk about is context and how within within your prompts and this is really the power of these models is context so instead of this time just saying happy birthday to you I'm going to say happy birthday to you and here's a great handbag gift idea and I'm going to say generate and what it's going to do it's not going to come back with their Happy Birthday song it's going to come back here and say hey I'm sure any Lady of your life will love in your handbag for the best as a gift so you can kind of see now I've changed the context and it's coming back with some interesting text about this and this might be something the beginning of a plan for example or maybe something you want to put onto a Blog um now if I take this whole idea a bit further I'm just going to copy this over here and then this time what I'm going to tell um the GPT 3.5 in this case I'm going to tell it how I wanted to behave and in this case here I'm going to say act as a marketing assistant and write a marketing plan and a fun tweet for happy birthday to you and here's a great handbag idea and we're going to click on generate and now what we'll see is that this will come back with a marketing plan and again as I mentioned around that word example now you've got a great starting point for a potential campaign that you might want to in this case it was handbags Etc but it comes out with some useful information okay there's a target audience that's kind of useful um and down here there's a fun tweet it's your special day celebrate and start with a new handbag so you hopefully you kind of get the general idea about how this is working okay so the next thing we're going to do is go on to another demo that I've got now this demo this was kind of slightly more on the business oriented one this demo here is kind of getting a bit more on the technology side about how you can use these within your apps so what I'm going to do now is go across into something called chat completion so I'm going to select chat completion and what I'm going to do is I'm going to set a system prompt when the screen pops up okay so you saw me paste copy that message so I was pretty interesting that the wrong place okay so that's jumping around a bit what I've got over here is I've got a system message and that system message says you are an assistant designed to extract entities from text users will paste in a string of text and you respond with an entity now the example with this be that you might have done speech to text and they might have said my name company phone number Etc will be embedded into that text and what I want to be able to do is extract that and some useful data that I can flow into my application um so that's what I've done there so I've set um a system message which is set in context for this chat so I'll confirm that so I've basically saved that okay so the next thing I want to go and do is I want to go and put in some text so imagine this text is again coming from a call center application someone and someone put an email so I put in the user message and the user message in there was hi my name is Robert Smith I'm calling from contoso I'm interested in learning about blah blah blah can you please call me back on this phone number and um Etc so that's what the the chat was so I go across to the the playground and I send that message to the model and remember I've set a context I've said that you're an entity extractor and here I'm getting back a bit of Json and and here's the entity and again this would then flow through my system so you can imagine I'm a bot or something like that or you've got email coming through or whatever this however this message is flowing through and you can now start extracting stuff out of out of that that you might want to flow into your system and it's inside here I can view the code um you can see this is the code that made up this chat um and I can look at the raw the the raw version of this and you can see I've got a role which is system and there goes a set in the context and then a role of user now what's really important to understand here is that these models do not maintain State they do not understand context so what happens when you when you're communicating with these models you're re-establishing the context for this chat and then so you so what's basically having every time you're chatting with us you're sending the system message you're resending this this user message and you're resending this as a user message and then the assistance come back so what's happening back is that there's no context of state that's being held in these models you're re-establishing state every time you you communicate back with these models okay so the next model is I'm just going to clear that chat and the next model next time I'm going to show again this is definitely getting more into the world of geeky um but this one here and I have to confess I stole this one from a demo that I first saw on YouTube and I remember thinking how on Earth is this working why does this work so this case here is imagine you're a Microsoft SQL Server I type a command and you reply with the results and no other information or description just the result okay so I'm going to set that as a system message we're going to come back over here to the playground and I'm going to set the playground up here and I'm going to save this I've called it I'm culling the model that I'm using the chat gbt model that I now want you to think that you are that you are a Microsoft SQL Server and then I'm going to come back over here I'm going to put an exec who am I so this is just a standard bit of a SQL Server oops I'm going to use it something called who am I and it's going to come back say domain username but that's fair and you might be thinking to myself well why is this because check gbt isn't a SQL Server what's going on and what's happened is that chat gbt and all these models have been trained on the documentation for SQL Server so the SQL Server documentation and it knows if you go and do that the most likely outcome of that would be the domain and username okay so I've come across here and I go exec um elevators perhaps I'm going to paste this in and what it's going to do remember it's a statistical model and these are the most likely databases that you'll find for the documentation so I've got Master Etc so these are a fair examples of the type of database you have in there okay the next thing that I do is I'm going to create a database come across here and I'm going to paste crate plus create database customer and I'm going to do that so that's completed successfully and I'm going to say I'm going to use this so if you were a database person you would know about this I'm going to use the customer database and that's great I've changed the context to that that's perfect I'm going to create a database and the database I'm going to create is called users and it's going to have age and name those are the fields in this database that's been created successfully I'm now going to do an insert across there insert that and then okay that's worked successfully now I'm going to come across here and I'm going to say select for all users and paste that in and it'll come back and say that I found kit where I've just that was the user I just inserted so again what I want to get across with this demo is that context matters and you can see what I'm doing this I've chat I'm adding context and these models they have an understanding of SQL the documentation and this is the most likely outcome behind that if I go and look at the raw dates you can see that this is the conversational chat that's been happening every time I put in a chat and re-establishing context with the model and it's coming back with the most likely set of text that that I would expect so it's not a database server it's just saying this is the most likely thing that I would expect to see within the database from a database query um so that's kind of interesting that's the one I like now the last one I'm going to show you if you've played around with this already um where is it over here now this is Dali and the one I like to do so this is daily so this will do is generative image so I can come along here and say draw a super hero cook and the style of pop art so I'm again putting in context and you'll see I've got creative mode set on so you can all use this you just go to bing.com and you select chat and what it will do is it'll come off here you'll get a bit of a bit of a um your a prompt to say your image has been generated it'll tell you a bit about what it's going to do so it's going to create a style of pop art it'll even tell you a bit about hey pop art is a movement that emerged in the 1950s and 60s characterized a bit of a bit of a keep you interested in why this has been being generated and you can see hey look I've now got a superhero cook and we'll put this one and that's kind of what's going on so again you're free to use this it's part of Bing and it's a fantastic service I've used this for both for work and I use this both for um uh just general data day as well so it's kind of fun already so we've done that alrighty so the next thing we're going to talk about is just some things to think about it now I've said this a couple of times and it's deliberate the really important thing to remember is that open Ai and these models check GPT though they might seem like human their capabilities and that's part of their interest it's really important to remember that they are AI models they're mathematical models they're statistical models and they're making a prediction as to what the next word or next token is going to be so it's really important not to humanize what's going on because it's ever so tempting you look at these things and you start getting to this world about why is it behaving like this it's really important to remember that generative AI it is not intelligent it is not until deterministic and it's not trustworthy so those are really important things now on the trustworthy angle the way that I tend to think about this I think about if you've got a a business process for example that you want to automate if it's low risk and maybe it's a bot and you just want to return back excuse me um information about a product that's a relatively low risk activity and that's probably fine with something like that so you think low risk automate with AI but high risk collaborate with AI so for example maybe you're a doctor and maybe you're writing a referral then you would not want to just have it automated you would want to collaborate with it you want to look at what is generated does this make sense in the context of this patient that I'm seeing and and you want to collaborate with it and make sure that you're an expert in that field yes this makes sense but as you can imagine as a doctor without being automated generate this get a huge Time Saver but it's a high risk scenario so you want to collaborate with it so that's kind of want to get across um with that um and also always work with that responsibly now we're also doing a lot of work around what we call content filtering so in bing.com your notice so you might have read about there are content filters in there we do not want people to be putting in hateful things for example or you know hateful things into these models and having them return models uh hateful output in the business context you do not want these models that someone may for whatever reason be putting in hateful things inside an application inside the context of a business and having it returning back inappropriate content back into a business context we do not want that so what we have with you have this content filter models you'll see that in bing and you'll also be able to use those in your Solutions so those are those are coming through at the moment um into preview so those are some of the things and again this emphasis around responsible AI okay so you might want to take a screenshot of this um so there's some useful resources there so we've got the documents there we've got services um how to go and build AI Solutions data and privacy around AI Solutions Etc and how to generate generative models work um really useful set of resources so please fire away and take a screenshot of that I'll leave it there for a moment and I'll grab that okay um there's something called AI business school I again encourage you to take a screenshot of that some really useful resources for that down at microsoft.com AI Dash business school um and you'll find again um just take a screenshot of this uh and then go and search for some of these resources um you'll find a lot of a lot of content out there um some really well regarded Microsoft learn modules around introduction to Azure open AI what does it mean you'll see a lot of work around what's called prompt engineering a lot of things like that a lot of really great resources to get up and running to build applications what I wanted just to kind of emphasize as I showed you the playground before when these demos keep in mind the idea behind the playground is for you to test out ideas that will eventually find their way into applications so this is a way in which you can go and test out your prompts is that kind of getting what I expect is it is are these prompts working the way but the ultimate goal behind this playground is for you to use that code that you can see over here and use these prompts that you've been playing around with to go and generate intelligent apps that's fundamentally what the playground's around so I didn't want you to think oh great do end users go and use this no it's for people building apps tuning the prompts and getting those to work um in a simple way for applications okay so I think that's pretty much what I wanted to cover um so next steps go check out open AI access the the AKA dot Ms slash um access access to resources and the first time you sign up for open AI Services you will also need to fill in a responsible AI form um as I said we're we're super cautious about this and you should be as well we want to know what are your scenarios you could do and we don't need to know the infinite detail about it but we just need to know that it's a reasonable thing to be doing with AI and you're not out there to cause harm basically that's what we're trying to do um okay some questions on here okay so what is the benefit of using Azure open AI over open AI I think I probably answered that up front the main difference is that open Azure open AI is much more about Enterprise Readiness all the security all the things the security promise that you'd expect when you're an Enterprise using services in the cloud you want to make sure you've got things encrypted secure code security secure keys um your debt your storage your data is encrypted those are the things that we're doing behind open AI Azure open AI so hopefully that answers your question you're completely free to use open Ai No one's saying don't use it but just remember that if you're Building Solutions for Enterprise or startup buildings for Enterprise Enterprises are probably going to be wanting those models to run the context of their Azure subscription with all the things that they want and expect from Azure so hopefully that answers that question um yeah so when I say when you're going to sign up for a um open I haven't mentioned that you need to do the responsible AI there's a question here what role do you think third party Assurance has a part to play in the AI industry and I would suggest that actually has a strong um role to play so for example there are a lot of um open uh open source projects that are there to look at responsible Ai and the shareness of data that you're using to train your models so if you go into Azure AI machine Learning Studio and you go into the responsible AI dashboard you will see that there are a lot of open AI models that that you can use there to look at your data and to scale your to look at your data and understand the fairness now that is absolutely uh open source third-party models and solutions which are playing a very active part and the process of building models which are hopefully going to benefit Humanity so that would be an example there so I would say there's a big part to play with that you'll also see and sorry I quite remember we have just announced a build um an open set of at uh a Marketplace if you want of open AI sorry of open source models that you can run in azure so we absolutely recognize that there are some fantastic work that's going on in the open source world and we want to make sure that everyone gets to benefit from that and so those models you can run those in Azure Etc so there's a tooling open AI there's a tooling for AI to make sure around responsible data and there's also these open source models are just amazing out there um is so the question now is the Azure open AI service only available to Applications to Azure Enterprise customers aren't are non-enterprise customers able to use this yes non-enterprise customers are able to use the services as well yes so yeah definitely yeah okay so let's answer that question I think that's probably about it Dave I think so as well yeah well look thank you very much Dave uh incredible session lots of information uh I've put up on the screen some extra learning for people if they want to continue their learning they can scan those QR codes and I'll take you through to some more learning piles and modules around Azure open AI um but on that I would like to thank Dave Glover for joining us for delivering such a fantastic talk and uh this will be available on YouTube if you've missed something on the Microsoft reactor YouTube so jump on and catch up on anything you've missed but with that I bid you a good morning good afternoon and uh good evening wherever you are in the world thank you very much all right thank you Michael and thank you much for joining us for tonight's on this morning or evening or whatever you are in the world for the session thank you very much thank you

Channel: Microsoft Reactor

Views: 4,854

Keywords:

Id: NARe_yKaDsY

Length: 61min 47sec (3707 seconds)

Published: Wed Jun 14 2023