Generative AI for the .NET Developer | .NET Conf 2023

[Music] now of but we're we're 1 hour in so we've got 23 hours or I suppose if we're we're looking really at the time it's 22 hours and 57 minutes left of the the live stream um for today um and well I guess it's time that we start looking at well we we'll bring on our next presenter uh uh wrong button that's the that's the button I one I I'm doing doing all the all the back of house stuff and front of house stuff uh myself today so just balancing and hitting the right buttons that's uh it's all the fun part of it but anyway um let me go grab the next speaker hi folks um I'm back again that's right the the next speaker is well it's me so if you thought you were going to be getting away from me for at least the next half an hour uh that's a little bit unfortunate because yes uh I am going to be sticking around uh and it is time for me to to jump on uh for my session and this is going to be a really fun juggling act because I am producing the live stream at the same time as presenting on the live stream uh and also moderating the chat to see if um if there's any questions that come in so do feel free to ask questions on the chat if you're watching um live on on our channels um because uh me and the other hosts will be keeping an eye across those um those questions that are coming in and foring them on to the presenters um in the Q&A as we hit that but anyway let's crack on with uh with my session which is the one that I've most been looking forward to and I hope that you have to as well um today I'm going to be talking about generative AI for the net developer um if you've only just tuned in for um for today's session my name is Aron pal I'm a principal Cloud advocate in the net on aure advocacy team here at Microsoft um and uh the focus that I've had over the last six months well I mean I suppose this seems to have been like what everyone's focusing on is AI so I want to talk a bit about that and what it means as a net developer to be doing AI but rather than just jump into a bunch of slides I'm going to jump in with a demo so let's come over here to vs code um and I've got a couple of demo applications that we'll actually have a look at how they're made later on but we're going to start off by just running it so let's go net run uh so we'll fire off this application going to start up in a minute let's collapse off the sideb to there make everything nice and big maximize the screen real estate that we've got there we go I think that's that's readable enough Okay cool so this is kind of a um a chat gbt like experience but running as a net console application uh so let's say what is the history of.net um send it across a question or a prompt uh and and then this is going to then come back with an answer for us uh so this um this chat experience we've built here it's got an Australian personality that's why it's kicking off with good day mate uh got to bring got to bring some more Aussie to the live stream for today and then um yeah we can see it's it's hitting kind of the keynote I'm not going to read what it's saying on screen because well that's a lot of text and it's not very interesting to watch me read you can read just as well as I can but yeah it's the key points of NET Framework came out in 2002 alongside visual studio.net um it's talking about know some of the other other milestones in there around Net call the releasing of that being crossplatform being open source um and then the unification that happened with uh with net F where NET Framework and net core um merge together into the single net platform that we have today but you also notice that this is a little bit old it's it's not knowing anything kind of Beyond Net 5 or it's referring to Net 5 as as looking at that future um so this is because we're using a a model that's not cut of up today doesn't know about 6 7 and definitely doesn't know about 8 given that that was only released yesterday so let's wrap this up um from a demo standpoint and we are now going to jump back into our slides uh and talk a bit about Ai and then kind of how this sort of an application gets built but naturally whenever someone asks about you know like AI is like is is it going to take our job like did AI actually write the slide deck that I'm going to be presenting today um and that is something that you'll have to try and figure out yourself by know sticking around for this presentation is is how much is this a talk that I wrote and how much was a talk written by me just waiting for some form of generative AI to give me back a prompt but before we can really dive into you know the the um the nuances of of what um gener is and that kind of stuff I just want to talk about some of the like fundamentals of it and it's like where everything when it comes with AI starts with a model it's this is really the the initial things that we have when we're working with an AI um and I I'm I'm not an AI researcher I'm not uh someone that is like from a data science background I'm a developer I'm a consumer of rest endpoints is the way that I describe my usage of AI and um when so when it comes to the way I think about it I think about the model as kind of it's a database or or humanize it it's the brain that we have um and that is basically it's something that we're feeding data into and the more data we feed into it the more quote unquote intelligent it can become and and the resulting of working with that model can give us better answers this could be textual data it could be image data whatever the case may be and that's what we use to build up our our model our starting point when it comes to working with our AI platform that we're building on top of it so let's talk about a couple of different forms of AI that we can work with or that we have worked with because AI is not new like we've we've had AI around for decades it's just it's become a whole lot more prevalent in the last particularly the last 12 months most people actually work with AI on a daily basis or or engage with a forms of AI on daily basis whether they they're kind of cognizant of that or not and the most common way that people work with that is through voice assistance whether this is like an Apple Siri in your watch or your phone Google assistant in you know the the Android devices um I have Google homes all around my house I'm going to hope that I don't accidentally trigger one during this talk and we'll hear it from like a room across I'm resenting or if anyone's listening I won't say the trigger words for Amazon's device because it's nothing more fun than having those trigger often um in people's houses while you're presenting live but these are these are obviously forms of AI but they're um they they tend to use a variety of different components that make up the AI landscape to give you something to work with things like natural language processing or NLP where we are taking the spoken text or sorry spoken words and turning that into text that can then be consumed by our model because models they don't have ears they're not going to hear what we say so we have to turn it into something that is then getting represented by our sorry consumed by our model underneath so natural language processing can it understands the text it can break that down into the words that were actually spoken within it and do things like understand sentiment in that so where is the emphasis placed on it so that you can do things with um uh the the intent that people are M are stating in spoken language we we know the the difference in using spoken language as a way to communicate with someone versus just purely text and how that can be differently interpreted another form of image generation sorry um uh AI is image classification so in this case we would have a model that's maybe been specialized around you know understanding what an image of a dog looks like the characteristics that make up a dog and then as a result of that we would then train it on a specific type of a dog say a Chihuahua and what is the characteristics of a Chihuahua versus a generic sort of dogs um and then as a result of that we we we build up a a new model A specialized model on of um particular kinds of images that we're classifying it with and then we would feed it new images you images that are captured in the wild using our application whatever the case may be and see how well our our model is capable of detecting what we have with inside of that so in this case we would then feed it in a bunch of pictures of you know other chah and muffins and and see which one it detects as uh as an actual as the dog and which one does it detect as a muffin and then in the case where it gets it incorrect we would then um adjust the training data so that it is more capable of correctly detecting the the dog with inside of um the images that we provided and this kind of the way things like your face detection in your phone um and stuff like that works is that they understand the constructs of a human face and it will be able to um detect a human face but then you train it specifically on your face so it knows you versus your partner versus your kids versus the random person walking down the street behind you trying to unlock your phone and this brings us into generative AI so generative AI is like it's it's the thing that has really reshaped the way that we converse about AI as an industry today um and I think that there there's kind of one graph that really illustrates why that's the case and it's the the indication of of adoption through the the speed of it attracting users so where with chat gpts um it reached a 100 million users within the first two months of its launch and we can see other what I would very much consider disruptive Technologies in our industry things like the internet there's no denying how the internet has changed the way we um we we work as with computers and the way that we think about Computing uh and it took seven years to reach that same level of prevalency with inside of um within globally uh like seven seven years and you know we like I said you know how impactful that has been but then chat jbt did that within two months and and the amount of awareness that it has with inside of the general populace is is pretty uh pretty unbelievable for me I've been around tech for a couple of decades now and and seeing how aware non-technical people are of this big shift in technology is pretty uh impactful you go and ask any non-technical friend or family member and there's a good chance they've heard of if not use chat GPT or or something of that ilk so when we're talking about generative AI naturally we have to talk about chat gbt because it's the thing that's really popularize this term in the market of of generative AI um now it's also important to to note that Chad jpt is it AI Chad GPT is an application built on top of AI from a a generative AI platform from open AI um Chad GPT is it's just the interface to it to it because at the end of the day the model and in this case um a GPT model or a generative pre-ra trained Transformer model which is what open AI open AI has produced it's just it's sitting there it's it's full of information it's got connections between that information it understands how to to draw conclusions from um information requests that's provided to then give you responses or or completions as we call them in um generative AI terminology like it knows how to do all that but it's just it's sitting there doing nothing until you prompted until you give it a request and that's what chbt does it gives you that interface to it so like under the hood it's it's just sorry as an application it's just a conver a way to converse with this um this model and it does it in a much more natural style than I said I've got Google homes around the house and if you try to ask questions you've got to make sure that you're triggering them with the right set of keywords to detect what you're trying to do figure out the intent whereas um chat GPT and and these GPT models that it uses are more capable of of figuring out what you want based off of that but at the end of the day chat jbt is a very generalized model it it doesn't do well it can do a lot of different things you can get it to summarize a movie in the form of a Shakespeare um Sonet you can get get it to um generate you a workout routine based off of your current level of fitness and the availability of a gym um that you can uh you can get to you can get it to write code for you there's kind of no ends to what you can get it to generate um but because but that's because it's a generalist it can do pretty much anything but that puts the onus on you a lot to make sure that you're coing it to give you the kinds of things you're actually outputting which actually wanting as an output um and then that brings us on to the idea of um of specialized models and I'd be remissed to not talk about GitHub copal because it's one that is very well understood by um probably a lot of people that have been watching whether it's been GitHub Universe last week umite if you've caught any of those other sessions you've probably seen get up copilot presented at netc over the last couple of days as well but get up co-pilot is a an approach to a specialized model so it's it's specialized around just generating code complete or or completions that are centered around um coding problems that you're trying to solve and what's interesting about get up Cil it is it's using the exact same models as chat GPT it uses a GPT form model which you can use with Jack GPT um I think chat GPT is 3.5 gbt 35 by default um but if you're on the um paid subscription you can use um GPT 4 and there was some announcements a couple weeks ago about some changes to those models and things like that from open AR themselves um but but because it's it's usually the exact same thing that we get out of um chat gbt the question is well why would I use one versus the other like if if chat jbt can do exactly the same things as GI up copot what's the difference well the specialization is where it comes in it's not that GitHub co-pilot has been trained on additional data but it's a lot more contextually aware if you ask it a question of I want um I need a particular type generated well how does it know what type to generate well it looks at the codebase that you're working with it sees that you're in a c project so it's going to generate you a a c type rather than a typescript type definition because it's just going because typescript maybe is the one that it would think is the most likely thing or the most common response that it could give you it understands context it understands the the cursor position in a file the open tabs that you've got the types that are loaded um via um references with inside of the code base so that it can give you a more accurate response that's around the particular code that you're working with now you can do this with any other large language model that but the thing is you then have to provide all that context to it and that's really where specialization over the top of GPT models or these large language models um really comes down to how much information you can add to the request or the prompt that's coming up from the end user that brings us to the question of well what's what's the role that Microsoft plays Within all of this particularly like what's azure's standpoint because like I'm I'm in the net on Azure advocacy team so other than like I think it's kind of cool like what's the what's azure's relationship with this well at Microsoft we've produced the a net SDK for working with open AI um or Azure openai if um you do if you would prefer uh and and in that case so we have produced an Azure open AI service that goes along with um can be a comparable to uh open AI itself the the main difference is well that we're in partnership with open AI um we're not trying to produce our own models we actually provide the same GPT models that openai produces produce they provide them to us um and we then have them as availabilities within theide of azure um because open AI just wants to make sure that uh artificial intelligence platforms are available to as many people to like they just want to benefit people by having these AI platforms and in Microsoft that kind of fits very well with our mission statement of empowering every person um on the planet so it's a very nice Synergy um relationship we've got there so like I said we can take these pre-train models from open aai um and then with azra open AI service we can add uh extensions on top of those based off of your own data you you bring in uh your own organizational information whether it's data from a s database from files that you've got with inside of um document storage and you add add those but they're only for your instance of the server we're not using them to retrain GPT models or anything like that they are just additional bits of information that are only that you can query with inside of your application when you perform requests and get um responses back and you can then contextualize around your organizational information organization's information so that you can um get you know hopefully more accurate responses that are relevant for you in a business rather than it just going to generate out something completely random um obviously ignite is also on it at the same time as uh as what we're doing today and we had a heap of additional announcements around um the AI services that we've got you a AI um and a whole bunch of stuff like that which is well beyond the scope of what I wanted to cover off today but I do recommend checking out some of those sessions if you want to learn a bit more deep dives into things like how do you bring your own data into um an AI system that you're building uh stuff like that oops wrong button okay so let's jump back to our demo and have a look again at how th that that application we had just there before like how did it actually work so we'll come back into vs code and let's let's build an application from scratch I've got about I think 12 minutes do you reckon we can build an application Live um from scratch I I reckon we could give it a try so let's do net net it's going to help if I actually can spell things so we'll do net new and we'll create one called netc demo go to netc demo and we'll copy uh I'm just going to copy across the app settings um from previous because I don't want to uh accidentally publish my uh API keys or anything that for for open AI on a recorded stream because then I have to go through and and cycle them and then find all the the applications I'm using them in um okay so copy across our app settings that's all done uh then we'll doet add we're going to add some reference we're going to add reference to a shared project which is just got um some some shared types add package um oops add add no net ad package um so we're going to use the hosting extensions just to to run this application and then new get install packages all the things do. net add package and we'll add the Azure a. open AI um uh new get package so this is the um it's still in preview the the open AI package unless I got like G today and I haven't seen that announcement um I've installed the preview version so hopefully hope that's the most upto-date version we've got there all right uh and then we will find our application in here some here we go here's our net comp application excellent and let's just delete all of that and we're going to need to do a few things we're going to scaffold out um obviously loading um uh starting our our application host we're going to need to connect to open AI Azure open AI service and um then we'll start building out some prompts all right so we're going to start with a IH host Builder and let's see if in hense wants to to fire up I don't know if it wants to just yet and I do actually have GitHub copilot running as well so it may jump in and try and complete things for me but unfortunately it looks like I'm actually going to have to type things out myself or things are really going to freak out and I'm going to realize that I'm running out of time and we will actually just start by copy and pasting some code because this is not like this boiler plate code is not that interesting all we're doing is we're using the host Builder we're going to just start that up we going to configure that with our app settings file and using the just environment variable for developer versus non-developer settings uh and then we're going to configure some services so the first service is well we're going to load up our configuration file so we can pass that in through dependency injection and then I'm going to create an instance of the open AI client now here I'm using U just an eum to determine whether I'm working with Azure open AI or open AI uh and that's because this net SDK that we've produced for openai Works whether using a open AI service or open AI directly so you can use that interchangeably with inside of your application um it kind of it can it can flick back and forth between uh between the two so that's that's all set up um and then we will create a new class uh okay looks like intellisense hasn't properly loaded so let's just restart Visual Studio code and we're going to have to create our host builder in a moment there we go all right now let's see if uh if everything is going to activate um this this is the fun thing of like doing a talk like the day after a release of um of a new um RTM of.net is that I'm not quite sure whether the stuff I'm running is the preview builds or the production ready builds um I think I might actually still have some of the preview stuff running on my machine uh order of execution problems so let's see is uh gbt no all right I think I think my attempt to do a live coding demo is going to to full flat on its face so instead let's um let's just have a look at you one that I prepared earlier and talk a bit about how that works um just zoom that out a touch so we can see a bit more on screen okay so here's that file that we loaded up opened up before and I've got this um console GPT service so this is the um uh the hosted service that's going to run when our application starts up uh we've got a application hosting lifetime so this is something that is going to um control the the start and stop of the application and then like nice cleanups if we're worrying about disposing of objects we don't have anything that we need to dispose of so that's not particularly impactful we've got our open a client that's coming in so our application itself is not aware of are we working with um Azure open service or open um directly and we're just going to capture that along with the settings as some um variables I could turn that into a primary Constructor let's do that so it makes makes me feel like I'm actually writing a bit of code there we go I've I've sort of written some code we re well we refactor some code there uh all right now what's happening like the guts of what's happening is inside of this start async method here um so we're getting back our settings which is just going to have information in there uh for when we call out to uh our uh completions endpoint to get responses back from our large language model um we're going to create some chat completion options so there's a couple of different ways you can work with um generative AI uh chat completions is the kind of thing that most people are familiar with it's it's what you do when you're working with you know GPT is you're you're having a chat and you're getting completions back from that and we can control a bunch of stuff in the way this works um the maximum number of token so how many things are we sending across uh as um the request to our large language model we can restrict that so that we don't write war on peace and send that across and then it results in a massive amount of slowdown as it tries to generate a response um or we can just leave it to the default maximum amount that the model will temperature temperature is used to control the well if we um hover over and get the the um typ information it's used to control the creativity of what the model is going to generate back now um it does support a value of of kind of zero to two um but anything kind above a one um in terms of temperature is going to get some pretty creative responses the the things that you're going to see out of that are they might look like words but they're not necessarily words and they're not necessarily constructed sentences or anything like that good rule of thumb is you want to sit somewhere just below one um I tend to do about a 7even temperature um when working with it and then we've got some other things around penalties and uh and stuff like that but the most important thing that we can provide in our completion options is messages so the messages are the prompts the things that we're sending across to our request to generate a completion and there's three primary categories of messages um or or roles that the message can fall into we have the system role we have the user role and the assistant role the system role is how we set up the for lack of a better description the personality of the um conversational experience that we're building so in this case I have a system prompt here that says you are a friendly assistant named netbot you prefer Australian English as your language and you're an expert in the net runtime an expert in the net runtime and the C and F programming languages response using Australian colloquialisms and slang that's why we saw things like get a mate and stuff like that in the response that we got before because we gave us some personality so we're setting up this just initial set of options um write out the the starting message and then we are going to then uh we are then going to um read and from the user so that's where I typed in the message um if it was nothing continue if we said goodbye we're going to aort and to quit out um otherwise we're going to add the prompt that someone put in as a user rolled message so this indicates to our um our model that this is something that you need to try and answer from so this is something that the user is requesting a response to uh and then we're going to call out to get chat completions async we provide it with the model that we're using um so I could be using gbd 35 35 turbo gbd4 etc etc the options and a cancellation token cuz this is an async um execution uh we're then just going to check to make sure that we got some um some choices back so did the completion like did we actually get a generated completion um if it doesn't have any completion it's just going to you know I don't know what to do otherwise we're going to iterate through the responses and write them back out so let's fire this back up again and we'll come into our sample application and we'll do net run uh let's just full screen this up again see kind of what this uh like this kind of personality that we we've put in there like how does that uh how how does that really influence the way that we can ask it think so we Zoom that back in and we'll say uh should the word color have a u in it so let's see yes as in Aussie in Aussie English it should have a u in color favor favor humor um but if you yeah so so you can see how we've we've given it kind of this um yeah like I said personality to it you we've given it context into what it's capable of doing all right but we've also said that it's an expert in net so let's say generate a person type with a first and last name so let's see what it's going to do with that I haven't told it to generate C or F but as you can see that it's done that because um because we we gave the context to this um this um chatbot or the this this experience that you know its personality is you know it understands C and F so it's naturally going to give back those as a response um now let's say uh I want C only from now on so we'll give it another prompt and say um create me a list of five people all right so because we said I only want C from now on um it it it understands that we've added that as a message this um in there and we're keeping a a history of messages that we've previously asked so it's not generating F anymore because we said don't generate f it also knows to generate the person using the type signature that we have up here there it is because that's still in our history those have been pushed in as a system messages that we can keep track of uh and we can work with so that's um and and then we can obviously keep asking more questions like how do I filter and so on and so forth and it would keep responding back and it understands this history and it's going to keep using that information in there but we're nearly at the end of the demo for today so just to wrap up um when we talk about AI there's a lot of different things that it can mean when we're building AI based systems but right now generative AI is the thing that a lot of people are looking at how do we use these large language models to have an experience where we're generating answers based off of information we can give it context is key to get more accurate information back um at Microsoft we have AZ AI open AI service which has the gbt models from open AI that you can work with you can a layer with your own data to give your business context to it um but it's not actually training those models on that data uh so you're not having to worry about your business um aspects or business Secrets getting leaked out anyway if you want to play around with that sample application uh that I just had there and there's a couple other samples inside of that of using um other parts of the openai SDK you'll find that at github.com arenow conso GPT and to get started with generative AI it's really just a single HTTP call away like that's what these are just rest API call you don't have to become like an expert in how models are built or anything like that um I just want to do a shout out next week we're kicking off um a hackathon uh called the great. net8 hack uh you can find that at aka.ms Haack together.net and we're going to have some prizes for uh for AI based Solutions as well as ctive Solutions and things like that but that's just about uh me done for for now uh I'm GNA hand it back to today's host uh but if you want to learn more please do reach out to me I'm also writing a series of Articles around generative AI for the net developer on my website at Aron pal.com that's all the rest for me back to our host

Channel: dotnet

Views: 13,902

Keywords: .NET

Id: yc0Zl_UXCY4

Length: 32min 45sec (1965 seconds)

Published: Fri Nov 17 2023