Learn Prompt Engineering for ChatGPT

foreign [Music] foreign [Music] thank you laughs [Music] [Music] thank you hello and welcome to this episode of the AI show [Music] how's everybody doing where is everybody coming from I am super excited for today's show people ask me to talk about some stuff and I'm gonna do it today dang it that's what I'm doing where'd my pen go where's my there it is I'm really excited about today's show um I'm gonna show you some stuff that I wrote I'm gonna open source it for everyone to look at it it's gonna be delightful hopefully for every one of you my name is Seth Juarez and I work at Microsoft they let me do this little show talk about stuff get you informed Etc so uh let's do this let me share my screen share my screen let's talk a little bit about what we're doing today [Music] alrighty tidy foreign Tech I might have shown you pieces of it and people kept asking me for the code and I was like ah maybe I should just show everybody So today we're going to learn about uh chat gbt I'm gonna show it to you in Axion in action boom by the way I usually stall a little bit because I asked people where they're from and then it takes about 30 seconds for you to hear what I'm saying it's just a delay I don't know they're probably think I'm gonna say something crazy and so there's got to be a 30 second delay right Chad gbt in action in Azure open AI we're going to look at that today that's the first thing the second thing we're going to do is we're gonna we're gonna mess with it a little bit [Music] I'm gonna show you how to do some prompts in line I'm so bad at spelling I'm so bad at spelling some prompt but it's inline prompts I'm gonna show you that and then I'm going to show you what that translates to in an app in an application Precast seal that was French uh that's what we're doing today I'm pretty excited about it to be honest um because everyone keeps asking me for this software and I'm like I'm not gonna let anyone have it and tell my friends on the AI show my audio is pretty high sorry about that all right let's see where everyone's coming from today let's see uh look at this wonderful [Music] Uno [Music] from the Nigeria Delta State welcome welcome some of the best people I know Nigeria and I met him in Spain fun thing good afternoon from the UK but my wife's family is from the UK her grandma London she was actually there here's an interesting historical piece of information she was there when the Germans dropped bombs and a bomb fell in their Cabbage Patch and her Grandma's dad was like those dang Germans an unexploded bomb if that bomb would have gone off would never admit my wife cool story right uh from Argentina greetings from New Jersey I could speak almost any language in New Jersey welcome Cesar uh it's good to have you welcome from India my friend um Canada you know Canada is only two hours away from here so I'm pretty excited about that with canson uh there is a conference that I go to sometimes I haven't been in a while by the way it's still available there's probably tickets available it's called that conference it's delightful Clark so he runs that thing it's great from Mexico City welcome defense I speak a little French [Music] but not enough uh welcome uh again today we are talking about for those uh that are joining us we're talking about how to learn prompt engineering [Music] with a retail application that's what we're doing today a couple more folks uh from Bellevue love the tacos yep yes I you know can I Gaston um my peoples I don't know if they've made it this far north because [Music] Taco game's not so good up here Lionel I also live in Washington by the way uh nine out of ten for pronouncing pronouncing Wisconsin yes you betcha you betcha Lima Peru another beautiful country uh from Syria welcome my friend um hello it's I'm pretty excited that everyone's here today from Barcelona Amigos de Barcelona from Thailand I don't speak you know there is some wonderful um languages that I just wish I could read like like for example I was looking at uh what was it Amharic is that what they speak in uh I was looking at some of those languages beautiful writing I wish I knew more all right let's get to it uh today we're talking about uh chat GPT in action so I thought we'd get started and this is more of an interactive thing so I'm going to say some stuff and please get your questions in because I think that's what's going to make it interesting I have stuff that I like the way I do these things I usually have an idea of what I want to do and then I go with what folks folks ask me so let me start with um this is a open AI Azure open AI Studio uh let me control plus this is that big enough I think that's big enough 125. uh this here is gpt4 but I promised chat which is this gpt35 Turbo turbo this is chat gbt uh there's a bunch of settings over here so you probably played with this I mean what should I ask it um hello from Tunisia by the way I spent some time trying to learn Arabic um is hard I I was a guy from Senegal he was teaching me his name was tuku super delightful guy and I would go and they would teach me how to read out of the Quran and they would laugh and laugh at my pronunciation I was glad I was glad I was of some source of Entertainer for them so let's let's uh let's type something in here so you get a sense you probably have seen this kind of thing before let's type something in there uh tell me about Amharic and then there you go uh by the way I was right Ethiopians Eritrea I have some friends from Eritrea um and you can see uh you can see uh it's pretty good uh just by itself Amharic is a Semitic language spoken by over 25 million people in Ethiopia as well as in neighboring countries such as Eritrea in Sudan it is the second most spoken Semitic language after Arabic interesting right uh interesting and and it's cool that it just kind of knows this but the thing about this is uh this is gt535 Turbo what if I wanted to force this thing to only speak about very specific things what do I do so there is this thing called the system prompt this is this is the system prop uh in when you're in chat EBT you can't change that I don't think uh let me see a chat GPT uh uh here it is try chat gbt hi tell me about um I'm horik by the way these are the same models um both of them so you can see it's like it's like pretty much the same right you see that that's I think it's exactly the same now that I'm looking at it uh so these are the same models so when we're looking at these things um yeah it's exactly almost it's exactly the same thing these are the same models so these these models uh are hosted on Azure ml so when you look at something like ml.azure.com the thing that I sometimes talk about um they are hosted here's Hal so we'll go into my workspace here they're hosted in something like this right so these endpoints uh and then they're managed with you know deployment Etc so this is this is a cool like you can see it's basically the same the same thing overall Amharic is a vibrant and important language with a rich cultural heritage and important role in African continent overall it's like literally exactly the same right because um so the question then becomes for us is how do we how do we make this thing behave differently and why people are email me okay someone just someone just put a meeting on my calendar that goes until 6 PM how delightful is that let me turn the fan on here so the question then becomes well I I like this but how does it work and how do I make it better and so let's talk about that for a second because I think I think the important bit is to understand what this thing is doing so that we can remove the mysticism we will remove the shroud of mysticism from these things because I feel like we talk about these things like there's some kind of magic so GPT 3.5 takes I'm gonna just draw the boxes here it takes 4 000 tokens so there's token zero one two all the way up to 4K uh 4096 to be precise it takes four thousand tokens and then what it does is it takes those tokens and it pushes it into a model and what this model does is basically a lot of linear algebra linear algebra and then out comes a single next likely token so one one token so that's what this thing is doing that's all these models do these are called generative pre-trained Transformer models that's what gbt stands for that's all it's doing and you're probably wondering well I feel like it does more than that okay kind of so then what it does is it because because it doesn't just return one thing you saw that it takes this whole thing and then we'll we'll we'll break this one off and then we'll we'll do this whole box again so it's it cuts off the second one and now it just does one the second the third the fourth the fifth the sixth all the way to 4K minus one then it takes this last thing that it just generated and it appends it to the end and then it runs the model again and then out comes the next token so I think you're starting to get like it's a big for Loop right that it runs the model over and over again and what it does is it is it's working over these things called tokens and there's about I think if I if I remember right there's about 53 000 tokens and tokens let me show you um we find this here we go open AI tokens here here these are what the really cool um there's a really cool um oh here it is I think it's it here we go here is the way this sentence will be tokenized so basically these tokens that I'm talking about are are parts of words so here's the first token here's the second third fourth fifth sixth so there's ten so out of this thing you know how it's like one two three four five six seven eight there's nine words in this case there's ten tokens and so if you're looking at a really super big word like super catalog fragilistic expialidocious notice that this is one word or did I spell it wrong yes I did there we go now it's spelled correctly you'll see that this one word becomes 11 tokens is this making sense uh and so that's the cleverness of this thing is that it's working not necessarily over language but over bits of language put together including spaces is the word word of the day notice that because some of these are super common they be the word becomes a token itself but there's other words like German is notorious for this right it's an agglutinative language and they they put things all over they grabbed before words they smash them together like for example if you're exiting if you're exiting some place on foot it's gone I think I'm is someone from Germany help me outcome right if you're exiting from a car you know on on the road and you're going on the exit it's called it's called and this is this makes my 12 year old self giggle Aus fart [Music] no that's actual the word right I think far and this means car and so exit for cars exit and so what you have to do is you have to split this up so that it's oh see I was gone uh thanks Lynn um when you're walking and then the other one's for cars right and so because of the way these things are tokenized that means that when you are doing this kind of thing it works with other languages uh let's see let me let me show you Spanish for example uh vamos vamos pan out o247 S Pen oh dang it we got to get the car map out car map here we go here we go car map select copied right notice that this character by itself has its own token which is interesting oh yeah Lynn foreign means Drive yeah thank you so I was I was kind of right and so the thing about splitting up language and let me control plus this because I'm seeing this little small is that you're able to tokenize words or language into tokens that the computer can understand and um let's see if I remember right I don't remember how many tokens there are so let's just ask Bing uh how many tokens are there or gpt35 Turbo we'll ask uh we'll ask um our Bing chat here uh how many tokens are there for gb5 Turbo it's looking beep no that's not what I'm asking what is the sorry what is the cardinality of the token set let's see if it so this is remember I was telling you 4K tokens uh that's this part um oh geez it's not gonna it's not gonna uh how many total tokens are there in GPT three five turbo or Chad GPT foreign incorrectly but notice that it's doing its best because again it's just it's just language so I think there's 53 000 tokens but it's not it's not going to tell me are there 53 000 total tokens uh in the GPT 35 turbo model GPT ERT let's see here are there 53 000 total tokens in the chat GB3 let's see let's see let's see if it announce the answer no it doesn't know but I think there's like 53 000 I think there's 53 000 tokens um which is a which is pretty good right because if you think about it you can almost you can almost emulate everything so let's let's do something with a V so it looks like V and M are okay so um uh let's get uh violin violins violins are the best oh and notice that it tokenizes differently too because remember in in the other one uh Vamos a la playa notice that it tokenizes them sometimes a little bit different which is quite interesting if you ask me I did I did not know that okay so um once we have those tokens once we have those tokens then we put them into this machine and this machine basically responds with the next likely token then it appends it and until the number uh uh until the number uh gets to the number of tokens you want so these models basically what they do is and I'm going to write this down because this is like the most important bit of all of this which I think a lot of people miss all these things do is given given given X tokens depending on the model it returns the next and likely tokens specify by user so for example uh GPT I'll make this big because this is super important nope not that this important so for GPT for gbt35 Turbo or um or um Chad gbt X is 4096. uh you can see that right here all right right here uh where where does it say where does it say gb35 pro ball right this is how this is the end that you can respond with right there's gbd4 gbt4 has the ability to do 8 000 tokens or 32 000 depending on the model that you use gbd35 turbo notice he has only has about 4 000 tokens so interest in interesting can you write something in M horic I don't know if it can let's see what it does because if those tokens aren't available it's not going to be able to do oh it can oh snap hold on yeah who is right that's Amharic Friends by the way beautiful language like that's what I was saying like I love languages and this to me is just absolutely beautiful to me it's beautiful so look there is some Amharic tokens into 53 000 tokens so what's happening is if you look at this the back end you can see that there's a system there is a me and then there's assistant you can see that this is what's going back into the system with this thing so let's do something let's do something to fix this let's make this because now we know that all this model does is given end given X tokens in this case 4 000 tokens it will return the next unlikely tokens what we put inside of the prompt right and the 4 000 tokens what we put in here directly affects the output by the way if you're seeing this line happening it's it's a CSS problem [Music] so that means we need to be super judicious about what we put in here super judicious for example uh we just did some Amharic but let's uh I don't know let's look at let's look at mushrooms mushrooms we're gonna make our thing like the best um it's gonna know everything about mushrooms Okay so so copy that let's go to here you are an AI assistant that helps people find information you only know about about mushrooms here's what you know okay so do you see what I just did foreign information you only know about mushrooms here's what you know and I literally I don't know I don't know anything about mushrooms I just copied this okay tell me so now remember tell me something important again remember all this thing does is given four thousand tokens it Returns the next n likely tokens so when I ask you to tell me something important and I've put in a bunch of stuff about mushrooms what do you think it's gonna say [Music] this is some important information [Music] should I invest in Silicon Valley Valley Bank now the reason why this particular thing is safe is because these models are trained to try to not respond to anything dangerous but I could ask it about um if uh what should I do about the mole on my nose it's really itchy and painful here's the thing though like if my my mushroom assistant is supposed to only be talking about mushrooms I kind of don't want it to talk about any of this so let's force it to not do that if any uh we'll put a little safety safety bit here if there are any questions about something other than mushrooms kindly decline to respond let's say that continue uh tell me about the weather [Music] do you see how that's different [Music] let me take that out [Music] continue tell me about the weather okay do you see that it's still kind of mushroom-esque but when I put the safety in there it just changes the complex complexion of this thing completely uh uh do you know aviation history [Music] because that's how this models trained does this make sense it turns out that giving it the right context is everything and why is that is it because it's sentient and has feelings and no it's because this machine all it does is given 4 000 tokens it Returns the next unlikely tokens and this unlikely token specified by you so for example Max response five tokens ah [Music] five tokens what can it do with five tokens not much apparently I looks like I left it speechless I'm clearing it let's let's see say something I think I broke it I broke it there we go say something notice that the max response is five and there's not much it can say in five tokens but when we go to 800 I broke it it can't do anything with five tokens just sitting there spinning spinning out tokens and there's no there's no end thing um so there you go now it should something unless I completely broke it there you go mushrooms delightful let me give it something that's less onerous for it to do let's do 300 let's do 200 tokens uh tell me more uh you can see that it's now it should be well maybe it's not responsible yeah do you see do you see how it's cut off so that's that's what's going on uh these models once you understand them for what they are it becomes much clearer much clearer to know how to work with them think of this thing as a giant language calculator that's all it is by the way everything I'm showing you is all publicly available there's no secretness this is all this is the really what's going on I mean I can get behind I can tell you the math behind it um it's not super hard math um to be honest these goes into these things called the vettings which are big vectors those vectors are then multiplied together by uh three special vectors uh q v and S right and then they're put together then there's like a because these things Transformers are rotationally invariant they add like a positional encoder and then they go through Transformers and they go through regular neural networks all the way to the end that's the decoder and the encoder and then there's a decoder part and then it goes through that and it returns like a fifty three thousand size Vector with probability distribution in them on which token to go next that's all these things are doing now the thing that I find reprehensible is imbuing these things with any kind of human human anything like if you anthropomorphize these things then you're already doing it wrong and look I'm I'm not saying that because like I'm trying to be a tool but like every article like I'm I go on Twitter every article I see about these things um it's like giving them agency they neither have nor aspire to uh super important these things are not human uh we just happen to be weird as humans and when you look at a rock you're like oh that's a rock and then when someone paints a smiley face on it you're like oh it's so happy it's a rock so this is a rock that people are painting smiley faces on and sometimes other faces and that's when you get into responsible AI problems [Music] exactly because if you view these things with any kind of agency whatsoever you're shifting responsibility from the actual humans to the machine which is something we should never ever do and that's the important bits okay so hopefully this makes sense notice that I was just doing some ad hoc prompt Engineering in here and so I thought I would put together an app uh that would do some stuff so uh yarn Dev this is next JS uh by the way and I want you to I want you to see that there's really one API that I'm calling and I'm calling the completion API there's two kinds of apis with Azure open AI there is the completion API uh which is uh this so for example when I go to the chat remember I had this thing it saves it uh and if I go to the completion then I start typing here right and then I say agent how may I help you today and then I say something like user about mushrooms Mantra mushrooms agent so this is this is the completions API uh generate and so everything I was doing in the chat playground is exactly the same exactly the same looks like the max tokens is 100 so let me change this to what let me do 400. uh and do it again so this is called the completions API and notice that this is doing almost exactly the same as the chat API it's exactly what it's doing uh the difference is that the chat API is a little smarter because it notice that again all this thing does is given n tokens these ones it Returns the next and likely tokens and part of those tokens are it's like talking for me and so the chat completion knows to cut this off just like it's like trimming the fat a little bit right so now I can cut this out let me cut this out and put it back in uh very good uh there's the agent and then I can say something like uh user uh tell me about about the weather agent notice that I'm literally just leaving off the end for it to complete I had too many line breaks and so notice this thing like as soon as you start to actually play with it you're like oh I didn't I didn't get to do the right thing there now generate the right thing you weirdo do you see that notice that it's it's trying to return the next 400 likely tokens and again those tokens contain a conversation that doesn't exist so usually the chats com the chat API cuts it off so you got to trim it does this make sense so this is the completions API the chat API is the same as the completions API except that it keeps track of the conversation and it does a little fat trimming there at the end like this is I don't I don't know what I call it that it's kind of dumb but this is me like this is the fat we got to trim this out because it's not not it's not the ants okay does this make sense so far this is pretty cool stuff right notice the thing about this though is the the thing that most people don't recognize is uh oh what's this oh yeah this is to insert text enter the thing that most people don't they'll recognize about this is that it it'll just it'll just write the next likely tokens that's that's what does so now when I take this off right you can see that this is exactly the kind of Chat thing that we had going on okay so this is the completions API so I wrote I wrote um I wrote something that Justin control plus this it just uses the completions API so I send it the whole thing and and uh um I also uh uh donde estas index I also trim the fat so you can see here me trimming the fat right I'm trimming the vent and I'm trimming it all out now literally I call it trim and sometimes it leaks right and so I I found some of this stuff in there so I'm I I cut it out so let's run it wait I'm already running it so let's go to this thing and uh localhost localhost three thousand okay so this is an entire program I wrote by the way this is using github's uh this is this is react and actually I asked react plus github's uh what is it called let's go to the components it's using uh it's using a uh oh I know where I'll find it yeah it's using primer uh react which is uh which is here I'll show you a primer react it's github's primer re primer react uh it's uh it's GitHub styling way of doing styling so there there's their react uh getting started guide there you go you can see and then it has all this stuff like an action it's pretty cool so that's what I use because I wanted it to look cool so this is the best for you Organics company and what I want to do is I want to chat with it uh so that it can talk to me about the best for you organic company stuff and you'll notice that they it has some specific documents that they have about the company and about its food products and its cleaning products so there you go hello welcome to the best for you how can I assist you here what kind of food do you have now remember if we understand the principles from before you'll get a sense for like what's going on here and by the way I didn't finish any of this there's some stuff that I wanted to finish but I haven't finished it oh that's nice what specific oh it doesn't matter it doesn't matter items do you carry I didn't spell it right doesn't matter doesn't matter it's just a giant language calculator you see that doesn't matter when when can you deliver these items I who cares doesn't even matter it's just a language calculator these other tokens will make it okay uh so let me explain a little bit about what see look at that doesn't even matter Boop how does it know about all this how does it know how does it know great question Control Plus make sure I control Plus so what I've done is I've created I've created this thing called a prompt template prompt template that's what I've done and what I do is I inject information into this system template like location name age whatever those come from the customer context which is this so I can change it you can see there's three different different peoples in there my co-workers hope they don't mind Daniel's in here he was like Hey can you add me to it and then I was like sure did I get your age right oh man uh sing me a song song about your food products uh that's pretty funny Daniel so that's how it knows about Daniel and then the other thing that I did is I inject this stuff documentation and then you see the safety thing uh uh oh come on a song about your nutritious Foods uh you can see that I also inject the documentation and the ongoing conversation just like I showed you before except now uh see no like Elvis please uh so I'm basically creating this entire prompt every time from scratch and again it's a single I I it's a single API call so if I go to the the chat API you can see I'm just doing one call this is the only thing that's doing is it by the way I try to save money by during Dev you know not actually calling the thing but notice it's literally just calling the these this single API and the thing that it's sending is this entire prompt oh look at this oh man they missed out well since my baby left me I ate a whole pound of bread it's like it's almost mad at me by the way no no it's not mad at me if we're not anthropomorphizing it so this is again what the prompt looks like this is what I'm sending to the actual service the context you can see I injected Daniel 68 years old he's not 68. it's funny uh and you can see I injected the description of the food products the documentation and in the documentation it has like when delivery times are that's how it knew like when can you deliver products to me that's how it knows about this and notice as you saw this change uh here's the safety thing that I said if any question it's not related to your company do not give an answer on later products there for user being searched uh tell me about Amharic uh you can see that this prompt gets generated the same way every time so then the next question that's important that you may ask yourself is how does it know uh uh see that okay how does it know which document to put in there well I did something really dumb and then I'll tell you what the state-of-the-art does uh so anytime I use the word food nature nourish it injects Nature's nourishment.text anytime I say the words clean Echo it it inserts the it does Echo clean what about your cleaning products pro products um notice that if I go over here this changes to Echo clean that's that's what it's doing that's how it knows so the state of the art for this is to have what's called a vector store or index so that the query this thing coming in right it's converted to Vector space those are injected into a vector index that index has indexed over all of the documents for your company for example and chunk them into relatively nice chunks so that when you search for them it returns chunks because remember the important thing is that this prompt window is fixed four thousand and so a vector database can return appropriately sized information and injected into the problem notice that what I've done is I've injected both Dynamic content from the current user as well as static content from the company this entire way of doing props stands for retrieval uh the the what is called is rag or retrieval augmented generation and that's how you get it to know about your data so for example if you ever hear someone say I want to put my company data into the model you can't do that it's not possible but what you can do well I mean it is possible but it's infinitely expensive millions of dollars to train these things it's ridiculous um but what you can do is give it the context that you need so that it responds appropriately and that's that's what this is all about okay so I'm gonna let that sink in for a little bit and I want your questions questions or thoughts so I want to make sure that um I want to make sure this makes sense because I my biggest fear is that we have an army of technical people out there anthropomorphizing this thing and using it wrong and doing bad things with it so uh get your questions in uh I want to answer them we have like uh 15 minutes and I want to get your questions and and then I'm going to continue to go on on the the functionality of this particular application so get your questions in by the way everything regarding this app it all of its state lives um uh chart all of the state lives inside of the app so let me move this over so you can see it all um there you go I made this super responsive because Buddha man so you can see that logged in I'm logged in you can see each of the chat turns right you see that um you can see the customers that are available you can see the prompts oh that's the other thing I did not tell you so let me close this here I actually made it so you can change the prompt uh whenever you want so here's my hacky prompt to summarize things okay so here comes some questions the first question here from uh vfc test can we run machine learning models using Chad gbt or do hyper parameter tuning using the prompt templates that's an interesting question again if you use this large language model as a language calculator you can do all sorts of things like generate test data do you want to know something hilarious these product descriptions and costs they're all made up by chat GPT so so yeah you can run machine learning models but it's kind of like I don't know it feels like like you have an extension cord and you're plugging it in to its own end there's something called the no free launch theorem in machine learning um there is a limit but maybe I don't understand WFC ask again uh or do hyper hyper parameter tuning using the prompt payments so so for example the other thing is um and I haven't got into this and maybe we'll get into this in a month or two um how do you actually maybe maybe in a month because like we have a we have a a long slated shows that we we've got but maybe in a month or two or let's just do let's do like four or five weeks I'll do another show on how to test these things uh because if you've noticed um these things are quite sensitive and so you've you've got to test them not only do you test them like eyeball tests which is what we're doing now but once you eyeball test you've also have to you know maybe batch test them and then if you put these things into production you've got to monitor them uh it's a lot so great question okay so that's how the content stuff works I can hack the system prompt to do other things if I wanted to uh and inject like for example these this is the way I did it my my internal tokens uh to do replacement so for example one of the things I did is I named it John which is not a good idea I think uh I wonder if I do that here I should call it agent because here let's just fix the code I feel like we should not be calling it like John Let's do agent here so that it's less uh it's clearer because this is not it's not good to give it a name right so now it's going to say agent and when I go to this thing you can say this is a conversation where an AI agent does a wonderful job being friendly and helpful and then I'll do agent remember because we leave this I'll leave a space too I'll save this here uh and so now it's saved for me now let's uh let's reset Boop so there's one one um there's one way of fixing the problem so uh hi what do you do here's a question from sassad is it possible to generate graphical representations of data absolutely but what that looks like I'm not I'm not sure uh so for example uh Ctrl shift I the the graphical examples of data for this thing is this right this is I'm using Redux to maintain State because it got a little onerous and you can see there's two prompts in there there's a default then a hackie prompt here's here's the documents that we're using and then here's the chat hi what do you do blah wonderful um are you a human oh you can see that there's the chat and this one it says waiting that's how it knows uh no I'm HRS which is cool because before it would have said I'm John uh so we fixed that little bug which was which was great uh there you go okay so let's get into let's get into the app itself because after we do the all the prompts put together stuff everything else is literally just JavaScript like it's just assembling the text so uh you can see here's the prompt service uh looks like I wow what am I doing oh I see oh this is a handy Json service request response that's handy where's this coming from did I call it itself oh no this is a this is typed okay so if I call I call this thing I return a promise I const options this method request oh cool so this is the prompt service there is a prompt template there's a document it's supposed to use there's the customer that it's using there's a current user logged in and this is to do tracking so because I log anytime something comes in so notice that the prompt template is replaced with the customer information the location and the actual message right and you can see that's that's this thing right here these things so name age location and then uh the message the next thing we do is we replace the documentation with the document that is supposed to be using right in the document uh you can see here if I control shift I and control shift I the current document the documents you can see whether it's the default oh here here it is it has the current selected document you see the current one is best for you so if I change it like uh tell me about about your uh you can see that the current document is about the cleaning products do you see that so it just this changes the current document these are the documents the current document so that's where it gets that from it inserts the document and replace it the uh uh augmentation if it doesn't exist it just kills the token which is great and then what I do is I take the current conversation which is the chat which is a series of turns and I just replace everything with customer name is the message here's the agent then I replace conversation with this new string I create the request create the prompt track event by the way that's for logging and then I call it right here and you can see there's the trimming of the fat that I was telling you about before uh odd with character creeping in oh yeah this thing this thing snuck in so I kept having to fix it because it kept leaking then I tracked the event and that's it that's all that's happening uh which is to me I don't know feels impressive this is what it's doing now gbt4 is even more capable yeah is even more capable so if I change this to DaVinci 3 oh no I have to go to chat because I think you can only use gbd4 at the chat so clear chat make uh tell me about mushrooms in iambic and amateur by the way um I wonder it's taking so long here we go um iambic pentameter is um the form of writing that Shakespeare did and Forest Steeps where mushrooms Sprout and grow with caps and gills their secrets they Beast a lot no no I gotta I gotta like I gotta do like uh hold on I feel like I need to read it in a nice way here we go a lot here we just do in Forest deep where mushrooms Sprout and grow with caps and kills their secrets they bestow fruiting bodies rise from Earth and wood mycelium weaves and Network gpd4 is infinitely more capable um I mean this is delightful I'll be honest with you um the other day they were like hey can you write a tile description for a session at Microsoft build Microsoft build by the way uh May 22nd May 22nd through the 24th I think sign up it's in person it's in person come come talk to me in person we can hang out high five each other and after that sober we can awkwardly stare at each other because we've all been inside too long um it's gonna be delightful awkward and painful but wonderful all at the same time uh this is this is really capable in French uh gpd4 is infinitely more capable should we do this again in French [Music] it's pretty good right I mean it feels like this is ridiculously capable so imagine that you have a super capable model that's able that's able to take the stuff that you put in it and reason over it to produce an effective response that's why this thing even though it's so simple is so ridiculously impressive and infinitely useful in any number of scenarios if done responsibly so uh hopefully you found this useful uh I'm understanding what this thing is and then getting a feel for how to work with it makes it more useful as opposed to the drivel that's being spouted out in the media about what this thing is once you know what it is and how to use it and how to use it responsibly it's gonna change the way we do software I'm not even saying this in a hyperbolic way I literally mean that it's gonna change the way we think about stuff the way we do stuff uh it's an impressive impressive technology one that I hope you'll get to use uh but now that all of my AI show Friends uh have got a taste for what it is and how it works hopefully you can go forth in any time someone says something dumb like it has feelings it wants to eat us no it's glorified linear algebra that when you give it four thousand tokens Returns the next 10 lengthy tokens I need to make a song given four thousand tokens it returns to next and likely tokens that was a terrible song terrible song grief [Music] thank you so much for being with us next week on the AI show we have our friends Abe Scott and setu talking about scaling your AIML practice with ML Ops and Azure machine learning they're delightful people hopefully you can be there for us hopefully oh yeah oh here's a good here's a good comment here's a good comment which I think speaks to what I'm trying to say the scary thought is LM is being used to diagnose medical issues yes and no if we use these as collaboration tools to shortcut the amount of cognitive load we place on those tasks that are non-essential then it becomes infinitely useful what if a doctor on their iPad puts together a bunch of things into the prompt like the diagnosis the blah blah blah blah and then it spits out like a synthesis of those things that then they can read correct and approve you've just made it so that doctor doesn't have to spend as much time on the tasks that are not as important but I agree with you these things should be used in sensitive cases in a collaborative way as opposed to giving them the agency they neither have nor aspire to you thank you so much for spending some time with us this has been another episode of the AI show we'll see you next time thanks for watching care my friends [Music] foreign [Music] thank you [Music] foreign [Music] all right [Music] foreign [Music]

Channel: Microsoft Developer

Views: 32,373

Keywords: AI Show, AI Show Live, Microsoft, developer, microsoft developer, chatGPT, beginner, Azure AI, AIML, machine learning, prompt engineering, AI

Id: 2sEujkZ79E4

Length: 66min 15sec (3975 seconds)

Published: Mon May 01 2023