A language for LLM prompt design | Guidance

over the past couple of weeks I've seen this prom engineering courses propping out on marketplaces all over the internet as if all of a sudden these gurus have learned their inner workings and are now the Authority or something they didn't even know existed a month ago so I'm calling it out proper engineering is mostly BS and these courses are mostly a scam so don't waste your time and I'm going to show you in 30 minutes what real problem engineering looks like and it's probably not what you think they are or what the gurus want you to think they are it's not about writing detailed instructions in clear English grammar or those kind of nonsense I mean that would help but don't go around charging 500 on your power engineering courses for that because it's just not right so what is prom engineering from a software perspective it's basically a way to get your language model to Output what is expected of it the correctness the factuality the relevancy to what you need this output for so if I'm asking language model for a Json object I want it to be accurate to be correct but I also want it to be valid Json if I'm asking a language model for a SQL query same thing it needs to be a valid query syntax and not just notationally but also semantically correct so the first step is to introduce some sort of prompt design framework a pseudo language or templating language to help you express your intent to this language model now I'm going to be using guidance which is an open source project by Microsoft that allows you to do this problem development using something that feels like a templating language so first step let's install guidance and then let's hop over to its GitHub page to see how it works so I'm going to activate my environment and I'm going to say peep install guidance and that's about it so this is guidance on GitHub enables you to control modern language models more effectively and efficiently than traditional prompting or change let me zoom in a bit guidance programs allows you to interleave generation prompting and logical control into a single Continuous Flow matching how the language model actually processes attacks so it also says here the simple output structures like Chain of Thought and its many variants have been shown to improve llm performance so that's what guidance is let's go into our code and write some code and see how all of this fits together so let's go ahead and just open up a new file to work with I'm going to call it demo12.py because I'm just been using the same repo now you if you want to go ahead and download all of this stuff it's all on my GitHub repo I have a YouTube series long running series I think it's like now video number 11 number 12 now where we just look at all kind of things you can do with llm we can learn about what an LM is learn about embeddings learn about Vector store and all of these building blocks of LM right if you want to learn about that go to GitHub go and look at the Rapport you can start you can Fork it or make a copy of that you can also follow along the series on my YouTube channel all of that is free all of this stuff data sets code all of this is free on my GitHub and on YouTube all right and if you're not new here then you kind of know what to expect so let's begin by importing all environment variables so I have an EnV file in here then it has my open AI API key and stuff now you don't actually freely need to use all API you can use pretty much any LM that is supported I'm going to give you a few examples here but I want to show you that if you have your environment in your EnV file what you do is you can say from.f so you want to bring that in you also want to install this if you don't have that then you say from load.f then you just want to say load.m all right so this is how you actually look at all the EMP files and bring in your keys here and random variables write your keys and all your secrets all of that and you're loading them in and now you could go ahead and really start to print your stuff out if you have your environment various you can print them out right now we installed guidance we need to bring that in as well so let's go ahead and say import guidance okay so this is something you need to install as well so that's the only two things you need to do for this video now I'm going to give you a really simple example um before we begin right so first of all I need to tell this guidance what llm to use to power all of these programs together and so I can say guidance and I will set my llm and this there are plenty of choices here I'm going to show you a few choices now if you look at this I have a great open AI I have LM mock mock is just a mock-up just for testing open AI Transformers there's plenty of choices in here you can use a few I'm going to show you some examples later I can show you some maybe open source ones as well but I'm going to first use a text Devin C right and then later on in this video I'm going to switch it to another LM and I'm going to show you what happened with a chat based model for example like uh chat gbt for example that's also one option you can use so let's begin with this now if you've seen my open AI video you know what happened next is that you just basically call open AI you pass in your prom you said well using this model go and get that result or if you're seeing my Lang chain or Lama index video you know what to do as well it's pretty similar with flow here right now this guidance is all about this prompt and gear it's all about prom design right so let's take a look at how we want to quite we want to see something like call the program and I want to pass my prompt so I want to pass in like for example you know what is one plus one or what is your question you get what have you right now this program is not defined so we need to define a program and so the program will look something like program uh it goes to guidance and let's set that up so if I want to ask what are the let's say the top 10 most common commands used in the OS operating system so I want to ask this question so I could use this and in here I'm just going to use a simple string to say what are the top 10 most common commands used in the and then let's say Linux operating system now I don't want to have Linux being hard-coded so I want to actually pass this in so that the user calling this program could specify that so I could say OS equals to Linux and then now I don't have to high code this anymore just use templating to say OS right if you work with ginger or Django you've probably seen this before floss use ginger Django has the right symbol the header bar syntax if you use any of those handlebar syntax languages you send this kind of a bit quirky but you know that's what it does here you're saying this is a template so it says fit this in if the result equals this like a pretty result and that's kind of the simple hello world right um basically called OS equals Linux what are the top 10 most common commands used in the Linux operating system now of course after each spot I'm not showing you anything fancy about guidance at all you're like what is the point of guidance at all if you're going to just do this you could just do it in Manila Python and just call open Ai and you're right so let's go ahead and introduce some maybe layers by layers of complexity so the first thing I want to do is I want to give it an example of that uh maybe start off with preparing some text there so I want to say something here is a common command and I want to give you some examples so I'll put things into a single quotation because I use double quotation up here so here is a common command and this is what I want to do again you see that and the bottom bleeding the curly brackets and I want to say generate so Jen is a way for guidance to tell the LM to generate stuff in here right and whatever results in here you could give it a name so kind of think of it like variable so if I want to call these commands I can do commands and if I do that now I could go ahead and add commands to this this would be the key that would be a key core commands and that is from here this is whatever generated it's going to see in here and later when we print off this it makes sense right then I could say Don't just show me one but show me 10 of them um and and then you stop at what so I want to say stop in this case here again single quotation I want to say stop at the end of the quotation mark so this is saying that if you generate a sentence this is where you end it so giving it a a way to know like okay this is the end of one iteration now move to the next one right and how many of them do I want I want to have 10 and of course you can stop here but if you want to change the temperature if you want to do anything of that sword you can have it here and say oh I want to change the temperature to 0.80 or 70.75 um it's up to you all right just to make things clear I'm going to add some dashes and then we're gonna save all of that we're gonna just make sure we're in the right environment clear the screen and then we're gonna run this so demo 12.qy that wins003 can't be mapped because there is no darvins there is only wnc so let's run it again sometimes the error is so obvious and you're like ah how did I miss that right so let's run it again all right and this is our answer so let's see so first of all it prints the result commands and this is the commands in here and it takes all of this and you put them into the value of this key so result kind of a dictionary with a key of commands match to this one and that's the value so you have LS list directory contents CD change directory and we move files copy files okay so all of this is not bad all of this is pretty good so you see over times the answer gets a bit more random but it's still the same structure is there so number one is the ls then the CD then it's m we then CP then it's RM then it's man man stands for manual and then grab and then pseudo fine and then CH Mark that's for you to change permissions and stuff so same thing here right l number one LS then CD and stuff and each one of them give it a next line so that's that's okay but let's say if I want to ask for only the top five so I'm actually going to change this to n equals to one now I'm gonna ask just for one and if instead of top 10 I'm gonna ask for top five so what are the top five most common commands using the OS operating system and now I'm going to move this to a next line so this is not just a here is a common command but now this is going to be care of the top five points right or maybe just be more explicit common commands temperature is still going to be the same now the only one thing I want to change here is if it's going to keep on going how do I cause it to stop now I could change this stock here to say that you stop when it gets into number six so I would say something like when it goes to I skip the n and then say six dot so what this is going to do is it's gonna find this and you say this is a sub character when you come to n6 then you want to stop there because then it results in the top five so this is one way to add some sort of assurance to yourself to say that hey by the time you reach to n6 and you know that because I printed 10 iteration and each of the iteration it always get to this point you see n6 and you see n6 in here and you see n6 in here again you see n6 in here again and six again so you always see the answer so I I'm putting n6 in here because I see that that's the pattern and then I escape that by adding a backslash in front so here I'm saying that you want to generate that five so go ahead and generate as much as you want but only up to the first because once you get to the stop you want to stop there so let's run this program and see what the difference this result would be compared to the earlier results run that and now we have the what are the top five most common commands used in a Linux operating system so first I print the commands this is here then I print a result so you don't actually need this if you don't want to you could just remove them but here's the result right LS used to list the contents of a directory CD used to change directories and we used to move or rename files our M used to delete false men used to view the manual page for command that's pretty good and it's pretty tidy and you see that the stock works here as well right if I put five then you will stop at the fifth because it would just stop right here right before I am man so you see llcd and we are M all right then you can up the temperature a bit to try and experiment on your own I'm going to keep that back to six so that was just a basic introductory but let's move up now and actually see some of the guidance syntax and it's pretty well what you can do you know in terms of the problem engineering problem design kind of things it gives you very clear ideas of how you could improve the quality of your output and you can specify them in this very clear templating language so I'm going to show you some examples here now if I go back to top 10 most command common commands used in the OS operating system I could maybe add a few more details here say I want you to provide a one-liner description for each command right now the other simple things I can do here is I could do this block and every time I start I use the pound sign or the hashtag right the brown sign I said I could this is a block and then I want to end the block and to end the blog I will do something like this so that's kind of how you create a encapsulation and a block of logic in here right so if I want this block to be hidden from the output so I don't want a result to include that because if you look at the terminal here it always includes that tax here right so he says uh what are the top five here a top five now if you want to hide that I could actually hide that by specifying these two hidden equals to true so this is useful for if I want to prompt engineer stuff but I don't want it to be included in the output of my of the result so I don't want that right I could do this so I could use this as a way to maybe specify some examples I could say a few examples maybe a few example commands would be now to make this a bit challenging I don't want it to have like one dot so for example I wanted to have this syntax of like bracket like this I wanted to have this bracket or maybe this one even right I said um give me something that shows like a PWD print the current looking directory and we move the file or maybe we can just copy here later we just used to remove our rename file so use to move or rename false just literally copying what I have in the beginning there right in the earlier iteration in fact I don't even need to I can be even lazier and just ask it to generate more for me so I could also use the Gen here so what this is going to do is like okay I'm gonna give you two examples but I want you to generate let's say another two examples so here I want to say give this name examples you can name it anything you want honestly let's see name examples then I want to say n equals to two so this is going to say try to look at my examples here and try to complete that try to generate another example and then of course you want to say stop when you meet a quotation mark so stop there and then you want to say maybe give it Max tokens here so it doesn't generate too much I just maybe put 20 just enough and finally up the temperature a little bit just to experiment okay so here I'm asking it to First generate this but all of this should be hidden so even if you generate that it's not going to actually be shown so if I want to see that I really need to go ahead and maybe add a few print statements so this will be result but I want this result to have the key of examples because mind you in the result itself this is not going to be included so I need to accept specifically specify the key here examples in order to see what is being printed in here now I could say okay here are the command commands so this is kind of like a warm-up this is kind of warming it up it says this is what I want you to do this is my expected output now here are the common commands so by doing this I basically give it a few more examples to prompt this is the format I need because maybe I'm marking the assignment I need this to be a set of format or maybe I'm using this in some sort of parser there is a downstream pipeline you know the process tags and stuff so maybe there's something and there's a reason why I wanted to be in this format but by prompting it first I give it more context to work with and give it more examples now I can say go ahead and generate that for me now I don't even want to leave this for a chance because for some reason this is really important I needed to start with a square bracket and then one and a two and a three so for for some reason that's just very important to me so I could do I could say take each one of them January each and then immediately close see what I like to do is I like to immediate because kind of like HTML you know when you start opening block you want to close that closing tag so opening tag closing tag and this I give it another name I'm going to click commands so I could go all the way down see if you print the comments out for me as well if you'd be so kind bring this down go back up and then now you could generate um I don't know 10 iterations I guess 10 is good because we ask for top 10. and this is where you want it to generate your examples right so you could do a gen so I could say something like open quotation then I could say jam and then generate the examples so each one of them is just gonna be one iteration so generate one iteration and so the way I would do that is I'll say this right this refers to sort of this iteration of this Janice then the stop is still the same though right stop is just basically again the end quotation mark I could now specify some sort of structure around the output so suppose I don't want it to generate this kind of output anymore right I want a description to be after the description so I could say description and then now I could generate another one so I could say Jan in the description stop so I could do it like this so instead of having it generate everything into one line I could say okay put a comma for the description so that way it's more possible for Me Maybe I'm writing some sort of regex I said look at look for words and I said take out the keys and then description should be another one now there's many reasons you will do that but here I sort of guarantee its structure by providing that in fact I could even guarantee the structure of this by adding a few more tricks here so one thing I could do is I could say s square bracket and just to follow the format of this so this is the format right so I add this in and here is my one two three each so what I can do is I can use the special index our syntax to say it starts from index so the first time it's going to run this genius it's going to be zero and then zero and then you generate a one of the command and I give it a description and then one and then description again the command description to the again the command the description and this is really nice so let's run all of this as a sanity check let's say python demo 12.pi and now it's done so you have the zero one two three right so if you look at the dashes here now the first one is the result and the result is just gonna be this what are the top 10 most command used in the Linux operating system provide a one-liner description okay so first observation is that it pass the Linux into this one and it generates this but it doesn't generate a few examples because if you see what happens it skips off all of that it skips all of that it goes straight to the here or the common command so what happened to this block so why is this line printed but not this line well because you put everything into the block hidden equals to true so none of this actually gets printed out all right but then it jumps all the way to the here are the common commands and now let's see Zero because I say it starts from zero right and then it generate the ls it says description is the list directory contents that's correct CD change directory PWD print working directory and make it there and so it generates all of them up to 10 of them so it starts stops at night all right then it says here are the top five most common commands so one thing you realize is that now the top five most common command also changes because now it uses the format that you see up there so this is great because I prompted from here I probably in here and I give it a description and now I said well from there generate the top five most common uh common command and now I give it this kind of structure that I will ask for suppose that in your prom exercise suppose that this is really what you want then you can also hit quite all of this into the block structure so sort of this like this so you're hiding everything into the hidden equals the crew block and now you're only getting the top five most common all right in fact you probably also hide this right so you also don't want to show this you just want to go straight to the answers so there's also one use case for that now let's move everything back in order now I specify hidden equals to true but I did specify examples in here so if I print out the examples I can actually see the answers in here so this is actually hidden right so I said well this is the first one PWD prints the working directory and then and we use to move a rename file and we literally stands for move but I also need to generate two more and it generates two more so the first one CD changes the current login directory LS lists information about false and NSC and then now you're getting another one CP copies are far from location to another and the four pseudo executes a and it run out of tokens because here I'm only giving it um tokens of 20. so it runs out of tokens it didn't complete the entire fourth command for me right so you can actually use this key value by looking at you just to pull out what you need so you can pull out exactly what you need from the commands you can say commands you can say I want to pull out this um so you can do that as well this here doesn't seem to make a lot of sense because I'm putting them the same so this is actually all writing that so you want to have a commands too for example um in this case I'm just going to delete this one now suppose I want this to be a teaching program so this is a CLI tool right I'm giving you some tips for the day but I also want to give you a quiz I said well you want to test yourself you want to see you know whether you truly know Linux commands or not right so I could say something like Chris of the day and then ask you to answer a simple quiz I said okay maybe now that you think you know Linux maybe I can ask you a few questions so explain the following comments for three points I'm going to change that as well in a second now I want the commands to be somewhere from the earlier comments so there are 10 of them and I want to say randomly pick three of them pick three of these commands and put them in here and ask them to increase the user so I don't want to show the description I only want to show the the command I want to say explain LS to me explain MV to me explain CPA to me all right so here I could now use the each and then immediately close to each all right and what's really cool about guidance is that I can even pass a normal function so I could say something like kick let's say pick three now P3 doesn't exist but I can specify that in Python code right and I say pass the commands into big tree so commands is actually from here I say pass it in the big three but where is bigquery from that's not even defined so in the program I could have a Pick 3 and just to keep things simple I'm going to use a Lambda I generally don't recommend using a Lambda I generally I recommend writing a function and actually naming a function but in this case here I'm just going to say take set X so here we're gonna find the commands which is from here which we printed out earlier right these are all the commands so I'm saying take all of them take all of them and then set them why set them because I don't want to have any repeat so I'm here I'm sort of trying to give it more Assurance to make sure that my output doesn't contain any duplicate because sometimes LM tends to do that it tends to give you repeat um you know query so you say LS and we CP and then you go to LS again I don't want that so I'm saying take the set and this is very common to sort of deduplicate the stuff right and then just randomly take the first three or something this whole thing I'm gonna call it a function called Pick 3 and then I pass a picture now pick 3 take one argument which is just X and that is going to be from commands which is from here so that's really awesome you can do that so now in the each block you expect to have three of them because zero one two right you expect to have three of them this is the Pick 3 command uh which you wrote yourself now I could also specify my own stuff so in this case here I would just copy what I have up there which is the index instead of actually putting them in square bracket I'm just going to put the dot so this is going to be one dot but it's going to start from zero dots which is pretty weird because the index starts from zero so you're gonna have like 0.1.2 so that's not very conventional so I can actually say plus one so the zero plus one is going to be one so it's going to begin with one dot and then two Dot and three dot it's gonna it looks more natural and we're more used to seeing that right then the next is just like this this is similar to what you see up there so it's just prepending the one dot this 2. this three dot this in fact you're not even limited by how many functions you want to pass in so suppose I'm I want to change this and I want to say take this and insert a random Point here so I don't want to say three points because that's like hardcore who hardcore stuff anyway right so explain the following commands for three points now again I'm giving it some sort of structure I mean guaranteeing some sort of structural Integrity while giving it the enough freedom to use the RM to generate the free flowing languages and stuff within the constraints of the structure that I provided right so here I have the random points and how do I this is not specified anywhere so I'm gonna go ahead and also specify that in here so let me go ahead and use import random and now I'm going to say random points equals to random dot ran in that's random integer so just get me a number between one to five I don't really care just a number between one to five and then use that and specify pass that into random points and this is going to fit into this text here so explain the following commands for any random mod points if you can complete them you get these points that's it right just give me five things a little bit and before I run this I also want to have this being randomized because since we're gaming find stuff we might as well go with the full gamification so I don't want to have to have a hard-coded uh quiz flavor I want to change things up every day you open the program it should give you a different kind of flavor so I'm gonna specify some flavor text right up here when I say flavor text or quiz flavor and I'm gonna ask you to pick from one of this value so here I have quiz of the day right and then I have a few more I say tests your knowledge here is a quiz and then one that is slightly more snarky maybe you think you know Unix right so just four of them um doesn't matter you can have as many as you want so I'm just saying this is quiz clever then I'm gonna go in here and pass that in to my program which flavor equals the priest flavor so every time my llm runs I wanted to just select one for me right so just choose one for me and so for that I'm going to use the select syntax and then select what select a flavor so um you can quite quickly if you want to be consistent I can just let me flavor this is um you know whatever you name it this is kind of a variable name there anyway but here the options this is not up to you this one you have to specify what you pass in here so we name it grease flavor so that's a quiz flavor so what this will do this select is going to select from one of the Quizlet where is this from it's up here right so it's going to select one of the flavor and then it says explain the following commands for whatever random points this is a random number um the random number is generated from here so let's save all of this and take a look all right so command variable flavor not found and that is because this is a capital f so we will need to change this to a capital f okay let's run that and now it's done all right so let's take a look okay so here's the result what are the top 10 most common commands used in the Linux Operating System Program online description so here's LS list Direction contact okay so all this we've seen before um all right now here is a quiz okay so here is the quiz is actually a random number from a random selection from here so instead of pick increase of the day audacity knowledge it picks the third option which is here is quiz that's that's great and then it asks it gives you explain the following commands for random points included so again it uses my structure it allows me to just interweave my structure with the prompt it gives enough room for LM to just be you know be jumpy be creative but within following a certain specified structure and giving you some sort of output and that's really cool so here it is explain the following commands for four points but where is the four from that's just a random python um you Generate random number right and then now it gives you three different options instead of starting from zero you see in at the top it starts from zero see that at the top it starts from zero but instead it starts from a one dot so how do you start from one because I just take index and I say at one so zero becomes zero plus one becomes one all right then it asks you to explain the three now I don't have a description because you're not supposed to see the description you're supposed to explain them for me to get the four points and this is the kind of tooling that the guidance provides you so when it says it allows you to just control your language models uh more efficiently and more effectively this is what it allows you to do instead of just trying to do prompting or changing just one big problem you know in traditional when you do Chrome engineering you have this one big problem and you try to put everything in nature English and like uh you know please do this and please assume the role of this or whatever these guidance programs is a bit more robust it says you can interweave your generation you're prompting your logical control all of that into a flow into a continuous flow right and you know what's really great about this kind of tool is that now you have more assurance and you have some more guarantee that you can generate syntactically uh valid a format for example Json right now I could say something like okay use the comments above as input generate a fairly Json object that Maps each command to its description okay and just to help it a bit I could open up like maybe a beginning of that I can say okay I'm gonna start this off for you I will have different different keys so the first key is the you know the key here would be the Linux all right OS so I want this to be just OS this is kind of the same thing up here right taking the OS so again using that templating so that's the first key and then now this will be an array and within the array now I would say generate each so I can say General H and then General each one and here you can say commands you can name it anything you could say alt instead of just want to say object and I say number of iterations equals to one so just generate one now if you actually seen the Json before right in this array how do you specify an array you actually separate them by comma after each line you want to have a commas you say a comma B comma C but you don't want a comma to appear before a because a is the first item so I could say unless so basically indicating the first one unless it's the first one right then again immediately close the unless so unless it's the first one you can add a comma to it all right it makes no sense to add comma a comma B comma C but it makes sense after a so a comma B comma C comma right so unless it's the first one um then you know don't bother adding so close out the Jamich and then within the Janice you can now have the same thing again we explained this before you can just have your this so that's a nice looking um sort of opening for it right and you can ask it to complete that for you um You probably don't even need this even though it's smart enough to complete that for you so let me save all of that um move into my terminal let's run that so yeah we see all of that above um here are the top 10 most common commands and give you 10 of them here is a quiz now it picks the oh again because the third option but now instead of the random number generating the fourth you generate the first one point so explain the following commands instead of the Four Points you get one point now for chmod grab and PWD so if you think you're using Linux and you want to get more familiar using Linux try your hands on some of these quizzes and see if you can explain all of them yourself now let's test out the Json right so there's a Json okay so here's a screen uh here's a Json object and then there's a Linux okay that's okay and then there's a array and an LS list directory contents everything seems good it close out the array you see the thing is that with all of this uh facilities it provides to you you can be more assured about the kind of output you can get and this is not too much to learn right these are all pretty intuitive you know you can have a simple example laying around and it's pretty intuitive to learn so I would say you know this is a lot of value um and you know the only dependencies to just add guidance into it and that's kind of it no you can't really talk about top tip prom engineering without mentioning the idea of using rows on chat-based models like chat gbt so chat models have typically been tuned to expect this kind of like Road tax in the prom which is why you see a lot of problem examples they they start off with like you are a financial advisor do this for me or you are a real estate agent you know assuming the role of this you know do this so when you see Guru is selling you 500 courses of proper engineering it's basically nonsense Beyond just the first five minutes of telling you about the you know train of thought process the step by step and then you know telling you about roles and stuff and and really all you're doing is is the three or four different techniques right anything else basically just fluff right so specifying the row and say hey assume the role of a real estate agent or assume that you are a financial advisor and do this right or you're a university Professor do this that that's one and then the next one is basically step by step right you can sometimes people call this guided from engineering or use like fancy names like chain of reasoning or multi-step reasoning you know all this stuff but what you really do is you break down the program you do steps and then get your LM model to take these incremental steps towards the final answer so I'm going to show you both of these as well using a chat based model so I'm gonna actually duplicate this I'm gonna copy that um rename this so I'll delete almost all of this um just keeping up the first six seven lines of code and then what I'm gonna do is I'm gonna change this to instead of a text I mean C 0 0 3 I will change this to a chat based model and one of the chat based model is jvd 3x5 3.5 turbo right now I want to specify that if you don't want to use GPT for any of this some of you have left comments in my channel on the on the videos and say that I don't want to use opening igbt because it costs a lot of money and stuff well you can use one of the free ones right so you could use a Transformers this will go into all the hugging phase models you can use one of the hugging phase models so one of them is the stability IAI model and that's the step stable LM base Alpha okay so there are two in here so this one is the 7 billion model so you can have stable LM base Alpha 7 billion and the other one is a three billion model so pick whichever one that works on your computer right if if your computer you don't have that kind of GPU space you can't have a 7 billion model running then just use the three billion model one and then you say device equals a CP um I think 3 billion is probably light enough that it will run on most famous computer but if again you go into any error just let me know in the comment section and I'll be happy to you know give you more advice uh depending on what kind of Hardware you have right but if you can spare the 7 billion just go ahead and just do a 7 billion that's also one option this is the open source model you can use a Transformer it goes to hugging phase and put it on your computer's cache right so pick one I'm gonna stick to gbd 3.5 turbo because even comparing some of the open source model this one still gives me better performance and it needs to be a chat based model so that's why I'm sticking to a triple five turbo now the two different techniques the first one is assuming real basis like giving it a row right and then the next one is doing this step by step so I'm going to show you both so again let's let's talk about the 10 different Linux commands again to use the same idea but this time how we do this um in the context of a chat based model so unless a guidance right not gonna have anything in here yet but let's just call the function so result equals to program and that's just gonna call it right and again we have the OS equals to Linux so let's keep to that I mean you can change this to Mac OS I don't use Mac OS anymore I use Linux so I'm going to stick to Linux then I would like to just print the result to see what I get from here okay now let's focus on the guidance now you can specify rows because they're also available in guidance so I can say something like system this is where I set up the context so I can say something like you are a CS Professor computer science professor and teaching Unix systems Administration to your students actually not Unix but it should be OS telling GPT is like hey you are a CS Professor teaching Mac systems Administration to your windows no one actually do system email Max you either do it on Windows or Linux mostly it's on Linux and then I want to close that out so system so you can see the pattern here is generally very consistent right you start that off and then you close it and then you can now say okay now this is the user's input itself so the user who is chatting with you the GPT right that is the use itself so this is uh you know where you write your queries and stuff so in this case here the user is going to ask what are some of the most common commands use in again I don't want to hard code it so again we can say OS templating provide a one-liner description you could also maybe say list the commands and descriptions are one per line number then starting from one so instead of asking you to start from zero or instead of using that that uh you know this notation to you can you can use this as well you can say index and then plus one just sort of have one each but here I'm gonna try and hope that gbd 3.5 um which is chat GPT is going to be smart enough to not have me to do that but just gonna give me the one base indexing now I want to mention one thing about the Toyota sign I can also add clear design here and what does this mean so this is the white space control operator right so again it's it's all part of the handlebar syntax if you use game gel or anything like that you've probably seen this before so this allows you to remove the white space between the block itself so the tilde sign here says that remove the white space either before or after attack so I can edit here I can edit here as well right so there's going to be any white space um would be removed making the program looks actually a little bit prettier without including too many excessive white sprays in the prom okay so we explain the system we explain the user but what about the GPT itself now that is known as the assistant so we're gonna call it assistant I'm gonna remove the white space and here I just want it to generate I just say just January commands man right just general commands and maybe the max token should be 100 or maybe 60 or 80 just control it um so here are the system here is a user and here's the assistant that I want because this is a step-by-step process I'm showing you two techniques at the same time I'm showing you the row I'm showing you the step by step as well you can either include your chain of thoughts processor here TLT CO2 you can include those things you can also just maybe just continue asking prompting um so for example I can ask a follow-up question now I can say and take the user I could say wait look at all of these commands there which among these commands are beginners mostly to get wrong explain why command mic be confusing and then ask assistant to do the same thing so just repeat that again um but here you want to give a different name I want to say this is confusing commands and maybe this one should be a little bit lower 60. now if you can also specify a separate llm so there are two llms in play here so the first one is the guidance itself it needs an llm right to sort of use that to power this whole program but then there is also a second LM that you can control in here you can say well I want LM to be whatever you have here so you can you can have the whole thing being Transformers but this itself you know you want this to run inside open AI gpd3 for example all right so if you want to do that you can say check and then you specify your model this could just copy from here or you know copy from here just to keep things simple and keep the video short I'm just going to try and copy this one here but you could try different different models in here I'm just going to keep it to the same thing I'll just type this and then just have all of this into chat this also works right without running too much things let's just run all of this and see how it works all right so let's just go and say python now this is demo 13 now so that demo 13. py and let's run that we accidentally move out one of the user so let's bring respect so one of the things is that when you try to close this up really fast on the go sometimes you tend to make one of those two small mistakes fix that but the the error is pretty helpful perhaps the block command was not correctly closed so you found out that I did not really close it properly so it gave me that so the assistant starts the system you all see as Professor teaching Linux system so I could actually hide all of this I could say this is hidden it was because I don't really need to see all that but it's here already so what are some of the most common commands used in a Linux operating system and then here's the assistant and then it prints that out um give you explanations to each one of them and also notice that I said number them starting from one and it did that job correctly and then I have um user and then I have the assistant so user which among these commands are beginners most likely to get wrong so I'm asking you to look at all the top the top 10 in here you could you know maybe have something like specify a unique you know just like what you did there with the set list set um just like what you did here with the reset you could try to make sure that these are none of them are repeating in this case it's just a quick scan shows you that none of them are repeating but if you use a older model you may see a lot of repeating you know if you use gbd2 or jvd3 you may expect some of the repeating in here I don't know but in that case you could just specify a function that goes like unique you know just like choose unit or something and then you could just use it in your program but in this case it does okay and then you follow up the question with which among these commands are beginners most likely to get wrong explain why the command might be confusing show example code to illustrate your point okay so beginners are most likely to get the RM command wrong it is this is because the iron command is used to remove files or directories and it does not have an undo option if a user accidentally remove or deletes a file or directory using the RM command it cannot be easily recovered so this is kind of a step-by-step process instead of finding out this one directly I'm asking you to generate 10 first and then from there pick up like two or three that are wrong and then from there you know generate a follow-up question so you can do that in this kind of interface as well and all of this can still be within a template that you need to fit right so earlier we see you can even have this as a Json you can try XML you can try to export to CSV but you can now sort of provide a structure make it possible for you to interleave your generation your prompting and your logical control into a single Continuous Flow it gives you some examples here so give you an example this one uses text wency it says tweak the following probe to apply to model instructions and you give it a proverb um then you know here is the templating this is something that we've seen before select is something you've seen before as well um here is the select chapter one or two small issues so if you look at this for example I opened an issue some time ago or three days ago I said that the select example doesn't really work like that so if you put a select right now and you say select 9 or 10 or 11 ask it to pick from one of this value it doesn't really work it would just end up with something very awkward like this like 9 10 11 12 and then um 10 is selected so 19 11 12 10. this is not what you want so the workaround is to specify them as a separate option and then specify them into the options itself um so that's something that I showed you there so if you follow this video this will work but if unless this gets touched but right now there is no one who attend to the issue right now so some of the documentation if you were to follow it strictly it will not work directly so you will need to do some work around so for example this one that shows you the 10 or this doesn't really work and it's already an issue I created on GitHub then there's a chat dialogue this is the system this is the user and Anderson this is these are things that you've seen before then I mentioned as well that you can use one of the you can use any kind of uh model here so you don't have to use gpk if you don't want to you could use llama 7 billion as well so you set up Transformers that I've shown you that as well and then you generate a Json and then ask this guy to make a character so this is a character profile and you can then specify all of them and then you just have to call correct the maker and it will always give you this valid Json and that's great you know you can you can always have this kind of assurance that what you get here is at least gonna be at least you know at the very least it's going to be some sort of semantic Json maybe some of the values are not correct you know maybe the select Weapon It's not really not what you want but at least you know it's valid Json because you have some sort of structure around that using guidance and so what it ends up doing is it says oh here is what you want and a mantra I will protect the week so there's a lot of things here it's a pretty exciting project that you see nine hours ago there's a new merge pull request so yeah it's a pretty exciting project and so I would say if you want to spend time learning problem engineering don't drop any money onto any of this like oh you know also to write better prompts you know teaching how to write better problems on which really goes down to writing better English you know be more accurate in your descriptions be more detailed those are those are a bit um you know a BPS no no offense to those people selling those courses but you know I just I can't respect that kind of hustle what I encourage you to do instead is to pick up guidance um and use this as a starting template and just go and really learn about that and really try to think about how you would do prom design from a software perspective and the opportunities that comes with that so that's all I have time for today um hope you learned something new I'm going to see you in the next video bye

Channel: Samuel Chan

Views: 7,546

Keywords:

Id: k4Ejc3bLQiU

Length: 43min 15sec (2595 seconds)

Published: Fri Jun 23 2023