Self-Correcting AI Coding Agent + Prompting Deep Dive

today we are going to take a look at an AI coding agent that writes the code installs the needed packages executes the code adds new features and of course try to self-correct any errors preventing the code from running it's going to be a deep dive into how I prompted this system to work and of course we're going to do some testing so let's just get started here you can see a very simplified version of how this works so basically the user has to set a goal what kind of app it wants right and the coding agent is going to write that app it's going to try to execute this code in the ter and if the code uh is doing some kind of error we're going to send that code over to the self correcting uh part we have and it's going to try to take that error message and self gret send it back to the execute code command and try again and if it works we're going to run the code of course and after that we're going to take that code and try to add some new features and we're going to try to execute it again of course if we get an error again we're going to send it back here try to correct cor it and retry the corrected improved code uh if there's no errors we're just going to execute the improved code so that is very simplified how this works we're going to dive deeper into the code uh at the end here so I think we just going to head over to the python code now and start diving into kind of how I prompted this system before we do some testing okay so this part of the video is going to be a bit technical because I'm going to go slowly through how I kind of thought about prompting this system because uh I needed to adjust a lot of prompts here to make it actually work the way I wanted to so if you're not so interested in learning how to prompt this kind of systems you can just skip ahead if you just want to watch how this system performs when I run it uh so I will leave like a time stamp if you want to skip this part uh but let's get started here so you can see we're just going to go through the main function and when we have some kind of function here we're going to go up and try to look at them so like I said we're going to set a goal so I have a bunch of different goals here we're going to test so let's just look at an example here so we set the goal to display the stock price of Apple using a free API so that's is basically a simple goal we give right and we have two different system messages but this is because we need one function to your task is to remove all text that is not needed to run the python code because when we will try to execute the code we don't want any uh other strings or integers involved in theod Cod right we only want the part uh that is executable right and the the first system message is you're an expert uh software developer with expertise in Python your task is to create Creative Solutions so that is kind of the two system messages we will be using and if we start on the top here as you can see we have this initial code so this is the first code we are going to generate so uh we using an F string here so you're a super creative expert software Dev with expertise in Python uh I'm going to have to scroll over here so write a python code two and then we have uh this goal here set in like these brackets and the goal of course is the one we set here right uh and we also want to set our system message here so let me just fix that so we just going to do like this right okay and then we have the uh we pass on the system message so this is like the the first system message here right so not the special one and the next part here is going to be to clean up the code a bit so we're going to use this clean self correct function and we're going to feed in the code we wrote here right so if you see here here is kind of the function we have up here so the clean self correct function again we use F string so we're just going to pass in the code we just created from the initial code here right and your task is to remove all text that is not needed to run the python code so this is basically exactly the same as we put in our system message so for this function we are using the system message to right and that was kind of our special system message here right so I struggle a little bit with that but after I put in these two I put the system message to remove all text and I put it this self-corrected prompt here and clean self correct code prompt um uh it works pretty good to be honest and the next part again is going to be to clean up the code a bit more uh so we're just going to run this function called clean code this is just going to remove some we always get this uh python with this um yeah just some additional strings here and we just going to remove those and replace it with just a white space right uh if we take a look at the code here usually we can get this uh this python stuff on top here right so we just going to remove those so yeah that has been working pretty good okay so next then it's just going to be to print the code and then we're going to use use the find and install dependencies function so we're going to pass in the code we just created here and it's going to look for those import statements and actually try to install with a pip install every single uh dependencies we need to actually uh run this code and then we just going to save it to our code path and yeah then we're going to run this execute code with self correction if you take a look at that you can see that it is using this execute code function we have up here to actually try to run the code and if success we're just going to return it through and if we have some issue with the code then we're going to go into this part here that is uh going to try to self correct the code so the first part we have this self correct uh function here so this takes our code and it's going to take the error message too if you go down and take a look at this function here so this is the self correct function and we just going to do an F string again so the following python code has an error and then we uh pass in our code here and we're going to pass in the error message that actually comes from the terminal the syntax errors and please fix the error and provide the corrected code so that is kind of how we try to correct any error messages we get when we run the code right and then it's just going to continue up here in execute the code with self correction function so we just going to clean it again remove those uh additional strings it's going to print it it's going to save it and it's going to try to run it again so you can see we have some kind of Max attempts here so we can set we can pass in the file path and the max attempts so if we go down here now you can see I just set the max attempts to five so we're going to type five times uh to self correct before we just give ight and that was just the first part so this was to write the code and execute it and self correct if we have any errors and then we kind of come into our Loop here so this is going to be kind of where we add some new features so we just going to do an F string again so the current python code is the current code right and the current code is just uh going to open the file we have saved into our code path and we're going to brainstorm three new cool features and here you can kind of put down whatever you want so I just put down like colors UI animations styling improvements to add to the python code so that was kind of the prompt I used here and we're going to print the new feat we want to implement so we're just going to do an Fen again and just paste in the new features from the what open AI or our llm returns here and then we're going to run a variable called updated code so this is just going to pass an F string again to open AI uh so it's just going to be current python code we're going to feed in the code here and it's going to do features to add so it's going to list up the features we want to add and add the new cool features to the python code and write a new Full updated code and then we kind of go into the same Loop right we're going to remove all the text we're going to clean it we're going to print it we're going to install all dependencies and yeah we are back into kind of the loop where we try to run it again so I know this was quite a lot but uh uh I spent some time trying to prompt this correctly I'm was very happy how it turned out actually uh I initially had this um the other part of this system where I used a screenshot of the app running and try to feedback some information from the the vision model that we used um I kind of took it out because it didn't work so good I might Implement that uh later right uh but I hope this wasn't too bad to follow along uh if if it was too bad maybe I can do some separate video If people want to know actually more about this but uh yeah that is how this works it's not that complicated the only thing I had to work on was of course try to figure out what kind of prompts I should use here uh but yeah like I said very happy how it turned out so I just think we're going to do some testing now and actually see how this works in practice right but first let me just quickly plug my membership so if you join as a member here on YouTube you will get access to the GitHub Community where I upload all my codes you will also get access to the community GitHub and if you join today if you decide to become a member you can actually join the giveaway we are running out where I give away some Nvidia merch so we have this t-shirt here we have some stickers for your laptop we have this very cool coffee mug here and we also have this socks so if you become a member today you can actually join the giveaway it's too not too late yet so yeah see you there hopefully but now let's actually do some testing here so I just want to show you quickly like this is uh a Powershell terminal I created with this system so it kind of shows uh the Bitcoin price so it updates every 30 seconds it show kind of I have like2 bitco coin so it shows kind of what my value is in knock and it shows if I have lost or gained any money over the last 24 hours that was a pretty simple Powershell app I built using this system so I thought thought it was pretty cool so I thought the first goal we're going to set is just to create a simple snate game so we're going to start off using anthropics clae tree uh Opus API because it's really good for coding I've been super happy using it so I hope we can create a pretty cool snake game using this API so yeah I think we just got to head over to the terminal here and yeah let's fire it up okay so this is going to be the first iteration so yeah let's see how good this works now so yeah seems to be working pretty Okay we okay so I crashed let's try to play again yeah the score is working the snake is growing so pretty good okay so let's just crash that let's quit and see what kind of um improvements it wants to suggest here so it wants to add some colors to the snake and targets to kind of change up the colors I guess and it wants to add some levels and increasing difficulty so let's look out for that in the next iteration okay so let's see now so remember we are looking for some was it colors okay so we have an error here so unone local error cannot access a valuable snake speed so we're going to test uh the self correct functional to see if it actually works that's good by adding the Line Global snakes at the beginning of the Run game you should make it veryable accessible now the code should run without the Unbound local error okay so let's see if it correct itself here yeah okay so I'm very happy I have that for now I kind of got to show how the self correcting works here so let's try it now you can see we have a level we have some snake colors here that's a bit strange but pretty cool so let's see if we can get up in level here so that we have score two look at the snake so how do we get higher level okay so I crash let's try again okay so yeah I got to level two that means uh it's sped up so I guess both the colors and the levels are working so pretty good I want to try to quit one more time and see if we get any more improvements to our code here okay so it looks like we have some kind of API connection timeout here or something I didn't set up any try function so that was a bit stupid uh but I think we got to call it that for the snake game and move on to our next Golden and that is going to be to display the Bitcoin price graph using a free API for coind desk I also changed uh it to CH chat GPT 3.5 turbo so we can get some more speed on this uh it's much quicker but I think it's working pretty good okay good so let's head over to the terminal let's clear this and yeah let's run it okay so this is kind of our first version so here you can see the Bitcoin price graph uh maybe it's a bit small for you let me try to yeah we can do that and here you can kind of see the price on the left here and we have the date so yeah it's been turning quite High lately so yeah that was kind of the first iterations let's just close that and see if we can get some yeah uh Implement some improvements okay so here you can see the improvements is going to try to add a grid and style it's going to include moving average line and it's going to add an interactive zooming and pen okay I don't know what that is but yeah let's see if it works okay so you can see we got something a bit different here so if we Zoom this a bit out right so you can kind of see uh we have the 7 Day moving average so we have kind of have the price in yellow here and the red part is the 7 Day moving average okay so that was a bit different so let's close it again and see if it changes okay so here we can kind of see we got an error here so two tool tip set text F so and tered string literal so detect that line 39 so see if we can try to self correct this and run it again so this change made the hover function with the correct the code should now run without syntax error so let's see if that's correct and yeah you can see that time the self correct uh function worked pretty good so we got the code running now uh I don't see a lot of changes here actually but uh yeah uh at least the self correct function worked so I was pretty happy about that so yeah I think we just going to close this now and try to set a a more different type of goal right okay so now I wanted to do something in another language instead of doing python let's try to do something in go so try to create an advanced CLI password generator app uh using go so I just changed up everything we had from python to go right so yeah I'm just going to try to run this now and see if we can get some cool apps in another language I found out that I had to change up these install packages a bit because we have to to install go packages right so I just changed up that and now I think we are fine so let's head over to the terminal and run this now and see if we can get our app so yeah you can actually see the flag installed successfully mat R installed successfully so it did work so yeah we got our password here now we're going to try to H Implement some colors and some Progressive password generation whatever that is okay yeah so we generate a new password uh so it's running pretty fast here so callor for Progress bar animated password strength meter okay so it went ahead yeah that was pretty much the same wasn't it let's run it one more time okay that changed up a bit but uh yeah let's try another project and go and see if we can do something completely different let's try to create a web CL that scrapes the H to headlines from the URL the verge.com so let's see if we can do that okay so we're downloading some packages here and yeah that was a lot of installs but okay okay we got an error so see if we can correct it uh nope let's try again yeah it worked in the end I didn't expect that so let's take a look here if we okay let's let's run it a bit more and see what happens okay so that changed the color okay so we got an error trying to fix it uh okay uh I think I'm going to stop it here and let's see if we this is actually the headlines from the word.com okay so the headlines we have the perfect music streaming app does not exist witching scan to watch Sam al man reloads open AI so let's see here uh yeah the perfect music streaming app does not exist witching to scan Watch Sam ulman rejoins so open so yeah that worked uh it's not in a good structure but we got exactly what we asked for so yeah I guess it was completed so I think the last run there really showed off how good this self-corrected code function kind of worked we tried four attempts and it got it on the four attempt fourth attempt so I was pretty happy with that so yeah H hope you enjoyed this I know was a bit heavy kind of on the prompting side but I kind of wanted to take a video and kind of go slowly through how I think about prompting these systems uh so like I said in the intro if you want to access this code yourself just follow the link in the description become a member of the channel and you will get access to all of this and the community Discord So yeah thank you for tuning in hope you enjoyed it and I see you again on Wednesday

Channel: All About AI

Views: 13,195

Keywords: ai, ai agent, coding agent, ai coding agent, ai engineer, ai engineering, prompt engineeing, claude 3 opus api, claude 3, python, software engineering, go, golang

Id: h38wONkdZfk

Length: 17min 44sec (1064 seconds)

Published: Sun Mar 10 2024