GPT Engineer: Can AI Really Code a Complete Codebase?

specify what you want to build the AI asks for clarification and then builds it that's the philosophy behind GPT engineer it's able to create an entire code base from just a single prompt and it asks clarifying questions about that prompt in order to be able to determine what to build in this video I'll show you how to install it we'll build custom application and then we'll move on to what I find most interesting which is understanding how all the code behind it actually works so with all that said let's get on with things so this is GPT engineer by Anton Seeker over the last month it's had a huge amount of stars on GitHub and a huge amount of interest and the important thing here is it says it can create an entire code base but the way it does it is really interesting the fact it asks for clarification about what you're going to build so it's not a single prompt that you're giving it you're giving it a number of clarifying points so when it was first released the only way you could install this was by downloading it and cloning the repo they've actually got a Pi Pi package now so as this is installable with Pip and it's able to get from pipeline I want to keep my independency separate so I'm going to keep this in a virtual environment so we do python minus m VM for the VM module and then the name of the VM so I'm going to call that vmf which is slightly confusing but there we go um we can see that we've created a new virtual environment there and the directory that I'm working in and then I'm just giving you activates that um virtual environment so we do Source VM bin activate and then we can just do pip install GPT engineer and if we actually take a quick look at this we can see the dependencies on this aren't that great open AI is the main one here with typo being used for the command line everything else is fairly small kind of tools there so that's installed fine the next thing you're going to want to do is get an open AI API key where you do that is head over to platform.openai.com and in this drop down over here you can see your API Keys head in there and generate a new one for this particular project and then export it onto ground line so I've got that and I can just do export open AI API key like that and then paste it in I will be getting rid of that after this video is published so you won't be able to use the key you're thinking that way so next we need to create a folder that mimics this kind of projects example structure that has a prompt within it so I'm just going to create a Project's exam folder and then within that created this prompt file and a lot of these videos that are showing you how to install and use it will just go straight for this prompts as a demo what I might do is tweak this slightly so I've gone back to open Ai and chat GPT and so give me a project specification for space mode is game written python it should be specific enough to enable an AI to create it see if so and this is 3.5 okay let's take that and I'm going to copy that entire prompt into this font here so now I have this prompt which gives a long specification about how Space Invaders should work and we're just gonna say gpg GPT engineer projects example and see what it does see if it needs any more clarifying points from that what is the exact size of the game board how the cells certainly contain okay so I've answered a bunch of questions but actually got to the point where some of the questions were so specific that I didn't know the answers to them so I've just gone continue you answer them to uh to GPT engineer and we can see that it's creating creating the files now with it looks like using pi game and one of the things I think is interesting about the way these are created is depending on the complexity of the project means you're going to be charged more so the more files you've got to generate the more queries that will go on um and hence the more you're gonna end up paying on open AI calls iron at the moment able to use gpd4 API and therefore it's going to be costing me that little bit more because it um GPT engineer defaults to using gpt4 over 3.5 and fall back to 3.5 if you don't have access to gp4 so you don't need to worry about that now it's finished and it's asking me do I want to execute this code pip install the requirements and then python main so let's see let's say yes and actually see what happens all right so because I answered yes and I but I didn't press enter it didn't execute the code that's possibly a little bit um they probably should accept yes as well but anyway and it's asking me if it ran at all but I've not actually run it so so that has not gone well it's basically created a file with Main and a game and all the models that it's using for the game it hasn't created and I had a quick look over on my usage and the only thing I've done today on open AI is create this so all these clarifying questions and everything that's cost me 40 cents so that's obviously using gpt4 um and that bit more expensive but still for something that fails it could end up getting quite costly if you are going to do that a number of times anyway what I might do is change this prompt to something a little simpler so let's completely delete this and start over okay so I'm going to go with an API that serves dad jokes that's probably fairly hopefully fairly simple in comparison let's see where we get with that so it's asking me some clarifying questions so I'm going to say it's going to be stored in a text file let's say it just returns it as text I put test hopefully it can deal with that we only need to handle a small number of requests and it's just responding with a dad joke and that is coming from a text file so hopefully it's able to cope with that okay so this time it says if yes press enter otherwise type no let's just press enter okay so we have an app running let's have a look in the workspace see what that is like so you need a jokes okay so this is dead simple in looks like it requires a jokes or text file so let's create that okay Stars chat GPT for some dad jokes I'm a dad so I'm allowed to do this yeah so we can see from The Joker pie that is um returning from the each line a particular joke and then it's just stripping it and returning it anyway so I think what we might have to do is restart this oh that's interesting so if you immediately just go ahead and ask it to install it puts it straight into the virtual environment that you've already got so we've got all the requirements for GPT engineer but it's also bunged in within that because we've got an activated version environment it's also popped in our requirements in it so probably not recommended to go ahead and try and execute it straight off the bat because it just dumps it straight into that version environment as well which is not ideal okay so we need to go to joke joke okay cool so that is that is working which is good I won't actually dip back to the usage and have a look at that again and see how much we've used over and above so we use 39p on that Duff um call out to create the space inverters game let's see how much we're up to now so that is another 30 cents wow it's pretty expensive isn't it to do these things and if you're using gpd4 I recommend that you um configure it so that you're using GPD 3.5 while you're trying things out because it's a lot cheaper so yeah best part of a dollar on um testing there I think it probably typed more in clarifying questions and clarifying answers than I did then there's available in the code so you see we're only got like 16 lines of code there anyway it's still fairly impressive I think I wanted to move on to and talk about as the main part of this video is actually how it does this because I think the code is really interesting so let's clone this locally and take a look at it now I think this is fairly interesting because I've had a brief look at this and there's a few things that I wanted to comment on so the structure of this most of the code is within this gpde engineer folder if you take a look at this we have ai we have chat files we have collects DB domain learning Main and steps this first file we're going to take a look at is basically the structure of a conversation that is between GPD engineer and open AI so you can see we've got a bunch of convenience methods there for structuring the data format of the data that's going back chatter files file that we've got so this seems to be taking a conversation and figuring out what files need to be generated from it this collect file is basically some analytics by lots of things that um they're you I think this is probably what they're sending back as part of the learning stuff afterwards so it's asking did this work or whatever and this is a key value store so we have a bunch this is data so this called database and it's a key Value Store notice all these files are really small they're not very big um and so we have there's a number of different um databases there so there's some for memory logs pre-pumps input and workspace um domain I guess this is probably the domain that we're working in but I'm not sure just single line there so yeah this is learning this is uh again the clarifying questions at the ends so in this file you can see this is the main file so this is where we actually where it's actually constructing the kind of main commands that you're able to use on the command line you can see it's using typer here for this command line arguments and setting everything up make things simple um we can specify the model for a model in there so there's probably something that you want to do if you want to use 3.5 and perhaps get some less and less good code but cheaper um certainly the experience I've had here is that it can get expensive fast yeah so this is literally just running through all the thick bits of code that we've just seen and setting everything up and using it and then I think the main this is the main kind of steps all the different steps and I think this is customizable but it looks of things if you look in the repo so you can go back and decide which of these steps and how you want them to run and in fact actually they're all at the bottom here so you can actually so you've got customizable steps here and what depending on what config you're using and you can decide how that's configured so you could actually add your own ones here yeah so I looked at this and I actually totted up so you can see we've got 67 lines there like I said these are all small files 40 lines 40 lines 56 what struck me about this is the the complexity doesn't seem to be in the the actual kind of structures and how it works it's it's all fairly easy to understand how it's doing this it's actually in these pre-prompts so the prompts in order to get open AI to do what it wants to do you can see there's uh one two three four five six seven eight nine files here so with the code I tied it up and it was about 745 lines in total so it's barely anything really in the grand scheme of things here you can see these prompts some of them are quite complicated and you can see where it's generating um tests and you getting feedback it's just really interesting that actually the majority of the complexity seems to be in these prompts so going from figuring out how to write something in code to how to prompt open AI in order to generate code for you and as we've seen that code may not work where in the case of say the joke API it would have been far quicker for me to go off and probably look at the documentation and figure out how to read a file and do it that way because it was 10 lines of code or whatever and instead what we've actually done is we've spent a little bit of money and in order to get a code that is works but doesn't have any documentation and is a slightly buggy so the developer doesn't necessarily understand how it works so yeah let me know if you've had more success with tools like this um I'd love to hear your feedback consider subscribing to the channel like the video all that sort of social stuff and I'll speak to you soon new video all right bye for now bye

Channel: Ian Wootten

Views: 13,856

Keywords:

Id: bBFiQclvHns

Length: 15min 18sec (918 seconds)

Published: Fri Jun 30 2023