Claude 3 Opus vs GPT-4 for Coding - My Experience

so with Cloud 3 worth it for programming well let me tell you my experience I've been using Cloud 3 upus for the past two days or so and so far my experience has been really good now I've been using GPT 4 for like the past 5 months every day so I would say I'm a good candidate to compare it with Cloud 3 ERS I'm not going to run any benchmarks or do any type of analysis I'm just basing it off my experience and how it has worked for me for the past last couple days so I'm using this model through their official application which is cloud. now you have to pay 20 Buck to get access to the upus model but it is way cheaper than using the API directly which is the reason why I'm ultimately using it via this interface so now let's take a look at this first example here I'm asking it to refactor some typescript code to instead of throwing errors use a discriminated Union instead so here I say let's change this with returns a discrim minated Union with success true success false if it is Success then success true and then file to get URI string and if it's not success return the error which is a string and then I passed in the code Let's ignore the lack of formatting and now let's take a look at its answer here is the modified code that returns a discriminated Union with success true or success false and the corresponding file to get URI or error message and then as we can see it extracted the to type aliases for the union and it correctly specify the discriminator which is the success property and then it performed the actual Union upon result and then uses that to specify the return type to be a promise of result so so far this is great this is what I would do and well pretty much anyone now if I keep scrolling down as we can see it is using the discriminated Union so success true and then it passes the property in the case of false then it passes the error and the if I keep scrolling down same with this one same with this one same here and then finally within this catch block and if I scroll down it gives us the step by step of the changes it did so it creates the success result the error result and then the result and then it tells you that it changed the return type of the function and then instead of returning undefined the function now returns an error result so this is great well as we can see it is using the union everywhere and then here for the case where it is valid and this way the color of the function can handle the result based on the success property and access either the file to Get Ur or the error message accordingly now let's compare it to GPT 4 now here I'm using the API which is actually via cursor Dosh which is a fork of vs code which simply integrates GPT within the IDE so this is obviously better than using say the official chat GPT since you're using the API directly so there are less restrictions so the same prompt and as we can see it didn't separate them into different typ aliases it is all within one so bonus points to Cloud three and the reason is because this is not really readable for example what if the discriminator switches upon more than just two states then it becomes a huge mess and likewise with the number of properties what if you have more Properties Plus at a glance it is not really telling you what is the success and what is the false sure you can check and see okay success is true so we get access to this one else we get access to the error but with the other approach suggested by Cloud as we can see all we need to do is look at these type Alas and we immediately know what corresponds to what so that's why I would give a bonus point to Cloud now I'm not saying GPT 4 is in the wrong since it works but I would prefer the other approach by Cloud now let's see if it is being used correctly so return success true then success false then false here false here true here and oh no we have a throw error consider handling this more gracefully now if we compare this to Cloud as we can see it is returning so we replace the error we were throwing so again bonus points to Cloud three the reason why we're using discriminated unions in the first place is to avoid throwing errors that's the whole point and GPT outright es skied that logic here so it really does make sense so yes Cloud's answer is way better and that's pretty much it for this example now let's move on to the second section just to make sure that it retains memory so context so here we say perfect that function is going to be consumed in this one so I changed the name to download or get a file from get a file or download how about we refactor this hook and we do and then we have first of all rename it second of all do not misuse use effect so I gave it some extra parameters and then three get rid of the get local file function since it is redundant and download file already incompasses that logic and for use typescript so this is a test case to see if it can do these three suggestions so if it keeps the context of the prompt and also if it can convert JavaScript to use typescript annotations so here I passed in the file which is pretty short it doesn't have a lot of logic as you can see and as we can see we have here's the refactored hook and component using typescript so it uses the arguments and then the result now here is one mistake it is not being consistent with the type names so as we can see this is use file item and use file item result which does not correspond one to one with the name of this function so this should be used downloadable file result and the same with arguments but okay you can easily refactor that so let's not be that pedantic and then here it defines everything it uses the result property so result. success else this and then it refractor the other code and converted it to typescript so it is using the props so as we can see this is nice it is using the same name and it added the props postfix and then it uses consumes the hook which is the one that it just refactored and then it has all of this logic and that's pretty much it and then it has all of the things that it has done so one hook has been renamed from this to this to instead of use effect the download file function is now invoked imperatively when needed it is defined using use callback to memorize the function and avoid unnecessary Recreations so this is quite important because as we can see it is now also thinking about performance then the get local file function has been removed since its logic is already Inc compassed the code has been converted to typescript the download file function now uses the download or get file function and then the message prop in use downloadable file is now optional indicated by the question mark in message so now in my opinion it did accomplish the task I told it to do these four refactorings it wasn't that much and it successfully gathered the hook that we were using and it also took this file that I passed in here and it changed it from JavaScript to typescript successfully so now let's keep scrolling down so here we have how about we add an open file method to the use downloadable file and I pass in all of the code so you pass in the open file function that I wrote and then I pass in the refactored code so for the use downloadable file hook and then here as we can see it added it here so use call back and notice how it is keeping the react. use callback so here before that we hadn't used any react hook only for this case like the US state so so it kept that convention but once I showed it this file that uses react so the object instead of the structuring the properties and the methods well it cut and that and as we can see it maintains the exact same convention so as far as keeping the code uniform and using the conventions and whatnot that's great for cloud then I passed in another file I told it to change it from this weird mapping Factory component to an actual Factory component with a switch statement and as we can see it uses the switch file type and now here's the interesting part notice how here there are no types so this is all in JavaScript but if I scroll down it added this type alas file item props and then file file metadata and it imported that from this repository as same as this message item so we maintained the context for when I showed it this information so we got hey we're using the file metadata and we're using the message type and so it is now consuming it correctly in this type alas which is absolutely great so as far as context goes it seems to be doing just fine obviously this is a short conversation we would have to see how it does with maybe 20 messages but for the amount of information here I would say does a really good job now I have another test case here and this is mostly to see how cloud approaches a problem now the previous one was mostly refactoring and going back and forth but in this one I'm going to tell you to try to come up with a better solution so I passed in this code and then I told it as you can see I have not refactored the use effect yet now in react you should use use effect sparingly since it is not meant to be a signal Watcher but rather a way to synchronize your R state with external State however since we have full control of the state in this case it is a terrible practice to use a use effect here now I'm not entirely sure what the behavior of this component is exactly but by the looks of it if there is no source that means that the file isn't downloaded on the user's device so what it effectively does is if it's not downloaded download it right away or so it Sims this is the hook and then I passed all of the code and then this is the download orat file. Cs and then what do you suggest with do in this case maybe pass an argument to the use downloadable file like download onload Boolean or something of the like maybe you can come up with a better name that signifies its exact behavior and make sure that it only downloads it if it doesn't exist in the user's device and then as we can see first update the used on loadable file hook to accept a new argument let's call it Auto download so it did come up with a good name and then it used a use effect while I told it that it is an anti pattern and you shouldn't use a use effect for this so one last point to Cloud 3 now let's compare it to GPT 4 so I passed in the exact same prompt and we got this answer so I suggest naming the parameter something descriptive like Auto download if missing now I prefer this name over Cloud's name which is Auto download and the reason is because it does tell you exactly the behavior of this property so as for naming conventions bonus points to GPT 4 in this case and then it gave us the code but again it is using a use effect and well the solution is pretty much the same if I come down here it is invoking the download file function which we already defined here where we have download file so as we can see GPT 4 repeats itself Cloud simply used and referenced this function instead instead of basically copying this logic and pasting it here so bonus points to Cloud even though Cloud got the naming convention wrong it wrote cleaner code in this case now another thing we can notice right off the bat is that GPT 4 gave us the whole code meanwhile Claud just gave us the IR relevant parts so in my opinion in this particular case Claud wins and the reason is because why do we need to see the whole code I just need to see what the changes are in this case just the arguments then here which makes sense since it is using ARX and then here for the use effect meanwhile pt4 went ahead and rote everything from the very beginning so it is slower CL gave us the answer faster because well heed all of these unnecessary code repetition so bonus points to Cloud but obviously in some cases it is better for it to write the whole code since it already exists CL did a better job if the code didn't exist then obviously we need the whole code now as for the second part updated usage in image file component and here as we can see used downloadable file and passed it as through which is basically the same as Cloud did here so nothing new but again notice how CL gave us the relevant part it gave us the component which was the only thing in the code affected by the changes meanwhile GPT 4 went ahead and again typed everything out from the beginning so cloud is still faster while preserving the relevant parts from The Prompt now I told it I like it but again I'm not a fan of the use effect so here we tried changing it up and instead did it within the flow of the hook so if Auto download and not Source then download file and again I'm not a fan of this approach why because well every time the state changes or the consumer changes it is going to pretty much rerun this logic sure if there's a source then this wouldn't result in like an infinite Loop but it's better to be safe so now I did the same test with the GPT 4 and then it said modified use downloadable hook without use effect and as we can see there is no use effect but there is no logic for downloading the file if it doesn't exist in the device but if I come here to the second part it move the logic down to the image file so this is worse than Cloud at least Cloud kept it within this context of the hook and not within the context of this function which is better again because we're passing a par parameter to the hook which was Auto download if it doesn't exist and as we can see just got rid of it alog together so even though both got the implementation details wrong Cloud's answer is way better just for the fact that it tried giving another solution within the hook and not moving it outside which doesn't make sense since I never asked for that since the beginning so now we continue with the conversation so notice how we have this piece of logic I passed in the download or at file function which is the one with the actual logic and then here in use downloadable file which was the hook we were just talking about and then I said the source would be set to the URI of the stored file if it exists right so effectively with the current logic since source is initialized to null at first it would always download the file and then I told it I guess this state should be initialized with the data we get from the file system to make sure it exists if that makes sense and if it doesn't then Run download file so no need to have this dangling if statement within the hook that would go in the initializer so I had to give CLA some hints telling it hey we should be using this within the initializer and then I also told it about the behavior we want which is it should Auto download the file if it doesn't exist in the user device and then it said you're right the current logic initializes source to null which means it will always trigger the file download even if the file already exists on the user device to fix this we can modify the used downloadable file hook to check if the file exists on the user's device before initializing The Source state if the file exists we can set the source to the file's URI otherwise we can set it to null and Trigger the download if Auto download is true and as we can see it uses this use effect and a synchronous function and it pretty much copied over the logic from the function I passed it so from this one and so now at the very least the hook has the behavior we want so then I pass the prompt to GPT 4 to see if it now works and as we can see it told us yes we're going to change this to the hook but notice how there is no Auto download in the arguments so we just skip that all together and then here it has a function check file exists and then it does all of the check and then I use effect that executes this function whenever check file exist but it's within a use callback so it doesn't matter we're not going to run into an infinite Loop but then notice how it performed another change only attempt to download if source is not already set and this is not the desired behavior and why because what if the user wants to redownload the file for whatever reason then this is going to prevent them from doing so without this check the device is going to ask the user you already have the file are you sure you want to download it again that's up to the user at that point so GPT 4 produced another side effect that was not intended so as we can see from the code from clut this is way better he didn't output any side effect as we can see unchanged and this is pretty much the same all it did was create this use effect and as we can see it is using the Auto download argument so if it exists then set the source to the URI else if Auto download and then it triggers the download file function which again is not being done here it is just checking if it exists and that's pretty much it it is not downloading it because again it got rid of the Auto download argument altogether now I did write more messages with Cloud to refactor the code to create a reusable function for this to make the code cleaner and whatnot but with this we can get the gist of it now the final verdict what is my conclusion based off my usage and these two last days with Claud and basically 5 months or so with gp4 now personally I would say Claud is better than GPT 44 code I have to use it more I guess it's too soon to say that yet only time will tell but from what I've done with Cloud compared to GPT 4 I'm definitely going to be using Cloud more however gp4 has some things that are better than cloud and they can complement each other really well for example the naming convention that was done better by GPT 4 sure it is something quite insignificant at least when it comes to writing code that actually works but hey you never know maybe gp4 might come up with a better solution and so my recommendation to you is use both if you can I'm still going to be using cloud and GPT 4 together and since gp4 is more adapted compared to Cloud for example this ID does not support cloud and I'm going to use GPT for more in that sense but that doesn't mean I will abandon Cloud it's the contrary I guess I'm going to be using GPT for more for mundane tasks maybe writing some boiler plate and whatnot since again it is already integrated within the ID but for code that is more complex or requires refactoring or something of the like I'll be using Cloud although ad do let me know what your experience has been so far what are you going to be using GPT 4 or Cloud thanks for watching I'll see you in the next one see you

Channel: Lucas Barake

Views: 12,096

Keywords:

Id: 8j1QSsMea8A

Length: 20min 0sec (1200 seconds)

Published: Sun Mar 10 2024