Big BLOW to Copilot by CodeWhisperer

today I've got a showdown between GitHub co-pilot and Amazon's code whisper these are both AI coding Sidekicks I've been using copilot for almost a year and I love it done some videos about it you can check those out we'll put them head to head in a battle for the best coding assistant and see if code Whisperer is my new co-pilot uh that's not confusing at all forget I said that github's co-pilot has a pretty huge head start at least by today's standards and AI advancements it supports a pretty wide range of Ides and languages and costs 10 bucks a month for individuals but you can get it for free if you are under an organization that does open source projects and cold whisper is brand new and it has some ground to make up to catch up it also works with many Ides but not as many as copilot at least according to the docs but it does have some extra bonuses for you if you're in the AWS ecosystem like support for AWS Cloud9 and Lambda and it has a unique security scan feature that I demoed in my initial impression video which I'll link down below if you want to check that out whisper has two pricing tiers one we're all gonna love that's the free tier at least for now and there's also a 19 a month professional plan with some extra perks like more security scans and other things both of these support many languages but today I'm checking out typescript and if you want to see other languages in action just drop a comment down below we'll have a look at that too the first thing we're going to try is generating code based on the function name as I start to type with copilot I get suggestions immediately for the entire function completion I'm typing calculate here and it's already giving me suggestions rectangle I mean I didn't want that but what I want is days between dates and I started typing day and it already completed the entire function for me so all I got to do is press tab here but not not going to do that just yet let's go over here to whisper type in calculate it's assuming uh we're doing addition for some reason I didn't call the function add just calculate but it's making the assumption that we're doing addition let's do day I'm getting nothing let's do uh days okay calculate days we're taking in a date and a second date we're getting the difference between them now with copilot it's assuming that we're getting strings and it doesn't specify the typings in the function signature which is not that great or you can argue that it's great depending if you're mad polcock or not in code whisper it's actually defining the parameter types which is good in my case let's do um between dates I've completed that with copilot I'm getting some complaints from the editor and that has to do most likely because it's using any types and typescript doesn't like that here it's much cleaner we're not using any types we're using dates types also copilot generated vars which is a modifier you probably don't want to be using anymore unless you really need it you should be using consts and let's and that's what code whisper did it did it cost let's do calculate area I just typed out the whole function name very different results here folks co-pilot assumes that you're going to be passing in some kind of shape what the heck is this I don't have a shape to find why are you assuming shape not the best example Whisperer is assuming width and length which is what I would expect also co-pilot is trying to figure out whether you passed in a shape to be a circle or not I suppose it covers the circle case which is great but you also now need to Define shape whereas code whisper does not account for circles very different results so that was just by function name I sometimes do that but most of the time I write a comment to help direct co-pilot so I'm going to start off with a comment create a function that takes in two dates and Returns the number of days between them and I'm at the end of the line I'm going to press enter on both of these uh copilot what are you doing it's adding to my comment saying ignoring time of day whisper is already on a task it's already creating a function so I'm going to press tab here and it did not complete the whole function it only completed the one line so I have to press enter now it says your code here and for for some reason it's returning time from the dates oh okay it's getting the time and then it's calculating it so that's right copilot does after I accept that comment is it gonna actually it's adding to the comment this is something new folks this is something that I've just started seeing recently where it expands your comment indefinitely and doesn't actually get to the function this is a bad show co-pilot you've been so good to me what is this copilot fails I gotta I gotta say this is the first time I've seen something like this like this this major a failure it just keeps adding to the comment it's not actually creating the function bad copilot I'm a little disappointed all right let's do something realistic uh right now ai is hot open AI API I'm gonna ask these two AIS to go to the other Ai and ask it something and then return me a result copilot talk to code whisperer no I'm just kidding and we're gonna talk to the open AI API here we go function to run a query against the open AI API you using the GPT 3.5 turbo model and return a response come on copilot it's giving me the usage why is it doing this expanding my comment I just want you to do it am I holding it wrong here come on Steve Jobs tell me I'm holding it wrong create a function it's telling me what I need to hey we got some code so I added the word create to the comment and that seems to have helped us generate some actual code it's telling us to import the open AI API it's telling us to import the dot End Library create a function to run the query there it is are we going to have more there it is so uh a couple things not right that I see here but we're gonna get to that let's see what code whisper does and to be fair I'm gonna include that create function word in there in the comment okay okay couple things to dissect here copilot is not using the proper module system that I have in my ts config so we're targeting es2016 they're both targeting es2016 because I use the same version of typescript module system is common JS yes it's common js on both of them and as you can see we're using the require keyword if we take a look at what code Whisperer did it did not include any Imports or requires at all it left it pretty much up to you to know what open AI is so I'm gonna have to Nick code whisper for doing that I don't know what open AI is I mean I know what open AI is but maybe I don't know what it is and it doesn't tell me what I should be doing with it co-pilot gets a bonus here even though it didn't do the Imports correctly every step of the way told you what it was doing import the open Library import the dot End Library it kind of gives me a hint of what I should be doing next which is installing those libraries few moments later yes I'm changing accent of meme all right I've installed the open AI SDK this is helpful because now it's telling me import the open AI SDK so it somehow knows what we need to do it's most likely based on the fact that there's undefined symbol here but it could also be because there's now a new npm package in package.json so I'm going to accept that and look at that it's giving me the proper import statement for openai it imported configuration and open AI API but it did not connect the dots here for us unfortunately so now I don't know how to use the configuration and open AI API without going to the documentation figuring that out I'm going to try something else in code whisper I'm going to delete this function again and I'm going to have it regenerate the function based on the Imports and let's see if that helps that's what I was looking for right there because I've imported configuration and open aipi AI creating a new function for me based on those and there is the configuration that's what I need is the open API key and an instance of openai API let's do the same thing with copilot and see if this helps I'm gonna just blow away what it had so far and do the import first that the top co-pilot you're disappointing me you're just giving me comments what happened to you I need to start with an import statement here because copilot is not doing it so it's importing it from the wrong package it should be just open AI not open AI Dash API and it imported everything not specifically configuration and the API that we needed so let's see if it regenerates the function properly it's not completing the whole function like I'm used to it's still calling the old open AI complete instead of the create completion and I guess that's that so copilot fails here it doesn't even call the right functions from the open AI API are you using copilot right now let me know in the comments and let me know if you've already tried code Whisperer because from what I'm seeing it's doing a better job so far at least in the tasks that I've given it there might be other tasks where co-pilot is better it's got more Ides that it supports it supports more languages so there are some benefits to using copilot in the case where I tried it with typescript code whisper seems to be winning including having a more updated idea of the API that I was trying to use which is a modern API this is just one little tiny little slice of an example there's tons and tons of things you can do with these tools I'm going to be trying code whisper more often that's for sure have you experimented with these two did you see similar results let me know in the comments down below and if you like this video go ahead and give it a thumbs up and stay subscribed for more content like this I plan to use these tools throughout my testing so you'll definitely be seeing more of these tools and videos to come thanks for watching folks I'll be back [Music]

Channel: Alex Ziskind

Views: 101,663

Keywords: programming, software development, chatgpt, code, refactoring, code refactor, gpt4, gpt-4, copilot, github, github copliot, copilot-x, copilot chat, copilot for docs, copilot for pull requests, copilot for prs, copilot for cli, copilot gpt, copilot gpt4, copilot gpt-4, openai, microsoft, microsoft ai, github ai, code ai, ai in code, aws, amazon codewhisperer, code wisper, code whisper, code whisperer, codewhisperer, aws code, amazon code, aws code whisperer, amazon code whisperer

Id: FomyM8CynF0

Length: 9min 44sec (584 seconds)

Published: Mon Apr 17 2023