GitHub Copilot X tested with REAL scenarios

copilot X was announced and it has captured the attention of everyone in the world I personally never thought that I would be excited about the next evolution of copilot because I thought the first version was already quite magical and it does a lot of great things and it's a major productivity booster but then copilot X came and it changed the whole game in my opinion copilot will fundamentally change how we write software and it's going to introduce a major major paradigm shift to this industry in this video I'm not gonna be talking about copilot X from an opinionated perspective I'm gonna show you copilot X and what it can actually do this is a sneak peek into copilot chat to be more specific and I'm gonna be testing this tool in three different scenarios I'm gonna test its ability in helping me figure out how I can build an application with something I've never done before I'm gonna test its ability to understand existing code and give me reasonable explanations of what's going on in the code base that I have never touched before and I'm gonna test its ability at refactoring existing code that I'm quite aware of and I know its capabilities stick around this is going to be an interesting exploration none of it is scripted and it's all recorded For Your Entertainment and for educational purposes let's get started [Music] let's dig right into it so I already have my vs code Insider's editor running on the left hand side I have my terminal on the right hand side don't ask me why I'm not using the integrated terminal I I've been having an issue with this nightly build of the S code so I don't want to troubleshoot this right now I want to focus on the more important things and the first experiment we're going to run with is to try to create a GitHub CLI extension using go and we're gonna rely on copilot chat to tell us how we can do this and how we can write our entire code so this is the first experiment where we're trying to create something we don't have a lot of information about and we're going to rely on co-pilot to teach us everything from the ground up and in my opinion this is really the best use case so I already have a folder created for this extension I'm gonna go to the GitHub co-pilot chat window and as you can see it's already greeting me and giving me some suggestions for some prompts that I can start with and the first one I'm gonna go with is how can I create a GitHub CLI extension using go and let's see what the prompt or let's see what the response is so it looks like it's giving us some really nice suggestions already let's see the first one is install the GitHub CLI tool all right that's already installed create a new directory done initialize and you go module and that director using the go mod init command awesome so I'm gonna follow it as um I'm gonna follow with suggestions let's see go mod in it okay we need to provide probably a package name over here so we're going to call it github.com link which is my handle and then we're gonna because we call the GH something that's what we're gonna do we created the new module create a new go file for your extension and Define your commands using the Cobra package interesting so it's already suggesting suggesting a package we could use uh use the GH package to interact with the GitHub API and perform actions on behalf of the user nice I like the suggestion because it's not pushing Us in the direction of we have to build everything from the ground up it's actually pushing us to use and leverage the existing packages to build something better and then build the extension using the build go build command and test extension using JH extension list to ensure it's properly installed well let's see and oh it's giving us also some boilerplate for stuff we can start with it let's actually copy this whole thing I want to check if it's really so I'm assuming this is gonna be my main so let's create a new file here main.go so obviously for a complete go beginner this is gonna be there are some nuances that are hidden but because I've probably done this a bit before I am already anticipating what's gonna happen a little bit so all right so it's using some packages that are probably not available for me here and then it's calling it my extension and then it has it's adding a command this looks pretty reasonable to me honestly I don't know if it's gonna work but let's save the file and see and let's try to build this so let's assume that I know nothing about go how can I build this package and let's see you can use go build all right this will create an executable called my extension yeah we probably need to change the name because it's called GH something right so let's try go build first of all uh and it's complaining now the complaint is that we need to install these missing dependencies already we have the solution so I I would have normally asked uh copilot CLI what does this error mean but because um already go was helpful I'm gonna go with these suggestions all right awesome I think we are ready to try again go build all right I think this package doesn't really exist and this is where we need to be a bit more careful with how we do things right because it's suggesting that we download and install packages that we really don't know who are the authors for and we really did not expect so this could also be an attack vector that could be leveraged by certain malicious actors so it seems that we're not really able to use this package at all all right let's see what does this error mean and I'm gonna have copilot chat actually try to clarify a little bit what's going on so this error message means that GitHub and whatever this package is not found in your go module dependencies to resolve this error you can run go get your suggestion does not resolve the problem are you sure this package exists so now it's apologizing and it's revising ah interesting so my response got filtered let's try it one more time all right this time it worked and it's saying it seems that GitHub does not exist in the CLI module instead you can use this Google package interesting I'm also not sure this is the right path well what the heck let's see maybe it's on to something so I'm just gonna paste everything as suggested we're gonna go again to our terminal we're gonna run go build and again it's complaining about this module being missing and we're gonna and try to install it well it seems that this one exists and we have actually let's see is this working too many errors all right nice so obviously there's something going wrong here let's make an attempt at fixing these issues so first of all this is declared but not used okay that's that's clear and here it's creating a new client for GitHub enable repo override what is this one actually all right well this is already an example to show you how you cannot fully 100 rely on copilot chat to do pretty much everything and this is also a clear indication that these tools are really not ready to replace developers in any way shape or form obviously my prompts could be improved uh to be more specific and by that simple fact that means that I'm already sort of writing the code in the prompt or at least I'm I know what I want to achieve that I'm feeding into the co-pilot chat before it can generate something meaningful useful and functional this is a clear response that these tools are not going to replace software Engineers anytime soon however they are helpful in at least getting us started so I don't want to really spend a lot of time trying to create this extension and resolve all of these problems most likely I'm not going to need this and maybe this client works or it doesn't we need to go check the interface uh but this is probably an HTTP client that has been built by Google uh to Leverage The GitHub apis however there are better ways with which we can Leverage The GitHub API the CLI team has already published packages that allow us to create HTTP clients they allow us to also create graphql clients which are all very much more helpful in this particular case and they will allow us to do whatever we need to do they also handle very nicely tabulation they handle pagination API rate limits and all of that fun stuff as a first attempt to create something we have no clue about you've seen it yourself maybe we can improve this maybe not we will see I will leave it up to you when you get access to this feature to test things up to your satisfaction now we can safely proceed to the second experiment which is trying to use copilot X to help us understand an existing code base that we have no clue about and since Elon has shared the Twitter algorithm and open sourced it very recently why not try to make sense of it using copilot so let's first of all clone this uh open source repository on GitHub which is in the Twitter organization and it's called the algorithm I hope it's not that big and we're gonna wait for it to clone and it seems that it's working quite well all right perfect it's not big at all so let's go into the algorithm and let us open this inside our vs code to see how our copilot is gonna handle this so obviously I'm not really sure copilot can uh reason about existing folders without tabs being open I think when Tabs are open it can analyze and assess the context and give us suggestions based on that I'm not really confident that it's able to analyze a folder structure but let's let's give it a shot um can you see the files inside the directory what is it called project stamped the algorithm Let's uh make sure that we're in the proper directory let's give it a full path actually maybe maybe that'll help I'm not really sure what's going to be the outcome let's see so to see the files inside this oh it's giving me a suggestion of how I can see the files okay obviously it doesn't have that awareness so it does not really see the files on your system which is which is good in my opinion because I definitely don't want all of my files to be pushed elsewhere uh I only want whatever I'm opening to be to be analyzed by Copilot now let's maybe start with the readme so it's a set of services okay that's fine and then we have features this is also another example of why developers are still needed someone needs to make sense of this architecture and for us to be really able to navigate it we need to know where we can start but I think copilot is able to uh help with a few things so let's see home mixer maybe let's look at the service and it's written in Scala I have no clue how Scala works or I have really no I've never used it before so let's try something a bit drastic let's ask copilot what is this code doing and let us see if it can spit out something so this code defines an object called home mix alert config with some nested objects and Method it also Imports some classes so it's literally describing verbatim or like a literal description of what what is going on but I'm not really sure this is useful for us why because we need more context right obviously and I'm pretty sure also copilot needs more context and it's not going to do a lot of magic here the alerts include empty response rate alert yeah okay fine let's keep this file open and let us see here and let's be a bit more specific so for example uh let's ask it what is this class and Method doing all right so this is excala class named whatever that has a Constructor okay fair enough I mean that's useful like I didn't even know that that's how you define the constructors The Constructor in Scala like for a class implicitly so that's that's pretty cool okay I learned something new today the class has a method named get scored get score tweets response that takes two parameters request type and a parameters object fine the method returns a stitch Okay so so this method also calls another method of the product pipeline registry well I mean I I can read this in the code already sort of so it's not particularly helpful here but what I can do for example what is a stitch and I think this is where copilot chat will really shine it's really an explaining the syntax and the grammar of the language maybe even some of the quirkiness of certain languages they could be explained by copilot chat and I think this is where the real value is going to come in so in the context of the code you provided Stitch is likely a type Alias or a custom class defined elsewhere in the code base amazing so it's already provided us with feedback that this is not necessarily a language feature but rather it's more something that has is custom defined and because I don't know Scala that's obviously something helpful in trying to understand this code base and let's say however based on the name it's possible that Stitch is a type that represents the result of stitching together multiple data sources of processing types amazing I think this is very useful as feedback honestly even though we didn't really make anything groundbreaking yet um but I like this I like that I mean as a as a beginner or as someone new to a language and a code base I would love something that can create this separation or distinction between language features and between the custom stuff that we have built uh I think that's of great added value honestly so let's see over here oh interesting com.weather.com it's a fun code base it seems that the folks there are really enjoying themselves uh while writing this okay that's fine so I'm not really sure I'm not really I'm not able to think of a path where I can leverage co-pilot chat to do all of the heavy lifting for me maybe what I can do is I can feed it maybe they read me and ask it to tell me I can ask it stuff about the readme for example instead of me going and reading the whole thing so let's let's give it a shot all right so it's trying to explain the readme for me but I'm not really sure that's exactly what I want um give me a summary of the readme.md file that I have opened and let's see if I can get an interesting summary so the readme file provides an overview of Twitter's home mixer um all right that's useful the service is built on product mixer a custom Scala framework that facilitates building feeds of content okay interesting the for you recommendation algorithm in home mixer involves several stages including cat the degeneration future hydration scoring and ranking using an ml model filters and heuristics mixing and product features and serving okay I can I can use this I can use this to navigate so that's that's fantastic uh the pipeline structure for home mixer was organized around pipelines that split the execution into transparent and structured steps requests first go into the product pipeline that's fine each mixer recommendation pipeline may run multiple candidate pipelines to fetch okay so let's try something like what is candidate generation and where can I find the code that handles this let's see if it's going to help with that so candidate generation is the process of fetching tweets from various candidate sources awesome such as the early bird search index user tweet entity graphs or mixer and follow recommendation service this is the first stage of the 4u recommendation so I'm not really sure if it's hallucinatingly hallucinating this or this is actually coming from uh from the readme file but that's awesome and the code that handles candid generation is likely located in the scored tweets recognition pipeline config class alright that's perfect let's see oh so this class exists brilliant and it looks like it's quite an extensive definition which is which is amazing honestly like I think this is a fantastic starting point for going down this path and trying to understand whatever is happening in this in this code base you've seen it yourself I think if we have a combination of a well-written readme file as well as a good structured or well-structured project I think copilot chat is a fantastic tool that can help you navigate an existing code base and honestly it does deliver on the promise if you really can tweak your prompts in a way with which you can extract the Insight or the value that copilot chat provides so obviously there's a learning curve here and you're not going to get the maximum value from the get-go so if you start playing around with this tool a bit more and you learn the limitations of it and how it can derive context I think it would be a fantastic fantastic assistance and understanding existing code bases as you've seen now yourself alright now we got to the last experiment for this video which is trying to use copilot chat and copilot text in general to refactor an existing code base and I'm going to use a project I um launched or I published a few years ago which is pretty much a small utility that allows you to search your start repositories on GitHub and even though you can use the search functionality in the UI to do this I think being in the terminal is a much nicer experience at least for me and I would like to be able to search these bookmarked repositories because sometimes I star certain tools that can come quite handy when I'm doing a certain thing without further further Ado I've written this small utility in node or in JavaScript to be more specific and it's quite simple it's composed of two main files the cli.js which is nothing but a small you know module that's just gonna buy parse my arguments that I pass through the CLI and it will call the method or function that's called search in my main.js file and it will pass these options to it the search is composed of multiple functions that are pretty much going to call the API of GitHub fetch all of the start repositories it's also gonna cache a version of this response locally on disk so that we don't always make calls to the GitHub API this is useful whenever you have a large amount of start repositories the calls can be quite expensive and you could be hitting the rate limit with that also it would make the utility slow and here we are in this search function we're just you know loading the file from cache and then executing a search based on it and that's pretty much it spinning the output based on that search to the user so what I'm going to do here is I'm gonna try to have copilot try to refactor some of these functions and methods so first of all let's try the explain functionality right so let's assume we have someone new who doesn't really know what this code does so let's see if copilot is going to do a good job with explaining this small utility so as you can see here it's already figured out that this is an OJs module it searches through a user's star depository on GitHub for searching for a certain search parameter that's fine it uses the GitHub API to fetch the start repositories I think that's great honestly like it's knowing what I know about this utility I think it's doing a great job at explaining what's happening here and it says the search is performed using a full text search algorithm perfect that searches through the full name description and home page fields of these repositories that's very accurate and honestly like it's not very difficult to detect via the code uh because you can pretty much see it over here in this part but this is because I know the code already if you don't this is very insightful information that allows you to parse this code much more easily right the module is composed of several functions that perform different tasks like validate parameter construct API URL calculate Pages fetch Pages all of which is correct and then the module uses several packages that's also very correct mini search for performing the full text searches so on and so forth that is a very good explanation of this code honestly it's I think it's doing a bit of a better job here with this code just because I think it's because it's all contained in one file as opposed to the previous experiment we ran on the Twitter code base which is a bit more decoupled and decomposed and I don't know how much this stuff is going to influence how we write software in the future right because we've been using stuff like object oriented programming and implementing certain design patterns not necessarily for the benefit of the machine but for our benefit as end users and readers of the code I wonder if we're going to re-adapt our design patterns to now serve the machine who's serving us as opposed to writing human readable code from the get-go so that's that would be interesting to see down the line but for now let's focus on the experiment that I wanted to run which is trying to refactor some of the code so let's let's do this first of all I want to you know eliminate some of this stuff and I want to try to use async and a weight in this code like this is one of the first things I would try to probably introduce and maybe even get rid of axios because like what's the point of it at this at this point this was written like years ago actually is what it was a thing right now node supports all of these different you know ways of making HTTP requests and creating promises quite natively so let's see refactor this code to stop using axios and use fetch instead fantastic also use a sink await instead of promises go pilot figure that out so let's see what it's going to spit out over here brilliant I think this is I think this is correct so the URL is being constructed over here and then we have the headers that are being appended and we have a constant that is response where we're fetching we're calling the fetch function and we're passing the URL in the headers then we're getting the Json data and then we're returning it and we also added a sync I think this is perfect let's give it a thumbs up and let us just accept this because I think it works now obviously with this adjustment it's going to break other parts of the code because we need to await this function right now which isn't what we're doing or maybe it still works let's give it a shot npm install npm run build and let's give it a shot now so bin start search minus U link minus F let's say es6 or whatever and we're gonna see if this works okay okay perfect so as we expected this is failing obviously because it's saying fetch is not defined so let us continue leveraging copilot's capabilities over here and let us go to the chat and say what does this error mean all right fetch is not defined means that fetch function is not defined in the current scope the fetch function is a built-in function modern web browser so on and so forth how can I fix this error all right so the fix is to use uh node fetch and the alternative is whatever we were using which is axios that's fine so let us actually do the following and PM install node fetch maybe I need to upgrade my node actually is let's do this is fetch available natively no the fetch function is not available natively for node.js this is something I think we need to reconfirm let me do a quick search all right we're using version 16 which I think is the problem over here I think fetch is available in later versions probably available from version 17.5 onward all right we're gonna skip this uh because I think you got the gist of it the point was to show you how we can refactor certain the code that existing code that we have and show you how we can take it all the way I think by upgrading our node runtime over here we're going to be able to get this code to run probably we need to make a couple of adjustments also wherever this function is being called and fix it over here maybe by adding like an await and we can also ask copilot to do this for us but I just wanted to show you how some of the refactoring works and that's pretty much it that's all I wanted to show you for today I hope you enjoyed this video hit that like And subscribe button it really helps me a lot uh with this channel it's a funny coincidence that about a year and a half ago we're getting close to two years I published my first video and my first ever YouTube video and the start of this Channel and the topic was GitHub copilot and honestly back then I didn't really think I'm gonna make another video about copilot but it's just a funny coincidence that almost a year and a half later I am making a video about the next generation of this awesome awesome tool which I think is going to be highly impactful and I cannot wait for every one of you to get your hands on it because it's going to change how we write software and I'm really excited about what's coming up down the line thank you for watching and I'll catch you next time [Music] thank you

Channel: glich.stream

Views: 43,432

Keywords: github copilot, github, github copilot x, chatgpt, llm, chatgpt 4, software engineering, programming, coding

Id: s7AGkcSMiaI

Length: 30min 19sec (1819 seconds)

Published: Mon Apr 03 2023