FINALLY! Open-Source "LLaMA Code" Coding Assistant (Tutorial)

coding assistants have changed the lives of developers around the world but all of them require you to be connected to the internet and usually talk to chat GPT and that's a huge limitation not only because of the cost but also what if you're on a flight what if you don't have internet connection now we have an incredible solution and we can have our own coding assistant completely locally today I'm going to show you how to use cod but we're going to be setting it up with a local model powered by olama and it is open source so many of you have asked asked for exactly this the ability to have an autocom completing coding assistant that is run completely locally with a local model now I want to mention Cody is the sponsor of this video so I'm going to show you how to set everything up and you're going to be able to run your own autocomplete coding assistant completely locally it is amazing let's get into it the model we're going to be using today is code llama and we're going to be using the 7 billion parameter version and it works really well now I have a MacBook Pro M2 Max and it is instant now Codi comes in the form of an extension for VSS code so if you don't already use VSS code download it that's what I use as my main coding editor so go to code dovisual studio.com and download whatever version is right for your operating system once you do that open up visual studio code you're going to come over here to the left side and click the extensions button then you're simply going to search for Cody c o d y okay then you're going to look for Cody AI right here and we're going to install all pre-release and it is done so then we can close out of this window then over here on the left side we have the Cod extension so go ahead and click on that and it's going to ask you to sign in you can sign in with GitHub but I'm going to choose Google so I'll click right there then I click open I'm going to sign in then I'm going to click continue then it's going to ask me one question I believe sure let's dive in what do you use coding for I'm using it for personal projects and that's it then we authorize and then we can open in Visual Studio code okay now we're signed in and we're pretty much ready to go so by default it uses is GPT 4 for everything and it is completely free so you get a coding assistant based on GPT 4 absolutely free there are some rate limits for the free version and the pro version is free right now and I believe the next tier up is about $9 a month but what we want to do is use autocomplete completely locally so let's set that up open up your browser and we're going to be using olama to power the inference so go to o l. a download olama and install it once you're done you're going to open up Ama and you know it's open if you see the little llama in your task tray up in the taskbar next we're going to open up a terminal in Visual Studio code and the model we're going to be using for inference is cod llama 7B so what we need to do to download it is O llama pull Cod llama colon 7B dcode and then hit enter now I already have it so it shouldn't download it again but we'll see yep so I already have it it'll take just a few minutes to download it's less than 4 GB and then we're ready to go next we're going to go over to the Codi settings right there and we're going to open Cod extension settings you're going to scroll down and you're going to look for this setting autocomplete Advanced provider you're going to click on that and if it's not already selected mine was you're going to click unstable o Lama now it is experimental and they do recommend using gp24 but this is definitely more fun and right here you can select different models if you wanted to try this one but this is supported by fireworks as it says and we're just going to leave this as null because we're going to do everything through o Lama and then you can click out of the settings and we're pretty much done let's give it a try now to prove that it's running completely locally I'm going to open up this output and then over here I'm going to select Cody by Source graph and here we can actually already see it Cody completion provider unstable olama code llama so let's give it a try so first I'm going to just have it write me some code by starting with a comment so I'm just going to type Fibonacci method then I'm going to hit enter and then it immediately you can see how fast that was starts giving me the code so now I hit tab to complete I'm going to hit enter again to go to the next line and it completes the rest of it boom done that's it that is how easy it is and it's super fast let's give it something a little bit more complex now now if you did want it to autocomplete without starting to type you can hit this little light bulb right there and you could say ask to generate and then you tell it what you want it to generate right there so let's try that ask Cody to generate write me code that outputs numbers 1 to 100 I'll hit enter Cody is working and there it is very fast and we can accept we can retry and we can undo and also show the diff so that is super handy and I'll accept there it is and as I mentioned Cody is also really good at understanding the context of existing code so if you remember in a previous tutorial I wrote some code to detect faces in an image and this is that code so let's say I wanted to add some documentation right here add a new line and then it immediately starts typing some comments for me based on the existing code draw a rectangle around the face perfect then if we just highlight this code I can click here and say ask Cody to edit and let's say instead of saving it as always sunny face zero I'll say change the image name to ASF 0. JPEG and I'll hit enter and there we go I'll accept perfect and we can actually see all the output down here and if we look right here we could see provider model code llama 7B so this is a small High performing model now Cody has a lot of other really cool features that you can use for free they aren't currently powered by local models but I want to show it to you anyways so we can chat we just click on that button right there it opens up a chat and here it's using CLA 2.0 but if we select this drop down we can use any of these other models including one of my favorite mixol 8X 7B by mixol these are all Pro Models but as I mentioned Pro is currently free so let's select Cloud 2.0 so I'll say What is the this file about then I'll hit the add sign and do find faces enter and now it should tell me what's in that file yep python code detecting faces in an image and extracting them to save as a separate image file we can also easily add documentation so if we click right there it's currently reading the code and adding documentation in real time that's actually super cool how it does that and then we can hit accept right there you can also edit code with instructions so go ahead and click that and we can just say remove all comments and perfect it removed all the comments super fast too we can also have it explain code which I showed you and we can also look for smells code smells so if you have pretty robust code and you're looking for any ways to drive that code up this is a great way to do that and then probably the thing that I would use most often if I was still building in production environments is generating unit tests very helpful something I did not enjoy doing although of course very useful to have test for your code so there you go Cody is very powerful I'd say better than GitHub co-pilot because it has a lot more functionality baked in plus you can do autocomplete with local models so check it out go to cody. deev thank you to Cody for sponsoring this video and if you liked this video please consider giving a like And subscribe and I'll see you in the next one

Channel: Matthew Berman

Views: 93,674

Keywords: coding, ai coding, ai coding assistant, copilot, cody, cody dev, sourcegraph, llm, ai, chatgpt, chatgpt coding

Id: gY_E3QBZ-NE

Length: 7min 20sec (440 seconds)

Published: Tue Jan 30 2024