Code Llama: First Look at this New Coding Model with Ollama

today we're going to be checking out code llama meta's new large language model for generating And discussing code I want to see how it is for Python Programming and so in this video we're going to be comparing it to llama2 I'm going to be showing you how to set it up easily on a Mac and we're trying it out on a number of programming challenges and later on we'll even see if it's able to program the snake in play game so with all that said let's get on with things so this is the announcement for llama2 we can see that we have 7 billion parameter 13 billion parameter and 34 billion parameter models respectively they're all pre-trained on code specifically code and so we can use this for python C plus plus Java PHP typescript and a bunch of others as well so this is really great for us as developers we can actually use this potentially integrate it into other tools like vs code for instance um and use it very simply locally and because it's free and available to us we can just install it on our machines and make use of it you can even potentially use it for commercial purposes too to install this I'm going to be using Alarma which is what I used in my last video to install llama 2. it's a very simple way of getting set up with large language models on an Apple silicon Mac literally download the file head to Alarma download the application file install a new Mac you'll end up with a little icon like this in the top right saying that things are running once it's installed I've done a previous video on this so if you want to go and check that out I will link it in the description and then to install codel Armor All We need to do is tell Alarma to pull it down so the example that we are given is olama run code llama so let's go ahead and do that if we run code llama so this is very similar to something like Docker where you're pulling a Docker file we're pulling down a model from hugging face Instead This is a default model um I'm going to try out these python challenges first with the 7 billion parameter model which is the default which is 3.8 gig download that we've got there then I'll move on to the 13 billion parameter model if they struggle now the 34 billion parameter model I can't actually run on my Mac because it's a 16 gig M1 Pro which is not what they recommend they recommend having 32 gigs of memory and I don't have that okay so codel armor has come down okay so the first thing I want to do is see if it can solve these program problems that I gave it before first one was whether or not I could write a regular expression to match email addresses let's see I'm also there we go okay that's cool and that was a really fast response actually you don't seem much faster than when I was doing it previously so it's given me an example that looks good that's that's actually really good because on the Llama 2 one it just didn't work at all it said with llama 2 that actually it didn't perform particularly well programming challenges let's try this other one as well that I had which was um can you write Python program to find the largest to integers into digital juice from a list so it looks like we run but digging into it doesn't look like it's doing what I'm asking so if basically find some largest one from the list fine and then to find the second largest it takes one element off the end of the list it what if the maximum was within this smaller part of the list that's not going to work so let's see actually let's pull that up in vs code and see if I can make it fail okay yeah so that fails because that immediately prints out 1010 which is not correct that's a good start let's see if it does any better so I'm only running code llama there I'm going to try it with 7 billion parameter model which I've already downloaded in fact sorry it's a 13 billion parameter model and that is seven gigs confusing there we go okay yes okay let's see if this works extra close bracket there I'll brace well that works so that's good let's try some other numbers so it sets up a larger list it appends the maximum from the list that we already have it removes it from the list which is what you'd expect it to do because you want in the original code there if we go back to it it's just it's not the correct answer so even with that seems like a fairly trivial problem to me but it's still not able to solve it on that smallest model so those are the two challenges that I said first of all I kind of wonder if that regex now um Works legitimately but we'll move on anyway the next one so I want to check uh from prices python challenges which we can actually test online then so these are just a bunch of challenges that are on this website python principles I will start with some of the smaller one the easier ones and then move on to um maybe something a bit more complicated um so let's start with the very first one Capital indexes write a function named capitalings indexes what's nice about this we can literally copy that and then so write a function in capital indexes function takes single parameter which is a string here's an implementation okay let's do that run the code Your solution was correct you've now completed this challenge brilliant thank you code llama okay so that is one of the smaller challenges let's go for a little bit of a harder one I like this one about palindrome so let's do that a string is a palindrome when it is the same when bred forward and backwards for example the string Bob is a palindrome so alisaba but the string ABCD is not 100 okay write function name palindrome I've actually got the description right function named palindrome it's a single string as a parameter your function should return true if the string is penetrating around false otherwise so it's getting a bit confused because of the line breaking on this okay so yeah this is a fairly simple one because that's actually I can tell from that it's it's just saying give us a reverse of s what's what this uh list kind of interpretation means and it's just comparing it to S so that should just run and be happy oh no okay so it wants the name of the function to be palindrome solution was correct what's the example show all right so we've literally taken the one that is the most complicated let's try something harder so the last one is going to be a thousand separator let's see if it is able to do that so write a function named format number that takes a non-negative number is the only parameter your function should convert the number to a string and add commas as thousands separate for example calling format number one million should return one comma zero zero zero comma zero zero okay oh that's interesting okay so we got an error in there it is invalid syntax to the response that we got from code Lama is just an F string which actually works fine we can see it here and that's exactly what we'd expect so if I put in that if I put in a much bigger number it also breaks it up correctly um this particular test thing isn't able to cope with her which is kind of funny and so it just breaks and also hint convert the number to a string and enter you to over each digit iterate back so it's all start by reversing the string to make your life easier so the gold standard of a lot of these test videos seems to be always to program the game snake in Python so let's at least try that and see if we can cope with something more complicated I'm going to guide it slightly here and suggest that it should use Pi game so I generally don't hold much kind of faith in the fact that play game is a good test um I think you probably should do your own testing and it seems like every test seems to be using snake which I don't think tells you a lot about whether or not it's doing good programming we shall grab that and see where that works for some reason all these so we got a mistake here I can see that there's a mistake on here it seems to be putting in more brackets on everything and now well that's about I don't know so there there there it's put an extra brace around everything what is going on so let's create the environment let's open a new terminal okay so it's activated the virtual environment for us let's pip so probably game so we've still had to do some stuff um this is loading images so that's not going to work so I hadn't spotted that while we were testing it it was using images let's see if I can guide it a little bit more and ask it for just using pixels so I've said update the program so it doesn't use images but draws code blocks for the player and the snake again we've got these extra braces you can see here there's two on this side three on this side what's going on I don't know do you want to see if this works and again like I said immediately got these brace problems oops I don't actually want to delete more on than I need to okay let's try and run that python so yeah another race is unmatched uh why is that now okay so have I taken brace out where I actually wanted it so it doesn't need to do that yes okay okay well that fails um that is not working so we've actually got something it did actually run which is good but we've just got this dot in the middle of the screen so um we've got it working with basic functions but to do a whole game like this it's probably asking a bit much of it we probably could get there because it looks like it's got basics of it down but um yeah probably requires a bit more work so there you have it code llama is able to code python functions it's able to get the majority of them correct it even got as far as going further than the um interpreter that we were using on the principles website there and doing better than it could actually detect um but it's not able to cope with doing one of these larger games which is kind of to be expected really but uh go and play the code llama it's very easy to install with a llama go and check my video on that if you've not used alarmer for large language models before um and I'll see you in future videos soon remember to like And subscribe and all that sort of good stuff that really helps me out and I'll see you soon all right bye for now bye

Channel: Ian Wootten

Views: 4,844

Keywords:

Id: TWc7w5rvKrU

Length: 12min 20sec (740 seconds)

Published: Wed Sep 06 2023