Use THIS Today to Make Your Local LLM Smarter + Claude 3 Opus Tips

okay so this is a custom test I used to check the intelligence of llm so we are running it on chat GPT 3.5 I know the answer is 36 and here you can see it calculated that it was 32 apples so let's head over to my boosted llm this is also chat GPT 3.5 but we using the API so when I run this now you can see should use code analysis yes and the next thing it does is create a python code to actually try to fix this okay so the first attempt didn't work so let's see if it can self correct here code output 36 perfect after going through the series of I went to describe uh you end up with a total of 36 apples perfect so today I'm just going to explain the logic behind the system you're going to do some more testing and I'm going to show you how I use clo three Opus to add features to this okay so what you see here is basically how I thought about it this is not a new concept of course GPT 4 uses this all the time with a code interpreter so of course that is kind of my inspiration right the first thing we're going to do is to analyze the user's input query to find out if we even need to use code to improve the answer and if the answer to that is no then it's just going to respond as normal so we're going to skip this step here but if the answer to that is yes then we're going to write the code we're going to execute this code in the terminal we're going to take that output and we're going to fetch the user question and we're going to feed the code output into a prompt as context uh think of it bit more like Rag and this will enhance enance the lm's response and we can bring that answer back to the user so like I said it's pretty much inspired by the code interpreter but yeah you can use it on local models you can use it on any kind of model you want right before we do some more testing let's take a look at some key lines of code in this system so the first lines I want to highlight is the should use code function so this takes the user input right uh we set a system message for this so you are an expert in determinating when to use Python code would be advantageous to provide the best answer to a user's question and I want this to answer yes if it believe it's using python code would be beneficial and no if natural language is fine and the next part is just going to be this variable here where we use the open AI to answer this prompt so here we kind of feed in the user question right in this F string and then we kind of just go uh if there is a calculation or similar need to be done to answer the user question always answer yes and then we just going to return Yes in the analyst response so this is kind of what decides if we're going to actually use code uh in the query or Not Another important part of this system is when we actually trigger this should use code approach and we start to create our code uh it's very important that we save or store this output in a variable called code output so this is going to be the output from uh the successfully executed code right and then we have this variable called natural language response here we take the user's question and we take the context that is going to be the code output right uh so this is you can think of this more like Rag and then we just going to trigger like use the context above so this is the code output and the question uh and give a helpful answer to the user's question in natural language so basically yeah you can think of this like you take the rag that we bring in from the code part and we look at that and we use that context to give a natural language response right so as you will see in a few examples uh but I'm not to spend too much time on the code this time those videos can drag out forever uh but if you're interested of course as always just become a member of the channel I will put this up on the community GitHub where we have full access to this and the community Discord so yeah look out for that just follow the link in the description uh for now I think we just going to go and do some more testing and see actually what this system can do right so the first thing I want to start with is actually not using open AI or Gro uh what we have available here I want to use a local model on my system so I'm just going to copy this local function here that chooses a local model run from LM studio and I'm just going to replace instead of running from open AI we're going to run from our local system so let me just set that up okay so that was done I switched out all the open AI functions to our local model uh I went over here to LM Studio I picked the mistol 7B open Hermes 2.5 I started a server here and yeah we are ready to go okay so the first thing I want to try is just what is the price of Bitcoin today should use code analysis yes okay so we're g to use some code here that's good I was hoping for some kind of API okay that yeah that's promising so let's see here we are installing our packages we need request I guess and stuff uh could not convert string to float 72 okay I think that is the price uh let's try again here still could not convert string to float okay so we got it in the end that took some time code output the price of Bitcoin today is 72287 and then we get like the natural response the current price of Bitcoin today is 72287 perfect and you can kind of see here uh if we zoom in here you can see we are using the mistal 7B right if I zoom in here you kind of see it here accumulated 22 tokens right so I'm very happy with that let's just stay here on mral 7B so I wanted to try what is the weather Forest for London this upcoming weekend use an API to find out my API key is and I just gave them my free API key from open weather map I think it's called yes we should use a code that's good hopefully it can find the API I want to use so yeah open weather map it fed our AP key yeah that looks good so let's see now yeah uh the upcoming weekend weather forecast is showing overcast clouds with a temperature of 13° Fahrenheit humidity of 81% I don't think it's Farenheit though I guess it is wind speed all right I guess that's fine so yeah good job okay so I went ahead I switched to Gro so this is going to be very fast we're going to pick the M mixol A Time 7B model here and we're going to try uh something a bit more challenging so yeah let's just just clear this right okay okay so what I wanted to try was uh I bought one Bitcoin one year ago what was the cost then what is the worth today how many percentage did I gain okay so let's see if we can fix this okay so let's see here yes we should use the code analysis so let's see what happens okay so we are using a coindesk API here okay so let's see if this going to work I'm just going to let it run okay so we have an error here okay so we ended up with uh unable to fetch historical price that could have been API problem so nope that did not work let's try to change to chat GPT 3.5 turbo yeah so let's see if uh chat GPT can solve this uh we're going to use the 3.5 yeah like I said Okay so let's try the same again okay yeah we're going to use some code here good okay now we are using a different API so coin gecko so let's see if that works okay so we have an answer straight away okay so the cost of one Bitcoin one year ago was $ 24, $178 71,000 now okay uh percentage gain 195% this seems very reasonable I think that might be correct let me just quickly go and check that okay so here I am on the one year short so if we go all the way left here 24,000 something right okay if we see here 24,000 yeah perfect so yeah that solved it pretty good so yeah well done okay so the final thing I want to show you today is how much I love the anthropic CLA 3 Opus for coding so I'm just going to take you to how I kind of add a new feature to the code we have been using today so I just set my system prompt very simple your software de with expertise in Python your task is to assist the user so nothing fancy here uh I just type my code and then I paste it in my full code here right and let's take a look at uh my instructions here uh so let me just copy that so you can see it a bit better I want to add a new feature to my code the feature is the following one before executing the code I want the user to confirm with a y or n if the user presses y the code will execute if the user presses n we we should suggest a different code to the user after that go back to the confirmation stage can you fix that was all I put in right and when I ran this we got this answer so basically no here certainly here is the updated code so it doesn't go around like gp4 explain everything in clear text before so it just gives me the code and a short explanation on the bottom here uh okay so let me just copy that so let's grab the the function first let's go into our code here paste it in save that let's go back to anthropic let's grab the main function right okay good let's go down here uh I I know it's weird using notepad but it look looks much better on video uh okay so we did that now let's test it so let's just run the same query as previously so let's do the Bitcoin one okay so hopefully now when the code is written it stops right and ask for our permission yeah you can see do you want to execute this code y or n uh let's press n because that's more interesting okay so generating an alternative code so we're going to compare them now and see if they are uh different right okay so let's see here now so yeah we have the print statements here yeah you can see this is different we're still using the API so now let's try to execute that yeah perfect perfect let's see if it was correct this time yeah basically the same so you can see it took me like 2 three minutes to add a new feature to our code very simple uh hopefully this gave you some kind of tip but just wanted to highlight how good and how enjoyable it's been to using CLA 3 for this yeah so that's basically what I wanted to share with you today as always if you want access to the code I've been using in this video just become a member and you will get access to the GitHub repo uh other than that wish you a great day and hopefully I will be back on Sunday I'm traveling this weekend but I will try to make a video uh beforehand so anyway see you soon

Channel: All About AI

Views: 8,294

Keywords: local llm, local llama, python, claude 3 opus, ai engineer, ai engineering, mistral 7b, groq, llm, lm studio

Id: oP1_tp_TnWA

Length: 11min 9sec (669 seconds)

Published: Wed Mar 13 2024