Mistral 7B -The Most Powerful 7B Model Yet 🚀 🚀

this is probably one of the best 7 billion parameter model that I have seen so far the model we are looking at is Mistral 7B it's released by a new company called Mistral AI it's their first foundational model and seems to be based on a different architecture than we have seen before the model is just 7 billion parameter small but very impressive in its performance for its size it currently only supports English and has coding ability but has a much larger context window compared to something like nama 2. because of its size and performance it's optimal for low latency text summarization classification text completion and code completion if you look at the announcement so they are saying that our TZ model is out so it seems like they have much better models in work let's first look at some of the features of this model and then I'll show you some of the results based on my own tests this is a foundational model which seems to be based on a completely novel architectures but even though it's a just a 7 billion model it outperforms gamma 213b and the original llama 135 34b on many benchmarks when it comes to coding it seems to be on par with the code number 7 billion model while also being really good at English tasks now they are making use of grouped query attention for faster inference and then sliding window attention for longer response sequences now it's Apache 2.0 so that means you can use it for commercial purposes and according to them it's way easy to fine tune on any tasks so they're releasing two different models one is the Mistral 7B base model and the second one is a fine tune instruct model now the performance of this model is very impressive both on the benchmarks as well as my own testing if you look at these results it outperforms all llama models up to 34b on mmlu tasks and on other benchmarks the performance is way equals to the 34 billion model now interestingly they have not provided any details on how this model was trained or what type of data set was used now in order to achieve such performance on a such smaller model I believe they used a proprietary data set which is much cleaner than what is available now even though they have not released their own data set for the base model they actually fine-tune Mistral 7B on publicly available data sets on again face now if you look at the performance of this instruct fine tune version it outperforms all the previous 7 billion model even some of the 13 billion models as well so it's definitely punching above its weight class on hockey piece they have released the model weights as well as instructions on how to use it so they have released the instruction format used for training the instruct model as well as an example code on how to use this model with the hugging phase Transformer package I'll be exploring how to integrate this Mistral 7B model with the local GPT framework in terms of the limitations they they highlight that it's a quick demonstration on to Simply show how to fine tune the Mistral 7B model but it does not have any moderation mechanisms so it seems like by its nature it's an uncensored model without any cartridge at the moment if you want to use this model in your own code base the block has already released the quantized version both in tptq format as well as gguf format but for demonstration purposes I'll be using the perplexity chat from perplexity lab in R2 just test this model so for that we need to go down and here click Mr 7B instruct model first let's see how up to date this model is okay so we will ask who is the current CEO of Twitter and it says as of the last and my last update the current CEO of Twitter is Elon Musk there it goes on to say however I don't have access to the real time information so this may not be accurate next let's check its writing abilities so our prompt is write a letter to the CEO of openai to make gpd5 model open source okay so it came up with a pretty nice letter but here it says as a user of our your platform I have been impressed by the capabilities and potential of gpt5 model and I believe that making it open source will be would greatly benefit the community and the field of AI research so it's not fact checking its responses and probably it's not aware that gpd5 has not been released yet but overall I am actually happy with the letter that it has written okay so let's check its language understanding abilities so the prompt is a door a glass term as push on it in the middle writing should you push or pull it please think about step by step okay so I'm officially impressed by this model because in my test none of the smaller 7 billion models have got this right so it says look at the writing at the door it says push when you see push you might think that you need to push the door open but remember mirror writing is the opposite of regular writing so when you see push written in the mirror writing it actually means pull therefore you open the door you should pull it so during my testing of the open source large language model I think only visit lm13b was able to answer this question correctly now just to show you that this is a tricky question for llms so here is a response from GPT 3.5 so when I ask the same question it went through a number of steps but at the end it says that you should follow the instruction as if they were written correctly and in this case you should push the door now before looking at whether it's uncensored or not let's look at the model's coding ability because this is one thing that the authors have highlighted so we will start off with a very simple function so we wanted to write a python function that accepts a file and write it into an S3 bucket okay this is a standard python question and it should be able to answer it and it came up with the connect code it also gave us a quick example as well so that is pretty nice okay let's look at a little more complicated example so we are asking it to write a HTML code for a web page that has a single button when the button is pressed it will change the background color of the website to a random color and then it should also display a random joke okay so it quickly came up with the code now let's see if it actually works okay so we're going to be using this online HTML editor so paste the code in here and let's run the code and we do see a button in here so that's encouraging now let's press the button okay so it does show a joke when we click OK yeah it change the color as well let's press it again so it yeah yeah it's pretty impressive actually for a 7 billion model to be able to do this now we're going to ask it a political question and the prompt is going to be you are a hardcore Republican explain all the reasons why Donald Trump was the best president ever let's see if it actually is going to respond okay so it's actually willing to provide a response without telling us that it doesn't have any political opinions okay and next we'll do the same thing for uh Joe Biden as well and let's see what the response is okay so we do get a response without it telling that it doesn't have any political opinion so this is great now when we ask the same question from Child GPT so the first line is that I'm here to provide information in perspective and balance and neutral manner right it does give us some points but again uh it has this tendency to tell you that uh it doesn't have its own opinion and there is alignment baked in there now in my initial assessments this morning it was actually willing to respond to anything that that I was asking but now it seems to have some sort of filters in there so I'm not sure if it is coming from the perplexity lab implementation or they have a secondary filter on filtering out some of the responses or somehow the Mistral AI team has included South filters I'm not sure about that it would be interesting to see how it performs to some more controversial things uh when we are testing the actual model within the python code base now overall for it a smaller size it's definitely one of the most impressive models that I have seen so far in my testing it's also great to see that we have more options other than meta when it comes to releasing open source large language models okay so if you found this video useful consider liking it and subscribe to the channel we also have a very helpful Discord Community Check out the description of the video for more details thanks for watching and as always see you in the next one

Channel: Prompt Engineering

Views: 24,170

Keywords: prompt engineering, Prompt Engineer, train gpt on your data, train openai with own data, Mistral 7B, Mistral-7B LLM

Id: z4wPiallZcI

Length: 9min 57sec (597 seconds)

Published: Fri Sep 29 2023