Function Calling using Open Source LLM (Mistral 7B)

hello everyone welcome to AI anytime channel in this video we are going to work on an experimentation with function call so we're going to look at how we can make function calls using open source large language model we are not going to rely on open AI or close Source models to do function calling Now function calling Is Not A New Concept you know it has been in trends for last few months where open AI started the function calling capability with with their GPT 3.5 gp4 models Now function calling is extremely important when you have multiple features in your uh gen power system or the application that you are building for example if you want to connect with external tools plugins or some external apis how you can you know uh write different functions and that when you put a prompt when the end user put a prompt in your system how that can get automatically classified and invoked the respective fun function and I think that is really important to build a production level genni system mainly when it comes to you know uh rag or conversational interface where you have different data sources that you need to connect with uh you can do calculations you know you can do recommendations and things like that and it is fairly easy to do function calling with open AI models or other close Source providers because they provide you know this facility out of the box but we'll see how we're going to leverage mistal 7B in this video so I'm going to use open Hermos uh mistal 7B uh llm and we'll write some code to see if we are able to do function calling so I will write few functions that might be a dummy function but you know the underlying Concepts remain the same you can use weather apis you can use traffic apis you can use external uh plugins or uh tools like surf surf API or tavali and things like that so let's jump in and see how we can know uh uh perform function calling using open source llms and if you look at here I already have a few codes inits just to save some time because the model loading will take a bit of time but I'll just explain what I'm doing uh ignore this part of uh loc which gives some utf8 error so I just have been uh working with that now I'm on a00 GPU on cab uh fortunately uh right now I was able to connect with this runtime which is difficult to get in collab it all all redirects to v00 I have installed Lang chin uh I'm not using the new Lang chain 0.1 I'm relying on the Legacy version of Lang chain but you feel free to use the new version of Lang Chen as well few things getting the parcel I'll tell you what this does I'm getting some utilities typing Transformers torch uh because we going to work with open source llms we have langen thingy over here convert to open AI function so langen provides a high level abstract class that helps you convert your function to an open AI function and that is fantastic I'll show that I'll show show you that what do basically does and of course not only a function but you can have a high level class as well that you have created through penic for example and if you want to convert that to open AI function how can you do that that's what we are trying to do over here then we have penting to convert penic to openai function the first was to convert a python function to an openai function the next one is to convert a penic class to open AI function then we have some validations uh base model field validator we're going to use ptic here to write our classes and then validate it uh this is the model I'm going to use I'll show you that large language model uh model here let me just open that uh it's by technium open Hermos 2.5 M 7B and you can see I'm relying on M 7B uh it's a startup in France called mral AI they are doing a great job uh to strengthening the open source Community now you can see this is a model more than 100,000 downloads last month and I'm just going to use this large language mod model I already have written a function that helps me get the model name which is a string and you can see the bit of validation I'm using Transformers pre-train to get the tokenizer and the same Auto model for Cowal LM because we are using Cowal LM uh which helps you generate tokens okay it's not some mask language modeling it's bit different so we're going to rely on this class of Transformers and I'm getting the model and tokenizer you can see this is the way we have loaded the model we are saying tokenizer SL model load model uncore model name and we get the model over here you can see it takes a bit of time and it gets it now I have just printed the model which is a myal pipeline you can see the entire uh very high level uh layers and things like that now we'll start writing the code guys so if you face any memory error you can do a few things uh like you can clear the cache and do those things but I think it's fine so what I'm going to do is I'm going to write a class here so let me just write a a class and in this class I'm going to write book recommendation let me go come of come a bit closer book recommendation you can see I'm doing a base model and so I'm going to write three function these are all python function this might be dummy functions but you can just connect your apis as well if you have an external apis with this but let me first show you how how this how this is going to work now class book recommendation and then passing the base model and what I'm going to write next here is let's keep a doc string arguments uh basically to just have an explanation so I'm going to write provides book recommendation provide book recommendations based on specific or specified specified interest this is what I'm going to write okay this is my okay and now I'm going to have an interest you can see this is is right so let me make it a bit smaller it's 175 let's make it 150 uh can now it's fine now we have an interest I'm using ptic version one if you are using V2 you have to beat up you can make a bit of changes like field validators and things like that now interest which is a string and I'm saying okay the field looks like and I can just remove that it's fine now in field I have a description that description says uh the interest to base recommendations on I'll just make a bit change over here to LM can understand better question of user interest about a book this is what I'm going to write over here the interest is fine now I'm going to have recommended book which is nothing but the expected output I'm looking so recommended book and again this is a let's have a single which is a string so description book answer I'll write answer and this is a cach guys I think if you if you're working with the open source llms and if your function call is not working uh the way that you are expecting then you have to do better descriptions uh on the because it all happens on the prompt level right it's like you know it's happening some zero sort prompting and things like that but open AI is far more capable the the GPT 3.5 or gp4 they are far more capable than you know some of the smaller open source llm so if you're working with this open source llms for function calls make sure that your descriptions are up to the mark and if you see what I'm writing answers should be answer to recommend a book so I'm saying to model that answer your answer to recommend a book this is my recommended book and now a recommended book is done now I'm going to have a validator so validator and inside the validator my validator is interest so I'm I'm going to interest validate the interest which is the prompt that user will put it and validate not it's a decorator so validator interest and you can see this is how you can uh write your function So within that I'm going to write functions if you have your API thingy uh if you're connecting with external sources you're going to write here so I'm going to write diff interest must not be empty and you can see so it's a class it a value let's call it for you to understand better it's a field or value you can also call it a value and then it says if not filled and then interest must not be empty and then return filled so this is the first this is the first simple class that we have created for book recommendation which looks at the uh interest of a user that what kind of book that user is looking for and then this will be passed to Mr 7B and then M 7 look at your prompts and then execute the right function now validator interest class filled if not filled raise value interest cannot be empty return filled so this is fine must might be too deterministic let's call it cannot be cannot be empty okay now this class is fine I'm going to copy this and I'm going to create three different classes over here quickly and for the second I'm going to call this uh not book recommendation let's call it joke so we want to create a crack a joke or get a joke from Mr 7B based on so joke and then it says get a joke so I'm going to write here so let's do that so I'm going to write get a joke get a joke that includes that includes the setup and punch line this is what I'm going to write over here now in uh in the interest now this this should not be interest because we are writing setup and punch line so this becom setup and the next become a punch line so let's do that so I'm going to write here punch line fantastic now you can see the description needs to be changed again if we not changing description then that's a problem so in the description let's write question to set up a joke okay so question to set up a joke and then in description uh answer to resolve the joke answer to resolve the joke and then I say this needs to be changed because now validator will change from interest to setup because in the previous class we had interest now here we're going to have setup and I'm going to say and let's do a bit of more changes over here so I'm going to say is question ends with question ends with question mark so the model when model looks at it it will it will definitely say okay that's the question that I'm getting now question mark and class fi that looks fine now here I'll make a bit of change to do a bit of stripping so if ah if fi ah excuse me if fill and then I'm going to say if field because I'm looking at the question mark so it should be field minus one and not equals okay and then let's do this okay so this is a better way of getting but I don't need this uh Dot and now here I can say raise value error interest cannot be empty this is wrong let's say uh badly formed question and then you just return the field now this is fantastic now this this is also done so let's just run it you can see setup now I'm just going to copy this again uh book recommendation I'm going to write the third one which can can be your like let's have a music recommendation and you can also connect with your internal rag so for example if you have an internal rag which like you have a vector data Bas you know you have created a rag Pipeline and you also want to rely on external data sources like some apis you know some plugins and tools you can all bring it together in different classes and functions and then you can use I'll show you that how what next we're going to do and how we're going to use M 7B to do those function calls now in music recommendation and rather than music let's call it song okay song recommendation I'm going to write provide song recommendation let's change the book song recommendations based on a specified genre so let's call it JRE because interest might not be that relevant and now here in interest I'm going to write joner and here question to recommend this song question to recommend a song question to recommend a song and in the recommended book I'm going to write song here so let's do that so I'm going to write song and then description answer to recommend a song so let's try it out now this interest becomes joner because I have chain that and I'm going to write joner must not be empty so we are just doing a bit of validation with pantic over here joner must not be empty and then if not filed I'm going to write Jor uh Jor cannot be empty excuse me fantastic and I think we are done with this so let's get that here now if you look at on top we have a lang chain abstract class called where it is convert ptic to open a function so let's copy that here and I'll show you what I mean by that because open AI has a defined schema you have to it basically accept the function into a specific schema that open AI models like look at that schema and then execute the function and that's what we're going to do it over here so if you look at this convert piden to open AI function and if you pass any one of it so let's pass song recommendation here and let's see if that able to convert the ptic to an open AI function schema fantastic right if you look at this it gets you a name name is song recommendation you have a description if you look at if you have worked with open a function calls I have created a video that I will share the link in description you can have a look at that and this is the right schema for AI models now there should be a name of that function so we are calling it song recommendation then you have description that's called provide song recommendations based on a specified genre parameter which is an object properties genre description type string and then song answer to recommend a song typ screen and required is your Jer and a song fantastic right so now you can try it out for others as well let's get that and I'm going to call this as probably book book recommendation let's try it out so book recommendation yeah and you can see the book recommendation so in require I'm also saying that I need the recommended book because I need the output because a few times you know you can't sometimes you can't get the output as well but this is what I need now I have created some gist I'm just going to use those gist one is called let me just go raw here and contr a control C I'll explain that what I'm doing now this extract the function calls if you look at what it does it says extract function calls get the completion completion. strip has a pattern where you can have multiple functions and it uses a regular expression if not match return none if multiple function so basically this is this has been taken from open hermas as well they have given in their blog post now this basically extract the function calls okay so let's run this so I have run it and I will take the now I'm take I'm going to take maybe I can write also so but I'm just going to save some time and I will explain the function we're going to write the completion function so that is generate hermus and this is the important part so let me explain that now if you look at this what we are doing here we are saying generate harmers we are passing our prompt passing the model passing the tokenizer and then we have some config that we can override and we are saying in a function that function name arguments you can have an argument value argument value can have n number of AR arents and arguments value pair in the prompt I'm saying you are a helpful assistant with the excess of following function and we are passing the functions I can see it's in a dictionary we are passing the function over here it says and we just leveraging this convert identic to open a function class you can see it over here convert identic to open a function passing my joke passing my book recommendation and passing my song recommendation if you have n numbers of function that you have created and you can do that over here and if if you don't have a ptic but you have a regular python function then you have to use convert to open AI function to do that okay uh pretty much self-explanatory and I'm defining to use this function respond with multiple functions I have some hedge cases that I'm handling and then I have some configs you can see uh temperature 0.2 top P top K Max new tokens and things like that and then I'm just using model inputs passing the tokenizer getting the IDS and yeah yeah so this is pretty much Transformer thing so I'm just going to run this so let's run it and once you run it we need one more uh inference function and I'm just going to copy it from here where I'm going to pass my prompts okay so let's do that so let's copy it over here time and come and let's paste that now if you look at this we are saying generation fun we are using partial uh the fun uh this is coming from here here we have imported it somewhere over here you can see we are getting it from function tool input partial and let's come down we are saying these are my three prompts tell me a Jo joke song for inspiration and recommend me a book on crime tiller now these are like a bit static because we are not connecting with any external data sources but you can have it dynamically as well you have just to write you have just you just have to write that function okay uh if you're using ptic it's fine through a class if you're writing python function feel free to do that as well and you can just call it same way now for prompt in prompts completion functions let's print it out and see what it does now once you do that I'm expecting that I will get my output you can see fantastic right uh I didn't get it for song and we have to probably have to make some changes in the we have to make some changes in the uh description but I got it got it for name joke you can see argument it say setup why don't scientist trust atoms that's the setup and it's punchline because they make up everything so I joke function is working fine book recommendation interest crime Thriller you can see I'm asking about a crime Thriller recommend me a book on crime Thriller and it gives me the recommended book The Silence of the Lambs which is a crime thriller book which is fantastic right you can see I'm on 800 it took around you know it took around 5 seconds on average let's make a bit of change guys here tell me a joke or uh tell me a funny joke let's let's make a bit of changes and see if this is working fine but if it's not working you have to make the changes on uh you have to make the changes on the description and also have to look at how you can improve the prompt but it certainly works you can see it's going to work if you make it even more better it will work in a better way as well I'm going to write song for song for uh let me see what was the prompt or what is the description for that I just want to check it out once song recommendation based on a specified genre okay song recommendations perfect on a JRE so I'm going to say recommend me a recommend me a romantic song Let's see that okay if it's able to get it if not that you're going to improve that recommend me a book on recommend a book let me remove a bit or I'm just going to write a book on crime theer or let's make it crime theer a book on personality development and I want to see if it's able to retrieve it's able to generate anything okay so you can see that uh I at least joke I got the same joke joke and yeah fantastic so we have to see the song recommendation is not working but the book recommendation definitely gives me a different answer it says The Power of Now by eat to let's see if that's a book uh let me just have a look that on Google uh I haven't read that yeah you can see that's that's a book uh The Power of Now Okay fantastic so personality development you can see it's getting it right uh joke is coming the same joke because they make up everything why don't scientist trust atoms but I'm bit disappointed with the song Maybe I'm not putting the right description and the prompt recommend me an let's see that once recommend me an inspiring inspiring song and then I'll just have a look at my config uh use cash ah okay let's call it false uh let me call that false and make the temperature a bit higher run this you can see uh the same answer but now let's try it out once more but now you would have got the idea that how you can do function calls even if I'm just doing cache the song recommendation is not working but the rest of the two works we have to see how we can get uh the other outputs but yeah this is fantastic now this is what I wanted to create guys in this video how you can write your functions and how you can invoke that uh functions uh using an open source large language models based function calling ability and I will give the code in uh the description the get through gith repository and also I can give you a gist as well not a big not a lengthy code and I hope now you understood how you can do function calling uh using open source large language model I can see on a800 around 5 to 6 seconds time which is fantastic the we once we remove the the cash okay it took at least double of time okay but yeah uh let me try it out tell me and uh tell me a vulgar joke I vulgar joke I want to see ah I want to see one thing which is if it has that uh what I'm going to see here is that if it's able to deny the answer and when I say vulgar or any explic you can see wow fantastic now when we CH so right you can see the output also we are getting so now we started getting the output for all of the three guys which is why did the tomato turn red because it saw the salad dressing the most used joke uh when it comes to a vulgar joke right it's uh that's that's fantastic song recommendation you can see joner inspiration and it says say Eye Of The Tiger by Survivor is it a song I mean I don't know ey of the tiger oh wow Eye of the Tiger that's a song you can see on here on YouTube Spotify and things like that and book recommendation interest technology recommended book The innovators dilemma by Clayton m Christensen wow fantastic I loved this response this time all of our functions are working it's able to identify the prompt classify it on prompt level and then invoke the right function now imagine if you have your internal rag you have your external chat tools plugins you have different apis and you have written different functions based on the user's input let these handle you don't have to you know do a prompt classifier you know you have in number of samples and then you classify prompts using logistic regression or deep neural networks and things like that right let this handle and this is fantastic I loved it and let me know what you do with this if you are extending this uh and the credit goes to open harmers I have taken a bit of quote from open hermer blog post or their GitHub uh but I had made a few of changes on the uh things that I've WR but credit to them as well credit for creating this uh open harus version of the M model and uh that's it guys if uh if you like the video please hit the like icon and if you haven't subscribed the channel yet please do subscribe the channel uh if you have any uh thoughts or feedbacks or any queries related to this or any other videos of mine please uh comment that in the comment box and you can also reach out to me through my social media channels please find those details on Channel about us and the uh Channel Banner that's all for this video guys thank you so much for watching see you in the next one

Channel: AI Anytime

Views: 5,944

Keywords: ai anytime, AI Anytime, generative ai, gen ai, LLM, RAG, AI chatbot, chatbots, python, openai, tech, coding, machine learning, ML, NLP, deep learning, computer vision, chatgpt, gemini, google, meta ai, langchain, llama index, vector database, function calling, mistral ai, mistral 7b, mixtral 8x7b, mergekit, merge LLM, gemini ultra, function call, openai function call, function calling using open source llm, hugging face, function calling open source, function calling mistral, llmware

Id: MQmfSBdIfno

Length: 25min 50sec (1550 seconds)

Published: Sun Feb 11 2024