AutoGen + Function Calling + Open Source LLMs, Here is How

hello and welcome to Yu lab continuing our deep dive into the integration between open source llms and auto Jane this new tutorial will show you the useful and popular feature function calling within the autogen framework that significantly extends the capability of Agents using open source language models open source lrms shine in their ability to process and generate humanlike text at an affordable cost from local deployment or cheap API inference however when it comes to accessing real-time data or performing a fixed process to get an answer to a mass problem they hit a snack the introduction of function calling of open a GPT models drastically extends GPT 3.5 and GPT 4 capabilities by enabling them to execute external API course which in turn allows access to realtime information and decision making with contemporary data inputs to compete with GPT models on this feature some works are ongoing pushing the Performance Edge of function calling in open source language models including prompting strategies like the instance in L Smith and fine-tuning like what fireworks AI did in this tutorial I'm going to keep developing multiple agents LM applications in the autogen framework and using decent open source language models with function Calles to see whether they can generate and execute a function task like calculation for currency exchange I assume that you have already played with auto Jen for a while to create your own ELD empowered Mount agent conversation app with auto J and its predefined conversible agents and patterns developers can easily create an AI chat group crowded with different roles to handle complex generation tasks however Auto Jan's low-level framework only supports open a API to use open source language models we should make their inferences compatible with the messaging to open a API to do that there are a couple of tools that can help one type of tool is like VM and which construct the model influence locally with GPU support and another type using API inference like fireworks AI without local computational resource but with a little token cost in my last video I showed integrate fireworks AI compatibility API into Auto J aligning with open AI apis no need for a dedicated Handler there are five steps to do that step one input packages with pipe install P Auto J step two Define find LM config setting the model from fireworks AI in order to use open AI compatible API you have to adjust a base URL in the in the open a object and add your fireworks API key then set up the the environment variable oi config list in step four you have to construct at least two agents assistant agent and a mass user proxy agent the last step is to initiate a conversation by promting a mass problem for a generation the q1 72 chat model exhibits impressive generation quality a lower cost and comparable inference speed to GPT models fireworks AI introduces a function coding model enabling AI to access apis for realtime data and adaptive agent actions challenge es like intent detection and data formatting are addressed by their fine tuned code Lama 34 billion model which much outperforms prompt engineered function calling Solutions their released evaluation proves its accuracy adaptability in multi-turn scenarios and super intent understanding compared to gp4 the model name is FW function core 34 B v0 and you can try it in the playground today we are going to implement the function coding with this model to see whether the open source models can perform well in generating confirmed function calls in Json format please note that the full support of open AI function calling not the function calling assistant started from Auto Chain version 0.2.3 by providing decent decorators that wrap the function definition into Json let's see how to call it the first step you should upgrade your autogen package if it's older than 0.2.3 then you should create the LM config object we will create this object including what model to be used in this case we use the model from fireworks AI platform make sure you input the entire P name for the model the correct fireworks API key from your account and the base URL for the fireworks endpoint step three as usual you should create at least two agents the creation of agents in this example is quite simple because we only Define an LM powered agent chat Boot and a user agent user proxy to execute the code from generated Json formatted function course here we don't have to write complex system message to the chat boot to guide it function parameter generation format you only tell it to use functions to provide the answer step four here comes the most critical step to Define what the functions and the parameters that we want the language model to call this example firstly defines a help function exchange rate to return the exchange rate between US Dollars and Euro it's a help function so it will not appear in the function calling generation now we use penic to modeling an object with currency and amount attributes to the class currency it utilizes type annotations and field descriptions for validation now comes the newly released decorators of aoj here defines currency calculator function decorated with user proxy C reest for execution and chat boot reest for lrm for a currency exchange calculator the function takes a base currency with an amount and symbol and a quot currency symbol it calculates the exchange rate between the base and C currencies and then Returns the converted amount with the quote currency symbol by adding the two decorators the the chat agent knows how to generate calls to respond questions and the agent user proxy knows how to pass the function object to execution there will be a new section tools added to the LM config after successful definition of currency calculator which shows a proper Json format defining the function we want and following open AI function coding structure now we should do some modif ification in the source code of autojen to make sure the prompting template in autojen are fully aligned with this language model to now the code works fine with GPT models but unfortunately does not work well with FW function call 34b v0 model by throwing an error on name parameters in the message obviously it comes from the incompatibility of this open Source model with open AI models but don't worry I will show you how to make a workr for it in a very simple way firstly let's define a small function that deletes the name field from the tool core ID dict move this function to the top of the autogen source file conversible agent. Pi for a global definition if you don't know where is the source file of autogen in your machine type the pipe command to find its location pipe show P Auto J insert the D name function right ahead of this line of response generation under the function definition def generate o AI reply okay now we can start the conversation make sure you have successfully modified the source code and save then you can prompt the agents by input the message how much is 11223 in US dollars with one shot the chatut agent was able to generate the correct function currency calculator with proper parameters base object with eura for currency and 11223 for amount and quot currency USD after that the user proxy agent executed the function and printed the result of 12345 USD the chatut generated text to the user that 11233 is approximately 12345 us and then terminate the conversation in conclusion using the function calling feature with open source language models in Auto J is possible when the model is specifically trained for handle function calling tasks and the message format of the prompt matches the model's definition if not you should make a modification on this messaging format to make it aligned that's all for today for the tutorial and source code you can find the link in the description below don't forget to like subscribe and hit the notification Bell keep innovating and I'll catch you in the next one

Channel: Yeyu Lab

Views: 2,993

Keywords: AutoGen, Chatbot, OpenAI, GPT-4, Python, Open Source, LLM

Id: UIBerUGqHjc

Length: 10min 58sec (658 seconds)

Published: Fri Jan 12 2024