The future of AI agents is WebAssembly (get started now)

hey welcome back so we are going to have a lot of fun in this video we are going to bring together the power of large language models agents function call retrieval augmented generation and web assembly that's right we are going to dynamically load a web assembly function we're going to find that function using retrieval augmented generation and then we're going to have our AI agent execute that function using function calling and if you're wondering why I think web assembly is going to be become the defao way of doing function calling and tooling within Ai and the reason is it's secure by default I can dynamically load a web assembly module I can access the functions and by default it is pretty secure it operates within a Sandbox environment it doesn't have access to things like networking doesn't have default access to the file system these are things you need to give permission for web assembly to use which is inherently safer than executing a native function such as Ruster or python within your local machine and I think the possibilities become endless and not only that web assembly is a linga franer tooling language so I can write my code natively in something like rust or I can use something like assembly script or C and then I can compile down to web assembly and have it execute in that environment so we take away the complexity of which language you're going to execute and as I said much much safer than running things like Python scripts so to get started the first thing we're going to do is we're going to create ourselves a nice little simple AI agent I'm going to just create a folder called hello agent we'll just CD into that and the framework that I'm going to use today is llama index I know in other videos I've used Lang chain but because I'm doing retrieval augmented generation towards the end of the video I think it's easier to use llama index in this case so I'm just going to do a pip install at llama index and that will get that installed on my machine now that's installed I'm just going to bring up visual studio code and I'm going to create myself a new directory called tools which is where I'm going to Define some default tools to begin with and then we'll switch that across the web assembly later on in the video so the first tool that we're going to create we'll just add two numbers together we'll call it add tool and I will put that in my tools directory to Define that tool and llama index I just need to import llama index uh. core. tools and because we're going to be doing function colon I'm just going to do import function uh tool and then after that I just need to Define my function as normal so I will just say def ad uh we're going to have a parameter called a we'll have a parameter called B they are typ integer and we are going to return ourselves an integer there I'm going to be I'm going to give it a nice little description which says add two integers and return the result and if you saw my other video on how to create react agents you will know that this description is vitally important because actually that is what the tooling Frameworks are going to use to be able to understand what this function is actually doing there and then of course I'm just going to return a plus b now that I've got my tool defined I just need to call uh now that I've created my ad function I just need to Define my tool so I'm going to create something called add tool and then I'm going to set it equal to function tool and then I'm going to do a from defaults and then I'm going to pass in the function so it's going to be FN equals add and what that's going to do is wrap up a nice little function calling procedure that llama index will use uh when we Define our agent now that I've defined that I'm going to create one more tool and it's going to be called at multiply so we'll just copy and paste this we'll call it multiply and then with the multiply tool I'm going to do is change the word add from to multiply we'll change the pantic description there as well and then we will just do a multiply by B and change the function name so we now have two tools for easiness sake I'm going to use open AI today I'm going to use specifically the GPT 4 only model uh so that means I'm going to do a pip install llama index llms open AI in a future video I will do this with something like mistrial because that now supports function calling but for this video I'm just going to use GPT 40 um now that it's installed I am just going to create myself a nice little uh M file and then I'm going to put in open a AI API key and then I'm going to put in my API key within there so if you want to know where you can get your API key from you just need to bring up a browser so I'll bring up Brave we go to platform. openen ai.com and then I can just click click on API keys and I can create new secret key put in the name there and then all I'm going to do is copy that uh within there now that I've pasted in my open AI API key I can now create my agent so I'm going to create a file called agent. piy and the first thing I'm going to need to do is uh actually load those environment videos so I'm going to use M uh and I'm just going to do a load uh m in there the next Imports that I need to have there is actually the open aai agents there so I'm just going to do from llama uh index and I'm going to specifically have agent and open aai and then I'm going to import uh open AI agent and then obviously I am going to need uh from Lama index again and this time I'm going to have llms uh open Ai and I'm going to import open AI so that allows me to use an open AI agent and I'm going to be using the open AI llm so I can use GPT 40 next thing I need to import is the tools that we just built a second ago so I'm going to do from tools and do add tool and then I'm going to import uh add tool and then I'm going to do from tools uh multiply tool and we will import a multiply tool and as you can see I've got a little bit of issue with multiply tools not access so if I just come back into multiply tool I just need to change that add tool to multiply let's just save that and come back into our agent okay so now that I've got my imports in place I just need to load those environment variables I'm going to do that by doing a load. m I need an llm so I'm going to say llm equals open Ai and I'm going to pass the model is going to be equal to GPT uh for O next thing I need is an agent so I'm just going to say agent equals open AI agent I'm going to do that from tools and then I'm going to pass in the list of tools that I want to uh include so in this case I'm going to have my multiply tool and I'm going to have my ADD tool I'm also going to need to specify an llm so I'm just going to say llm is going to be equal to llm and as you can see that is pt40 specified there and then I'm going to switch on verbose so you can just see what's happening on underneath the hood and now that I've got my agent defined we can actually just ask a question over it so I am just going to say agent. chat and the query I'm going to have is what's uh 25 + 5 uh multiply by 2 and then we will just print out the response the reason I'm picking that as a query is what I want the agent to be able to do is recognize that it needs to do an addition and then it is going to use the add tool that we defined and then I want it to recognize that it's going to use a multiplication and therefore it's going to use the multiply tool so if we just save that for a second and I do a python agent. Pi you can see verbosity added user message the memory what 25 + 5 multip by two so the next part that you can see there is going to call a function and it's going to call the add function so it recognizes that it needs to add 25 and five together it's recognize the parameters A and B from our Tool uh and then it's added them and got the output 30 and then once it's done that it recognizes it needs to do the multiply by Tool and knows what the arguments there a and b and it's pass in 30 which is the output of the ad passed into two and then it's came back with 60 and it says the result is 60 so it's used two tools and the agent is came back with a pretty good answer which is awesome and that is function calling but that's not going to be enough for us because that's just using standard python tools and as I said we are going to use web assemblies so to do that we are going to use a tool called assembly script and assembly script is a really simple language kind of like typescript which will generate uh web assembly now in all honesty you could use other tools like C you could use rust and you can use whatever tool you want but I find assembly script is pretty simple and it's quite reflective of the web assembly that I want to generate so that's why we are going to use that today I have a whole video on assembly script I have a ton of videos on web assembly so feel free to go and check that out where I go into deep deep details of how uh web assembly works but for just now we're going to make this super super simple so I'm just going to create a new directory called uh hello assembly script uh we'll CD into that I'm just going to do an mpm in it uh that will just uh create me a nice default uh node.js project and what you will see is I have a package Json off the back of that and now what I need to do is install assembly script so I'm going to do an mpm install I'm going to install it as a Dev tool and I'm going to use the package assembly script so we'll just put assembly uh script and you can see that has now uh created nice little package Json and there assembly script is now a Dev dependency within that what I want to do now is just uh scaffold up a really simple assembly script project so I will just do MPX as in it and then Dot and that will now uh essentially create me a nice little uh assembly script default project and you can see there now if I look at this I've got an index file I've got uh some out of the box assembly scripts so here is an ad function which takes two numbers adds them together uh and then I can do the same for multiply Etc so if I want to compile this piece of assembly script this add function into web assembly then I can simply run uh within uh assembly script mpm run as build and then it will take that nice little uh index Ts that we see here and then if I go into the build folder you will see it's now created me a nice little release. wasm it's also created me a debug. WM awm file is a web assembly file it has actually also created a thing called wat and that which stands for web assembly text format web assembly text format essentially allows you to uh have a sort of lisp type language which uh defines the function and then what happens here is you could then take a web assembly text format file and then compile that down into to web assembly or a wasm file and as you can kind of see here this is the web assembly text format representation of our ad function so you see export add and then uh within there you can kind of see a function definition uh it's got two parameters parameter zero parameter 1 both of in32 and then you see here it does a local. get zero so it takes that first parameter so it just pulls that as a stack machine so we'll actually push that onto the stack uh and then it'll take the second parameter push that onto the stack so we have two parameters on the stack and then it will just add uh two items on the stat together so you know in this case parameter one and parameter two and then it will perform the calculation the addition and then it will uh push the result back onto the stack when you come out there and that will give you the result that is really what's going on in this web assembly text format I have a very detailed video on how all of this works go feel free to check all of that out but now if I just clear this and then I do an mpm start and then click on Local Host 3 000 you can see this number here three so it's actually done a calculation of some sort so let's just come back into VSS code for a second you can see I've got an index.html where it's adding two numbers one and two if I change that to 10 and 20 hit save and then we come back into uh our uh browser and I refresh you can see it comes back with 30 and what's actually happening underneath the hood here and if we just look under the hood of this release J you can see all it's doing is loading up that release. wasm is actually exporting the ad function that we created within there just as a normal JavaScript function it can be called as normal uh and then essentially it's going to compile and allow you to execute it so there's nothing more complicated than that if I wanted to I could extend the functionality so I could take this add function and uh I could actually create ourselves a nice little multiply as well so we'll just uh put in multiply and then rather than doing a plus b we'll uh return a times B and then if I come into index.html I can import multiply as well and then rather than calling the add here I can do the multiply which will now uh do 10 multip by 20 of course what I'm going to have to do is do an mpm run as build one more time and then I can do an mpm start again uh we can then click on Local Host 3000 and you can see it's done the calculation so the answer there is 200 and then if I come back in here you see it's multiply 10 and 20 so it's fairly simple to uh do things if I wanted to I could actually even manually create the web assembly so I'm just going to copy the release. wat and I'm going to change the name to math uhw and then in the same way as I've got this add and this multiply what I can do here is I can add a new function within there uh in this case I'm going to call it square and then rather than having in uh two par parameters I'm just going to have one parameter uh which is parameter zero and then rather than kind of remember the uh the pushing onto the stack so I push the first parameter onto the stack and then I'm going to push the second parameter and then I'm going to do the multiplication well actually I've only got one parameter because it's a square function so all I need to do now is just uh push onto the stack the parameter twice and then I can do the multiplication as I did before and then if I want to I just need to do another uh export here so in the same ways I've got an ADD and a multiply I just need to add a square now and then I can put Square here so this now gives me my manual web assembly text format but I need to convert this into a web assembly module a wasm file so to compile that I will need a tool called Web assembly binary toolkit or webbit and all I need to do for that is just do a brew install uh webbit on my Mac and then that will download that for me and of course if you're not running a Mac or you want to install it manually you can just go look up web it on GitHub you can pull it down locally and then you can make and build it on your local machine uh now that that's installed I have access to a tool on this called wat towm which takes a web assembly Tech format file and converts it into web assembly module format so I am going to uh take this from my build and I'm going to take the math.at I'm going to uh have a minus o which is I'm going to specify my output file name and in this case I'm going to build something called math. Wat W and if I come back into Visual Studio code you going to see there is this math. WM file that uh has been created now all I need to do is if I go into my release JS rather than loading release. wasm what I could do is uh change that to math. wasm and now if I do my mpm start one more time as you see it's still loading uh 200 there because it's uh still accessing the multiply function it's the same function that I used earlier earlier uh but of course that is now in my math. wasm as opposed to the release. wasm so you know that that's actually worked but if I want to test out my new Square function all I need to do is going into index.html change this to a square and I'll just add square into the list of uh functions that I am uh importing and now I just need to change rather than passing two parameters 10 and 20 I will just pass in uh let's say five and a square of that should be 25 and then finally I just need to come back into release JS and then you see the export here export add and multiply I just need to add Square to this list as well and now I can do my mpm start uh click on Local Host here and as you see 25 has now appeared and of course if I come back into VSS code uh come back into my index and if I want to put square of let's say seven I can just come back into my browser one more time and do a refresh and I've got 49 so now not only can I create web assembly using something like assembly script I can also create it manually using web assembly text format but of course you can always use something like rust or uh C if you want to create web assembly as well so now that we understand that one of the things I need to be able to do is load my web assembly using python so to do that I'm going to come back into my hello agent folder and I'm going to install a thing called wasm time which is essentially a run time for web assembly that you can run on the back end so if I just pip install WM time there that should be installed so now what I'm going to do is I'm going to take this math. W that I created earlier and I'm going to copy this into my uh tools folder on uh my hello agent project so I'm just going to paste that in here so it say do you want to pass that now my math.at is now uh part of my tools forwarder and hello agent so now what I'm going to do is I'm just going to call that web assembly that we created from python so we will uh create a new file I'm just going to call it main.py for just now so we don't mess up our existing agent so I'm just going to import now from WM time a few things where specifically it's going to be uh store module and instance and they are going to allow me to load up that web assembly text format file uh manually so I'm just going to say store is equal to store and then I am now going to have module is equal to uh module do from file and store. engine don't worry about this too much here and then I'm going to have math. wat um now remember I put that in my uh tools folder so I am just going to need to say it is in tools at math.at there and and then I need an instance of this again this is very similar to the JavaScript code that was created by assembly script and we're just going to equal that to instance uh and I'm going to pass in the store the module and we just need to have an empty Imports in there and then finally I need to export out my Square function so I am just going to say equals uh instance do exports uh I'm going to pass in the store and then I'm going to pass in the function that I want to access which in this case is square and then just to make sure it works we're just going to print it out and make sure it comes back with some sort of answer so we're going to say Square uh of five and that should equal and we'll put uh percent D in here so now we've done that if I H just clear this and I do python main. Pi you can see it's square of five is equal to 25 and then I can change this if I say uh square of three and we pass in three here we should now get nine so the cool thing about this is we are now using python to call our web assembly module now what specifically is happening under the hood is actually loading into square. wat file so it's using the web assembly text format it's then doing the compilation to create the instance once it's got the instance it's created the WM file um it's exporting out that square function and then it's allowing me to execute and call it just like any normal function so that's pretty cool so far we've got python to able to call our web assembly functions but actually I want my agents to be able to do that now as well so we're going to come back into uh vs code for a second we're going to come into tools here and I'm going to create a new tool and this time I'm going to call it a square Tool uh so we'll just call a square tool. pi and then this time rather than doing this multiply here what we are going to do is we are going to replace that with our uh Square function from web assembly so I'm just going to delete this I'm going to set this equal to square and then I'm going to specifically copy all this from main. PI and then I'm going to bring this into my Square tool now there's a couple of changes I'm going to need to make now you see here where it does print square equals 3 I'm calling square but I'm passing in my store here now I don't want to do that I want to be able to make normal calls so to make this work I'm going to change this form rather than just passing in the name Square what I'm specifically going to do is I'm going to create a nice little Lambda here and I'm going to set the parameter a of the Lambda to accept in uh score uh and we're going to call change this to scare function just so we can make this a little bit uh you know separated and now I can pass in store and I'll pass in a here so that is now rather than calling this Square function directly uh it is now going to specifically wrap it in a Lambda so I can just call Square normally now also to make this work I need to give it a name so um because the the function is a Lambda at this point so it doesn't have a name so I'm just going to give it the name squore Tool and now I can get rid of this print statement because I don't need that anymore so I'm just going to give it the name Square uncore uh tool and then that means that the agent framework is going to be able to see the square tool and use that uh when it's looking for it uh I don't need this print statement anymore so I'm going to get rid of this now if you remember in our tool definition you see this kind of pantic descript where we say add two integers which helps the agent understand what the function is doing I kind of need to do the same thing in my uh Square tool so what I'm going to do here is I'm also going to set a description and I'm going to set that equal to calculates and Returns the square of an integer now obviously I want to do the same for the parameters as well so I'm just going to create a new class called uh Square params uh I'm going to inherit that from a base model now if you remember from my Square function my parameter name was int uh and then what I need to now do is be able to provide a description for that parameter because I haven't defined that within my web assembly module so to do that I'm just going to use field I'm going to pass in uh the parameter and then I'm just going to add a description so it'll take all the properties that it's got today already We'll add a new property called description and I'm just going to say the integer uh to be squared now as you see I've got a couple of squigglies in this case so I'm just going to uh add from pantic uh base model and I need to do the same uh for uh field and therefore I've now correctly imported base model in field from pantic so that looks pretty good so now my parameters that I have defined my web assembly module I've added a description to that and I've obviously added a name and description to the function now if I save that if we come back into my agent. py now and then I just add a new tool so we'll say from Tool tools uh do Square tool and we will import uh Square uh tool you can see that's came up with a square tools not access there so I just need to change this from multiply tool to square Tool uh as I've forgotten to do earlier and now I can just add a square Tool uh to my tool list here and then rather than asking what's 25 + 5 multip by two I can just get rid of this and I can say uh sare 10 and now rather than running a remain. pi if I do python agent. Pi you can see it's came back with the square of 10 is 100 so I've now got my agents calling web assembly this is pretty cool so far right but I think I can go a lot further than this so I think one of the things that we can now do is actually we could dynamically load tools so I'm going to uh create a new file called tools. Json and what we'll do within tools. Json is actually dynamically Define what the tool is so in this case we're going to have a function called Square tool you see the description is going to be square as an integer the uh Wason file in this case is going to be called math. wat the exported function we're going to use is square and then here's a parameter which is a type integer and the integer to B squ why is this important because that means I can dynamically load my web assembly I no longer need to have that hardcoded anywhere I can just dynamically load my functions so you can start to imagine that if we implement this you could have a massive tool directory kind of similar to kind of mpm where you list all the tools that you need it's got a description of the parameters and then you can just pass in whichever tool that you can use do some function calling on that and then it's just going to work so um so to make this work one of the things that we will now need to do is we're going to need to take that square tool and we need to modify it so it's a little bit more Dynamic so we're going to call this dynamic IC uh tool. Pi uh and now some of the things that we need to get rid of is that square parameters isn't going to be much use to us because that's really hardcoded to the function so we want to be able to dynamically kind of load that the store is going to remain the same um and now one of the things that I want to be able to do is uh be able to get a tool by a name so I'm going to create a new function called uh get uh Tool uh by name and what we will do is pass in the name name of the tool that we want to load so we're going to say name is of type stir and then I'm going to return a function tool so one of the things that we are going to want to be able to do is I want to Loop through the tool section and then load every single tool that's within that so to do that I just need to create a nice little for Loop and I'm just going to Loop through all the uh tools in the config section there now I haven't already loaded my Json so I will just add that in here so I will just say uh with open um we will say tools do Json um we'll say as F and then I'm just going to say config is equal to json. load F there make that nice and easy I'm going to have to import my Json at the top here so we'll do an import uh Json so now that we're loing through the tool sections if it's uh the tool config section if the name is equal to the name that we passed in we're good uh now I can just bring this uh within scope Here and Now rather than uh manually specifying the W file I can can just say uh tool config and uh we can pass in the name of the wasm file and that means it will pull uh the wasm file from here and remember this is in the tools folder so we just need to do uh back/ forward tools math.at so it's going to load us from here uh so we've got that specified the instant doesn't change here but rather than calling it square. wasm we'll call it wasam FN um and again similar here where we do an inst exposes we don't want to pass in the uh the function hardcoded a square so again all I'm going to do is have a tool config and we are going to look for the exported uh function uh name and again if you look here exported function should be equal to square so this is just happening dynamically now like before I need a wrapper um so I can get rid of that store thing similar to the Lambda that I've got down here so I'm just going to call that uh tool uncore FN um I'm going to pass it a uh and it's going to be of type integer and then that's just going to return oasm FN so this function that we've got over here and we're going to pass in the store and we're going to pass in parameter a and that wraps it up so that we're not having to pass the store from the agent perspective here now rather than doing that kind of class for the parameters I actually need to dynamically load my parameters here I'm just going to have a nice little parameters model here so I've got Prime name and then I'm just going to Loop through the tool config in this pams and then I'm just going to create a nice little dictionary of Pam so all it's really doing is looping through this and then loading it into my pams variable here and then I've got this thing here called Dynamic pams and then I'm going to call create no Model H passing the name in the parameters and again if I just click on a squigglies here and add from pantic H create model that is now part of it so that handles the descriptions of the parameters and then now rather than setting Square tool I just need to return and function tool I can now set my FN to just being tool FN rather than having that Lambda function I don't need to worry about setting the name because uh it's now got a name there and description here I just need to set to toolor config uh do get going to pass in uh description and of course if I can't get it I'll just default that to uh an empty string there so I now have a dynamic tool and if I come back into my agent we can now go from tools do Dynamic tool import uh get tool by name I can now do a look up of the tool so I can say matched tool is equal to get tool by name and we will pass in a square tool now in my list I'm going to get rid of all of this from the list I'm just going to put Match tool and now we'll just say Square five and now if I run python agent. py one more time it will now do a lookup um and it works sare square of 5 is 25 so I've passed in my name I've dynamically loaded it there's nothing hardcodes I am dynamically loading my web assembly functions now if you think about that for a second that is pretty wicked right I have in here a nice little Json file that describes my tool Square tool squares an integer it determines where the wat file is math.at you can think about this I could load this from a database I could load this up from a big uh web assembly uh directory in the sky um I say what my exported function is square it will then compile it run it I say what the parameters are with the description and then the model is able to understanding through the magic or function calling how to interact with web assembly this is just gamechanging stuff you can imagine whole massive directories of tools up on the internet that we can simply just use and I think we are going to see the sort of mpm of uh web assembly for large language models coming soon and I think that's going to be one of the next big projects that's one of my big predictions here um now that that's working of course and it has all sandbox and it's safe so it's very very cool stuff uh but of course I am manually defining my tools at the moment you see I'm doing match tool get tool by name if I wanted to be super cool what I could do is rag search this right so if I had a big Vector database with a list of all the tools I could just do a rag search on this and then find the tool that I want to use rather than specifying that so that is what we are going to do now so I am just going to come back in here I'm going to create a new folder I'm going to call it data um I'm going to be super lazy I could put in the Json file and Etc but I'm just going to do this really quickly so I'm going to create a a file called tools. text and then all I'm going to do here is I'm going to have uh text separated lines which is tool name Square tool add tool multiply tool description squares an integer adds two integers together Returns the result multiplies in reality I want to be able use uh the Json uh capabilities of llama index but for Speed sake I'm just going to list all the tools here um and then what we can now do is look that up so if I come back into agent and I just add in a couple more Imports so I'm just going to do at llama index uh. core and I'm going to specifically import a vector store index and I'm going to uh take a simple directory reader so all I'm going to do is read whatever files in that folder I.E that text format and then I'm just going to uh load documents I'm going to say documents is equal to single directory reader uh I'm going to pass in the data folder name and then I'm just going to ask it to load whatever data is in that file now specifically as I said I should be passing in a file name in the future we could have an entire uh Vector database up in the cloud but I'm just reading files very very quickly to demonstrate this point and then I need to index it so I'm going to say uh Vector store index uh Dot from documents I'm going to pass in the document that we've 've got here now I'm going to move my query up here so rather than saying Square Five like this I am going to Let's uh put uh Square uh seven and then I'm going to pass query down here and what we are specifically going to do now is we are going to pass this query into our Vector database and ask it to look up which function we need to use so to do that we need to be able to query those documents that we just loaded and to be able to query the documents that we just loaded I need a query engine so I'm just going to specify query engine is equal to index which is our Vector store index that we just created of all the documents and then I am just going to say as query engine call at query engine uh do query and I'm going to ask it give me a list of tools that I can use to satisfy this query I'm going to pass in the query that we just defined here which in this case is square 7 and I'm telling it to just return the tool names and with that it is going to look at that uh tools. text here that we have it's going to take the the function Square 7 and it's going to look through this list and work out which tools will be able to satisfy that doing the vector search and using retrieval augmented generation the llm is going to help uh figure out what that final list of tools is going to be so I should get a list of tools so now all I need to do is do a tool name is equal to stir response now of course if I wanted to return multiple tools then I would need to do a little bit of a kind of loop through that and then uh get a list of the tools and iterate through that and then look up each tool individually uh but for easiness sake I'm just going to query on a single tool and now rather than calling get tool by name and then passing in square tool I can just pass in tool name here and then below that nothing else changes so so now if I run python agent one more time you are going to see rather than me manually specifying this it's actually taken uh the query Square 7 it's looked up our database and then it's calling to FN dynamically and it's called a square root of function and now if I wanted to I could rather than calling Square I could just pass in something different I could say 25 + 7 and now I could run my agent one more time this will error because I've not specified the ad tool yet so we get an error there cuz it couldn't find a tool tool there that's fine we will come back into my tools. Json I though I've defined it in my tools text I haven't defined it in my tools Jason that's okay I'm just going to copy uh the uh Square tool there we'll put a comment in here and now I'm going to add add tool and we'll say uh ads to integers remember I had already defined this in my math. w there we're going to say the exported function is ADD and in this case I need uh two parameters so I'm just going to uh put in B here and we're going to say the integer to be at uh the first integer to be added the second integer to be added so now all I need to do is change this from Square 3 to 25 + 7 if I just run this here you are going to see it's going to have a bit of a failure and the reason is is I've changed the number of parameters you see it's got too many parameters given one expected two so all need to do now is if I come back into uh my uh parameters for a second so if I go to Dynamic uh tool now if I just extend this to have a and b and of course I need to pass through b here as well and then I run this again you can see it's now passing through 25 and 7 25 + 7 = 32 so there's probably a little bit of more work here that it would need to happen to be able to uh uh have a dynamic wrapper rather than it being hardcoded to the parameters but you kind of get the point so I'll probably update that for a future video but all I would probably need to do is pass through quarg you know look at my sort of parameter name here pass in the dynamic parameters for the um the the tool function and then we could have something that's completely Dynamic um and then again if I come back to here change this from being 25 uh plus 7even put this to 25 multiply by seven or even say 25 multiplied uh by S we run this one more time as you see it comes back with an error so again all I need to do is extend this in tools here so rather than having an add tool I just add in my multiply tool now we'll just paste this in here uh we'll say multiply tool multiplies to integers it's using the same math. wack cuz it's already in there change it from add to multiply and we'll say uh multiply multiply we'll clear this run this one more time and it should be able to multiply the two uh numbers together and then you see 25 multiplied by 7 equal 175 and of course if I come back to this agent uh added to 7 save this and it's came back with 32 so now I have a completely Dynamic uh tools. Json file here you can imagine as I said before web assembly Central you know up in the cloud where all the functions are defined you just pull in a web assembly file that you need of course there's a little bit of work that I would need to do on this Tool uh function here but this is super secure sandbox Etc I really think this is going to be the way forward so I think this sort of rag directory of tools up on the cloud uh where you can dynamically load what you need and then have the agents be able to call the tools and have this massage uh function tool directory again there's a little bit of work that I would need to do on that kind of wrapper there but again I'll maybe do that in a future video now of course I've shown you this with open AI but of course this is equally going to work with mistol and the new function uh tooling piece so again I'll probably do another video on this but anyway I hope you're as excited as I am I genuinely think that this is probably going to be the way forward I think web assembly is going to be the function calling uh mechanism of AI I think it's going to be the standard way to do tools and this idea of a big rag sort of mpm style database where you can just go look up whatever web assembly text files that you need or Wason files pull that in and then be able to execute those tools um and that's probably going to change the way libraries are done in programming languages forever anyway I hope this video has been useful I'm super excited about this and I'll catch you on the next video

Channel: Chris Hay

Views: 1,544

Keywords: chris hay, chrishayuk, gpt-4o, llamaindex tutorial, llamaindex, ai agents, llamaindex agent tutorial, llamaindex agent, webassembly, function calling, retrieval augmented generation, rag, assemblyscript, openai

Id: KwF_ZB29ZB0

Length: 39min 51sec (2391 seconds)

Published: Sun Jun 09 2024