Create Complex Research Analysis with AI Agents using SLIM models on CPU with LLMWare

hi everyone welcome to today's video we have got an awesome topic an awesome demo that we're going to be looking at and what we're really going to be focused on is multi-step research and analytics how do you go beyond simple question answering or simple summarization to really do complex multi-step analytics preparing more complex types of deliverables using LL mware and using slim models and doing all of it s soe running on a CPU so let's go ahead and let's dive in first what what is a slim a slim is a structured language instruction model these are small specialized function calling llms that have been designed to provide structured outputs python dictionaries Json and SQL all of which can be handled programmatically and then seamlessly integrated into a multi-step process so the way that we like to think about this is where people are looking to really deploy generative AI for True knowledge-based Automation in the Enterprise it's not usually a single step in fact it's multiple steps in fact with multiple specialized skills that need to be brought to bear often times this can be represented as a linear pipeline but a linear pipeline that has some usual decisional in it of something comes in perhaps we want to extract some key information from that as a first pass maybe we want to use that information to do some type of lookup in a knowledge base but then typically we want to do some form of classification perhaps it's a specialized model or tool that runs some type of classification activity and based on that branches often send it to other specialized models that are going off and doing additional processing based on what we've learned from that classification at the end of this process ultimately it all needs to come together and then be delivered and connected into a business as usual enterprise process and underpinning all of it needs to be an AI ready knowledge base that AI ready knowledge base needs to consist of you know documents and files and unstructured information that's ingested parsed extract and text chunked at scale into some form of a text collection index that's then vectorized by running it through an embedding model typically an embedding model that's been fine-tuned and optimized for that domain and that industry put in a vector data store and then ultimately integrated into some type of intersection with SQL table data stores that often times are where most valuable Enterprise information sits so this is the big picture but what we're going to do today as we're thinking about the future and we're thinking about where AI is going think we wanted to take a moment and actually look at one of the most iconic kind of moments that ultimately help to shape the formation of the software industry and that's really the partnership and then subsequent rivalry between Microsoft and IBM the partnership between IBM and Microsoft in the early 80s is what gave us Microsoft as this dominant player in the operating system Esther relationship ultimately became more of a rivalry and a competition that ultimately is what shaped you know the direction and the early days of the software industry so what we're going to do is we're going to take a look at this as a research project first thing that we're going to do is we're going to create a knowledge base of Microsoft materials in this case for the purpose of the demo we're going to keep it pretty simple but we're going to illustrate for you how you can build a very large knowledge base around this and very quickly scale it we're then going to run a fairly basic query against it what we want to do is out of this knowledge base of Microsoft materials is we want to find anywhere that IBM comes up because that's really what we're looking for we're interested in wherever Microsoft and IBM are mentioned in the course of this knowledge base but that isn't really what we're looking for what we want to find then is once we've identified where Microsoft and IBM are both referenced we really want to look at the sentiment analysis of it because it's the Rivalry we want to find any of those passages where there's some sort of negative sentiment because that's probably revealing the Rivalry and the tension between the two parties and then finally if we've identified that it's IBM if we've identified that the sentiment is negative that's actually the research that we want to do and then we want to dive really deep and look at a lot of very specific things around that type of information that's ultimately the deliverable that we're looking to create so what we're going to do we're going to flip over I'm going to show you the code and then we're going to go run the demo really walking through this endtoend multi-step type of research activity show you how easy it is to start deploying these types of things all running locally on a CPU I flipped over to my IDE let's take a couple of minutes and really decompose the code just so you can get a sense of what the code does and how it Maps up the high Lev workflow that we just took a look at on the PowerPoint chart and then we'll go ahead and we'll run the demo we can really see the whole thing in action what this demo uses it it really capitalizes on a new capability in LL mware the llm FX class we're going to skip through the first part relatively quickly we cover that in a lot of other demos and tutorials but Step One is we're going to create an AI ready knowledge Base by parsing text trunking and indexing from you know documents that's actually the process of creating a library where the demo really starts is we're going to run a query of IBM we're going to get all of those results all of the various passages that are in that Library collection that refer to IBM along with all the metadata the pages the sources all that sort of thing and then we're going to instantiate our agent we're going to instantiate it as an instance of the lmfx class we're going to load a whole bunch of tools these are small specialized slim models that have been 4 bit quantized so that we can run and load them really really quickly and use them on a CPU we're then going to pass in our work the work we're going to pass to the agent are those search results and then we're going to run through um an iteration and the iteration is we're going to ask the agent go through that list of work items and run sentiment processing against it keep going until you can't get any more when you can't find anymore then break the process and then we're going to run a follow-up which is looking at all of the results and all of the analysis that have been prepared of that sentiment wherever we find sentiment that was negative those are the work items that we actually want to follow up on we're then going to iterate through that follow-up list and on that we're going to run an analysis of the tags emotions topics named entities and we're going to ask for a brief summary we're going to assemble all of those report by report by report assemble that with the source data the information of the original passage and what was found and then we're going to assemble and deliver all of that as our report hopefully that makes sense now let's go ahead and let's see this code in action Okay so we've created our library it created a bunch of text chunks that process is done and now we're starting to step through we've identified all the passages that had IB you can see we're running this sentiment analysis against it the agent has a journaling capability we can turn it off if you don't want to see this on the screen we've turned it on it's very verbose but it gives a really nice structured Journal of all the work that's being done in terms of all the various inferences and we'll come back and we'll take a look at that once the process is done but you can see it's going through every single passage wherever it's finding the sentiment and then wherever it identifies it is negative those are the passages that we want to come back and do some further analysis of so now we've started to do that secondary analysis looking for the emotions the topics looking for the named entities and running some basic summarization all of this is being done then on the items that we identified where IBM was present and where the sentiment appeared to be negative this is going to run for probably another 30 seconds or so and then once it's done we'll we'll take a look at the end deliverable and then we'll also come back and we'll walk through these logs [Music] so we're making good progress almost done it'll just take another second there we go so we've just finished the analysis and we went through 180 steps we ran 50 different inferences these were different calls to llms we used six different llm models the ones that are listed there all of this was running locally on a CPU machine so from a data privacy point of view and let's go and let's actually look at the output that we were able to produce so that's right here we're showing the selected reports of all the areas where IBM was found and the sentiment was negative and in each of these gives the sentiment it gives the tags the emotions the identification of the topics the key people that were mentioned There's a summary that was provided and then it's really cool we've gathered The Source information as well so we can see where that was and what page that information was on all of this then delivered as a set of reports with all of these structured Keys consistent element by element we can see the commentary we can see the key information this gives us a great basis then for the next step in our analysis of really identifying those places where IBM came up in a negative context and what were some of the contexts around it here this was around a joint effort to develop os2 here IBM didn't develop the processes very well here was a case around the joint development agreement in 1985 here in 1981 it extracted some key information around some of the competitors and their financials and then IBM was still promoting windows but again there was some source of underlying tension you can see the summary reports that were generated so now let's go back and let's go back all the way up to the top of what happened here we did our parsing we created our library we ran our query that identified 20 different text passages involving IBM we then loaded our tools we then started stepping through this analysis for each of those 20 passages representing IBM we were looking for a case where the sentiment came back negative you can see all the color coding gives us a view of the underlying confidence and some of the other choices that were considered by the LM at each generative step each of these then as we're iterating through all of these steps the steps are being counted and aggregated and we're iterating through every single one of the search results and then we've just completed that so it's step 87 we've completed going through iterating through all those 87 steps and now what we want to do is we want to take the passages where the sentiment was evaluated as negative you can see the list here Passage 2 4 5 8 10 and 18 those are the ones that we want to do this deep dive on and create the reports that I just showed you so you can see then we start to go to work we do that deep dive on each of those We Run The analysis of the tags the emotions what the topic was code quality was the topic of this one the people then that were Iden as well as the organizations not surprising as Microsoft and IBM were the two organizations referenced we then go on we ask basic summarization we get that summarization right here and then we go on to the next block you again see the same analysis the tags the emotions the topics the people that were involved so on and so forth iterating through all of those areas where we had identified IBM and a negative sentiment and then as we mentioned before we have our complete report that was automatically generated and again coming back to the code the code is deceptively simple we parse the documents we run our query we create our agent we pass the tools to the agent we then load work to the agent of those search results we iterate through as a first pass all the sentiment analysis we then follow up wherever the sentiment was negative for that follow-up list that subset of work items we want to run through these additional analyses of tags emotions topics NE R and getting brief summary all of those then get collated and organized into the report that we see at the very end so hopefully this brings it to life for you we've run through a pretty complex structured analysis in in a minute or two relatively simple 20 or 30 lines of code to do this type of multifaceted analysis with a clear structured output at the end and we've learned a little bit about the history of the software industry any questions please reach out to us on Discord please check out the example as always on LL mware and please stay tuned we've got a whole bunch more videos really talking about some of these topics of multi-step analysis slim models and how to really do all of them with L mware thank you everybody take care and have a wonderful day

Channel: llmware

Views: 1,513

Keywords:

Id: y4WvwHqRR60

Length: 12min 11sec (731 seconds)

Published: Thu Feb 15 2024