GPT4ALL-LocalDocs: Your Docs in a Chatbot on your computer!? This is private GPT... personal GPT!

hey welcome back to AI mistakes the video we produced last week covered GPT for all and it provides the capability of running a large language model on your own PC device what I didn't know when I made the video a week ago was that this space is moving in an extraordinarily fast pace and late last week just before I kicked out the newsletter which by the way if you haven't signed up for the newsletter check the link below I learned that a new plugin had been created for GPT for all this new plugin is called local docs and it's a pretty amazing piece of work essentially what it allows you to do is not only use a large language model on your own PC but also to index and use files that you have stored locally on that system to search and use for interrogation in working with the large language model so not only do you get a many millions of parameters model that you can run on your local PC without a connection to the internet you can choose a folder on that device and have it use those files in answering questions and queries that you ask with the chat bot now it's not the fastest system but it does work I spent quite a bit of time yesterday testing it out on my own files I threw in several gigabytes of data mostly ebooks and reference information that I've used in my development and professional Information Technology career over the last couple of years and it did come up with some good answers it didn't always come back with the answer that I wanted but I think you'll be surprised at what it's capable of already keep in mind that this technology is still evolving the local docs plug-in that I'm going to show you here is also currently in beta so this is a nascent space that we're going to be working with and learning about I'm going to show you how to install GPT for all and configure it for using your own local documentation and then we'll do a quick search just to kind of try it out and kick the tires I think there's a lot of opportunity here and this is probably not the last video that you'll see for me on GPT for all but this is again a brand new space really new things are happening here this capability is something that required quite a bit of additional effort to enable you had to build your own Vector database using it with a tool called langchang that would basically allow you to extend that large language model none of that is needed here none of what I'm going to show you is very technical and it's a pretty straightforward process it does take a little bit of time once you add and configure local docs it's going to take a bit of time to search the files that you have locally so that you can begin to interrogate them but I I definitely would encourage you to try this out it does work on PCS and Macs and Linux as well there's an installation link on the GPT for all website I'll provide links below the video on how you can access that information let's get started okay here we are with GPT for all loaded up we've updated our binary object GPT for all version 2.46 now the update was released earlier today as you can see on the screen a few release notes they continue to evolve this product at an incredibly rapid Pace I'm participating in the opt-ins for anonymous usage and analytics in hopes that that could help and improve the tool so I'm just going to accept this dialogue and we'll get on with setting up GPT for all now I I have played with this independently I've removed the configuration so that you can have an opportunity to see how it's done here but as you can see along the upper right hand corner here we have a number of options this is the data sharing obviously if you want to be working with data that is on your own system and you are really concerned about the privacy of that data you'll want to be sure that this is toggled to off so that it is not sharing your data back to the team that is developing the product I don't have any private information here so I'm going to leave that toggled on and as you can see they do ask you is this something that you want to enable and tell us your name if you want attribution for the work that you're contributing under the settings icon there's a number of things around temperature in the configuration model you can leave all of these as a default value I'm I'm leaving them exactly as they are what we're most interested in today is the local docs plugin as you can see it's listed as a beta and all we need to do here is fill in some values now I've not had an opportunity to change the document snippet size I'm leaving that as a default and it does have a default value of three document Snippets to process per prompt you know so basically what that means is it's going to look in three specific sources and cite them in the response if they are applicable so I'm going to go to the browse button here it's just going to load up my files I'll reach in here and pick up my local docs AI folder I've loaded some epub files those are ebooks that I've got on AI and machine learning so I'm just going to give my label here AI ml epubs so that I can know what the reference to this is and of course you can add multiple sources I cut down the amount of files I've included in this location in order to be able to quickly show you what that looks like and it still does take a bit of processing time what it's actually doing is it's building a database against those textual bits of information so that it can use them in fulfilling the answer to your prompts so we're just going to let this run for a moment okay it's already finished that was nice and fast really good okay we're going to click out of that and then we have one more step that we need to take we need to actually activate the module and the way that we do that is under this data collection object I'm going to go in here and it's taken the name that I've given it I'm going to check that box and now it's going to begin using those documents to answer the question so the files that I'm using in this case actually come from machine learning neural networks artificial intelligence related content these documents are a couple of years old so this is not cutting edge and it's entirely possible that some of this information is contained already in the data sets that have been loaded but that's okay because what we're going to do is we're going to ask a question and we'll be able to see from the answer that it gives whether or not it is using the documents we've given it in forming the answer so I'm interested in doing development of game AI so I'm just going to go in here and say I would like to get started and using Ai and the development of a game what are some key things I should consider when I begin to use AI in AI game programming what are some practical steps tips that I should use and again this is all in real time so I'm not speeding any of this up because I want you to see that it does take a bit of time to calculate that answer it it's using all of the information is available it is taking advantage of the existing model that is already loaded in GPT for all there is really just the single model that we're using for text generative AI right now but this application is really set up to be able to enable configuration so as new models are released if you are developing your own models you can load those into this tool and select them and as you can see at the top of the screen GPT for all J version 1.3 groovy that is the whole territory in terms of the data the version of the binaries and the model that is available for answering our prompt query so as you can see here it is churning on this work it is not an instantaneous response but the reality is if you're reviewing documents that would require an individual a considerable amount of time to review in order to form an answer even if it does take several minutes for this query to go round trip it's going to be worth this experiment because you can get a highly detailed and well-formed answer based on the documents that you've provided the other thing I do like about this capability is that it does specifically Source the information for you so it will tell you what document it came from and a reference for where in that document it found that information now in this case I'm using the Epub format because it's a highly compressed and efficient format that is used for e-reader documents just to reduce the overhead and make things go as fast as I possibly can in completing this demonstration but as you saw a moment ago in the introduction to the video there is a significant number of document formats that are supported particularly of note is PDF so really if you have a document format problem just get it to a PDF and it will be able to search it now as I'm standing here talking about this what we see on screen now is it's showing that it is processing the answer against the AI ml epubs data source that I gave it so it's enumerating that and it's trying to generate a response using the information that's available that the question I posed is intended for it to find in its source documents a more specific answer than would come from the model in fact the book is actually called practical game AI programming so there should be an answer that comes from that because I have a book that specifically addresses that use of AI and it does have to go through and process the files it does take some time for this to to work through and look here it's given us an answer sure here are some key points you may want to keep in mind as you start working with AI in your game programming understand the basics of AI programming this includes things like data structures algorithms and control flow you'll need a solid understanding of these Concepts before diving into more advanced topics like machine learning or neural networks secondly choose an appropriate programming language for your project depending on the complexity of your game you may want to choose a high level language like c-sharp or python or a low-level language like assembly or rust consider factors such as ease of use and performance when making this decision thirdly familiarize yourself with AI libraries there are many open source libraries available for various programming languages that can help you implement AI in your game some popular options include open AI GPT tensorflow and pytorch you'll need to choose an appropriate Library based on the complexity of your project and your skill level fourthly develop a solid understanding of machine learning this includes things like supervised learning unsupervised learning reinforcement learning and deep learning understanding these Concepts will help you create more advanced AI systems in your game number five experiment with different types of AI there are many different types of AI that can be used in games including rule-based systems decision trees neural networks and even genetic algorithms experimenting with different types of AI can help you gain a better understanding of how they work and which ones might be best suited for your game number six consider the impact on player experience as an AI developer it's important to consider the potential impact that AI systems may have on players experiences in your game this includes things like difficulty levels rewards and even the overall feel of the game make sure you're balancing these factors carefully when designing AI systems for your game number seven test and iterate finally it's essential to test your AI systems thoroughly before releasing them to the public this will help ensure that they work as intended and are not causing any issues with gameplay or player experience now I was able to read that response back to you as it was generating so when it's actually ready to give the answer it is sufficiently Speedy on a system that's got it in uh a 30 60 TI in in the PC and I know from experimenting with this that this is not taking full advantage of the gpus that I have on my system so it will work on a down level system but having received that answer what you're seeing now is the references and so it's actually providing some details around where it found the information so a couple of important details here it is citing the sources for the information and oddly enough it did not use the Practical game AI programming document it used instead deep learning with Keras deep learning for computer vision and machine learning other algorithms now if we go in here and take a look at the context it will actually provide us with some information now this doesn't work for an Epub so it's probably not a really good example and I used a PDF for this I think we would have got a different answer so we'll give that a try and see what that looks like but I think you get the overall idea here that not only is it providing an answer it's forming that answer based on the documentation I gave it and it's providing citations for those so that you know where the information came from all right so we'll give we'll give a bit of time we'll try this again and see if we can update some more all right so let's take another crack at this and this time we'll swap out the epubs change it over PDF now I'm going to add these as an additional data source I've already taken and removed the previous collection from the answers so it will no longer provide answers from this available collection and instead I'm going to add an additional collection we're going to choose our file Source again we're going to go back out to our location and I'm choosing the PDF folder so I only have the pdf version of the files from that same batch of e books that provide details around Ai and machine learning content now this is all information I've collected over time some of it I have read uh some of it I am familiar with but not all of it so this could be an interesting answer I'm going to try to ask it the same question again and see if the PDF context comes back a little better and it only took me about a minute to get that new source added so here in real time I'll watch the timer on the video and see exactly how long it takes to generate this response it's not more than five minutes though based on the most recent recording that I just created I think five minutes really is a reasonable amount of time for these queries and the speed is going to continue to grow and you know the other the other great thing here is that as the algorithms are improving the speed is going to improve you're also going to be able to throw more Hardware at it you know if you had a sufficiently large server-based GPU I'm very certain that you could accelerate this process and the reality is there's more than one way to solve this problem I think to a larger degree what we need to consider here as we're playing with this technology is is this something I need to put in the hands of a salesperson is there value in where this could be run and how we can apply it imagining a scenario where you have a sales person that is running around and doing things being able to have a conversation you know you sit down with a client and they're asking a difficult question you've got a new sales representative you might find yourself in a situation where they don't know the specific answer but if you've given the documentation to them that does provide an answer to a specific question they could be running this kind of kind of query on their computer in the background while they continue to have a conversation so you might consider this as a substitute to for example a phone in call for help about a specific question and if the tool isn't able to answer the question you still have all of the other avenues of exploration available to you so it's not a dead end but you can offer a more rapid response a more complete response in a shorter period of time by having this kind of capability that runs locally on for example a Macbook that the salesperson carries around now this is processing the answer and again I fully expect it's going to get data from our seven eight nine billion parameter model and it's going to include the additional information the problem that we have here is that I did not turn on not turn on the source so we're gonna we're gonna stop the video here and try again okay while I was away I went ahead and configured an additional data source we've now added a pdf version of that same folder and has basically the same content we've just changed it out ePub for PDF so you can see the context and then now that I've activated it I need to go into the data and turn on that I would like for it to provide information from this collection and we're going to ask our query and let it come through and as it's considering this response a couple of thoughts really the availability the ability for us to use this kind of a tool for a mobile salesperson or somebody out in the field where they could actually run this model on a device that they carry around with them could be tremendously valuable essentially if you're providing the documents and the details and then providing a way to search those that could really be an amazing set of tools where this deviates from what we're familiar with in search is that you have to find exactly the string and it's going to give you the raw data and you still have to form the answer leveraging this generative AI capability you can give it the raw information it will go and find that and extract the valuable information provide a citation for that source of information and give you back a response that you can then use in having a conversation so as you saw in the first attempt that we made it was actually generating a response and I was able to read that response back to you in a typical human form so you could see where just coupling this with a tool a screen reader that reads back the response you could even include this in just a conversation Exchange where you're having a conversation you don't even necessarily have to look at the screen but imagine you're having a client conversation and they ask a question and you have the information with you that could provide the response you could go to a tool like this drop in that client question and have the system working on answering that question while you continue the conversation with a client and then come back to this in a few minutes have it giving it time to write the response to actually provide that answer back to the client so you can kind of think of this as a junior member of the team somebody that's there to help bring the response together they can provide that additional search and composition capability research and answer we're already now working through our documents it's looking through those documents to find the answer to my question what I post to it is I'd like to get started using Ai and the development of the game what are some things I should consider when I begin using Ai and AI game programming what are some practical steps and tips that I should use so it's coming back with a response now and it's saying I should Define my goals before diving into the technical aspects of it choose the right tools consider which ones are the best fit and it's bringing up some additional sources like unity and Unreal Engine plan out your systems architecture decide how you want to organize your code and data structures within the game this may include deciding on a central Hub where all of the information is stored or creating separate modules in different types of AI behaviors and now as it's reading back this response to us it is bringing in information from the documentation that I provided it it is giving us a different answer one of the the deviations was one of the ebooks I provided it was doing game development in unreal there's this is actually a pretty well formed answer honestly we've got we're working on 0.5 now test and refine that's similar and as you can see here it found sources in one file which is Unreal Engine 4 AI programming Essentials and here's the context randomly choose an area within its location this particular example is simple but when you begin to use filters the power of eqs shines for example you wanted to get all the enemies within 500 units of your location and limit eliminate those who aren't so it's provided this string of text that is seemingly in relation to what it is that we are asking it to answer so it is able to research and take a look at the files you give it I think I need some more time with this but I wanted to share with you all what's possible and that this is an area of active development no coding is required the more familiar you are with the documents I think the more successful you'll be in ascertaining the level of value that this could create in your files I recommend approaching this from a a small practice standpoint when I loaded the initial data set in for myself it was almost eight gigabytes of data that I loaded in and that was far too big for me able to make a good determination of how useful this is so but self put together a small source of data ask it some questions I think there may be some amount of prompt engineering that is necessary to get good answers but I think this is worth looking into and spending a bit of time getting familiar because I really do see that this is the future of where the tech is going so let me know how you receive this is this helpful is this the kind of information you're looking for from me does this create value for you and help you determine what to do next as you're looking for opportunities to apply AI to what you do I'd love to hear from you and if you have a question you'd like to explore please reach out to me I'm open I'd love to hear from you in the comments if the video is good for you please give us a thumbs up and if you haven't already subscribed to the channel what are you waiting for click that subscribe button so you don't miss the next video and with that this has been AI mistakes let's go make some more and see you next time

Channel: AI mistakes

Views: 22,548

Keywords: gpt 4, gpt4all, gpt4all install, gpt4all-j, localdocs plugin gpt4all, localdocs, localdocs gpt4all, no code llm, generative ai, nomic ai, generative pretrained transformer, large language models, personal ai, openai, actually openai, private gpt, personal gpt, chat gpt 4, chatgpt, ai, artificial intelligence, gpt 4 python, gpt 4 api, chatbot, machine learning

Id: l-Ji31HlC68

Length: 25min 19sec (1519 seconds)

Published: Sat Jun 10 2023