AI on Your Local PC: Install JanAI (ChatGPT alternative) for Enhanced Privacy

in this video I'll guide you through the process of installing and using open-source AI models through a platform called Jan AI these models can be run entirely offline on your computer and offer enhanced privacy and data protection over the more popular online models like chat PT Bard and Claude when interacting with online AI models your data may be utilized to train the underlying algorithms this means that you have to be really cautious when sharing sensitive for personally identifiable information during these interactions but with the offline AI models I'll introduce you to your data remains within your control all you need to do is download the model and run it from your own PC even when you're disconnected from the internet keep in mind this powerful tool does require substantial Hardware resources especially Ram I'll discuss the specifics later on in this video to help you prepare for a successful installation and usage experience now let's get started started download the app head over to Jan Ai and download the appropriate version for your computer I'm going to select windows then go to the download history to install the downloaded file initially you won't see anything since we haven't downloaded any models yet so let's explore the hub download models from The Hub there are lots of models here and honestly couldn't tell you how each of the models differ I'm going to leave it to you to do your own research if you want to go down that rabbit hole but I'll make two recommendations the model that I recommend for most users is this one on top mistro instruct 7B it's relatively small in size and it runs quickly and efficiently as compared to some of the other models so let's download and try this first using mistal instruct 7B let's start out with a simple High at first engagement it does take a few seconds for the models to start up and you can see the system resource usage here this is my computer spec by the way I'll also drop it in the description below I'll give it a more meaningful prom and it responds pretty quickly and they do look really good and you can continue to engage with it okay now let's look at some of the options we have we can name each conversation let's say we have have one on YouTube ideas and another on productivity tips as you can see the conversation history will show on the left you can either clean the thread meaning it'll wipe the conversations within the thread or you can delete the thread Al together we can give custom instructions things like use clear and concise language and instructional tone I'll put the responses in number list if you have multiple models downloaded and ready to use you can select the one you want to use from here inference parameters here the higher number of Max tokens allows for longer more detailed responses while a lower number restricts the AI to Shorter replies it's a balance between the need for a comprehensive information and the need to limit the response to a manageable size I always have this cranked up to the max this I'm going to skip since it's for internal configuration of the model the context lengths refer to the amount of the input text or the context the model can consider while generating its response this is a crucial parameter because it directly affects the model's ability to understand and respond to inputs especially in complex or extended conversations again I have this cranked up to the max all the way to the left is the navigation bar we're currently in the thread window but we can move move to the hub to explore the available language models we have the system monitor here to see how much computational resources are being used and we have the settings menu where we can configure a few different options a quick side note here if your options seem slightly different from mine it may be because I have the beta version what they call the nightly release installed I'll talk more on this later using mix 8X 7B now let's take a look at one of the best models available on this Hub the mixol 8X 7B as you can see it size is 26 gigs it took me about 45 minutes to download but of course this depends entirely on your internet speed this model has stronger capabilities than the one that I just showed you both models are from mistal Ai and if you're interested in learning more about the difference between the two I'll drop a link to their website in the description below now the big problem with this model is that it's really resource hungry let's maximize the tokens here I enter a simple prompt and it waits for the models to start you can see the memory usage climb up to almost 100% by the way I have 32 gigs of RAM the assistant finally starts responding about a full minute later and the response trickles in very slowly I'll follow up with another prompt and check the performance in my task manager you can see that it's maxing out my Ram besides the interactions being really slow I was getting a lot of fail to fetch error with this model troubleshooting fail to fetch it seemed fail to fetch is a fairly common occurrence and there is actually a troubleshooting website to address this so I'm going to follow the troubleshooting steps and first install the nightly release click on the link then to the download page and get Hub then the windows version of the nightly build after downloading is complete run the install file and it's telling me that there is an update available so I'll hit download then restart wait for the install to complete and we are done with this part returning to the troubleshooting page we will now install Cuda since I am using an Nvidia GPU click on it I think you can probably close this form but I will fill it in then choose the appropriate options here for me it's going to be windows 64 bid Windows 11 and local then download Once the download is complete find the file and run it the install is straightforward after you restart the computer make sure to go to settings and enable GPU acceleration unfortunately for me it still takes anywhere from 1 to 4 minutes to load the model and I still continue to receive the fetch error that again may be due to my Hardware limitation since I'm still maxing out my Ram despite the GPU acceleration enabled so if you have a more powerful computer hopefully it'll work out for you I'd love to hear about it in the comments otherwise the mistro instruct 7B model has been working out really well for me and I've been enjoying it a lot I'm only just beginning to poke around Jan Ai and I plan to download the other models to compare if you've tried this tool before and can share some tips and tricks please leave it in the comments below [Music] thanks he

Channel: Andy Park

Views: 4,609

Keywords:

Id: CbJGxNmdWws

Length: 8min 2sec (482 seconds)

Published: Sat Jan 13 2024