Better Searches With Local AI

in this video I want to show you how you can do better searches online that are far more private than what you're probably doing now and much faster to get actual real results so what happens when you do a search online today well let's open up Google right now and do a search for what is a black hole we get all whole bunch of results sometimes we get an AI answer here as well which is something Google is adding on some searches but we also always get a list of results apart from that occasional AI answer nothing here is really giving us the answer to what is a black hole it's giving us a bunch of places we can go to find the answer now we're going to probably learn a lot in the process but it's going to take a while now and of course to get to this point we have given Google more information about our interests and they can use that information to let their customer to Target us with their ads this is going to happen with all the search enges Google Bing and others even even Duck Duck Go does this and I thought that was the whole reason most folks are okay with their less than Stellar results so maybe you ask the question to an AI model perhaps it's chbt or even llama 3 using AMA in the case of a black hole we're probably going to get a great answer but if your question is more timely you're sometimes going to get an answer that's made up these are the hallucinations we've all experienced ideally you want to search online then feed the relevant information to a model and get back an answer you want something like what you get with perplexity AI now before you go crazy and say perplexity isn't safe or local or private I'm just using this as an example and we'll come back to a potential good solution not a perfect solution but a pretty good solution we can ask a question now I just got back from an Alaskan cruise on the westerdam which is part of the Holland America line it's strange to say that when they were acquired by Carnival a while back and are based in Seattle rather than the Netherlands but anyway I want to know what ships are in the Holland America line right now perplexity gives me an answer that starts to answer this the materials on the site suggest that I might get a more complete answer if I uh show me the money pay up to subscribe to perplexity and get access to their Pro level but it's good enough for our example now what happened here well first perplexity tries to analyze what I'm asking for and then submits that to an internet search then the results are fed to a model and I see the results of that generation nice we also have some options to focus our search to specific information sources there's also a discover tab to see what others are searching for and a library to store your previous searches which is great but you'll probably notice that what perplexity is doing isn't all that hard but there's some problems we need to solve to replicate it the biggest is how do we do the search if we want to avoid the major search engines are there any options well there's an interesting project called something like search NG I'm assuming that's how they want you to pronounce c a RX search they refer to it as a meta search engine and from what I can tell they're actually doing the actual search on Google and maybe other places too but every request looks like it's coming from a different client so there's no way they can track it for the most part now this is best when you host search NG yourself somewhere but you get some of the benefits even running it locally and that's what you get when you install perplex Perle me is still in its early days there's a lot that isn't working but the one main feature is working great to get it up and running clone the repo edit the config file pointing it to olama and start up the docker containers using this command and that's it if you aren't familiar with AMA then visit ama.com and you can get up and running in seconds now open port 3000 on Local Host and here we are unfortunately you're stuck with dark mode so that may be enough for a lot of folks to just not even bother with it when you first launch the site you are at this search window on the left side there's a magnifying glass which is the Discover page like on perplexity but that doesn't work yet and below that is a little book which is the library page to store the previous searches but that doesn't work either at the bottom is a gear icon for settings click that and you'll want to set the chat model and embedding models to whatever you want I'm using llama 38 billion parameter model for chat and nomic embed for embedding why do we need embedding well it's creating a kind of specialized rag system this isn't where you upload files but rather it gets the search results embeds them and stores the embeddings in the vector database then embeds your question and finds the most similar results this is then used to generate an answer and just like perplexity it first analyzes your initial question question to figure out the best search query on the web to get the best first results now we can start asking questions what ships are part of Holland America in the results we see a lot of what we saw on perplexity we have a list of sources we have the answer generated by a model and we have other related questions we can ask we can also focus our search to the same areas that we saw on perplexity and uh well that's pretty much it for the product so far I look forward to seeing the author add those other two pages as well as enable a light mode to make it a bit more pleasing to use for well the rest of us is this a complete replacement for something like perplexity sort of but not really maybe at some point in the future it'll get closer but on the downside it's sometimes slower than perplexity but not always I'd say often the results are better with perplex mostly because they seem to be more complete in a lot of the other tests I've done but maybe if you're paying perplexity that Advantage for local goes away of course with perplexa everything's local well except for the actual search and everything is a whole lot more private so that's a major advantage for perplexa another Advantage for perplexity is that you can upload a dock and include that in your search results making the results even more specific to you that isn't available at least not yet on perplexa Ka but but I think this is pretty cool I really want to thank I have a cute nose on YouTube for suggesting I take a look at it of course in their comment they asked if it was good at Rag and I think I answered the question but if you're looking at a rag solution for your docks then no this doesn't do that yet what do you think have you used perplex have you used any other tools that perform a a similar function and and does that better or how about any other tools you think I should look at let me know in the comments below and I can take a look I also have a newsletter that I send out that you can subscribe to at techn evangelist. newsletter if you want to support me there's a patreon at patreon.com techn evangelist and there's a number of different levels for that well thanks so much for joining me again on this I have a lot of fun making these videos and I hope you get a lot out of them too goodbye Where's My Water okay stop and stop

Channel: Matt Williams

Views: 24,955

Keywords: ollama, ai, llm, localai, ai agents, artificial intelligence, machine learning, future tech, ai tools

Id: TkxmOC4HBSg

Length: 8min 29sec (509 seconds)

Published: Tue May 21 2024