This Chrome Extension Surprised Me

it seems that every time I look into a new front end for a llama I discover a new best favorite tool I looked at open web UI and I thought it was going to be one of the best front ends for AMA and then I found Misty and that was pretty much the best that I'd seen for a simple UI where it takes away a lot of complication of open web UI but now I found an even better simple UI for orama and installs in a completely different way compared to either of those two options if you aren't familiar with what ol is well welcome to the community olama is a tool for running large language models locally on whatever Hardware you have assuming your Hardware includes either Nvidia or AMD GPU of a recent vintage or you're running an apple silicon Mac to find out more about olama you can visit ama.com and there you can download installers for MACC windows and Linux you may also find install packages on brew and apt and other places but your best experience is always going to be with the official installers on the main website open web UI is installed using Docker there are options for installing it without Docker but you're going to have a much better time using Docker if you don't have Docker it's probably still easier installing Docker and then install open web UI on top of docker MTI is installed as I think it's an electron app but it embeds olama inside the installation so you don't even have to have Ama installed you know that was one of the original intentions when we first created AMA but some folks don't like that if you do have olama installed it can use the models that you've already downloaded but by default it's going to use its own copy of olama but this tool is an extension for Google Chrome and other browsers that Leverage Google Chrome it's called page assist though if you do a search online for page assist you're probably going to come up with a feature for procreate on the iPad page assist doesn't have the interesting branching options that we saw in MTI but it does share some of the rag options from open web UI and it's far simpler to use than either one of those so let's take a look at how to work with page assist I'll start at the GitHub repo scroll down to the read me and we can see the key features the first highlighted feature is a sidebar that can be opened on any page you can use it to ask questions about the page you're on I wish they started with the web UI as the top feature because that is the highlight of this app that is welld designed and works perfectly the third feature really is the same as the first feature another feature not listed is the ability to incorporate web searches in in your questions to a local model but it's really not the best implementation there so let's look at using page assist I'll start with the main UI for the app at the top we have a nice drop- down for the model we can see all the models you have downloaded next to that is prompts though as you can see here we don't have anything here we have to populate that in another section of the app down below we can ask a question why is the sky blue and we get the answer in the section above that text box below that we have a checkbox to enable internet searching here we can see the results of that search and we can also see down here the references that were used that were then pushed to the model to sumr on the right side is a microphone button and this allows you to use speech to text we'll see how to configure that in a little bit and next to that is an image allowing you to use an image with a lava model at the top left is a button to open the sidebar and that sidebar shows us a list of recent chats and at the top right we have a share button which is a little bizarre a link to the GitHub repo and then the link to the settings for this particular app in settings at the top we can set the speech recognition language and the language for the app and whether we want dark or a light theme and you know me I am never going to choose dark below that is what search engine we want to use when incorporating search we can use Google or duck ducko or soou which I'm not familiar with and then we can enable text to speeech setting the text to speeech provider and the text to speeech SP this was really cool 5 years ago but the system provider tends to be kind of lame and not really that interesting so I've just ignored it and then down to the bottom we have a button to delete the chat history as well as a way to export and import data let's go back up to AMA settings we can set the URL for olama and so this is just going to be Local Host Port 11434 and then some settings for rag you can choose the embedding model nomic embed text is one of the better ones to use and then we have chunk size and chunk overlap this is a little bit confusing here it's unclear from the UI whether this is based on number of characters or tokens or sentences but later on we're able to see that it's probably just actually characters which doesn't make a lot of sense it really should be based on sentences and then we have the prompts that are used for rag versus a web search so you can go in here and tweak them if you notice there's a problem with the way it works now we can come up top and click on manage models and this is actually a pretty nice UI for managing the models we can see some details about the models I can click the trash can icon to delete it and the recycle icon to repoll the model from.com you can also add a new model assuming you know the name of the model available now let's go to manage knowledge this is retrieval augmented generation so here's where we can add our PDFs our markdown documents text documents and other types of documents after we add a document we're able to see which embedding model was used to create the embedding and now you can come down to manage prompts so here's where we can add prompts that show up in the drop down at the top to help make it easier to add default prompts as you're working with different models and then we can come down to page share URL now this is from my perspective totally bizarre after you ask a question or a series of questions to a model you can choose to share that output and it becomes a web page that anybody else can look at I I I don't understand why you want to do this and and that's coming from somebody who finds it interesting to share their notes with the world okay so let's go back to the chat window and I want to ask a question from the document that I uploaded tell me about attention and there we have an answer that pulls information out of the document and tries to answer it in a way that would make sense to me let's see I added a prompt earlier so let's try age specific and I can change age to 10 years old but there's one more feature that they push within the GitHub repo and that's the side panel where you can ask a question about any web page so I'll bring up the Mac Rumors website and find an interesting article that I'd like to ask a question about here's one I want to know more about the apple pencil Pro that was announced at the Apple event just last week so I press the shortcut to open the panel and now I have to check the checkbox that says chat with the current page otherwise it won't chat with the current page and that checkbox goes away and the only way can bring it up is by closing this panel and reopening the panel so you got to remember to do that what are the new features of the apple pencil and we get a nice simple answer now let's see where things start to break down this with this I'll go back to the front page and now choose the best new features of the iPad Pro tell me about the new features on this page and so rather than telling me the new features of the iPad Pro because that's the page I'm on it's still talking about the new features of the pencil which is the page I was on when I opened the panel so let's try closing the panel we'll use the same keyboard shortcut but nothing happens we can't use the keyboard shortcut to close the page if I refresh the page it'll only refresh the main page in this browser I don't have a way of refreshing the panel and the only way to do this I think is to close the panel and then reopen the panel so you got to close the panel not using the keyboard shortcut and then really you need to open the panel using the the keyboard sharr so that's all a bit frustrating and then the panel doesn't update with the new page which is also kind of annoying according to the repo this works best in Chrome and brave and on edge and not on Arc my preferred browser is Arc it's just a whole lot better in every way but the panel isn't supported on Arc which is fine because the panel is a bit lame even on Chrome so I don't plan on ever using that but page assist is still an amazing tool just for that super simple UI that has integration with rag that allows me to ask questions to a web search even though that web search is often not that great but for just quick interactions with a model it's hard to beat page assist what do you think have you tried page assist have you tried a different app that you think I should look at let me know in the comments below you can also follow the other things I'm working on by signing up for the newsletter you can find that at techno evangelist. comom newsletter or if you want to support what I do visit patreon.com techno evangelist thanks so much for being here goodbye my water bottle is never here

Channel: Matt Williams

Views: 15,760

Keywords: ollama, ai, llm, localai, chrome extension, google chrome

Id: IvLTlDy9G8c

Length: 10min 31sec (631 seconds)

Published: Mon May 13 2024