Use Your Self-Hosted LLM Anywhere with Ollama Web UI

you might think that this is open ai's chat GPT app but it's not I'm actually running this UI in the large language model that powers it from my own computer at home and today I'm going to show you how you can take your llms and your data with you wherever you are in just a few lines of code let's go we're going to need a few tools to get started so I want to walk through what each one of those is the first one we're going to need is AMA I did a video on how to install that and use it and I'll include a link to it up above next is Docker which allows us to run predefined images inside of containers you can think of these is self-contained apps that can run just about anywhere I'm not going to cover installation here but it should just be a one-click install where you download their installer file next up is the oama web UI um this is a really great open source codebase that we're actually going to be running via Docker and this gives us all kinds of functionalities that you saw earlier it has chat history um audio input image input um model modification and a lot more finally you're going to need to download and make an account with enro and this a software that allows us to expose an application that we have running on our computer out into the open world through a URL that they provide I'm going to be doing this tutorial on a Mac but all these tools should be available on Windows or whatever you're running as well so now that we know what tools we're going to be using let's just start by confirming that we have Ama up and running to do that we just navigate to Local Host and then the port 11434 uh which is the port that ama exposes through their API and we can see ama is running so we're good to begin now we're going to use Docker in the command line to download and run the AMA web UI tool I'm going to paste in this command here and we're going to go through it line by line so you understand what's going on so first up Docker run is used to create and run a new container The Hyphen D is shorthand for detach which means that we won't need this terminal open to keep the process running it's actually just going to be running the background for us- p is short for port and that means that we are assigning our host Port 3000 to the port 8080 that's running inside of our Docker container um again do ER is like a self-contained app and so there may be lots of things running inside of that and so we explicitly need to say um which ports we're exposing to the outside world um you can also think about this sort of like an apartment address and a unit number so 3,000 is sort of the um overall apartment address that we would come to from the outside world and then once we're in that apartment we're looking for a specific unit and that's Port 880 so this ad host command is a little bit trickier but it's essentially doing the opposite of what we were doing with the port command so with the port command we were telling um our host computer how to find something inside of our container um with this we're telling our container how it can access Services outside of Docker specifically we're going to be using olama running on our host machine not inside Docker and so this container needs to be able to find that and so we're essentially just giving it sort of like an address that it can find arbitrary ports that are running on our host machine- V is Shand for volume and essentially we are creating a volume called AMA web UI and we're mapping it to an internal folder structure at app backend data Docker images uh essentially restart stateless um so if you started a container and stopped it any data that you would have saved that you didn't persist would be eradicated in between restarts and so what volumes allow us to do is take any data that we have saved or like any folders that we want to persist and map them to storage on our host outside of that now it's important to note that Docker manages the location of the storage so we can't just go to the AMA webui folder on our computer and expect to find data um Docker is going to kind of manage that for us but you can just remember that this is for persisting data between sessions and between restarts next up is the name parameter um Docker makes you name your containers otherwise it'll assign a name for you just randomly but we're just naming this one AMA web UI so we know what it is um it's also important to note that Docker doesn't let you reuse names so if you want to you know make another version of this you either need to pick a different name or you need to delete this uh container that we're creating right now next we're just telling this container that we want it to restart if anything goes wrong and finally uh this URL down here is the address of the AMA webui container that we're going to be downloading and we're specifying that we want the main branch of it if you'd like a deeper dive on the docker CLI um I can include a link to their documentation down below I'm also going to include a link to all the other tools that I'm using as well so let's go ahead and run this command and it's going to be downloading um all the different data that it needs there may be a lot of dependencies but if you were to run this again um and nothing has changed um it should have all this data cached for you so we've run our Docker command and it looked like everything worked but let's just confirm that within Docker so we can just do Docker PS and that will log out all of the containers that we currently have running um so here we see the name is AMA webui that's correct and it seems to be good um so what we can do now is run this port on our browser and see what we have running so here we are in Port 3000 of our local host and we can see that we have this great olama web UI up and running um it's going to ask us to sign in but because this is our first time doing this let's go to sign up and we're going to create a new admin user now it's going to ask us for our full name email and password but it's not really going to verify any of this so I can just go in and say test test atp.com password okay and so we're in um so we have this really awesome looking UI that looks a lot like chat GPT with a few subtle differences so the first thing that it asks us is to select a model and this is really cool because AMA gives us access to a huge number of models as you'll recall so I'm just going to select our fi model which we know and love um and then it gives us some example prompts down here that we can just click to get started but we also see that we can attach images or files if we wanted to use a multimodal like lava we can type in our message kind of as we would expect and we can even do a voice input if your browser supports that so just to see how this interface behaves I'm going to click on the tell me a fun fact about the Roman Empire that's pretty cool it gives me a really Snappy response but it also has some great features like like I can edit the response um if I go up here I can even edit my question I can copy the question or response um I can have the response be spoken and I can even have it be regenerated but over here on the left we also see that our chat was added to like a chat history and so every subsequent chat that we have is also going to be added to that list so we can find it really easily the sidebar is also where a lot of the functionality for the AMA webui lives so at the top this is where we create new chats uh we also have access to model files prompts and documents which I'm not really going to cover this time but you should definitely explore them on your own and if we come down here to the bottom and click on our profile we have the admin panel which is where you would uh create and delete users so for example if you wanted to bring on friends and family to use your AMA instance they could do that and we also have the settings menu settings menu gives you access to a lot of really cool features so for example in advanced you can change the seed you can change the temperature of your model these are like much uh deeper underlying parameters for different models um but speaking of models we can also download and delete models uh directly via olama through this really nice interface so now we've confirmed that we have a UI that connects to our olama instance and works on our desktop browser now let's put it on our phone so we can take it with us as I mentioned earlier to do that we're going to be using a tool called enro and this tool just forwards one port on your machine to a URL that enro hosts make an account on the enro website for yourself if you don't already have one and then once you're in you'll see the same page that I see I'm just going to follow these instructions to get started the first thing that we're going to have to do is to download the enro utility um it gives us this Brew install link if you have home brew but if not you can also use their website to find just like a regular download that doesn't rely on any external tools once you have that downloaded then all you need to do is configure your enro instance on your host machine with the off key that they give you finally once you've added that then enro is all set up and is ready to host your application on the internet okay so we're just going to take this command that enro provides for us and go to our terminal paste that in and change the port to the port that we want which is 3,000 and voila okay we see that our session status is online we have our boarding address here so this is what we're going to copy and let's just confirm that this works okay and so check this out this is the same interface that we saw except this URL is no longer Local Host this is now on the open internet and if we go back to our terminal we can see that uh resources are being fetched here this is really cool okay so now we have all the pieces in place and the moment is finally upon us what we want to do is take this URL for enro application and paste it into the browser of our phone I have that here I'm doing a screen recording so once we have that we just log in uh with our email address here which I have as test fake.com again it really doesn't matter what your um off is we can sign in and so not only do we have this running on our phone which is impressive enough but when we go over here we actually see that our chat is here exactly as we left it so we have our question we have our answer and we can still interact with it even if we're a thousand miles away as long as we have an internet connection we can interact with this UI and our llms running on our home machine I think that's really awesome one other thing that I wanted to call out is this web UI is actually a progressive web app or a pwa and that means that when you save it to your phone as a bookmark uh your phone will actually treat it as if it was an app um and so you'll be able to see it kind of full screen like I'm showing now just one thing to bear in mind you're going to need to have enrock running on your machine whenever you want to access this remotely so you might just keep it running in the background if you want to okay so that's it we now have a slick UI running with chat accounts Model Management and a lot of other really great features and best of all this can run on our computer at home on our phone or really anywhere with internet so let me know in the comments how do you see yourself using this are there any other uis for AMA that you think I should check out what other devices besides your phone would it be cool to run this on thanks again for joining me on this tutoral to see what else I'm working on please be sure to like And subscribe

Channel: Decoder

Views: 42,176

Keywords: ollama, machine learning, self-hosting, docker, ngrok, LLMs

Id: syR0fT0rkgY

Length: 10min 3sec (603 seconds)

Published: Mon Jan 29 2024