How To Run Vicuna Locally (Windows, NO GPU Required)

all right so this is gonna be a short video on how to run the new vicuna model um and for this tutorial we're going to be using llama.cpp it's kind of like the last tutorial where we used alpaca.cpp but this is a more up-to-date version so uh the first thing that you want to do is click the first link and it will download the vicuna model it's about eight gigabytes in size and then while you're downloading that head over to the second link and it'll take you to the releases page and then you'll just want to download the Llama repository so download that zip file unpack it and then you'll have a folder that looks something like this except you won't have this file or this file this is the model that you just downloaded so put it in the same folder as the rest of the files that you just unzipped and then after that uh head down to the description one last time and then make a new text document name it run.bat and then take out the text I guess well I already have one so but after you make a bat file oh also you might have to uh enable file name extensions to do that so then edit the bat file and then paste in what is in the description it'll look a little bit different than this because I have some other stuff but paste in exactly what I have in the description and then you can run the bat file and it will look something like this and if if you see all this then that means it's working if it says loading tensors for some reason it seems like this model doesn't actually use that much memory it only seems to use about four gigabytes for me and it doesn't use it uses about the same CPU as the other models use so um but yeah this is the 13 billion parameter model of vicuna and it seems to work just fine with four gigabytes of memory somehow but um that's pretty much it if you have any problems join the Discord and I'll be able to help you out there and thanks for watching

Channel: Spreadsheet Warrior

Views: 14,560

Keywords: vicuna 13b, vicuna, vicuna 13b 4bit, vicuna 13b github, vicuna 13b quantized, how to run vicuna locally, vicuna model, run vicuna locally, download vicuna, vicuna model download

Id: iQ3Lhy-eD1s

Length: 2min 30sec (150 seconds)

Published: Tue Apr 04 2023