Installing Llama 2 on Windows Using oobabooga Web UI

today we will be trying out the new llama 2 large language model we will first install the uber booger web UI but I will choose to call it ooga booga click on the one-click installer for Windows from the GitHub page to download the zip file I'm going to move it to a more convenient location and then unzip it then you can delete the zip file and go into the new folder and double click on the start underscore Windows batch file to start the installation once that is finished we will go to the meta AI llama 2 web page and click download the model here you will need to fill in all of these boxes and then scroll down and click this button after a couple minutes I received an email notifying me the models are available for me at that point we can go to this hugging face page create a free account if you haven't already this page will show this message here you can check the check box and click submit after about 25 minutes I received an email notifying me that my request has been granted after that we can come to this hugging face page and select the model to download I will choose the smallest one called llama2 chat Dash HF click on the copy button next to the name on this page then launch ooga booga go to the models Tab and paste that in the text box and click download here we got an error that says we are not authorized that is because this model is a protected model and requires authentication so we will need to make a modification before downloading let's check out the ooga booger GitHub page here it says to download a protected model we need to set these two environment variables using an access token from hugging face in hugging face go to settings and then click access tokens on the left on this page click on new token here we can specify a name I'm going to call it ooga booga and then click generator token and now we have a token let's go to the environment variables and add the two variables mentioned click on new and for the variable name set it to HF underscore user for the variable value we will go to hugging face and copy the name then click OK and create another variable for the variable name set it to HF underscore pass for the variable value we will copy the access token from hugging face paste it into variable value and click OK on all the windows now let's go back to the hugging face models page and select llama2 chat HF say that five times fast click on that copy icon again next to the model name to copy it launch ooga booga and go to the models Tab and paste it in the text box then click download last time we got an error saying it was not authorized this time it should install successfully since we set up the environment variables using the access token once the download finishes click on the refresh icon next to the model drop down we can then select the Llama 2 model and then click load and when that finishes we are done click save settings and let's go to the text generation tab to test it out you can type some prompt here and then click generate I'm going to ask it about uh ducks for my computer using this model with all the default settings it took a while and used up all of the GPU memory but it did eventually finish producing the results and that is all there is to it you can experiment with different models and ooga booga settings to see what kind of results you get enjoy

Channel: Natlamir

Views: 10,177

Keywords:

Id: rID74k_ejMk

Length: 3min 37sec (217 seconds)

Published: Sun Jul 23 2023