Programming AI tools: Enhancing Your Development Workflow with the New Code Llama

hello everyone and welcome back to Dreaming AI my name is nuke and today we're going to discover a newly released model by meta called code llama which is based as you can guess from the name on the family of large language models llama widely used by those who usually want to create a homemade chat GPT or even by those who don't want to rely on an online hosted Alim preferring one locally and if correctly fine-tuned is possible that is able to partially replace even the GitHub co-pilot which is still today the best development assistant powered by AI you can use it to write simple programs complete scripts convert code from one language to another and many other things first of all we will need our model since I don't want to use the base model because it requires too much vram I opted for the 4-bit quantize model downloadable from the repository of the legendary the bloke you can find the link in the description you can also choose marbles with higher precision like the 8-Bit ones but usually the four or four point one bit ones work just fine models quantized in this way have the ability to be run without the need for a GPU of course it's still better to have one so the inference is fast and we don't have to wait ages for a response to be generated there are also many types of quantized models and I chose the model in the new gguf format I introduced this month by the authors of allahment.cpp uh um I also chose the model called code warmer 13B instruct as it appears to be the most balanced one according to the screenshot on the official project page now if like me you're just curious to try the model with single queries without the need for chatting you can use the workflow we created in the last video where we use comfy UI to interact with various models place the downloaded model in the GPT checkpoints directory and load the workflow let's simply get rid of the image generation part and remove this string function please remember also to put the cached option to no reading on the books repository it seems necessary to adjust the syntax with which we interact with the model simply place this code in the prefix and this other one in the suffix and then select the correct model and try querying it let's try generating a simple python code for a Hello World and see what happens it seems quite responsive let's try converting this code to C and python code and see if it generates valid code foreign now I'd like to show you one of my favorite interfaces for interacting with Ella models it's called text generation web UI by Uber Buddha and to install it simply download the the zip file from the GitHub repository and extract the archive to a convenient location and Run start windows.pat this process is similar for non-windows operating systems as well now wait for the automatic installation to finish and place the model in the folder obaba Windows text generation web UI models now go to this address or simply press control and click on the link in the console the main screen of the web UI will open and go to the models section click on refresh here and select the model in question I'll set the parameters in this way being careful to adjust the GPU layers based on how much vram you have and what to use setting them to zero will allow you to use this model only with the CPU as I mentioned earlier uh finally press load to load the model and in this case I will not change any other options except for the NAX new tokens located here this will indicate the length of the model response then go to chat and try querying our model in a similar way as before with the difference that here we can potentially chat and use the generated context request modifications or other interactions with the previous code let's try foreign helped you in case you were searching a model to replace Chachi PT for coding or simply to satisfy your curiosity in my opinion the 13 billion one is not bad but we are not quite there yet the 34 billion is probably better but it continues to be too resource intensive for the General Public please consider liking and subscribing if you found this tutorial useful also if you have any questions please let me know in the comments below I'll be happy to help you out as much as I can and until next time dreaming [Music]

Channel: DreamingAI

Views: 3,613

Keywords: Codellama, code llama, oobabooga, text generation webui, metaai, ComfyUI, programming, python, advanced, text generation, AI, stable diffusion, artificial intellingence, dreamingai, ai news, best free ai, best ai model, dreamingai tutorials

Id: warjG-IGmbA

Length: 7min 53sec (473 seconds)

Published: Fri Sep 15 2023