Llama 2 - Build Your Own Text Generation API with Llama 2 - on RunPod, Step-by-Step

[Music] if you're looking to create your own text generation API using the state-of-the-art Llama 2 large language model you've come to the right place welcome to today's tutorial where we'll show you how as always we'll walk you through the process step by step and by the end of this tutorial you will have a fully functional text generation API powered by llama2 and running on run pod what you can request to download this model directly from meta we will use a quantized version of llama 2 available on hugging face to get started you'll have a runpod account yet you'll need to create one you can find the link in the description for those that account just log in navigate to serverless and then to my templates here you'll create a new template and a cited a name for the container image use this pre-built image I've created and hosted on Docker Hub you can copy the name from the helper file Link in the description I recommend bumping up the container disk size to 50 gigabytes next scroll down to environment variables add a key for model repo and paste the value shown once done save your template now head over to serverless then to my endpoints here create a new endpoint and name it select the template we just created for this tutorial we'll create just one worker I suggest enabling flashboot to minimize the cold start time then select the GPU type and click create once that's done let's hop over to postman for testing but before we do that copy the serverless endpoint ID in Postman you'll need to import the configuration file from the description after importing it click on variables and paste the values of your serverless endpoint ID and your API key you can locate your API key in settings at this point we'll need to modify the prompt text as lama2 expects a specific format you'll find the prop template in the description and on the hugging face model page replace the section labeled prompt with your text let's use tell me a cat joke you're sending the request the model will begin downloading for a production environment you'll want to download the model in advance and store a network drive to avoid worker tasks from downloading the model while the model is downloading let's take a moment to remind you that if you need help with your text generation API or any AI Project Feel free to reach out in the comments would also offer dedicated paid support education training implementation and architectural services to assist you if you have suggestions for future tutorials reach out and let us know once the model is ready it'll generate a response there you have it our cat joke now let's try generating some python code impress it isn't it if you found this video helpful please like subscribe and leave a comment below we appreciate your support until the next

Channel: Generative Labs

Views: 10,404

Keywords:

Id: Ftb4vbGUr7U

Length: 5min 4sec (304 seconds)

Published: Mon Jul 24 2023