How To Use Meta Llama3 With Huggingface And Ollama

hello all my name is krishak and welcome to my YouTube channel so guys as you all know meta Lama 3 is available right now uh in both 8 billion parameters and 70 billion parameters and through this you will be able to solve various use cases like text generation question and answering and many more things now uh to get started how you can probably run this uh entire application or how you can actually use this particular model in this particular video I'm going to focus on that there are two different ways that I'm actually going to show you one is through AMA and one is through hugging face kaggle one is also there with respect to that particular excess but I did not get the excess right now so let's go ahead and complete this two that is one with hugging face and the other one definitely with AMA so first of all I will go ahead and click on get started in this so in this you will be able to see three options right metal Lama 3 is there metal Lama guard 2 is there Meta Meta code Lama 70 billion is there but I will just get the model from these three platforms meta AI is not available right now in India so we'll go ahead and click on hugging face okay so once you probably go ahead and click on hugging face so here you will be finding a link which is called as metal Lama so this is where you can probably see all the models there metal Lama 3 metal Lama 2 family everything is here now let's say that I want to use this specific 8 billion parameter so what I have to do go to this particular page and there will be one form to actually fill it out so you just need to fill out and just uh s uh you know submit it then after submitting it you'll be granted the access to this particular model so this is the first step that you really need to do in order to work with this particular model itself right so this is the first thing that you definitely require okay unless and until you don't get this access you won't be able to work with hugging face okay the other thing is that through the kaggle itself so if you probably go ahead and see kaggle right and uh with respect to this c c if you go and see this particular URL so here also it'll be probably asking you to uh you know so here you can see all the overviews all the models everything you you'll be able to see but still I not got the excess if you probably see Lama 3 over here right so I have still not got the excess it is taking some time but over here you'll be able to see that okay fine once you probably get the excess you can select 8 billion parameters and create a new notebook and based on that particular notebook you can start writing your own code since kaggle also provides you Jupiter notebook uh sorry uh GPU so you can specifically work with that okay so here are some of the inputs see some of the codes are there and you can probably execute it but as I said I've got the excess of hugging fish so let me show you and after this I'll be showing you with respect to AMA so here is one notebook file that I have actually created with the hugging face so I've installed Transformer hugging face and torch and here you'll be able to see that I'll also go ahead and install accelerate okay so accelerate is also basically required once I probably go ahead and take accelerate then I will be requiring hugging face token or hugging face API token so that that I will be you able to use with both hugging face Pipeline and hugging face Hub itself right so that is the reason I'm just making sure that I use both the token and this is the code right this is the code here you can see import Transformer I'll be giving the model ID that is meta- Lama meta Lama 38b and then I can use transformers. pipeline where I say that hey I really need to use this particular model for text generation and then the model will be equal to the model ID and the model arguments is nothing but torch T type torch uh float 16 that is the kind of quation that I'm doing and device map is nothing but Auto and as soon as I probably execute it hey how are you doing today so here you can probably see hey how are you doing today I'm doing well I'm a little bit tired okay so it will take some amount of time to probably load this you can probably see chunk by chunk it will try to load it and here you'll be able to see all this text generation RW if you want to try more things apart from text generation so what you can probably go ahead and search so let's say I will go ahead and search hugging SP uh text generation okay text generation so if you probably go ahead and see this or text generation I'll just write hugging face pipeline okay and in pipeline also you have multiple things right so if I go ahead and click on Pipeline uh within the pipeline you have multiple multiple steps that you can probably do okay so if you go down over here see you have Transformer pipeline all the task that you can actually do all this tasks you can do audio classification automatic speech classification I tried out already question answering I have actually done summarization I have actually done so it is absolutely working good and here also you could probably see with respect to the text generation so text generation is another one technique over here uh somewhere it should be available uh see text classification text generation so you can probably check out this specific Pipeline and get all the info and that is how we specifically work with hugging face okay so you can probably give all the things and automatically you'll be able to generate it okay so this uh and obviously one thing that you really need to note that meta Lama model is Lama 3 is not a multimodel it is a llm model so anything that is related to text you will be able to do it okay anyhow I'll be sharing this particular Jupiter notebook in the description of this particular video and again other pipelines that you want to verify you can actually verify now the other thing is that obviously um if I go ahead and search meta Lama 3 uh GG UF OKAY ggf model so if you go ahead and see this uh now you'll also be able to find in this hugging phase right uh so you'll be able to find quantized model also uh you can probably research for that or I'll try to create another dedicated video for that I just wanted to uh probably convey that information but let's go ahead and see with respect to the command prompt which I probably tried with AMA obviously one problem that we definitely find out that uh this meta Lama uh 3 is a huge model it's 8 billion parameters and obviously we really need to do quation and all so AMA provides you this specific feature where you can actually perform quantisation so here all you need to write is AMA run Lama 3 and here you can see that it is downloading it is downloading all the entire model over here initially and whenever I ask any question hello it's nice to meet you is there something I can help you with or would do you like to chat I asked who are you I'm Lama and AI assistant developed my meta AI this this this I was trained now we'll go ahead see some of the demo with respect to Lama 3 so here I will just say write me uh python code to swap or to perform binary search Okay binary search so here you can see I have probably been able to get this and it looks amazing uh everything is probably provided over here and understand the token size is quite huge 50 trillion tokens 15 trillion tokens data set that it is proba trained with okay so uh tell me uh poem on generative AI where um to a five years kid so I'm just asking some question random question so here you can see quickly I'm able to get it now if you are able to access with the help of AMA then definitely you'll also be able to create end to end projects uh for that you can obviously see my Lang chain playlist uh if I probably show you the code uh what I feel is that you just need to write the model name as Lama 3 and I think everything will be working it can be an end to end project it can be anything as such but you need to install AMA over there right so I think uh these are the two ways that I probably got the excess of one is hugging face and all so hugging face I've already told you we need to install Transformers hugging face Hub torch and all accelerate and then we can probably call this particular model use our Pipeline with some task or some task like text generation on or Q&A whatever I want and with respect to that particular pipeline whenever I ask any question I'll be able to get the response so in short it is probably generating the text again we can also change the settings and generate more text if you want uh over here itself in the pipeline section uh where we have that token size also so I hope you like this particular video this was it from my side I'll see you in the next video all the materials will be given in the description of this particular video yes this was it bye-bye have a great day

Channel: Krish Naik

Views: 27,701

Keywords: yt:cc=on, llama3, llama3 tutorials, llama3 demo wwith huggingface, llama3 demo with kaggle, llama3 demo with ollama, krish naik generative ai

Id: LA-hZDnn5Hc

Length: 8min 27sec (507 seconds)

Published: Fri Apr 19 2024