Fine-tuning A LLM Using PEFT | Generative AI with Hugging Face | Ingenium Academy

in this video we're going to take our fully fine-tuned summarization model Bart and then we're going to fine tune it some more using PFT and I want to show two things one that theft parameter efficient fine tuning actually works and two that it's a whole lot faster than full fine tuning and you'll see that there's about a 33 percent reduction in in time to train and so first and foremost let's go ahead and install all of our packages so everything's installed I'm then going to load in our model that we fine-tuned all right so that took a second to load in the model but we've got it now let's load in our data set so everything's loaded as you can see we're loading the Samsung data set it's got a train validation and test set just like before we're going to prepare our data set for training the same way we did before and so now that our data set is created let's go ahead and check the shape remember we're grabbing every hundredth example to create a smaller data set and here we have 148 examples in our training set and so now let's create our pepft model and so I'm going to pause right here so we're going to be using Laura as our parameter efficient fine tuning technique without going into too much too much depth here and explaining these parameters this number right here is the rank and it's the rank of the matrices that are used in Laura and if you're not sure what rank is we do have videos on linear algebra that Explain rank so definitely check those out um these are some other parameters and here we're going to tell Laura that this is a sequence to sequence model that we're going to be using puffed on pepft is a library that was created by hugging face and now to create a PFT model what we do is we get our model and our Lara config and then we create it using the git pepft model function from the PFT library and so let's do that and it may take a second but here we are and now what we're going to do is we're going to connect to hugging face so that we can push our PFT model to the hub if someone go get my access token like before and log in the token is valid great and the PFT the pef training arguments and trainer operates the same as the trainer that we use without peft I just labeled it path training args and pef trainer just to know that this we're using PFT and we a stand we what we did right here is I set the directory in hugging phase to be Bart CNN Samsung PFT instead of Bart CNN Samsung fine-tuned and so just so I know that it's puffed and what we're going to do is we're going to instantiate everything if you notice we're going to train it for five epochs you may have to do a little bit more than that to be able to see the results that I saw but nonetheless give it some time theft takes less time per Epoch because if you notice when we print out the trainable parameters we're only training 1.14 roughly one percent of the total parameters of the model and so each epoch's not going to take three minutes in my case it took around two minutes and so I've already trained this I'm not going to run it again but after you have created your PFT model or after you trained it you can now push it to the hub if I go to my profile and go to my models here your PFT mod when you push your pep model to the hub you're not pushing the full model all you're pushing is the PFT adapter you can literally think of it and this will get you pretty far as a an adapter that you plug in to your plug on top of your model that's been trained and fine-tuned to your data and so what you're going to do whenever you push whenever you push to the hub and then you want to pull it back down what you're going to do is you're going to pull your base model which our base model was our fine-tuned model and then we're going to load in that base model and then we're going to attach the PFT adapter because you can see right here it has the adapter model and adapter config we're gonna put the adapter onto the model by calling PFT model Dot from pre-trained from the PFT library and then we're going to load that model in and let's generate a summary using my pepft model and so it's going to be on the same dialog and summary that we've been using before so that we can sort of see how it's learning in more detail and as you can see it says Hana as command if she has Betty's number Amanda can't find it so she asked Larry to call her that doesn't happen so the model is still getting this part wrong right so maybe we need to go further and do some more parameter efficient fine tuning training for some more epochs I did five in my case and the end right here says Hannah is going to text Larry to ask him to call Betty now here's the thing last time it said Amanda is going to text Larry and and now it's saying Hannah is going to text Larry the model has learned that Hannah is the person that's going to be texting Larry not Amanda so it has made an improvement and that verifies it it's still not quite at the quality of the ground truth summary but it's getting better and it's learning slowly but as you can see pepft is a lot faster we can train for more epochs and that improves our training process quite a bit and so play with this train for more epochs and definitely learn more about pepft you can read the documentation on huggingface and now let's hop into the next video

Channel: Ingenium Academy

Views: 829

Keywords:

Id: vt3KiTgLFDs

Length: 5min 36sec (336 seconds)

Published: Tue Sep 19 2023