Train your own LORA model in 30 minutes LIKE A PRO!

in today's video I'll show you how to train Laura so what is Laura first of all and why do we need to train it if you ever tried to generate images with consistent character pose or object in stable diffusion you might have noticed that it's not that easy and Laura helps to solve this problem Laura stands for low rank adaptation and this model uses low rank adaptation technology to fine tune uh stable diffusion checkpoints Laura makes it easier to train stable diffusion on such Concepts as characters poses objects or even just different artwork Styles and after you train your model you can export it and reuse or share with other people so you can find plenty of Laura models on such websites like civit AI but the best fun is to create your Laura model yourself it requires smaller amount of pictures and lower effort compared to other models so if you have your your data set ready you can do it in like 10 or 20 minutes and that's what we're going to do now so the first and the longest part of Laura training is preparing the data set you will need 15 to 35 pictures of your subject they must be in different stance poses conditions whatever uh pictures should not be the same and uh you can see I prepared already a few pictures of my parot drais and we're going to train Laura for drari the parrot I know that in my previous video we were talking about uh not safe for work content uh but I decided that I'm not going to use uh such kind of photos in my Laura because YouTube wouldn't be happy uh with that and instead I will just use my Parrot uh to train the Laura so you see I try to pick different pictures in different uh backgrounds and places and even on my shoulder or like on my desk I picked 25 pictures of drais and I cropped them to 512 pictures um to the square size as you see uh if you don't know how to do it in uh there are plenty of graphic editors um that you can use online or just like uh applications on your laptop I just use default uh Mac preview tool there is a adjust size menu here so you can adjust your size you can also select an area on the image and then crop it so that's what I did to my photos and then I just went and described every picture uh first I placed the specific tag for my Laura custom tag custom word that will trigger the Laura and then I just described what uh is on the picture so if you look at the uh second picture is just thear mirror and his wooden perch not much to see so here in the text file I added all these uh words and I did it with every image so I have 25 images and 25 txt documents that are all named the same so first image named one and the txt file named one so uh now when we have our images and uh descriptions for them ready uh we need to place them in specific directory you can see that that this directory where these pictures are living now it uh named 10or drar the par where 10 is the amount of repetitions that we're going to use to train the model and the car par is just the name of my Laura and if you go one level up there is just the name of my Laura the root folder for the Laura so uh that's the initial preparations before we actually started to train the mod model next thing we're going to do is uh to find a correct notebook to train uh the Laura I used K SS and I used the notebook from the user uh l q Ruf I am not sure how to pronounce it so um I'll just open it here that's the original um notebook that I used and it worked for me so uh I'll leave the link uh to this notebook below so you can also use it you need to go to the file and uh save the copy in your Google Drive so let's do that and it will open a new tab with the copy of this notebook why I prefer to save the copy in my uh Google Drive the original notebook can change something can uh be updated there or something can break and it might stop working for you it's always better to have the working version of your um notebook book so I'm going to uh rename it demo Laura training uh and I will also give you the link to this one uh to this version that I saved on my Google Drive because I don't know when you will watch this tutorial uh and maybe at the moment when you're going to watch it it will be uh the original one will change and there will be different sales and it can be confusing next what we going to do is just to go through the cell of this uh notebook and uh run them uh one after another with our configuration for our Laura uh we will not run all of these sales because some of them are optional and we don't want to use them so I'll just show you the necessary cells that you need to run the first one will install uh python dependencies uh just don't forget to click this um checkbox Mount Drive it will connect the notebook to your Google Drive where you you will save for your Laura so it will ask you to connect and to give the permissions that's what you need to do I'm allowing it and now it will take some time while it is running and installing the uh the dependencies the the execution of the first cell uh completed and the second one is uh file explorer I'm not sure what is it for because it didn't work for me uh you can of course still try to run it but I don't think it really needed um so we are going to the second section where we can download our model so uh we have this um uh drop down with different models you can choose which one to download I will use the uh stable diffusion first dox model um and I will use uh just a stable diffusion one uh there are different ones like uh any Laura and Anime and Etc uh but we don't want to uh generate any this time we want to generate we want to train it on just um basic stable diffusion model so we will download this model we can also download custom model from hugging phase but for this tutorial we don't need to do that so we will skip this second cell we also will download this vae it is optional but let's do that let's download stable diffusion one and just note that after we downloaded uh models it shows this path so please save it somewhere in case if you accidentally refresh your collab H because you'll need it in the next uh steps so save both of these links we will uh Define our uh local train uh directory uh so this is uh the default path in this notebook but uh it didn't work for me uh so what I had to do I had to edit it a little bit uh so it uh looks not in the content folder but in Drive slash my drive then I can access this uh directory on my Google Drive so I'm uh pointing uh training data directory here it is done then the next one on zip data set our data set is not zipped so we don't need to do that image Scrapper again we don't need to do it we have our images prepared data cleaning we skip it so um this captioning as well we already captioned everything uh so we just go to the custom tag and in custom Tech we have few settings there is a an extension we have txt file so we can just leave it like that our custom tag will be dra Caris the par and here we will specify all and uh we have to run this section of course and then we're going to training model uh and before we actually start executing this let's go to our our Google Drive and you'll see uh this uh Laura folder with few other folders so in this train data folder you can upload the data set that we prepared so I'll just upload my drar par folder here if you go back to Laura you see there are a few other folders the regular uh data is a folder for pictures that are not your parrot and you can add them if you want but it's not uh required I will not add any and the output is the folder where you actually get your luras so so now when we uh uploaded the pictures we can continue with our notebook so model config uh we can add the project name it's going to be again drar is the par and petring uh model name or path here we can paste the path that we got here where we downloaded our model so copy this and go and paste it here and it's this the same you can do for your vae again uh going here copy this file and inserting here and the output de we also need to fix because you remember I changed the path here where it was our training data so I added this piece drive my drive and I'm going to do the same for this one I'll hit this check box and I'll run this model config so it tells you that it is uh yeah so the next uh part is data set config thata set repeats is 10 and we said that we want 10 repeats so we leave it as it is activation word uh it's going to be our dra Caris the parrot and caption extension again txt resolution here you should set it to your uh pictures resolution you can see it starts on 512 and you can be even bigger even higher but I will leave 512 because it's just going to be it's is going to be faster yeah I'm going to leave it uh 512 because it's just going to be faster um and yeah we're going to run it as well and the next one is Laura config Network category yes living uh Laura here this uh convert Di and convert Alpha we don't care about them because they are for other models for Laura we can set this network d I am and network Alpha so I'd set it I'd set it to one to8 both of them um these uh options you can experiment with and figure out what is working best for you I'll set it to these ones for now and uh this one I will not specify I will leave it empty then Optimizer config I will leave everything on default I think yeah so let's run it as well and then training config uh we will leave we will leave low Ram checked and sampler um you can choose different Samplers here it is the same Samplers that you see on stable diffusion vui I will leave with uh DD IM because this sampler Works faster uh the way it works it can just uh skip steps during the training we still don't lose much in quality of training so I'm going to use the this one number of AEP box 10 badge size four train badge size six we can leave them as it is with the exception of this one uh save every uh n EPO time um this will uh save your model your Laura after every repeat of training uh and we don't need it probably to set to one because we don't want to save it every time maybe we could save it to like two at least um so um save model as save tensors yeah everything else we can leave the next uh cell is the actual training just um hit this button and if everything is configured right you will see the output here that uh will tell you that your training is in progress and will output some logs and stuff if you see errors here on this step it means that you probably misconfigured some paths uh in the cells before so just go and double check everything you see it found my 25 image files uh so everything is right and it started to train them it will take few minutes so I'll pause the video and we'll back to you later when it is done our training is finished and now we can go to our Google Drive and in output folder you will find these files uh the this uh files with the numbers it is uh from different EPO steps uh and we are interested in the last onear the par save tensors and we going to download that and um then we go into our uh web UI uh you remember in the last video I showed you how to upload luras here so we're going to models and we're going to Laura and in this uh Laura folder we just uh upload our drais the par at Laura okay I just want to rename it because it has these version number it was not the first one that I save in my downloads folder so we uploaded our Laura in the right place uh now we can just go to our collab notebooks and uh find the one that we used in previous video and uh run it so again uh and yeah don't forget to disconnect uh your Kaya notebook because if it is stays connected it uses your collab resources so I'll disconnect this one and I go and run our stable diffusion web UI if you are not familiar with this one just go and check out my uh previous video where I explain how to install and run stable diffusion on Google collab so uh again we are just uh running all this stuff and uh here I will select uh 1.5 stable diffusion version because this one was the one that we trained the uh Laura 4 of course you can train it for the stable diffusion Excel but uh it will just take more time to train and to generate images and for this tutorial I want things to be quick okay so we are waiting for ourselves to execute and when the last one will be ready we can test our Laura okay so um our um our stable diffusion UI is ready up and running let's open it uh and here uh if you go to Laura tab uh we can see our luras available here and there is our drais part Laura and if you click on it uh you see it adds the uh trigger here to text image um field so uh let's try and uh generate something with our new Laura and I'll add a negative uh prompts as well and uh then uh uh I want to go and find some nice uh prompt with a par something artistic um so uh here on prompt hero we can find examples for different images prompts you can see that uh pars are not the most popular topic so there a bit of awkward stuff um but I think yesterday I saw one nice picture yeah this one I like this kind of impressionistic style and let's copy this prompt and let's add it to our stable diffusion I don't want to add dolphin and Cat I just want to generate a parrot blue parrot with red big sitting on the tree and I will leave all these um styles here as it was in the original prompt and I'll try I'll set the BGE count to four and I'll hit the generate button just to see if our Laura is going to work and you see here here we have the weight of our Laura it is one it is the maximum like 100% so what we can do we can change this number from 0o to one and it will uh help us to Define how much of our Lura we want to actually apply to the images so now I used the maximum one I used one it's like 100% And even on the preview we can already see so uh uh so we can see the results and honestly I'm Amazed by results that they're pretty good and that's kind of a first iteration um these first and the last one they are very good the second and third they have some kind of artifacts we could use them these two pictures and they look very much like trares and uh what if I say set it to 0.7 let's try we can still recognize drais in these pictures but they are a little bit different we gave our model a more freedom more imagination to use uh and what else we can do we can switch to different sampler we can try this erer a it is actually a very good sampler for Artistic Styles I really like it I use it for when I need to do some sliz and when I want to get ready realistic uh images I use these uh DPM Plus+ SD caras so this is the erer and yeah some of them are good like the first one is pretty good um second one added more birds one bird is a pirate it has only one eye um yeah so uh again the first and the last are pretty good and you remember we trained these Laura for stable diffusion uh 1.5 uh but what if we try to use it with some other model for example the dream shaper that I showed you in the last video let's try to switch it and try to generate something in this model okay so we changed the model and uh let's try to generate something with the same prompt first uh okay dream shaper gave us some broken results so I wonder if it's just uh the sampler is not very good for this one I'll try with my favorite DPM SD caras and let's see if there will be anything better okay the results with this prompt are still a little bit awkward but I think that it is probably because of these uh tags and our prompt so I'll try to remove them okay these parrots are a bit strange Let's uh uh try to change it maybe to nine and see what will happen uh okay they are slightly strange um but I think even with these results we could potentially work for example if I choose this one and send to image to image uh I like the uh composition I like the Shadows and lightning so I want just to fix it uh that it's a bit you know uh wonky uh so what I'm going to do I will uh add some sampling steps and I will resize it by two so it will add more nice details uh and the The noising Strand I'll set it to somewhere like 0.4 um this image to image TP is very interesting one and of course I'll make the separate video about it to show it in more details how to work with that but now I want just to try to improve these picture that we got from another model and you see now it looks much better uh it still has some artifact on the bak and some strange stuff here uh but in general we could improve it we could just fix these small things and that pick is not that bad uh so what else I want to try uh yesterday I was playing with this model and I created some interesting pictures uh so to save the time I will just use the peni info and send it to text to image uh to show you the prompt I was using uh so it was half woman half parrot assassin with wings and Feathers highly detailed digital paint art station concept art mate sharp Focus illustration and etc etc some Styles and of course our Laura and Trigger W so with this prompt we can try to switch to dream shaper and we will try to generate okay so it generated a warer woman with the wings uh and the feathers are kind of the same color as the drais feathers but still there is no bird so let's uh set it to four maybe it will work okay there is a lot of flying woman uh women but there is no real one that I want to see let's try just Griffon uh okay these are interesting and I really like the one in the left bottom corner uh where drais is kind of sitting here with this girl so I'll send this one to image to image and we'll try to improve it a little bit so let's uh resize it by two and The noising Strand let's it set it in the middle and let's try to generate uh okay now we are getting the better version of picture uh and uh the only messy part is here with this hand and we could probably try to add bad hands here and then go to generation and and make the noising Strand a bit more and to see if you're going to get something interesting okay so the girl is nice but the parrot is a bit broken here I'm probably will set it a bit more to let the model to change the original picture a bit and you see I like how it changed the clothes uh here it is kind of a feathered armor when here it was just a Tex style oh wow this one is even more interesting I'm already curious what else it can do yeah the hand with a parrot is a real problem here because it can't paint the hand properly so we could probably add add hands to negative prompt okay here it even uh added some clothes to a girl girl's hand that's ridiculous uh okay this one looks quite funny it is like three birds um okay these clothes are a bit ugly but in general it looks good and I like the background so I will uh send this picture to uh image to image again uh and uh what I'll do I'll go to in paint and now I'm going just to fix this close and I go in paint masked original and then I'll put here just bird close and I'll try to regenerate it we don't want to resize it by two though so let's try uh and okay it is a little bit better now but still this clothes are too big for this bird small Maybe okay it is still not completely the drais close one maybe if we add Laura back it will make better yeah that's a tricky part when you almost have a perfect picture and you need to fix a little detail it can take some time but all right it repainted something it added a hole to this um thingy and uh anyway I don't want to waste too much time on this little piece of picture so you can see that if you trained your Laura you can use it with different styles different models and result can be quite interesting and through a several iterations you can get quite um amazing results I hope this video was useful for you uh let me know in comments what do you think and uh if you have any tips and tricks about Laura training also share them and don't forget to like And subscribe on my channel thank you

Channel: Code & bird

Views: 20,051

Keywords: lora ai, lora ai art, lora ai tutorial, lora ai training, sdxl lora training colab, sd lora guide, stable diffusion, stable diffusion tutorial, train ai model, kohya ss, kohya lora training, lora model, lora model stable diffusion, train your own lora, LORA, train ai model with images, low-rank adaptation, lora training, lora training free, lora a1111, lora 2023, ai tutorial 2023, best lora settings, realistic model lora

Id: oksoqMsVpaY

Length: 30min 12sec (1812 seconds)

Published: Mon Oct 09 2023