SDXL LORA Training locally with Kohya - FULL TUTORIAL // stable diffusion

sdxl or stable diffusion extra large 1.0 is here and it can already create some stunning AI generated images it's a model that people can base their checkpoints on or train extra networks like loras which is exactly what we will be doing today checkpoints are Standalone and they are often large files while Laura's they are smaller files that are faster to train and can be used on top of different checkpoints kind of like filters that are professional at a certain style object pose or characters likeness today let's train our own Laura to do it we need to install Koya SS GUI it's an open source project that provides tools and scripts for training and fine-tuning models Laura's extra networks and so on this project supports Windows Linux and Mac OS platforms and offers tutorials and document rotation on the Project's GitHub page all the links that I mentioned here can be found in the description underneath this video you can check out their documentation for installation guides for other platforms today I will be just covering Windows to install the necessary dependencies on a Windows system you need to have a few prerequisites first of all python then get these two you probably already have installed if you're using stable diffusion locally or if you watched one of my two installation guide videos if not check them out the third one is the visual studio you just click the link it downloads install it pretty straightforward there's nothing extra that you think about there after that's done you have to create a new folder on your computer I called mine Koya then go to the path of the folder type CMD press enter and copy this command line it's from the project GitHub page once that's done you have to change into the Koya SS directory and then execute the setup.bat command and it will do its magic or you can close the window and just open setup.bat from the folder itself now that you have the setup menu open you want to install Koya so you type in one and enter for torch pick number two and just wait a little bit it's going to install a couple of things that are important for it to run and if you already have some of them it's going to update to the required version at least that's how it was for me now you have to answer a few questions and we're almost done in which compute environment are you running I chose this machine because I'm running on this machine which type of machine are you using no distributed training do you want to run your training on CPU only for me definitely no so I type in no do you wish to optimize your script with a torched enamel no do you want to use deepseat no What GPU should be used for training on this machine I typed in all now for the last question if if you have a GPU Nvidia series of 3000 or newer you can pick bf16 if not then fp16 is the one you should go for all right now that you're all done you can pick option number five and start Koya in your browser or you can go back to the folder and open the GUI dot bat file and it will give you a link to go to the Headless false error is alright don't worry about that one and this is the interface now that you have the software ready the next thing you need to prepare are your images for your Laura training use images of yourself other characters objects Styles but keep in mind that the better the image quality the better your Laura and your final AI generated images will be so don't use something really pixelated blurry and Tiny in size here are some ideas for finding images you can go to the Google images and change it by size large and you can find some really good stuff in there or you can go to Advanced image search and choose image size larger than 4mp or larger than 6mp but the higher you go the less options you will have so larger than four should be good you can also go to royalty free websites like unsplash pixabay and then find images there there are royalty free and totally cool to use for commercial purposes by the way you can also look for websites with high-res movie screen caps I find this option to be less talked about but I think it's super useful especially if you're trying to train a character it is probably one of my favorite ways to do it and you can also go to a website like Wikimedia Commons there's really so many places to look for images just make sure to look for images with different angles emotions lighting distance from camera clothing hairstyles colors and so on variants is very good as it will allow you to generate more varied images afterwards now for this video I decided to train a Laura on Mr disturbies for entertainment purposes only you've probably heard of this guy he is an incredible Creator and now that you have all the images here's a tip you can select all of them press f2 on your keyboard and rename them all at once I think that's a bit better than having all these random numbers and words in the names it shouldn't cause any issues but better safe than sorry right so as you can see I have 23 images here but how many images do you really need for characters you can really use as few as 10 high quality images to train an effective Laura to replicate a specific style you will probably need about 20 images plus to be safe I'd suggest starting with 10 to 20 images and adding more photos if the results don't meet your expectations now in the past it was common practice to crop images you prepared to like 512 by 512s using websites like Burmy but it's not longer necessary and you will get a higher quality lower without cropping the images because Koya has this option that allows you to group your images by certain ratios for training purposes keeping your original images with more details and more information you're not cropping it out anymore so I definitely suggest using this option and not cropping but if your computer can't handle large files you can crop the images or you can use a compressor to compress them now let's talk about captioning the images it is an easy task and it can be done using koias utilities captioning so the most common ones I find are the blip BL i p captioning and wd14 captioning so you can choose whichever one you like more then you just select the location of the folder with your images so in this case I'm using wd14 captioning I typed in Mr B for the prefix and I didn't adjust anything else and I just let it run and as you can see the folder is now being popped populated with text files and here is a little tip from me so the tag frequencies that are showing up I like to copy all of them and I like to create a new text file and put them in this new text file so that I can later reference them in my prompt creation I've never seen anyone else do this so I'm not sure how helpful this dip is but it works for me so I thought I'd share it with you and then in this case I created three folders images log and model then I dropped all of the Mr Beast pictures into the images folder now before we move on I also made a Laura for this video that is based on a style or a Vibe if you look at all these images you might notice that they have this dark atmosphere with some bright spots in certain parts of the image so for this lore I did something a little bit different I have gone and used the blip captioning and I added a prefix dark Shine the prefix in postfix are not necessary but I find it very helpful if you're training a character or an object to make sure that every single image has that caption so table diffusion knows what you are trying to train that's how I like to think of it as you can see the folder populated with text files now I don't find that the prompt is good enough for most of them so what I like to do is open each one at a time I look at the image and I think okay what would I type in to stable diffusion to get this and then I add it to the prompt so the difference between the blip and wg14 captioning would be something like a woman with pink hair and blue eyes and wg14 is one girl solo pink hair blue eyes so depending on how you like to write your prompts if you use damper tags or if you like to just use regular sentences you will pick one or the other if you don't want to open the captioning like this which I still think works very well you also have an option to install Brew dataset tag manager so you just have to download the zip file right there I created a new folder for a borrow that is a tag manager and I drop the zip file into that folder and I extract it here once that done I run the poor data set tag manager and you can see it looks like this and then I go to file and I select the folder with my Mr Beast images and text I can also go to view and preview and have it somewhere here so once I click on every image I can adjust the tags you can see the image tags are in the middle and then all tags are on the right so these are all the tags that are used within all these images there's a lot of helpful tools here like paste a tag everywhere or delete attack from everywhere so this could speed up the process if you wanted to play with it but using just the text files is also good enough so if you remember I created three folders for Mr Beast and you can do that yourself or you can also go to Laura tools deprecated and then it will help you create these folders and everything properly so for instance prompt you have to type in something what you're training or who you're training and if you're training a famous person it is a good idea to type in their actual name because potentially stable diffusion already has information about them or has been trained on some of their images so it will only help your Laura for Mr Beast I would just type in Mr Beast for this style I'm just going to write dark shine because I can't think of anything else at all now for the class prompt it's your classifier like women men object person style dog and so on for trading images you open the folder with your prepared images repeats is how many times they will be trained 20 is a safe bet if you put more than potentially the quality will be higher but also it will take way longer so I like to go with 20. now regularization images okay I never use it but a lot of people do and they say it's helpful so I believe them plus it takes less time to train without them so it's really up to you if you have a folder with many high resolution images of your class fire you can use it here now the destination training directory is where you want to generate these folders for training and where eventually you will have your Laura being trained to after clicking prepare training data all the folders you need should be generated in the selected destination folder images log model and you will also see that it says something like 20 underscore dark shine or 20 underscore Mr Beast this number is added from the repeats you chose please don't rename it koi needs it to train your images now to the fun part we're almost there Laura training so we have here in this tab three different things we need to take care of we need to take care of the source model folders and parameters so the source model we have to pick the source model are you training the stable diffusion 1.5 or stable diffusion extra large 1.0 select that and if it's sdxl model make sure to also check market now we go to the folders your already have everything prepared whether by creating it yourself or through utilities select your image folder but don't open it don't select the 20 underscore dark shine no just the image folder so it says dark shine slash images then we have dark shine slash model that's your output and logging dark shine slash log make sure to add a model output name name it whatever you want and now let's go to the parameters this one is the most confusing out of everything we've done I think there are so many options in this section and so many variations of the settings that I will show you will work just as well there's no one size fits all but I will show you my settings and try to explain what most of them means so you can make adjustments that will fit your Laura best so Laura type I select standard train batch size now this one's interesting for a person I suggest one because it gives more time per each image for stable diffusion to train on so for Mr Beast I selected one but for a style you can go higher and for this Laura style that I'm training currently I selected four it will speed up the process it will look just as great if not better than if I trained on one image if your computer can handle it of course Epoch is how many times you want to repeat the training on top of the original Laura I always go with 10 so I can then test all the loras and find the perfect one save every number of epochs if you leave it at one each Epoch will create a separate Laura file if two then only Epoch 246810 will be saved and so on to save space you can do too but if you can I suggest saving every Epoch and then deleting the ones that didn't work out Mixed Precision safe Precision I go with bf16 but if you don't have a 30 XX series you know like 30 90 for example or newer then you need to use fp16 instead number of CPU threads per core leave at 2 cash latents and cash latents to disk check mark both of them learning rate I suggest inputting a value between 0.001 and 0.004 for Mr Beast I picked 0.004 and for the style I picked 0.003 LR schedule we should select constant and warm-up of zero percent Optimizer add a factor that's the best for sdxl but that also means we need to have some extra arguments I will also leave them in the YouTube description by the way you can also see and copy all of these settings from my blog createxaya.com I posted an article on this exact topic max resolution 1024 by 1024 is probably the best option but you can also use 768 by 768 to save on vram but it will produce a bit lower quality images but not by that much to be honest so for Mr Beast I use 10 20 24 by 1024 and for this style I use 768 by 768 just because the original images are actually smaller as well enable buckets check mark it and you won't need to crop training images text and unit learning rate input the same number as in the learning rate so if we used 0.004 in the learning rate we put the same here no half Vie check mark Network rank so the larger the number the more details the Laura will retain but it will also produce a larger or Laura file size like by a lot it would be the size of a checkpoint sometimes I use 256 Network Rank and one network Alpha for Mr Beast and then I used 32 Network Rank and 16 Network Alpha for the style which will produce a smaller size and it will also look good so as you can see both of them work you can test it for yourself finally in the advanced settings make sure you check Mark gradient checkpointing and don't upscale bucket resolution this configuration will use about 20 Gigabytes GPU memory I have a 3090 and it works well I hope it works for you as well because I know how frustrating it can be if things don't work out or you don't have enough memory fun fact I had a 30 80 and I couldn't train Allura then I switched to 3090 and now I'm able to do it good luck as you can see this one's training the Laura and it took me 42 minutes and 31 seconds 16 images at 4 training batch size now all of your trained loras will be in the model folder for me it's the dark shine model you can copy them and paste them into your stable diffusion models Lora folder then open your stable diffusion and let's see the results so here I opened that common tags file that I told you about and I used a couple of tags from there just to remember what we train on and see how it would work out and and here is the result well it looks somewhat like Mr Beast but not entirely that's fine this Laura needs a bit more work which I will not be doing because this was just for fun I also tried some digital painting Styles illustrations and so on dragons too but that didn't work out now let me show you how to test which Epoch is good and it's my best friend XYZ I will tell you why it's my best friend in one of the later videos but for now let's just type in a prompt and see the result here is an image without the Laura I like to always go before Laura after Laura alright so what I'm looking for is shiny hair Vines and just dark mood overall now we can go to the Laura option in stable diffusion and just select all of your lorus copy them use the XYZ plot paste all these loras and add commas in between now it should go through each Laura one at time and you won't have to monitor it or change anything so just let it run and see the results I think the result with any of these loras looks much better than the original result but I'm just looking through them and trying to see which one is the best and in this case I think it's Laura number seven and eight and then I do that a couple of more times test them all and find the winner in this case I think the Laura that worked out the best is number five and number eight so I can delete the rest and just use five and eight or just leave one if I wanted to and the Laura is trained when I do the same with Mr Beast I find that it is 100 Laura number eight for style is often a bit later in the epochs that it's starting to look really good but for characters you never know and when you're looking at the character you're looking at two things one is the likeness that the Lord our producers of the character that you were training on and two is the ability to be stylized that's very important because if you want to create a painting but Laura is over trained it will always produce a photograph instead so we're looking for a happy balance where the character looks just good enough to be somewhat recognizable but you can still make it into an oil painting 3D rendering and so on so I think here it was super evident to me that Laura number eight was the best option as it produced the best results all throughout my tests even though sometimes it still failed but that's okay I hope this video helped you understand how to train your Laura and now you can go and have fun training them on your own only your imagination is the limit here you can train Elora on anything that you want hope you enjoyed this video and hey if you're still watching check out this one next keep generating cheers

Channel: CreatixAi

Views: 29,801

Keywords: stable diffusion, stable diffusion tutorial, ai art, sdxl, lora training, kohya, sdxl lora training, stable diffusion ai, sdxl 1.0, stable diffusion lora, stable diffusion xl, lora stable diffusion training, kohyass, sdxl lora, automatic1111, lora stable diffusion, kohya lora, sdxl 1.0 lora, ai, stable diffusion ai art, kohya stable diffusion, sdxl automatic1111, kohya gui stable diffusion, kohya-ss, kohya ss, CreatixAi

Id: d4QJg4YPm1c

Length: 19min 58sec (1198 seconds)

Published: Sat Aug 19 2023