Training SD Models with AMD GPU's in DreamBooth! Surprisingly fast results!

hey there hom labbers and Engineers Fe engineer here so today we're going to take a look at creating your own custom models in stable diffusion and we're going to actually go over quite a few different things here but we're going to get AMD gpus running our own custom Training to create our own models let's go in order to train our own models we are going to take a look at this extension called dream Booth it is probably one of the best known ones for training you can set it up exactly the way you set up most things inside of automatic 1111 come over to available click load from just type in dream booth and because I have it installed it doesn't show up here but you should see it there's just a single button go ahead and click it once you install it will end up showing here inside of your extension list go ahead check for all the updates and then apply and quit once you're done and then restart before we get too deep into setting up and training models I wanted to go over if few different notes that I've had over the past several days while I've been debugging this figuring it out and getting it to work the first note is that from what I can see it may be difficult to run this with anything less than 12 gab of vram on an AMD GPU I'm pretty sure inidia cards get to use Transformers I'm pretty sure that Transformers helps cut down the memory usage significantly for NVIDIA cards unfortunately meta does not seem overly enthusiastic to support AMD cards with Transformers at all so AMD cards cannot use Transformers at least now and because of that it ends up being very memory hungry when training also there is a package called bits and bites it doesn't really support rockem although there is a library that somebody made that has compiled bits and bites with rockem I have not tried it yet I don't know that might help with memory it might have no effect at all the current one for rockem is considered generally unstable so if people want to try it I will try to find it and grab the link my recommendations when it comes to training models is to start simple it is somewhat complicated and easy to end up doing things that either will not work or will only have I don't know minimal effects starting simple you can always train more train longer so I would recommend starting simple because this seems to be more of an art than a science I tried training sdxl models it blows up my computer basically every time I run out of vram entirely I have 24 gigs of vram I run out entirely it entirely freezes my computer most time I have to do a hard restart and actually push the power button to get it to turn off so I would recommend people don't do sdxl at all people are entirely free to try but I have not been able to get it to run overall I think training has gone in my opinion surprisingly quick it's faster than I expected so I've also been able to train luras but I haven't seen them honestly be all that effective along with training I would say this is one of those things your mileage may vary a lot depending on which model you base everything off of depending on what you're trying to train sometimes the you'll see the effects very quickly and very easily and other times it'll be very difficult to see the effects okay so that's it let's uh let's train our model one of the very first things I want to call out here inside of stable diffusion web UI when you install dream Booth you very well may need to actually edit a one simple file if you go over here to extensions go to the dream Booth extension you will then find a requirements.txt file go ahead and open that up and inside of here you will see this diffusers and when you initially install it it will say greater than or equal to 0.231 there is a newer version of diffusers out it's 0.24 this seems to have problems for some people so I would recommend changing this greater than or equal instead to be equals equals and that will ensure that it installs diffusers 0.231 so go ahead save this file after you make that change in order to actually install the new requirements we're going to type source V bin activate that puts us inside of our virtual environment from there we are going to pip install minus r extensions SL sdore dream Booth requirements.txt since I've already installed it you won't see it on my screen but yours will then go and flip over that diffusers from the 0.24 one that is probably installed if you just installed dream booth and it will ensure that you have that 0.231 version with that then go ahead and reload and restart stable diffusion and when you do that you will now have this dream Booth tab up at the top in order to create our own model here under settings we are going to go over to create give our model a name I am going to use a 1.5 stable diffusion model so you'll need to choose the appropriate one but anything stable diffusion 1.4 1.5 will fall under this v1x if you're trying out a stable diffusion 2 model or an sdxl model make sure you use the appropriate one and then the source checkpoint is going to be the model that you are using as a base to build on top of so with that I'm going to choose dream shaper 8 I have not actually done it but if you hit the show Advanced button right here there's an option here to unfreeze the model and I have never unchecked it but it does say that it can increase your vram usage so especially if you have a lower vram card I would try and uncheck this first what we want to do first is get a model to train successfully and then we can obviously go back and add more steps and do more things so since we have this all set up we're going to hit the create model button you'll see it starting to do things and this sometimes takes a minute or two when when I was testing with sdxl this actually took quite a while for stable diffusion 1.5 models it doesn't seem to take very long so okay it says the selected model Fe test model since we've created it now we can see in this select and we'll see Fe test model is here here so this is the model that we are now planning on training against over here in Concepts I took a bunch of pictures of just my kitchen countertops they are a quartz countertop and this is effectively what they look like there are not really images like that that I've seen stable diffusion create so I wanted stable diffusion to be able to create a similar pattern and for that I have a directory of 12 images and it happens to live at slome slj desktop and it they are all inside of a folder called cropped so all you need to do is put the directory of all of the images that you want to train on it doesn't need a forward slash at the end the prompt this is how you are going to effectively trigger whatever you have to show up if you are training a person I would recommend not using exactly the name of a person like Bob and instead use something that will not be a word normally that stable diffusion Associates with anything else I've found you can do things like put something like fe-h and since I'm training like a marble pattern you could put fe- marble you could also just put something like Fe marble if you were doing something that maybe you wanted to have multiple different prompts that it gets associated with you can add more things put things like kitchen counter tops in there again if you're doing a person you know you could would start to have your images associated with very generic keywords like girl you can have it be a custom one like Fe girl which is not going to be a word that stable diffusion is going to associate with anything else the more images you train on the more likely you're going to get good results if you're training something like people you need people you need pictures of the person or people from quite a few different angles quite a few different you know viewpoints as well as more variety and clothing and stuff like that positions you know hands arms legs so effectively more will give you better results but to start I would recommend just keep it very simple and if you are doing a person try doing a face and make sure you get pictures of them looking head on to the sides with their head up with their head down you know their head cocked to the side Etc this will give you much better results in the end because stable diffusion will do a lot less guessing on what does that person look like when they're actually viewed from the side if you have a picture for it it will know exactly for me I'm going to set my prompt as f- test marble so this is what I want to use to trigger stable diffusion to know that I am trying to get that marbly pattern I'm going to leave instance tokens blank I'm going to leave class tokens blank I'm going to come over to class images I'm going to just zero this all out I don't need any of this it doesn't we're not using this at the moment so just go ahead and get rid of it you shouldn't need to do it only if you put a directory in here and then only if you have this uh class images per instance raised up above zero should it use anything sample image prompts this is is something that at the end when you want it to generate an image with your prompt in it this is how you do it luxury kitchen luxury home kitchen Fe test marble counters so once we create our model it will try to generate an image and when it tries to generate an image this is the prompt that it is going to use to generate number of samples to generate you can drop this all the way down to zero if you want or you can just have one you can change the sampling steps if you want again if you're on a lower vram card in some ways it may make sense to just zero zero this out anyway because this will effectively take more vram now we come to parameters inside of parameters especially when you have Advance checked there are a lot of different options using EMA adds more vram caching latence uses more vram I would uncheck those this 8bit atom W this requires bits and bytes that's why in the beginning I talked a little bit about bits and bytes you can leave it on this and it will not work unless you have the specific bits and bytes compiled with rock M for everybody else I would recommend just use torch atom W if you don't have that bits and bites it's automatically going to move you over to this anyway I have found that this mixed Precision of bf16 seems to work checking full mixed Precision also seems to help if you are training a model you want to leave train unet on leave everything else exactly as it is intervals the training steps per image this is effectively how hard it is going to train each image for starting out just leave it at 100 the save model frequency I would bump this up a lot and it's because I believe it ends up using more vram and it could end up causing your training to stop since I'm training 12 images and each one I'm doing 100 steps I've typically been putting this at about halfway through so it'll save halfway through and then it'll save again at the end the save previews frequency I would put it all the way down to zero again it's one of those things it can end up eating a little bit more vram and to start out just zero it out batch size I would not change anything learning rate I would not change anything leave that as it is Laura I did train a Laura honestly I didn't think it worked very well I was not impressed so I would recommend starting out not to train luras but you can train them and if you wanted to train them all you need to do is check this box when you check this box you will need to go back to the model and then you will need to select a Laura to train against so I'm going to uncheck Laura for image processing we're going to leave the max resolution at 512 you can try lowering this down that may help especially if you don't have a lot of vram or if you end up running into vram problems prior loss do not touch saving you can go ahead and play with these I haven't done a whole lot of testing but the defaults seem to work okay and this will save the EMA weights you can try using a half model I have not tested it quite yet but I would generally speaking leave these as they sit inside of image generation I've found using native diffusers and UL ancestral discreet I have not had any problems I have had problems when I've tried different ones this one seems to work fairly well and then down in extras don't touch most of this but if you wanted to so the images that you use will very likely be larger than 512 and what you can do is run this and effectively say slome SL John slash desktop slash images to train and then the destination path home slj SL desktop SL cropped you can create a directory to put your cropped images inside of you can also put the directory of the images that you want to train and you can actually have it crop them right here that way it doesn't do it while you are actually training then if you have any problems or anything like that you won't run into it because they'll already be cropped and if you notice in my Concepts my instance images I'm using the directory with all of the cropped images and with that you'll want to hit save settings so now every everything should be saved so if your computer decides to blow up you should have all the settings at least saved after that go ahead and hit train as we can see training has finished it has done 1,200 total steps it took almost 6 minutes which in my opinion seems extremely fast and now it went ahead and generated this image and it used the images that I had to make these countertops now you can see that back here it looks kind of like trash let's try and generate a few images and see how it does so here I generated an image and I generated it with my model and the prompt I put is a Chef cooking in a kitchen white quartz counters white quartz is what my kitchen counters are technically made out of but if I change white quarts to be Fe test marble counters which that was the keyword and we go ahead and hit generate these do look a lot more like my counters I don't know what happened to the Chef cooking in the kitchen so there you have it that's how you can train a model your your mileage will definitely vary again this is a bit of an art form more than a science I would absolutely recommend start out very very simple and you know move along to potentially more training steps you know use EMA and things like that but to start out you know try to use as little vram as possible and try to get it just to work before you then try to perfect everything with that I hope this has helped folks uh you should now be able to train models I know some people have been having some troubles with it I know some people haven't been able to get it to work I've been able to get it to train these stable diffusion 1.5 models fairly reliably I haven't had very many problems with 1.5 models I would highly recommend before you train a Model start your computer up don't do anything else go and train after you finished training a model entirely reboot your computer you can try just shutting down uh stable diffusion but I would reboot entirely clear out all the vram so that nothing is held over because I've just found that the vram is a big problem with AMD cards it gets used up it doesn't get released it you know it ends up using using every single megabyte of vram and it can crash your computer and freeze things up so when you want to train start it up do your training shut it back down restart it entirely and then come and play with it and stuff like that I hope this has helped folks have a great day thank you so much for watching my videos home labbers and engineers I create and edit all these videos on my own so any likes and subscribes will massively help out the channel and allow me to continue creating content to help people if you got value out of this please consider liking and subscribing to the channel to be notified when new content drops if there's something I've not covered but you would like to see a video on it please leave a comment down below and again a massive thank you to everyone I hope you have a great day

Channel: FE-Engineer

Views: 2,938

Keywords: Auatomatic1111, AMD GPU, ROCm, Stable Diffusion, Dreambooth, Training SD Models

Id: gDFEtFohMFI

Length: 26min 55sec (1615 seconds)

Published: Sat Dec 02 2023