Stable Diffusion on Low VRAM GPUs - Token Merging, Xformers and other Methods

so whilst on paper stable diffusion requires a graphics card with 8 gigabytes of vram you can actually in practice try to run it with less than that and there are a number of techniques for doing that that you might want to test on your system if you have less than eight gigabytes of vram we've looked at X formers so this is the web ui-user.bat file which we used in installing stable diffusion in a different video we've moved on now to a situation where we've got slight adjustments you've got echo on either that's an adjustment that I made from Echo off it just means that these commands print as they are being processed inside of the command command prompt and um yeah I just prefer to see things sometimes as a stable diffusion is starting up you'll notice as well exformers has not this is the one that we put in here in the command line arguments this guy will install xformers it's going to run X formers each time that we run stable diffusion it reduces the amount of time but also can reduce the amount of memory required by by stable diffusion so it is beneficial in that sense it can reduce time and memory requirements you'll notice that there aren't any quotation marks around exformers that's because inside of Windows I find that you don't always need to put those quotation marks sometimes it's just a little bit easier to see what's happening without quotation marks let's go to the command line arguments and type in dash dash Med vram Now if you have less than eight gigs of vram typing this in Saving it and restarting stable diffusion will allow it to run with less memory so if you're having problems with memory insufficient memory errors trying this will let it run slower but with less memory it breaks down the processes in such a way that they take longer to run but they use less memory less vram they use overall I think as much memory but I think they start using system Ram now instead of this one which doesn't hurt performance too much there's another one that we can try which is Dash Dash low vram now again would save this restart stable diffusion and what would happen if we had this there is that it would it would run a lot slower so there'll be a devastating impact on performance but it would run with low vram how low we don't really know I've heard four gigabytes is possible with automatic 11 11. okay so those are two ways of running with less vram xformers will increase performance the med vram and low vram will reduce performance let me delete this here so that and save the file so that it doesn't actually save and run next time and that's because I don't want to run that because it slows things down you should only really use low vram in really extreme situations where you absolutely need to use it I'm just going to delete a few lines in for a bit of fun we can see something here we've got some remarks so inside of Windows if you put two semicolons before some text it actually doesn't read those when it's running the file so that's a simple way of commenting out some of the stuff inside the file so that it doesn't run each and every time let's save that and take a look at another technique for reducing the amount of vram required and here we are with some images that I created before this one was created using xformers and there's a there's a small disadvantage with xformers which is that although it runs faster with less memory it can from run to run change the way images render so if you I think let me see yeah this is done with xformers these are six images and the six images are exactly they're done with exactly the same parameters nothing changed so from one run we have this image next one slightly different image and that's just X formos that's how it works you have to accept a bit of variation in how it renders notice that some of the images don't seem to change at all when we start using merge tokens which is something which is built in to stable diffusion if you have a latest version of staple diffusion you'll have merge tokens it is different I I did the these six images using merge tokens cranked up to the maximum and the results were very different you can see with this one we actually lose a person what's happening so with merge tokens it is designed to introduce some changes but it does speed things up but what I found with these six images was that although this one's much more detailed this one's a lot less detailed it took just 10 seconds less every time it took about one minute 10 seconds to get this about 1 minute 20 to get that what's going on I think what was happening was that I had basically chosen settings where I wasn't going to see much improvement with the new merge tokens did a bit of extra research and what they suggest is it works best when you're dealing with large images and it also works best when xformers is not operating so I turned off xformers and then worked with the larger image and let's see what we got I decided to test uh merge tokens using stable diffusion running at 1280x720 very complex very very complex prompt which was designed to test how the system responded so we have large images we've got a prompt which just tries to get the software showing off something that's going to be a little bit striking a little bit unusual uh running with merge tokens turned on and running without uh X formers as well so that we could just test the performance of exformers against the merge prompt uh and what we're trying to do is to get it to produce images like this where there's a kind of interaction between the the the person another version of the person we've got some kind of uh mirror or some kind of window and it looks as though there's an interaction taking the place where it's almost as though they're coming out of the uh out of the out of the mirror that was the idea it was something creative and it required very very careful prompting now the idea was to see which rank with the lowest memory requirements but also to see which ran fastest so at 1280 by 720 and using just xformers we were getting about 1.15 iterations per second and that meant we took 3 minutes and 49 seconds to do all six uh all six renders and for the most part I think the outcome was pretty good it's kind of the stuff that I expected in terms of the the quality of the output I'm pretty satisfied with exformers and yeah I think that's going to be the standard so for the total amount of time taken without exformers and without token merging we got results of 6 minutes and 41 seconds and the overall appearance was very similar to the appearance of the X formers with some slight differences in the appearance so with token merging set to 60 percent we got results that were familiar but somewhat more flat less Dynamic than the other ones that we got this is a good example you've got a guy's hand holding it looks like a sort of flat kind of a Flats monitor screen mirror some kind of device the the the fingers and the hands were fine I spent a bit of time prompting uh good quality hands and fingers here another great example so with the token merging you can see that we've got this odd plant coming out of the the mirror so the prompt included beautiful mirror beautiful framed mirror which is also a portal to a different dimension so you can see the girl kind of coming out of the mirror that's fine that's Portal to a different dimension this plant here I didn't ask for I asked for IV uh and some of the other ones they put IV there but they also put this plant this kind of fur plant it was also there I don't know where it came from but it this doesn't look quite as beautiful as I had envisaged her eyes look a little bit wonky that's not a consequence of token merging that's a consequence of something within the model that I'm using and overall I found token merging that this is an example with exformers same seed you've got this beautiful Ivy things are a little bit more coherent the colors that was the other thing I noticed with the colors here the colors are much more vibrant a little bit more muted here that change in color took a number of renders to discover the colors were more flat with token merging and that was probably the most noticeable thing out of all the things that I noticed Now to turn on token merging you go from text to in text to image up to settings which is a separate tab then you look for optimizations and in optimizations we have the token merging here so we can start from disabled we can go all the way up to 0.9 which is the highest and it's a good idea to experiment with some of the ones in the middle and there are also other token merging options here as well for other purposes once you've decided you can hit apply settings it will confirm the setting and there you are ready to go with token merging [Music] [Music] thank you [Music]

Channel: Pixovert

Views: 10,604

Keywords: low vram stable diffusion, gpu memory stable diffusion, optimizing automatic1111 stable diffusion, stable division, token merge merging, stable diffusion optimization, xformers, performance on 4GB and 6GB systems, stability AI automatic1111 stable diffusion, xformer vs token merging, stable diffusion increasing speed, Graphics cards stable diffusion, VRAM GPUs stable diffusion, ai art

Id: 3Y8xr7d_OUo

Length: 11min 9sec (669 seconds)

Published: Wed Jun 28 2023