Stable Diffusion Transparent Generations: SD Forge LayerDiffusion

what magic is this images with transparency even in the glass today I'll show you how you can generate images with transparency using stable diffusion Forge UI go to the extensions tab then click on install from URL since it's a new extension that they are still working on it may not appear in the extensions list at the moment of recording therefore you need to install it from this URL the name of the extension is SD Forge layer diffusion click the install button and when it's ready you'll see a message indicating that it's been installed along with the path where it was installed if you encounter any issues you can simply delete that folder from the extensions go to the installed Tab and press the apply and restart UI button after the restart in the text to image tab if you scroll down you can find layer diffusion there to enable it just check the enabled box it's also available in image to image but at the moment of this recording it's not working yet let's give it a test to see what it can do I'm using the Juggernaut XL model version 9 for the prompt I'll add a 3D fantasy Shield the sampling method is DPM Plus+ caras but if it gives you problems with this extension they said eer a is more stable for the size I'll use 1024px then I'll enable the layer diffusion extension and hit generate to get our first generation with a transparent background the first time you use it it will download the models it needs in my case it downloaded three models but when I tried another option it needed another model however that download got stuck somehow if that happens to you here's how you can install those models manually on this hugging face page you can find all the models that the extension tried to download you'll need these for different options and some of them are quite big so it might take some time to download them you can download them using the down arrow download button and save them in the following location go to the web UI folder then models then layer model and here you put all the downloaded models now let's continue from where we left off after you have those models and all run correctly it gives you two images one with a square transparency grid behind it which is for preview purposes and is not actually transparent and the second one is the shield with a transparent background let's try another one to see what it can do depending on the prompt and Seed some results are pretty good While others may still have errors let's try the second option for this one it uses another model from the list I showed you earlier called conv injection I believe it might refer to convolutional injection according to chat GPT attention injection integrates attention mechanisms for capturing long-range dependencies while convolutional injection incorporates convolutional layers to capture local patterns and data but I have no idea what those mean since I am a designer and didn't need all those terms from what I saw the model is bigger for this one and I think I got better results with it however it's hard to compare since even if I use the same seed and prompt changing from attention to convolutional injection changes the actual generation let's try a prompt for a fantasy Blue Bottle a 3D render what's cool about this is that we actually get a transparent bottle um which background remover cannot do so I think that's pretty cool I hope they can improve it further to work with all the options I'll show you later where it has some problems let's change the prompt to a pink bottle and see what it generates I'll open the bottle in Photoshop to confirm that the bottle is actually transparent not just the background I'll drag a shield behind and move it around to see which parts are transparent you can generate pretty much anything like robots or people for people it's ni nice that it gets a nice Contour around the hair which is pretty difficult to remove the background in other software let's try another option um called from foreground to blending and this will allow you to add an image uh with a transparent background for example let's drag the robot that I just generated since it has uh transparent background now for the prompt um you'll want to mention what should be in the background of that image you just dragged for example I can add a battlefield and some explosions and now when you generate you basically get the foreground which is the robot in this case with a new background each time to show you the next option I will temporarily disable the extension and generate a winter landscape quickly now I will choose from background to blending option and drag that winter landscape into layer diffusion in the prompt I'll mention what I want to uh uh appear in that scene like a Snowman when I hit generate I get a snowman but with no background because I forgot to enable the extension let's enable it and try again now each time we generate we get a different Snowman on that background they say that the extension is still in work so it still has some errors for example when I generate an orange bottle at 1024 pixels all works fine but when I try different sizes like 1200 pixels I get an error and you can find the error in the command window probably that's why it wasn't released on the extensions list yet so you can wait until it gets a better version or just try how it is and use the things that do work as soon as I change it back to 1024 pixels it works how it should unfortunately the highres fix doesn't work and this would be quite useful it doesn't give me an error but the generation is not how it should be and the background is not completely transparent you should also know that all currently released models are for sdxl models for SD version 1.5 may be provided later if demanded as they say on their page you can find more information I will add the links I used in this video somewhere in the comments uh they say it's a work in progress and that for now image to image doesn't work but probably in a week that will work too you also have more examples and tests that you can try for different options on their page there's also uh a tab with issues where you can uh report the problems so the developers are aware of them I can answer questions about design but I don't know much about code I also want to thank you for your support I finally got my channel monetized after more than a year of work leave a like if this video was helpful thank you and see you soon um with a new video tutorial

Channel: pixaroma

Views: 9,472

Keywords: Stable Diffusion Transparent Generations, SD Forge LayerDiffusion, sd-forge-layerdiffusion, stable diffusion transparent background, transparent background ai generations, create transparent png with ai, generate transparent png with ai, stable diffusion tutorial, stable diffusion, stable diffusion ai, webui forge, forge ui, stable diffusion forge ui, pixaroma, stable diffusion extensions, stable diffusion transparent png, stable diffusion tutorials, stable diffusion xl, png, ai

Id: c03vp7JsCI8

Length: 7min 1sec (421 seconds)

Published: Mon Mar 04 2024