Layer Diffuse in Forge WebUI & ComfyUI

have you ever wanted to generate transparent images using stable diffusion well now you can and it's all thanks to layer defuse you can also manipulate foreground and background images so let's get into it this works in both comfy UI and Forge and while I'll be showing Forge first it's still worth watching for comy UI 2 as most things apply generally meaning it's useful to know whatever the interface it's worth noting this is currently for sdxl only though it looks like there is is a 1.5 version on the way to in Forge go to install from URL and paste the URL install and then restart models will automatically download the first time they are needed and get saved into a new layer model directory which is under models layer diffuse will appear in both text to image and image to image tabs but that's still a work in progress so for now I'll stick with just text to image in the examples they use DPM Plus+ 2m SD caras another good one is also Oiler you want to lower the guidance scale as well probably to around five note that only certain Dimensions work a good set to work from is the standard sdxl ones you may already be familiar with but from my testing it seems happy with values which are multiples of 64 hence something like 1024x 824 will give you an error there it is type error non- type object is not iterable oh no but if I change the resolution to 1152 by 768 it's absolutely fine you'll need to expand the layer diffuse section tick enabled for it to work and you can also pick one of the six available options the first two are different methods to achieve much the same thing an image with a transparent background let's take a look at those now attention injection gives us this slightly green bottle and you get two outputs one with a checkered background there so you can check the transparency yeah and the other one is the fully transparent image without the background do note that at the time of the making of this video only the merged checkerboard image is saved by default in the future this will likely be different as there is already a pull request to change this and of course the images are in your gradio temp directory anyway so you could always snag them from there right where was I oh yes first off you can see not only do you get a nicely centered image but what isn't immediately obvious is just how good the background removal is let me show you what I mean if I generate another image I'll use the different type this time just so you can see what that one does and now we've got a slightly more clear glass bottle but with some interesting Reflections now normal background removal doesn't handle these very well there's the fully transparent image so these little Reflections and things normally not very good but you you can see in this case it looks very nice indeed the other thing that works great is fine detail such as whiskers and hair if I go to the fully transparent one there you can see check out that tail it's a little bit tricky to see with the checker board but it's it's a little bit better on there but you got the whiskers coming out very very fine details you get to see it in a bit more detail in comfy UI when I show The Mask as well but basically it can make very nice masks all right so say you now want to add a background to that transparent image a bit like those old-fashioned artists used to do back in the days of things like Photoshop if anyone remembers that the next option from foreground to blending is what you can use how about a nice Forest the first thing to do is pop the picture into the layer diffuse section there and then up at the top you can prompt for whatever you want in the background okay let's generate see what happens there we go we've got him in a nice fantasy forest and of course if you want an entirely different background just change the prompt and whatever you want will magically appear there he is now inside a bakery as you can see lots of fun to be had there but there is even more what if I just wanted that Bakery well that's no problem for the next option from foreground and blending to background though there are a few more steps to this one first up drop the image you just generated into to the new image input box next change the stop at Value to 0.5 then they also suggest changing the sampler to a non DPM one such as Oiler for better results and now when you do the run the radent has gone and you've just got the bakery the final two options are similar but this time they're focused on the background to slap something onto a background just pop that in as the layer diffuse image and select from background to blending and a prompt for whatever it is you want to add on top boom there he is our rodent spellcaster just like we did before if you select from background and blending to foreground then use that image in the new input image box generate again and this time we get just the rodent we generated and the background has been removed all right that covers the basics in webui Forge let's take a quick look at comfy UI a bit like we did in for you can install this via manager using the install via git URL option just enter the URL there install it and restart just like you did in Forge as well also like in Forge the models will be automatically downloaded the first time they're needed and are also saved into the models layer model directory now this is just an example workflow so I can show everything at once but it is available to patreons already should you wish to use exact ly the same one down here the diffuse apply node is for those transparent images and you can pick either method for the decode there are a couple of options I've got one up here where you get a little mask and this one is a little bit simpler but the main thing here is just to show you the two different masks so I've also added a rem BG group in here and I think if I zoom in a little bit you can see we've got some very finely detailed whiskers on there which are slightly present on there but haven't got it on that side the boots are a little bit more detailed so as you can see the mask is quite good quality for the next group I've taken that image I just generated of the rodents and added some cabbages to the background exactly the same again in the next group but this time I've changed the layer type from foreground to background thereby allowing me to add a large tree on top maybe I just want that tree though which is exactly what happens in the next group and the diff apply node once again this has a foreground and background layer type option you can of course have a little bit of fun with these as well and make weird stuff so here I'm removing the tree from that image we generated before and replacing it with some cabbages or indeed with a cyberpunk style rodent does it work well with other nodes such as IP adapter yes it does there are got my Mage rodent and I want to use him as a base for one with a bonnet and some leather armor which comes out perfectly excellent stuff and you know what else is excellent this next nerdy rodent video

Channel: Nerdy Rodent

Views: 9,114

Keywords: nerdy rodent, ai, ai art, best ai tutorials

Id: En9EOlDfldU

Length: 7min 47sec (467 seconds)

Published: Fri Mar 08 2024