ComfyUI Fundamentals - Masking - Inpainting

okay hello and welcome to a comfy UI fundamentals tutorial and we're going to be covering masking today now I'm asking is a process by which you use a black and white image or a grayscale image to select an area for use as you you might need it so you can use it for cutting one image out of another image you could use it for selecting the area to do a particular process on you can use it for all kinds of stuff like that and it's a very useful thing to use in Comfort UI or SD in general and anyone who has done any impainting will know that you are using a mask in order to do in painting now the mask is often created by you painting a thing but there's no reason you can't load a a mask from a black and white image on your hard drive you just load it using a load image as mask you could use a load image node but it won't allow you to select what channel to use in here which which is not really a problem in some cases especially if you're working with Alpha channels but um for ease of use it's better to use the one that's intended for it because it's a little bit more flexible for our purposes so if we bring this down here you can load up something like this using the load default I've the only difference between a normal default setup and this one is I have a bus here and the bus is basically it's just a way of sending information through your workflow in a um understandable and organized way all right so with your key sampler you can do this really cool thing where if you right click on a preview image you can copy the image in a preview image and you can paste it into another image node so by doing that we've given ourselves something to work with now this will work with any image you could put any image you want in here and do this and if you right click on this and go open in mask editor I'll open this window and you can actually paint areas where you wish to select so if we want to select this bird here we can draw over the top of it and we can save to node now this area is now masked it's an area which is selectable and we can use this area in order to do things in this particular area of the image so one of the ways of doing that is we could use this mask too um let's grab up how about we use a I think yeah there's a in masquerade nodes there is a mix color by mask node now this generally accepts masks as a image thing oh and that's because some custom nodes they won't use the green mask um noodle I'm not quite sure why they do that but in order to avoid any problems with that we can get a mask to image which should give us what we need in order to make this function so if you ever come across a mask which is blue you should be able to fix that by just doing this now this is mixing color by mask which means if we select this we are sending this node the bird image and we are sending it this selected area of the um the bird image as well and if we set this up to that and then we send a preview image out and we click prompt you'll see that it's turned this area into red so that's one of the things that um it's one of the very useful things that mask will allow you to do is to turn an area into a certain color obviously this is not always what you want say you want something that is red but you still want the original image you can use another node which is in the standard set of nodes the post processing blend node and so now we're going to blend this image with this image again and we're blending it by half now if we put that out again and we've made our bird more red now we can tweak that higher and it gets more red or we could tweak it lower it gets less red if we go pretty high and then we can also change the blending option so we could use screen for example we could use um soft line now let's use multiply see how multiply has made it kind of dark the whole image darker the problem with blend is it's not one area I think there's a image blend by mask somewhere we'll just see if there is and yeah there we go if you use this you could only blend it using the mask as well in fact I might as well show you that so if we grab the mask from back here plug this in and we'll plug this one in here again and then we'll plug this one in here which we're emulating this one but we are now only blending it in the area where the bird is preview the image and run it again and if we now you'll see that the background has not changed color the problem with this node is it doesn't have a blend mode on it which I find very frustrating but it does allow it only work in the masked area so if we do that it'll just change the area where the bird is so that can be really useful if you just want to change the bird's color you can do this kind of process now I'm going to show you how you can use this process in order to in paint so in painting is something which everyone often finds themselves using in order to fix hands and fix messed up faces and that kind of stuff and so in order to do that kind of process you need to have access to sorry we could do it two ways there's two ways to do that we could use a a encode for in painting in which case we plug it in like so and we grab the mask from here and then we send it through to a key sampler C control V I wish I could control shift V these so I can just move the whole lot and send them but uh someone in the infinite wisdom didn't code the re-root nodes very well no offense to come few ideas sometimes you just can't do things because uh of technical reasons all right and so now we have a sampler and now we have a VA sent to the sampler and this they encode for impacting is specifically used for this task which is clipping areas with a mask and rendering so oops I've rendered without actually having an output which is a No-No so let's add a preview and we'll also need a vad code and Faye all right so I'll just put this here and now to generate it all right so you'll see that what it's done is it's re-rendered the image with the bird now you might think that's fine that's that's great that's what I want from a thing if you want to generate hands but you notice that it's doing something odd right now where the bird is not actually fitting within the area that you have just attempted to generate it's actually being cut off of this section because it's like a little window and it's just got hard edges on it and it'll cut it'll try and fit but it's not always going to fit so as you can see there's some glitchiness to it sometimes it works quite well but there's better ways of doing this and I'll just show you one of the other problems with this is if you want to just change your original bird a little bit with this method um say a tiny amount it's not going to work because that's not what it's one it's asking you to do when you create this process it's actually sending this gray image and then denoising that selected image which basically means it's like it's cutting this air and going I'm going to denoise this area but I'm going to denoise it completely differently to the main image you had which means that this small section may be attempting to generate a different bird and a nest image than the one you have which means the bird could be completely out of out of this hole and not even there it's uh yeah it's very frustrating if you try and do it this way and and a lot of you will have tried to do it this way because you type in pain and you think that's what it's that is the thing that would be doing it but um this isn't actually a doing painting at all in fact this this encode vehe thing um is almost completely useless for actually doing proper in painting because of this entire turning the thing completely gray bit here so let's get rid of it we don't want it it's dead we're going to use a different one we're going to use a where is it how to be in latent latent I've got to find it first all right in pain there it is set latent noise mask so this is actually how you do the implanting that you would have been you would have probably used on another another user interface which is how it it uses some or most of the um image that is actually already there in order to do your in painting so you connect your load images mask to your satellite noise mask and instead of at zero this being blank instead it's mostly the original image when it's at a loading noise value when you get higher it changes even more which is much more like you would be after if you were trying to fix things like hands and faces and so on so as you can see it's it's um it's the better way of doing this it's the correct way of actually doing this and for a long time I was actually using the other method and thinking oh this is busted come for you later than how to do it properly but um no there's just a different node and it's not named in a way that would make you think that it actually has anything to do within painting which is kind of frustrating but there you go so that's how you can do in painting on comfy UI for those of you who have been wanting to be able to do in painting for a long time so you can right click on here you can open your mask editor find your hands and then go okay save to node and then it'll just run and it'll redo the areas you've selected all right so now that I've shown you in painting what else can we do with masks that's really cool I've shown you how to modify an image with a mask but what about if you're too laid it in paint with with your hands what if you just want the computer to do it for you wow that's where some custom nerds can come into play and we can use a node called clipsec there's a few nodes that do stuff like this some there's quite a few different um custom nodes which you use some form of clipsec or some kind of an image recognition thing and what clipsec does is it scans the image and it selects Things based off of keywords sir if I have a bird in a nest and then I decide that I want to select it I can then have it select it for me so now it's going to Output once it's given a nap there we go it's automatically selected the bird so I don't even have to do it myself and if I now want to make use of this I could send the bird area to here and generate a new bird of course so I could just keep generating different birds in the same nest if I wanted to um but no we can go even more complicated than that what if we want to change what is in this image well we can run a different prompt over here let's create a yeah let's create clip text encode now clip text encode is usually way back at the start of your image but it doesn't have to be it can be anywhere that you need it to activate so you can actually send it to a new node and we're going to say a tiny cute dragon in a nest all right so now it's going to attempt to turn the bird into a um a dragon just by having selected it this way that does not look like a dragon to me try again please that looks like some creepy monkey thing I put the word cuteness it's doing weird little things but um as you can see you can replace the thing in there if you have this on high enough maybe if we go higher it'll do it no it's it's making a nest with Pokemon in it basically but as you can see if you just use this you can change the things in the image if you use the thing from before will you change the colors over here you can also do the same thing you could use this to do all kinds of crazy stuff using clipsec you could use the mask Editor to do the same thing uh you can use clipsec to select other stuff like ground and maybe it'll generate I'm not 100 sure if it'll detect the background properly but there we go it's selected this stuff in the background you could tell it to select the nest so as you can see it's it's pretty it's it's pretty useful I mean obviously it's tried to deselect a little bit of that but you know there's a lot of bird in the nest so okay it's regenerated the entire thing um but as you can see it's a very powerful tool to have at your disposal the ability to use masks and there are some different tools for using masks I'm not really going to go hugely into them but if you have a custom node set called masquerade nodes you can do so much more than just color by mask you can chop one image from one image into another image you could paste characters from a bunch of different generated things into another image so if you generated like um do I even bother make an example if you if you generated say a bunch of characters like um people from The Walking Dead for example and then you generated another scene with um a post-apocalyptic City and then you went okay and cut the characters from each one of those thingos and pasted them into your cityscape you would then be able to just create things with multiple characters quite easily so anyway very powerful tools hopefully that's given you a head start on how masks can be used and some of the really cool things you can do with them thank you for watching and there will be more tutorials coming I'm probably going to do one to do with face fixing at some point soon for when you generate stuff where the face is just completely out of whack more for mid-range stuff where I think people are further away because people close up generally comfy UI and SD you've got that handled these days but uh yes um I'm not going to copy this workflow because I think you should be you should build it yourself if you want to understand how it works um if there are custom nodes in here if you have come for UI manager you should be able to find stuff like hang on I'll just come on load up you can search in here for things like mask yeah masquerade nodes um clip sag clip seg node is in here you can just search in here for the ones you need but they're pretty easy to find all right hopefully that has been helpful to you thank you for watching and uh okay make some cool stuff please

Channel: Ferniclestix

Views: 23,909

Keywords:

Id: g9JXx4ik_rc

Length: 19min 19sec (1159 seconds)

Published: Sat Aug 05 2023