EASY Inpainting in ComfyUI with SAM (segment Anything) | Creative Workflow Tutorial

hello everyone and welcome back to Dreaming AI my name is and today we are going to learn how to do some in painting with comfy UI without manually drawing editing masks to do this we will use a technology recently released by meta called segment anything to it it's possible to extract elements very precisely from an image we will also use impacts custom nodes you can find a link to install them in the description I was planning to start from an existing example available to us on GitHub from the impact Suite I in fact by going to the dedicated section for interactive Sim we can save this image and load it as a workflow uh however I reflected on the fact that there are really too many things will be changing in this example and it's certainly faster to start from scratch so let's open a new workflow and begin our journey into the Fantastic world of in painting with Sam uh starting with a blank warflow let's start with the usual model loading pay attention to what model you will be using as not all of them can performing in painting correctly at this point let's add a load image node to load the image we want to modify perfect uh right click on the image and select open in Sam detector this will open a window where we can decide which elements to include in our mask to do this click on one or more parts of the element and then click detect to verify that the selection is successful or you can use the right click on the image for select the point you want to exclude from the resulting mask be cautious because contrary to what you might think many times fewer selection points you use the better it is also use the confidence parameter to adjust to the correct identification of the object in the scene if it's too imprecise when you've achieved a result you're satisfied with close the window by clicking save to node how the gray area will be the part that our model will replace if you notice imperfections you can always correct them right clicking again on the image and selecting open in mask editor this way you can modify the mask created using the right Mouse button to erase and the left Mouse button to draw now just for visualization purposes let's convert the resulting mask into an image and display it on the screen uh continuing with our example let's encode our image with the models vae now let's add a new node called set latent noise mask this node will take our encoded image the resulting mask from Sam's work and combine it into a single latent now move down where we'll set up the usual nodes for creating a new image place the two positive and negative prompts where we'll describe what we want to appear in our mask and finally add the K sampleware to which we'll attach our two prompts and the latent resulting from the operations above the key difference slides in this last step usually we would have added the empty latent image node which fed our model A Space full of noise using masks however the model will only be given a part of the image where changes can be made in this example I wanted to use the set latent noise mask as the Handler for this input as it allows for a more uniform and seamless image rendering there are also other types of nodes like they encode for in painting but it's much more rigid in terms of the area where in painting can be done and might cause some inconsistencies between the base image and the generated part let's change some K sampler parameters similar to the one that the impact examples suggest us foreign decode the resulting latent noise and display the final result in this example I wanted to modify the background of this image done if we wanted to modify more of this image while preserving the the changes made you can simply copy and paste the image using comfy ui's clip space like this look I think that once you become familiar with this method you can truly modify anything with minimal effort well that's all for now I hope this tutorial will help you learn how to in paint images via Sam please consider liking and subscribing if you found this useful also if you have any questions please let me know in the comments below I'll be happy to help you out as much as I can and as always keep dreaming

Channel: DreamingAI

Views: 12,574

Keywords: segmentanythiong, sam, metaai, ComfyUI, easy, text generation, AI, stable diffusion, artificial intellingence, dreamingai, ai news, best free ai, best ai model, dreamingai tutorials

Id: SMOM1bIY5yA

Length: 6min 45sec (405 seconds)

Published: Fri Sep 01 2023