HowTo better inpaint with Differential Diffusion in ComfyUI AI, awesome results, precise controll

Hello everybody welcome to this new tutorial today I want to build a differential diffusion workflow with you sometime ago differential diffusion could implemented into comi and I think it is uh really awesome gives you a much better control over your in painting it uses a map which is somehow like a mask to Define which areas should be inpainted in which amount dark are areas are not inpainted and the brighter the map gets the amount of in painting increases so it can be used to only slightly change an area in your image or you can also inpaint everything in this area also the transition between not inpainted area and your inpainted area is much smoother differential diffusion can be combined with your common models there is no need for a special in painting model now you can simply use it parallel to your normal model in my case I use sdxl there is no extra training needed or special fine-tuning the results are really great and it's really really smooth and it's much easier to use it for in painting and exchange areas in your images I really like it it is easy to use we use some custom notes in this workflow I use a note from the confi essentials notep pack and I use the Gan blur mask note from the impact pack the rest is implemented into comi itself yeah let's begin so let's start from scratch a simple checkpoint loader we choose a load checkpoint simple I use the sdxl base model I will link this below but I'm sure the most of you have this model already installed we need our prompts our prompt text positive and negative um you should use a text which is fitting to your input image uh and to the area which you want to repaint uh in my case I take this one some Android cyborg helmet thing dramatic lighting polished carbon fiber and so on and as a negative prompt I only put in Gold because sdxl gave me golden helmets all the time so to prev this I take gold into the negative prompt now we want to load the image I use this beautiful female Android and I want to replace the head with a helmet so let's paint the mask for this simply right click on the image and open and mask edit door let's increase the pen with the mouse wheel and then simply paint over the area we want to replace quite simple make it a bit bigger so that the helmet can have more space safe to note and there we have our mask it's called map in differential diffusions but I will call it mask now I want to use a gausian blur on the mask so that the area around the mask is much smoother this will really help to use the advantages of differential diffusion because the transition between both parts of the image is much smoother so let's choose this Goan blur mask I take uh 100 here so more blur and sigma like 50 play around with those values depending on the results you want to achieve then we need the in paint model conditioning note and let's connect everything posi itive prompt negative prompt the pixels the mask and the vae um let's have a mask preview to see how the Blurred mask looks like as the next step we can now connect everything to our K sampler positive prompt negative prompt the latent and now for the model we need the differential diffusion node this is doing all the magic we connect this side to the model and the model sdxl to the model input then we need a VA decode as always and connect the Vie and now we need our image preview or save image depending what you prefer and that should be it I keep the settings standard if your results are too noisy play around with a CFG value for example that might help in many cases let's have a look let's generate and there we go I really like the results I can see no transition it's really smooth like it's one piece of image so let's do another one it's quite fast as well another one so really happy with the results I only tried sdxl but I'm sure other models will give similar great result I hope this tutorial helped you I will link the differential diffusion paper in the description below as well as all the custom notes and the model and I hope I could help you with this tutorial please subscribe to follow my channel like the video that would help me a lot I hope I will see you in the next one goodbye

Channel: Neuron

Views: 3,454

Keywords:

Id: DVaakZ0pa6M

Length: 7min 49sec (469 seconds)

Published: Fri Apr 12 2024