EASY Outpainting in ComfyUI: 3 Simple Ways with Auto-Tagging (Joytag) | Creative Workflow Tutorial

hello everyone and welcome back to Dreaming AI my name is n and today we are going to learn how to perform out painting with comfy UI I'll be very honest I've spent almost 3 weeks researching The Universal out painting method and I hope I've come up with a solution that works well for everyone for now I've opted for multiple methods within the same workflow so that you can choose the result that suits you test in fact we'll be exploring three out painting techniques today that I hope will meet your needs as for the nodes we'll be using they include the need of pad image for out painting node the custom nodes of comfyi Ina nodes comfy UI fit size and my comfyi n Suite you can find all the links in the description as always let me start by saying that using the native pad image for out painting node already produced es acceptable results unfortunately it has some mutations especially with images of very low resolution where the final image may appear degraded particularly in the faces due to some annotations of the vae since the face portion is too small for it in these cases as we'll see we can use a node called image composite Mast uh you can also try increasing the resolution of the image but the higher the resolution the more likely we are to have inconsistencies in the parts added by the model so you'll need to find a balance great let's start our workflow with the basic nodes which include load image load checkpoint 3 Cas sampler nodes and the usual two clip texting code nodes we fill the negative one while for the positive one we convert the input text as we perform another operation on the positive sign I was tired of having to rewrite it every time while maintaining coherence with the original image so that the out painting wouldn't go wrong so I decided to use a very good tagging model called Joy tag which I just implemented in my Suite of nodes mentioned earlier to use it we load GPT lorder simple leaving Joy tag as the checkpoint name and connect it to another called GPT text sampler which will require the image from load imagees input just to show you the result I connect a node that will show us the generated text and also connect the positive clip we created earlier with joy tag you can also choose the number of tags shown by modifying the max tags value but for this example I'll leave it at 50 which is more than enough I want to point out that if you want to use sdx all models it might be better to use moondream this is because unlike the 1.5 models that prefer tags sdx all models are better at interpreting actual descriptions something that mream is very good at to use Moon dreams simply select that model in GPT loader and connect a string node to the sampler's input promp so you can ask the model what you want about the image both Joy tag and moondream will be downloaded on the first execution of the node and therefore do not require further steps to be used just a little patience the first time lava models are also supported but require a separate download for which I refer you to the instructions in the readme Amy official repository okay with that said let's connect conditioning a model as usual to all the case ampers and continue with our workflow by introducing the node that will perform a rough resize of the image when it's too small the Noe is called fit image and resize when we want to use it we'll need to enable the op scale and set the minimum size that one of its sides must reach this note is here just for safety so I'll deactivate it and I include it because I find it very convenient to be able to change the image size on the fly to test if the out pinning works better with different sizes we'll connect the output to the most important node of the workflow which is the pad image for out painting where we can choose how much and where we want to grow our image additionally we can also choose how much Feathering the mass created will have um this ladder value is very important as it will greatly influence how coherent the model creates between our original image and the generated edges in this case I want to increase my image by 184 pixels on all sides and I want the Feathering to be 43 now as I mentioned the methods we use are three for the first one we simply add a way in code and a set weight noise mask node to apply our expansion mask to the imagees Laten we then use this final latent as input for a first case sampler additionally I set an incremental seed in sampler scheduler and do noise based on some tests I did previously you can easily change them according to your reference but remember that the doo must always be less than one if you want the model to fill the mass using the content of our final weight next we add a vcode and a preview um lastly we add an image composite masked node to which we connect in destination and mask the outputs of our out painting node while in Source we connect the output of the Bae decoder uh moving on to the second method where I wanted to experiment with two nodes fil master areia and blur Master areia we connect some preview images for debugging and complete the node exactly as we did for the first flow uh the first node will fill the mask created by our out painting node with the selected method among those available I preferred Navy your Stokes we'll connect the image and the mask from our up pinning node to this one the second node will then add blur to this filling I set the blur intensity to 61 which seemed good enough here I always connect the same maskus before but in the image I connect the output of fil Mast area we leave both falloffs at zero because otherwise it would create a border that would detach the central image from our mask perfect let's complete the flow exactly as we did before finally let's look at the last method before executing this workflow a few days ago in my desperate search for a technique to perform perfect out painting I came across a video by a Creator named Rob Adams explaining how to perform out painting in a way I had never seen before the video is very interesting so I'll leave the link in the description uh essentially this method involves repeating a certain portion of the image taken from the ends pixelating it and filling it with noise then using it as a background for a mask the result is a kind of image to image somewhat like what Phil Nast area does but much more elaborate what struck me the most is that it seems to work really well which is why I decided to create a custom node that saves me from doing all the steps you see in the video giving me a very similar result I say similar because I think I still need to work on it a bit but for now I'm satisfied I named the node image for out paint Advanced as if you want to use this technique he can replace the stock one to use it simply connect the original image as input and select the various parameters as we would have done in the various nodes described by Rob atoms we have the mask expansion on the sides and Feathering like in the official node noise which is the blend between the cropped and stretched image and a generated noise image pixel size which identifies how large the pixels of the image pixelation process are pixel to copy which identifies the size and pixels of the portion taken from the image on the sides and which will be stretched for the entire expansion size finally we have the color correction section which will only affect the Mask Part and not the main image for noise unlike the video I noticed that with the method I used I have to use very small values if I don't want too many details in the final result for this example I'll always stable No.1 in conclusion we complete this Flow by practically doing the same thing we did for the first one the only difference is that in the case sampler I decided to use this sampler and schedu out and I lowered the D noise to 58 as higher values gave me results that I wasn't satisfied with all right give me just a moment to sort out this mess and then we can finally execute our workflow okay let's take a low resolution image as an example just to show you the issue I mentioned earlier and run our workflow uh starting from the beginning we can see that the prompt was correctly generated uh moving down we have outliner masks generated by our two final floods and finally here are the results um starting with the first one you can clearly see that the original result has a very degraded face while in the second column where we applied the composite it's perfect this is because the composite simply fills the part not covered by the mask with the original image however this results in a more pronounced seam between the generated image and the original one the second one is also fairly pronounced but overall good remember that the second column is just one of many options to compensate for the degradation of the original images face as you could very well use nodes like the face detailer from Impact pack to improve it or use paint to remove the seam now let's move on to the third and final result this result in addition to being perfectly integrated has an exorbitant amount of detail compared to the other two images the grass in fact has managed to expand even beyond the boundaries in a coherent manner as has the rest of the image of course the original image here is perhaps the most degraded but remember that it was still a 512x 512 as I mentioned before an interesting thing I discovered is that increasing the noise dramatically increases the details I won't lie to you I think this method in this specific situation makes the other two seem like failed attempts obviously it doesn't always work in fact in the various tests I did the first two often Pro to be better as the details could sometimes create unintended things in the image which is why I wanted to keep all three also always keep in mind that all three technique can be improved by adjusting the denoise of the T sampler or the feather the model used also plays an important role and if you are you know out painting images you have previously generated you can continue to use the same model to maintain maximum coherence and that's all for now in the future I hope to find other techniques that make out painting even easier but for now I'm uh pretty satisfied with these please consider liking and subscribing if you found this tutorial useful also if you have any questions please let me know in the comments below I'll be happy to help you out as much as I can and until next time keep [Music] dreaming

Channel: DreamingAI

Views: 1,933

Keywords: outpainting, joytag, moondream, custom nodes, workflow, generate images, ComfyUI, easy, basic, texttoimage, AI, stable diffusion, artificial intellingence, dreamingai, ai news, best free ai, best ai model, dreamingai tutorials

Id: xHvtwqtOhgE

Length: 13min 26sec (806 seconds)

Published: Sun Feb 25 2024