ComfyUI: Area Composition, Multi Prompt Workflow Tutorial

hi I am Mali and welcome to the channel composition in an image or Art Is Fundamental it brings all the elements together and affects how the image is perceived take this image for example what if I told you this is not one prompt but it's actually four extra prompts over the base prompt I first Define the background then Little Red Riding Hood along with the house the sky and lastly the sun look at one more image here I put a prompt for the background landscape the dragon and the bird I also use control net to define the bird let me show you how to use area composition multi- prompting and control net all together in comfy UI this is an advanced tutorial and you require basic knoow for comfy UI if you are new and want to start with comfy you can check out the channel for other comfy videos that cover the basics all the PNG results and the Json workflow files will be available in the YouTube members only post to start off you will need the comfy manager and two custom nodes in the previous comfy videos my colleague has already explained how to install the comfy manager via the command prompt I will leave the link for that video in the description below in the comfy manager search for visual area conditioning and install it the tutorial also requires the ultimate SD upscale custom node after installing both nodes restart comfy via the command prompt okay so I will start by creating everything from scratch it's better for understanding the workflow process for the first workflow I am using the Rev animated checkpoint all the connections for the clip text and code will connect here instead of the checkpoint node multi- prompting requires you to have at least one base positive and negative prompt these prompts are used for the overall theme or background of the image sometimes you may require multiple sets of Base prompts for multiple K sampler passrs for the current workflow I am going to use only one set an advanced sampler is not needed for this workflow so we'll be sticking to the basic one since this is a non sdxl checkpoint let the resolution remain at 512 only here you can use vae decode tiled as well the tiled version is used for larger resolution generation however there is a clear-cut quality difference in colors and contrast when using vae tiled for SD 1.5 if you upscale higher than 2048 comfy will give an error and try it using VA decode tiled it is better to generate the images in 1024 and then upscale at 2x at a time using ultimate upscale this is a simple prompt basically I am looking for something like a Little Red Riding Hood walking by a house surrounded by a forest a simple workflow let's generate the prompt and work from there the main sub subject character red writing hood is missing yes I can improve it via prompting but there is a better way with area composition and multi- prompting I will Define each subject and object within the image and get complete control of the composition I would be defining a separate prompt for the subject the house the sky and the Sun for that we need four extra clip text and code nodes the base prompt should only contain the background or the theme so I am changing it to include a forest landscape clear skies and sunset I will Define the rest of the elements in four additional prompts the Red Riding Hood in prompt one the wooden house in the second prompt for the third prompt I only want the sunset changing the sun's position within the composition would be very cool and for the last prompt I have defined the sky and the time of the day now I will add a custom node from Dave main 42 called the multi- area conditioning the multi- latent composite or the noise composition method is very different and I will cover that in another video in this node always link your base prompt to conditioning zero only positive prompts are supposed to be connected to these conditioning inputs I must add three more conditioning inputs since I have four extra prompts to do that rightclick and select insert input below zero since this is the base the width and height should be equal to the resolution generated from the first sampler switching the index will switch to conditioning 0 1 2 and so on and I will Define the resolution and the location of each prompt for the Little Red Riding Hood I will keep the resolution the same but make it so that the subject appears on the right side of the image I want the wooden house to appear on the left also I am defining a taller area so the main subject appears proportionately one thing I noticed while using area composition is that smaller subjects are a bit harder to place them accurately a more General left right or Center works better for the sky I am just going to Define it within the top 1/3 part of the composition the output from the multi- area conditioning should connect with the sampler's Positive input let's generate and see what comes up huh I got something red let me try and fix this one way to do this is to increase the prompt strength in the multi- area conditioning node let's try two and see what happens perfect however the Sun is not fine let's increase the prompt strength for the Sun as well okay this is way better and I also like the house it Blends better the overall image here however is lacking details what I want to do now is pass this image through a second time via the sampler to try and add some details to it for that connect a latent upscale node to the first sampler double the resolution here now connect the second sampler node only connect the base negative prompt to the second sampler don't connect the base positive prompt node we need to create another positive node for the second sampler the reason why we are creating another positive prompt node is because the text prompt for this node should be comprehensive it must include all elements from the base prompt and the additional four prompts together I broke up the original prompts into five prompts when when the generated image is repass through the second sampler a comprehensive prompt is required to guide the AI to construct and add details to the image I have included the forest Sky Red Riding Hood the house and the sunset in this prompt what you put in this prompt will impact the final image keep it as close as possible to the combination of all the prompts for the first sampler connect this new node to the positive input in the case sampler the Schuler also makes a massive difference in the final image let's keep it at normal with a Deno value of 0.3 I am not satisfied with this image it's pretty off I want to try and see what result results exponential gifts oof so this happens with Caris or exponential let's try increasing the denoising strength way way way better anything above a 0.44 to noising strength gives you sharper and better results I can improve the image with another pass through but this time I will use the ultimate SD upscale node I will be using the full hearty Remi upscaler for all the workflows in this tutorial similar to the second sampler the base two positive and negative prompts should be connected to the upscaler also it is to be noted that in each pass through the seed value is essential a different seed will give you different results it is advisable to randomize the seed at each pass through finalize it and then move on to the second pass through at this stage experiment with the Deno value start at 0.3 and move up too high may cause unwanted details in the image perfect and beautiful before I move on to a new example I want to show you how you can rearrange the elements and get various results I will now try and change the position of the subject and the house it won't be the same image but it will be something very different and creative doing this is very simple first I will select prompt one and move the subject to the left then prompt two and move the house to the right for now I won't touch prompt three or four you can see I got a completely different composition using the same elements let me try one last thing I will revert everything back to the original position and try to move only the Sun the original generated image has an artifact here however this is corrected during the second pass through and with the third pass through using ultimate upscale many details are added you can further improve on this but I wanted to show you what you can achieve via multi- prompting I am using the graph of the previous workflow here I will replace the sun in the third prompt with a volcano and try to blend it into the image I will use control net for the same when making changes to a specific additional prompt make sure that changes also reflect all the base prompts drag and connect the relevant prompt node to the apply control net node add a load image node and connect connect it with a pre-processor in this case I am using a stable diffusion 1.5 checkpoint so I would be using the anime line Edge pre-processor for the same remember that sdxl control net nodes are different and work only with sdxl checkpoints the pre-processor output will connect to the apply control net node input connect the base negative prompt to the apply control net negative input make sure the control net model is the same as the pre-processor the positive control net output will connect to the multi- area conditioning input there is no need to connect the negative input since the same prompt is connected to the case sampler you can use multiple control net with this area composition I method the workflow can be as complicated as you want comfy is really good with that you think of something and throw in nodes here and there to get the composition you want this is an sdxl workflow and I am using the law mysterious checkpoint the principle is the same as the previous workflow but I would be using a an edge pre-processor the image is also wide resolution the resolution in the multi- area condition should be the same as the empty latent resolution otherwise the results will be weird I am also keeping the prompt strength value at one it's advisable to start with one see the results first and then increase accordingly I wanted a face off between a magical bird and a dragon in a lush Scenic environment having all the strengths values at one produces the environment and nothing else in this case I will only increase the strength value of the missing prompts nice the control net canny was only used for the bird as it was challenging to get the bird to face the dragon via prompting this workflow is simple I am deleting most of the nodes as it only requires two prompts one is for the subject which is a portrait photo the other one is the main background I am using the realistic Vision checkpoint for this one this is an AI generated portrait photo and we'll use the midest depth control net model here if you have extra conditioning inputs you can rightclick and remove all unconnected inputs this time I am using a separate negative prompt for the second sampler it's it's not necessary to do so also note that when you use multiple checkpoints you can actually combine the negative prompts into one and then pass it to the sampler via the condition combined node for the background I have defined a nighttime Venice location and placed the subject on the right again the subject doesn't show with a strength value of one this doesn't always happen but it depends on the prompt the checkpoint and the dimensions chosen in the area composition node in some cases I had to increase the value up to five for the subject to be fully shown have a look at the circled area without a second sampler pass through the image blending is very bad I found that passing through two samples is a must in most cases yeah some of the images won't have this issue also sometimes passing it through a third time is too much for this image I just used an image to image upscale via the ultimate SD upscale node let's move the subject to the extreme left let me show you a problem you might face while using this tool if I move the image to the center see what happens now this issue is is not fixed with any strength value the most straightforward fix is to change the seed the AI is so confused here it Blended the forehead with the lights behind and made a crown in the second pass again there is a blending issue but it gets fixed in the second pass so there is no need to change the strength value or the seed I hope this tutorial was helpful and you got to learn something new in comfy until next [Music] [Music] time

Channel: Control+Alt+AI

Views: 8,422

Keywords: comfyui, comfy area composition, comfyui area composition, comfyui multi area conditioning, comfyui workflow, custom nodes, comfyui tutorial, comfyui nodes, comfyui stable diffusion, comfy stable diffusion, comfyui guide, nodes comfyui, comfy ui, comfyui sdxl, comfyui custom nodes, comfyui easy workflow, custom nodes comfyui, comfyui upscale, comfyui upscale latent, comfyui upscale workflow, comfyui node workflow, ai comfyui, comfyui ultimate sd upscale, comfyui explained

Id: NPkSa1y0GLM

Length: 33min 42sec (2022 seconds)

Published: Sat Nov 18 2023