How to create consistent character with Stable Diffusion in ComfyUI

hey everyone I'm going to show you how to make a consistent character with stable diffusion in confy UI uh I'm going to put the resources links in the description below uh you can download the workflow you can follow the link to download the models and install the work and so on uh first thing you need to do is to go to the link I provide in the description and then uh it has a workfl link uh on this page uh it has the link for automatic 111 and also comy UI so what you need to do is to go to my website and then click on confy UI of this workflow and then download the workflow using this download button so and then save the workflow uh after you have downloaded the workflow go ahead and load the workflow uh in comy UI so you can and drag the workflow and then drop it to come UI and then it will load so uh after you load it uh you would likely see some rack boxes because you have some uh note uh missing so what you need to do is to first go to manager and then go to install missing custom note and then if there are custom notes that are used in this workflow that is missing so so you will see a list of two to three custom notes here so you would need to install them and then restart your compy UI and then after that you're good to go and restart uh and then you should be able to see the worklow just like I do and if you don't see this manager button and then you would need to install config UI manager so in my website I have some instruction on how to install it and and if You' like to have a video um instructions to show you step by step how to install comy UI manager so feel free to let me know in the comment section okay yeah so after that um you would need to download seral models so um in this tutorial I I'm going to use sdxl model uh but uh what I find is that not all the sdxl model Works uh for this workflow so um I recommend you to start with Proto XL if this a workflow that I have tested and I know that it works with this workflow um so I'm going to explain how this workflow work and then after that I'm going to show you how to use it so um this workflow first use uh control net candy control net to fix the the composition of the image so what you need to do is to uh upload a composition image like this so you would drop it to the control net [Music] preprocessor low image note like this so it is upload and then you can see that there's fre by3 G of um character uh at different buing angle so what comy UI and Sil diffusion does is it will use control net candy to extract the outline of this workflow so that uh you will be able to uh copy the outline of this image and then after that um we are going to use IP adapter so to copy an image uh with a face so I'm going to use this image so I'm going to drag this image and then drop it to the IP adapter canvas so we are going to use phas ID plus V2 so this ID adapter is using uh inside phase to extract the face on the image so you only copy the face and the hair so it won't copy the rest of the image so that is something that uh you need to pay attention to and then after that uh the workf is pretty much good to go so I will run it uh because it take a while to run and after and then in the process of running it I'm going to explain what happens so now the workflow is running and what you will see is that um after it runs it's going to run through the phas detailer note so in compy UI we don't have the a not uh in as in automatic 111 that automatically fix faces but it has something very similar it's called pH DET teller is in the impact pack Uh custom note so you should have already installed it during the installation process when you install the missing notes of this workflow so what it does is would detect the phases in the original image produced by stable diffusion and then it will go for one by I want so you can see that the detected mass is here so it accurately detect the position of nine different faces uh what it does is it perform automatic in painting to automatically fix the faces the reason you need to do that is that because these faces are pretty small compared to the image because you have nine faces in a single image of uh 1,24 by 1,24 pixels so this is simply too small for the stable diffusion model to render it correctly so as you can see um the eye is a little bit messed up um and as the nose in the original image but after you use face detailer to fix it what it does is to just crop off uh one of the face and then in paint it uh at a certain denoising strength and then uh because it's a higher resolution so it can render the face accurately after that it would beze reduce the size and then put it back to the image and then it does that one by one so you will see it doing the rendering nine times here because there are KN faces detected okay so that is the gist of um how you use the workfl so to sum up so you're going to select the model Lo the checkpoint model so make sure if you're first time using this workflow use the poition Excel model like I did um and then after that upload the control net image and then also upload the IP adapter image so you can use any IP adapter image one depending on what phase do you want to transfer to the image okay so next I want to show you how to customize it so if you want to customize it my advice is that uh you would want to change the prom and then but before you do that uh you can um mute the pH DET the reason is that uh this pH DET take a long time to run and also it doesn't really change the global composition such as the hair color um or the uh background or the croing and so on so uh in comy UI there's a trick that you can press crl M to mve any note so go ahead and click on the face detail note and then press contrl M so the shortcut is the same in on Windows or on map uh so it's both contrl M so you don't need to press command m in Mac so and then after that you can change the prom so for example um I want to see the original ha color I'm not going to use B um in the prom so I remove it and then after that I press Q prom so um let's see what happened yeah so you can see that the hair color changes and then do something similar to your reference image so that's the beauty of Ip adapter uh in IP adapter face so not just the image control the final image but also the prom so essentially you have two conditioning that are more or less independent and complimentary to each other you can both use an image to control your final image and then you can also use a prom to control a final image so if you read the teal reports so these two mechanism um somehow independent so so they have trained um separate cross attention Network in order to make this image uh conditioning a lot better compared to previous attempts so uh definitely you should take advantage of it uh control the image with both and reference image uh and also control the image with the um prom so um and then after that you can go go back and turn on the face detailer again so press crl M after you select the note and then you can C PR so one thing you may have noticed is that um the K sampler didn't R run uh so that is a trick that I to use in confy UI you can also consider using it uh always use a fixed seed uh that will save you some time in rendering so what this fixed parameter does is that after um you have generating image if this seat haven't changed so this case sample result basically the same so it won't be run again instead confy UI would use a catch result so that it would going to save you some time that only thing you need to do is to rerun the phas detailer note so as we have here yeah so after running it uh nine times automatically you can see that now we have um high resolution phase that is being fixed compared to the one that in the original image so you can go ahead and right click and save our image in your local storage yeah so that's all I want to show you today uh to sum up this is um workflow that you can create consistent character uh with com UI a lot of people ask for it in the previous comment session uh in the same workf but in automatic 111 so I'm also going to put the link uh to that video in case you're interest in a workflow for automatic 11 and 11 so feel free to uh go to that video and follow the workl yeah so that's all I have today uh if you like the video please like And subscribe thanks for watching

Channel: Stable Diffusion Art

Views: 5,937

Keywords:

Id: r5YTu9TvEro

Length: 12min 37sec (757 seconds)

Published: Fri May 24 2024