The Truth About Consistent Characters In Stable Diffusion

the truth about achieving a hundred percent consistency in stable diffusion is not exactly true you can get 80 to 90 percent of the way there and today I want to show you how to do that so to get started you always want to start with a good model something like realistic Vision Photon absolute reality when it comes to people these types of models are always a good start in terms of consistent faces the easiest way to do that is to actually give your character a name in my example here I have La Lisa tisson Katie Dobrev totally made up and the reason why I use two is because individually when I use these names they have certain characteristics that I want to put together so you can use one name or two names if you're not good at making up your own names they're random name generators all across the net I'll leave a few in the description below you can also State national ability to keep ethnicity somewhat consistent and secondly you'll need control net if you don't have it installed I have an installation video I'll also leave that in the link below we'll also touch on root just a little bit so the first thing you want to do is come up with a prompt and a look that you like so I ran a few Generations here just to develop some sort of style and look and I wanted a really simple black sweater in jeans don't pay attention to the hands right now this isn't the focus for this video so from your batch of images let's say you run 10 or 20 you should get at least two types of characters that look fairly similar and then once you decide on a look that you like you can import it into control knit now when you're just experimenting in the beginning I encourage you to start simple with something like this black sweater jeans try to be specific on clothing as possible but that's where the challenge is consisting clothing can be really hard to do so let's open up control knit and we're going to import the image we're going to use I encourage you to use a full body shot or at least from the knees up make sure you have it enabled you can use Pixel Perfect if you like but the one we're going to use today is called reference there isn't a control net model to download you just simply have to select it we're going to set the control weight to one you can experiment with that typically 0.7 to 1 seems to work pretty well for what I do and the only thing I want to point out is I'll leave this on balance so that we have this style Fidelity option available this is going to help with the consistency so I'm going to leave it at 0.5 for now and I want you to pay attention to the style of this image here we've got a pretty plain black turtleneck sweater simple jeans some blonde highlights and dark brown hair and of course we want consistency in the face let's take a look at the generated images so here's the reference photo that we used very similar difference in color and lighting but that's okay in this image we have a different pose the hands could use some work I know but face is consistent and she's actually holding that turtleneck sweater here's another one in a different location although we aren't missing a hand here the jeans are staying consistent the clothing I would say is acceptable here's a few more examples slightly different pose from our original so considering we haven't done any Lora training dream Booth training this is actually very simple to do and it gets you 80 to 90 percent of the way there now what you can do is change in the background and the surroundings the only thing I'm going to change here is put Pier at Sunset and I'm going to use the same reference image and now we have the same character at the pier at sunset same turtleneck sweater every now and then we'll get this type of sweater with the circular neck and that's fine too because we can grab that whole batch and use it for different scenes we have the turtleneck sweater back here again with this circular type of neck my point is with this reference control knit it does a lot of the work for you and you can easily change the background the locations the outfits with very little effort now it's not 100 perfect like in the hands here for example but again there are ways around that now this is for AI generated images can we use this for real photos absolutely so first let's get rid of this image and I'm going to import a real photo and this time I'm going to use the extension root now I'm not going to go through the installation process I'll leave a link in the description below you just have to follow along however I do have an installation video coming up very soon it's very simple to do you take your reference photo to use as a face you click on enable and that's pretty much it now if you have multiple people in the image you can separate it here with commas you could put like one comma two for example but in this case we don't need it we just need to enable it I'm going to leave the prompt the same but now it's going to generate images that look just like April whom I've worked with many times in the past and the beautiful thing about this is that if I have a shoot with somebody and I don't like the location I can use this method to change the environment the location even the outfit now this time I notice it's putting earrings on the image which which is fine we could impaint those out but for the most part if we switch between these two the sweat letters look pretty accurate probably should have chose a different color we do see a variance of sweater here the face the hair pretty consistent the look the jeans like if we really look at the jeans here see the buttons the little small details this one there isn't a button this one there is that's slightly different so it's those little variances that are going to happen now when that happens this is where you can increase the style Fidelity slider typically I find 0.75 even all the way up to one can help further with that consistency but sometimes it's not needed so for now I would say to just start with this create a whole bunch of images naming your character and then what you can do with the reference control knit trait is utilize those images and those characters in different poses different environments and slowly piece together a story as mentioned in a future video we're going to dive deeper on more the Aesthetics like the hands and the faces and putting other characters in the same scene now if you're wondering how to optimize automatic 1111 for sdxl if you have an 8 gigabyte graphics card or less make sure to check out this video as I go over that in that video until the next one my friends I'll see you when I see you

Channel: Monzon Media

Views: 60,382

Keywords: consistent characters stable diffusion, stable diffusion consistent characters, stable diffusion, automatic1111, how to make consistent characters, how to create consistent characters, consistent characters no training, controlnet, controlnet reference, roop, roop extension, ai art, ai image generation, stable diffusion consistent character

Id: WQsUeyzq-VA

Length: 6min 59sec (419 seconds)

Published: Sun Sep 03 2023