Consistent Characters in Stable diffusion Same Face and Clothes Techniques and tips

hello this video is about creating consisting characters in stable diffusion so we can define a consistent character as a person who has the same face same clothes and different poses and different backgrounds now creating 100 consistent character in the stable diffusion is impossible because stable division is inconsistent by Design to create 100 consistent characters with clothes you should use blender or other 3D softwares not stable diffusion however it is possible to achieve high level of consistency which can be convincing enough now to have the same phase in stable diffusion we can for example create a sample prompt which is a portrait prompt so that we can use it later with after detailer we could also have a consistent face with help of rube now if we fix the seat we get the same face but when we change the prompt the phase changes also this reduces the flexibility of the output so it's effective sometimes but in most cases it's not now for example we create a portrait okay this portrait for example we try to make it as detailed as possible so in this location for example we can and add a random name for example okay we don't need to watch this but having such a name will help stable diffusion to make features from random characters which are most likely learned by SD as well okay so clip will separate this name into something like this for instance and try to create get some features from uh from similar tokens that exist in the stable diffusion it's just to create a random face so we don't really need to do this okay so we try to make the prompt as uh clear as possible as detailed as possible for example if we are satisfied with the with a certain Trace in general we move it to after detailer now after detailer will help us to create a consistent face for any character and we will enable after detailer now without of the detailer it's really difficult to get full body shots with the good faces now the next step is to create full body shot now we create a full body shot or Cowboy shot and enable after detailer now we can also create a new character using loras for example for instance we can mix different we can mix different loras such as Korean Japanese Etc with our prompt to have our unique character produced for example we have Korean the likeness and you put it with 0.3 and we watch Latina for instance 0.2 Etc so we can mix different characters to produce a new model that will have a consistent face for example we want this loras inside the after detailer for the the facial so if we take a closer look we can see that all these characters generated have the same face okay that we have planned so this is a consistent face now another method for having a consistent face without of the detailer is using rub now we track the face that we want here for instance and generate the pictures we press enable now in general root can produce acceptable results in general it's used if we want to do something like deep fake or have a face of a real world lesson and drawn onto the characters now we can see this is the result produced by Roop we can see that the faces are a little bit too gloomy they just like look that pretty in comparison to after detailer so this is about consistent faces and now we move to consistent clothing now consisting clothing is more difficult because some clothes are more complicated than others so for example for simple clothes this could be much simpler now if we try to generate the same with the same using the same problems with different seeds we will get changes in the clothes this is why we need to use control net in order to improve the proximity to our clothes so even with simple clothes we can produce almost similar clothes however some changes may occur from one picture to another for example in this picture we have one pocket one pocket over here we have two pockets one pocket two buckets strap Etc so even with simpler clothes some changes may occur without the use of control net even with control net controlling the other clothes might not always produce perfect results so achieving 100 consistency is almost impossible and stable diffusion now in control net there's something called reference now reference is not a control net model it is only a preprocessor however it helps to produce pictures with the same style as the input picture now ideally we should actually because if we want just the style of the clothes so we should ideally remove the background test the generation now it's better if I wanted the style of the person itself it's better to remove the background using Photoshop it can remove the background or using automatic 11 background removal tools for example here we have the background removed now we drag a drop it here choose enable reference we keep balanced then we generate sample 10 pictures to see the result produced when the background is removed so the generated picture is now if we look at them we can see that all of the pictures generated basically have the same cloth style and regarding that the plants all the jeans they are the same we can see that we all have the same build the point is that with the reference it's possible to achieve better consistency when it comes to clothes now this is the produced result for sample 12 images we can see that the star will be preserved in most majority of the pictures however two pictures for example have white shirt we can see here for example the details of of the inner shot are the same basically the outer jacket are very close to each other which is really good acceptable for stable diffusion now we can improve this process by improving our prompt for example we can say something like wearing short yellow winter jackets and add some negatives so we will described The Prompt better hopefully that we would get more consistent images now we can see here for example that they generated the images are more consistent basically we have almost the same code style the same shirt under the code and the same genes okay we don't have a belt in all of the images which is really good now one of the pictures did not well was not produced as I wanted okay but this is good in general because this is like 92 percent uh consistency now if we increase the level of details in in the prompt we might get better results we can also manipulate the control net parameters such as the style Fidelity Etc and all the control net weight and hopefully that we can produce better results now in this example for here for example we can see that in all of the Globes we have the same jacket the same pants almost the same shirt and all of them all most of them do not have a belt so that the style have have become more like the original reference image and we have the same face because we are using after retailer now in order to have the same clothing style with the same face of the person that we have designed before we can use multiple control Nets and the first control net for example we can put a paste picture and the user reference model okay then in the second control net we can use control net with open pause for the open pose I will use a ready to use open balls such as none if I have a picture I can use DW pause full for example which is slightly more accurate than open pose this is just a preprocessor then we run the picture and we get a picture with the same face and almost the same clothing style now we can generate for example using a patch folder and try to produce a large number of images now it's also possible to use Laura's Insight after detailer and in the prompt this will help us also to increase the consistency of the output to always to always have the same phase for example here we can see that using different clothes that were not shown to the subject we can always have the same face of the person now of course as we explained in previous videos that inside the main prompt we would put the Laura of the person with the lower weight and in the portrait we would we would put it put it in the full weight in order to produce the highest details possible so as a conclusion achieving 100 consistency and different character include character clothes and scenes and stable diffusion is impossible but we can get good enough results with help of after detailer loras and control net and have a good day

Channel: How to

Views: 53,323

Keywords:

Id: AqnJeGPDgsY

Length: 9min 15sec (555 seconds)

Published: Mon Sep 04 2023