Welcome to Kleebz Tech and another video in
my continuing series on Fooocus for stable diffusion. Today, I'm going to be going over
some tips for consistent character creation, but mostly, I'm going to be diving
into face swap. Now, don't forget, I have many other videos on Fooocus,
from install to poses and much more, so check those out. Before I jump too far in,
I don't want to get your expectations too high, like other videos do, and tell you, 'Well,
this is easy, it's simple, it works perfectly, and we are all happy,' because that's just not
the fact. It works pretty well if you have the right source pictures for one thing, and you spend
a little bit of time tweaking things, then you can get pretty decent results, but it's not just
a simple click the button, boom, you're done. First thing I'll get into is with consistent
character creation, just some tips that other people have, and ones that I might use. So, one
of the popular ones is to use celebrity names with different weights. So, for example, in this one,
I generated one image. So, let's say I wanted to create some source images, and I want it to look a
certain way. I could sit there and take different celebrity names, mix them as I put in here, you
know, I have Carrie Russell, Natalie Portman I picked out, and I applied different weights, and I
just stuck with a simple prompt, not too complex. Now, one thing I do recommend is, when doing
this, is to turn off the Fooocus V2 because that does add different terms to the prompt. So,
I like to avoid that, and so I have, let's say, my first image. So, let's say I want another
one, but this one I want the person smiling. Now, in here, I'd make sure it's the same seed, so I've
unchecked random, and then I'll generate one more image. Go ahead, and we'll have our two images
to start off with. That's one way that people do try to get consistency. It does work pretty well,
like I said, if you stick with the same model and everything. If you switch to a different model,
you're going to probably get a different outcome, or LoRAs and things like that. So, whenever you're
doing this, you want to stick with the same thing. I find you start off, you get a consistent,
multiple different faces, and then use those for reference later in the face swapping. Okay,
so we have another image. So, this is just to give you an idea. We have two different images,
and you could create multiple different ones, find the ones that all look identical. They're
usually, I find, this way works pretty well if you don't switch models and things like
that. So, that's one way that you can do it. The other way that I'm going to use, I'm going to
use for this one to show, is to actually instead create a grid of four. So, in this one, I've
got everything already set up, a grid of four images of the same beautiful woman with four
different expressions, red hair, that's all. I just went with a simple prompt, nothing complex.
On this one, we could use the Fooocus V2, that's not an issue for what I'm doing here, because
we're going to actually create one at a time. So, let's go ahead and generate this.
So, the nice thing is, when we do this, we're going to get a grid of the same person.
Now, the problem I have is I haven't found the best prompt to get all different angles
with different expressions, things like that, if anybody knows a good prompt for doing that,
then that would be great, put it in the comments. Okay, so I'm going to go ahead and just, well,
let's see what this next one looks like. Nah, I'm going to go with the one we got.
Okay, so we have these four. This is all one image, so it's going to be the same person.
Now, preferably, if you can get different ones, like with the mouth closed, mouth open, but
we'll start off with this one for my example on how the face swap works. So, at this point,
let's say I have the images, these are the four that I want. If you wanted to go this way,
then we go to the input image. I drag this down, I want to upscale it, I'll leave the same prompt
in there. Now, upscaling 2X, or these two options, will slightly modify what it looks like. So, if
you're very set on the facial details and stuff, I would normally just go with the fast upscale.
You can go with the upscale 2X, that's entirely up to you. At that point, I'm going to just go
ahead with the upscale 2X myself. I'm going to pause my recording while I do this because
I can't upscale and record at the same time. Okay, so we have our images upscaled, and now
what I find the easiest way to do this is just to open the image in Paint or something
like that, and then we can just go ahead, we will select each one and crop it, and then I'll
save it, and then we can just go ahead and save each image separately. Once we've gone ahead
and saved that, we can crop the next one and just keep going through until you have all your
images split apart. I won't go through all that. So, once we have our images split apart, then
we're going to want to go down into the image prompt area, and under the image prompt tab,
you do want to make sure at the bottom here, if you don't have any of the face swap options,
you want to check off 'Advanced' all the way at the bottom, so you'll have all those options. Then
at that point, we can take our images that we have broken apart, and we can drop these in there, and
you want to check these off to face swap as well. The default settings you're going to get when
you do the face swap, I don't find are usually heavy enough, but you're going to mess with
those and tweak with those. This will give you a pretty decent result, but you'll probably
have to crank these up. Now, I don't find weights over one really help, but that's just been
in my testing. The stop at consider that is, it's going to use that, this right here is saying
for 94% of that generation, stick with the face. So, it's going to be hard if you want to adjust
the look of the face, the emotion. You may have to lower that to get the results that you want.
So, for these, I'm going to just put these up a little bit. I'm not going to crank them too much
because I do want, for now, I'm going to put a priority on one of these over the other, and then
in here, we'll just go ahead and type up a prompt, a woman having coffee in a coffee shop, and we'll
just go ahead and generate a couple of images. Now when we prompt, we don't have to
do anything else because we've already checked off the options, so it knows it's
going to use the image prompt in the face swap. Whatever image it generates will
use them. We have our images generated. Let's see how they compare to the source image,
just for reference. Looking at those results, I'm actually pretty satisfied with how that came
out on the first settings I used. Like I said, you may have to tweak those settings to get
exactly what you want. One thing I suggest is, once you find the settings that work well for what
you're doing, go ahead and take a screenshot of this area so you can remember what those exact
settings were for the results you wanted to get. Now, there is going to be some difficulty when
you create some of these because all these images, you'll notice the look is almost identical to
the original face. Having more images, I feel, gives you a little more flexibility, but if those
images aren't showing different emotions, it's still not going to solve all your problems. So,
let's try another one. I'm going to demonstrate how it can be a little more difficult to get
what you want. I've used another image here; these are all slightly different angles but all
with the same emotion on the face - not happy, not sad, just pretty much indifferent.
To get the results we want is going to be a little harder if we don't have some
variations in that facial expression. So for this one, I'm going to say,
"a woman smiling and having coffee in a coffee shop." Let's go ahead and generate
those images. Now we have our images generated, and if you notice, for the most part, it got the
face correct. It's not perfect; the eyes, I find, don't carry over perfectly. That's one thing
I've noticed. I created one with green eyes; the green didn't carry over, the blue did.
These aren't the exact same shade of blue, but that can also have to do with the weight we've
applied to the images in the face swap features. But, as we're going to notice here, there's not
much of a smile. The reason for that is it's much harder for it to generate that smile because
the original face didn't have a smile on it. One thing you can do is increase the weight in
the prompt. That's one way of producing better results. The other option is to reduce the
weight and the stop at. The weight is more about how much it influences the final image;
the stop at is when it stops influencing it. So, if you set those to lower amounts, it's going
to give it more time to actually generate what it wants. The downfall will be that the
face may not be quite as exact, so you're going to sacrifice some of that resemblance
to get different emotions that you may want. Now, this is where it's actually useful
if you had different images that did have different expressions. You could
apply more weight and higher stop at to the one that's smiling when you want
the person smiling. If they're sad, you'd use the sad one; if they were
angry or happy, or any of those emotions, you could use those images to apply more weight to
get the results you want from each one of those. So let's generate this one again. Now, I have
reduced the weight and stop at. I've also increased the weight of the smiling in the prompt,
and we'll generate and see the difference. Okay, so we have our images generated. These are our
first images. As we can see, on the left, we just went with a smile; we didn't change the weight or
stop at or anything. And on the one on the right, we changed the weight, we've changed the stop
at, we've increased the smile in the prompt. Now, the one downfall I've noticed is the resemblance
went down slightly; the eyes seem a bit dimmer. These are the sorts of things that you're
going to have to tweak and adjust to find what you're willing to sacrifice to get a look
that you might be trying to achieve. That's why I said it's a lot easier if you have multiple
images that have different expressions. You can increase the influence of those expressions,
whereas with the other ones, you can't. So, let's show you one more example here. I've
got some other images; we're only going to use two for this one. Initially, I am going
to use a higher weight on this. We're going to set this right to stop at one. We're going to
set that one lower; this has a lower influence, and we're going to do the same thing with this
one. I'm not going to actually put any facial expression in because, technically, that should
pull in from this one—the same, pretty much the exact same facial expression at this point where
I have the stop at and the weight set so high. So, if we pull these up and look
at the comparisons, we'll find the original image. This is the one that we
had more heavily weighted. As you can see, the eyes did not carry over; pretty much
everything else carried over. That's one thing in the prompt I'd probably want to
put the eye color if I want a different eye color. I didn't notice that as much with
some of the other colors; green seemed to be the one that didn't seem to carry over
as much, so I'm not sure on that one. So, we'll see in both of these, we pretty much
got the same facial expression from that. I'm going to go ahead and change this. So,
we're going to actually down here, we're going to lower this down, we'll put that down at a
lower weight and a lower stop at. This one, I'm now going to increase up to one and a higher
weight, and I'm not going to change anything else in this prompt, and we're just going to
go ahead and generate again, the same seed. So, we have our images generated, and we'll go
in and look at these. Now, this contradicted me a little bit because, as we can see, one of those
images didn't give me the same smile as the second reference image. Now, I could probably adjust
the weight more and change that. I could also put a big smile, but that does give you an idea
of how you can adjust those weights and stop ats to change the influence of which ones of these
pictures. Now, you might ask why I just put one in there. In my experience, so far, when I only
had one in there, I didn't get as good results, even if I put the other ones at a lower
weight and only gave priority to one of the images. I still always found I got better
results by using multiple images down there. Now, you can still use the other functions
down here as well, along with the face swap, so don't be afraid. That's why I don't mention to
use all four; you might want to use something down in these other ones, so you can use two or three.
I didn't test the difference between those two much, but I do know more than one image definitely
had an impact on the results that I was getting. Another thing I do want to mention is,
let's say we had an image here. So, we have this one on the left; you can
use the face swap for inpainting as well, and I actually find this useful. Let's
say you get an image that you generated; it's not exactly what you wanted, and so
you just want to adjust it slightly. So, I'm going to come down here to my main tab, and
I didn't change anything in the image prop that I'm leaving as it is. I'm going to pull this down
into the inpaint tab, I'm going to actually paint over the face. And one thing you need to do if
you want to use the face swap and the inpainting is you have to go under the advanced section,
check off "Developer Debug Mode," and then in the control options, you'll find "Mixing Image
Prompt and Inpaint." So, if you enable that, now we can use the face swap on the inpainting.
Let's say we had this image, but we want it to be a bigger smile. I could reduce the weight of
this for one thing, and the stop at, and we left that one as the same, and I'm going to put "with
a big smile," and now we're going to generate. Okay, so we got our images generated. Now, I'm
not a real big fan. I don't think these came out as good as I would have liked. So, let's actually
get those green eyes, put a big smile. You know, I'm going to put the weight up on these because
I actually found that the face lost some of it, and I think it's because we're not using
as much of this image as I'd like to. So, I'm going to actually try this again,
using the same thing, using the inpainting, and I've added the green eyes, just
so I can show you we should be able to get those green eyes if we wanted
to. But you're going to have a hard time with a consistent character getting
the exact same shade of eyes, so you might be better off trying something like brown or
something like that for standard eye color. So, I'm going to stop the recording,
generate it because I can't upscale. This does upscaling when it generates. So,
okay, now that we have tweaked our results, I can actually see these look better right from
the start. The one on the left is the one that we originally had the lower weight the first
time we did the inpainting. The second one on the right is when we increased the weight of the
second image. We didn't change the other image; we only changed the now. Let's pull
that up so I can show you here. So, back in the image prompt, we had increased
the weight of this one, which we had originally reduced, and then we left the one on the right
here the same. It wasn't so good as far as the original image of the person. The one on the
right is, as we can see here, if we look at these images, this one is much closer, and that's
because we had the higher weight on the image. So, you really want to play around with those to get
the results that you want. As I said, you're going to have a harder time getting a different facial
expression. If it's drastically different, then you're really going to have an even more difficult
time, but that's something that you can play with as far as what the inpainting goes with the image
prompts and everything else. It's just a lot of tweaking, a lot of playing around until you find
exactly what you want, and when you do find the settings that work, keep track of those settings
so you can use those in the future as well. Now, one thing I will mention is this: I've had
people ask me, "Well, can I use the face swap and then do a completely different style?" Yes and
no, because, well, let's go in here. I'm going to actually check off all of these. Let's go with
the, oh, whoops, adorable 3D character one. So, I'm going to turn off the inpainting for this, and
we don't need that in there. Okay, so we currently have this set up. I have the face swap set up just
as before. We'll increase the weight on these. That's fine, and we'll go ahead and generate.
Okay, so we have our adorable 3D character generation, but when we look at it, we can tell
it's not really that good of a 3D character. It really comes down to, from my deduction, the
weight. The higher weight on these will impact that because these are more photographic. So, if
we were actually to reduce these down, let's try again and see what we get for results. Let's pull
those up. As you can see, we now have more of the look that we were probably going for with these
than we were with the other ones. So, if you want to apply certain types of styles, you may have to
lower the weight and the stop at to get exactly what you're looking for. Another option is to use
a different model, something we can do as well. So, for example, in here, let's say we went
with the anime one, we'll go with anime here, although I don't even think I need to at that
point, but now let's go ahead and generate and see what happens with this. I'm not going to change
the weights or anything. Well, you know what, I actually am for this because I expect the results
to be a little bit different with this because we're using a different model. So, there we go.
As you can see, definitely changing the model does have a huge impact, but you can still use the face
swap to bring that in. You may need to reduce the weight, although I find if you're using a model
that's trained, like this one's trained on anime, so it's going to do much better. You won't
necessarily have to reduce the weight as much. Okay, and one last thing I discovered while
messing around with these things, you can, when you're doing the Styles, so in here I
have that adorable 3D character. In previous, we didn't get the results that we wanted. You can
actually, when you reduce these weights enough, as you've seen down here, I've reduced them, you
know, a decent amount, and I've changed the prompt to "a woman with green eyes dancing," and I have
the adorable 3D character. Now I can go ahead and generate, and we'll see different results than we
got last time because we lowered those weights. Here, we got much better results this time than we
got previously because we lowered those weights. So, this time, they actually do look like
3D characters instead of more photographic. So, a lot of it comes down to the weights
and the stop at. The stop at, remember, that's kind of where it stops having
the influence, and the weight is how much influence it has overall. So,
by adjusting those with any of these, that'll impact a lot of your results, and
you really want to mess around with those. I think that pretty much covers what I wanted
to show with using face swap. The two biggest factors that I have found to determine how
well face swap will work is the quality of the source images and adjusting the stop at and
the weight to get the results that you want. If you found this video helpful, please do
consider hitting the like button since it does help. Don't forget to check out my other
videos on Fooocus as well. Any questions or tips on getting better results, definitely leave
those in the comments, and have a great day.
