ComfyUI: Consistent Face MOC | Stable Diffusion | Deutsch | Englische Untertitel

Hello and welcome to this video in which I would like to exchange a little bit of life time against knowledge . I looked at a bit of face swapping last week and came out with a workflow that actually works quite well for me and I thought it shouldn't be too complicated and it should also be easy to install because I think on the channel we have a wide range of ComfyUI users, i.e. all newcomers who come into the topic up to more advanced users and I didn't want to do such a complicated workflow now, I thought it would definitely be easy for me, but then I came out with this next workflow and I just want to put it to the discussion that is such a face swapping mock so my own creation what I had found so far I never really liked it and that was too complicated for me in that direction, so I just show it to you and yes we can then go into the discussion or you can write to me how you do it or how you find it whether you achieve better results or not whether you think if you try this workflow here that it is always the same person who comes out in the end he still has a hook let's see if we run in there in the course of this tutorial but let's just get started and we'll build the workflow ourselves together and I hope I remember how I made it to install the custom notes give the ComfyUI manager a swap there you can already see the note that I have used and that is this one here the simple face swap note and that's what I just like that you can install it can download it can and it works that was pretty important to me at this point I'll leave that again because with the others, for example, if you look at the notes for ComfyUI and so on, it is a well-known framework for face swapping here, however, it is often mentioned that you have to install visual studio here c++ build tools and something like that and that's what I actually wanted to avoid I don't want to install any additional software or anything else it's not software it's tools SDKs in that direction that's what I want to avoid that's why I chose this note here and it's also very interesting that it is said anyway there are so far only 128 by 128 models in the area so that was published that is very small so and that's why this face swapping technique is also not necessarily suitable to exchange large pictures with faces but we'll get to that in a moment if we look here in the root notes, for example, we can already see that here the in swapper 128 o n n x is used that is also the or this 128 model and so from the size of 128 by 128 that's why I stayed with the decision I just use it now and we'll take a look at it further if we need a few other things that is probably the quality of life suite the impact pack with the face detailer that you can see here on the notes and the links are below in the description if you are missing custom notes and otherwise if I have provided the workflow and you can try it out here you can also go over install missing custom notes and reinstall them we just start at first I take two loader from the tiny terra notes that I would like to take it with pipe loader for a moment so you have to cut very briefly I had forgotten to start the recording program because it shows us which combination of keys I press down here so it was a simple control c control v at the point that was nothing dramatic good from here we go into spy sampler I already enter the model or we give both samplers the same model check point name to input and it works now by the way that if you have now created such a separate input here you can now double click on it and you get a primitive right away that's great or I think that's also extremely practical we take the epic realism connect it once with both samplers and then we load the same model for both samplers I said we need two samplers here I stay with euler and normal so that I don't have to adjust the sampler names and the scheduler for each sampler later so the upper area should serve us for creating a face and then we can directly hang a preview image here we will need that anyway later here we can also turn on the preview here, too, that means we say here I take a fixed value here any seed and I say here 40 year old woman with brown hair and blue eyes dreadlocks I leave the negative at the free point or simply for safety we can already enter our two standard negatives well the notes down here I switch off again that was the control and that clicked too quickly control m ok once through or load the model once and then let it slide I have forgotten something and that is white background I would like to have a focus on the person himself and we say here at the place close up off that should not be an underline but a binding line so that's good I think that's okay it's characteristically okay good from this point we will now turn on the lower bar we say we first want to have this prompt here that we have here in front of us not have to enter again and again we do that by adding a text note so we copy this prompt here out and in there and then say at the point we definitely want to put it in the positive input and now we are looking for a replace and here comes the quality of life suite to wear and that does not fit here again moment what am I doing here which one did I have there replace string no it was not replaced text it is sorry but then you have to go through and here we can say we want to convert the text into an input then we can hang it in here and then we can hang it in there in the positive good that means this is our person now we also have old and new here we now say at this point our old value will be his underline person and the new value new input there we hang our text that means we need a new note of it and now say at this point only under line person and the whole thing we hang in the text now we only have to take out the wide background here down here and pack it up here what we are doing at this point now is we take this text and replace the underline person by that and that gives us the possibility if we now just hang a text debug under it here that hopefully shows us that yes now we have close up on our prompt stop plus white background that worked and that serves to describe our character or our face for the sampler down here we also take the positive in the input we also take a replace we replace the same thing with new also our person and take as text but that what I just noticed is that we also have to take out the close up here above and put it here that means up here at the point we get how do I have two empty signs yes I hate that up here we get close up before the old woman blah blah blah white background and if we do that down here moment that is now our text here we can now say underline person and then make different other prompts for example here we can then say absurd rest masterpiece high detail intricate 8k something like that but we can also say we want to have different variants let's take a wildcard and can say walking through what's going on here through the street waiting for the train what else can you do in jazz club something like that that means every time you run through it takes a random value from these terms and renders our picture but nevertheless we have here in front of us because we can exchange again the description of our person at the point that we can now try out a positive attempt to render so that works quite well what we are doing now is we go back here in an upsampling process I just take the high res fix scale I can already see the height of the picture at 712 so that we can get a little more photo size here we go here in an upscaling process and also take an upscaling sampler option and the new one down to 0.5 so now we get different pictures here again and again that are then upscaled that works and at this point here I then build the face swap node which is basically not difficult to understand here you just have to say we want to have the picture that is down here the image what we want to change that is that and here we have the face and there we can now take the face from the upper sampler and instead of this picture we now send this picture to upscaling to take a look at what is happening here I would say we are hanging another preview image node up here because that is our actually is already the third step we can also take a preview image node here that is the second step that is the third step and here we have the fourth step if you let it run then the sampler runs through that is the picture that we got here okay good point with the face swap node I would have to enter a 0 again down here these are the indexes of faces in the picture that means we have also got a bad picture but I don't know if that would be recognized as a face but that would be 0 1 2 3 4 5 we want to have an ideal case only one person on the picture that's why we do at the point 0 and 0 so we want to use face index 0 from the picture and face index 0 from the one that would then have to work if I let it run the image is swapped apparently the preview node is updated up there a little later it is upscaled and what we will see there is as soon as these nodes have been updated here I also have the size now I can put them together a bit so we now have our original image so our original face better said we have the image that was initially created by our first sampler we have the image after the face bob here you can already see that the face was transferred to the small picture and here we have the up sampling the up sampling of course everything breaks down a bit again what the face bob has done but what is good at this point is that we can send the face to the right proportions and as I said at the beginning we only have a 128 by 128 pixel model available here that means if we would do that on the upscaling then it would be super pixelated because this face is simply scaled up without k.i. but is simply enlarged and if we then chase it in the upscaler we get the same effect that the upscaler creates its own mix out of it and we lose the person again so how do we get to the point now that we can make the face consistent and for that I thought of the following one thing I still like to do here that is an image sharpener I just have to see which one we take there are even simpler ones right here from the post processing suite here I would like to take the image that we got after the face swap and sharpen it a bit before it goes to upscaling and 0.3 is usually enough but now we come to the point back here and here we will first install a face detailer from the impact pack and we have to connect it once here we say now we want to have the image we want to have the model the clip dvae the positive the negative well now I have hung myself here so negative and of course we need a bbox detector just search for the ultralitics detector and a sam model we can use the sam loader here and with that we have that we can now add another preview up here at the point or we hang the preview down here because that's supposed to be the last picture so and here the trick happens now that is actually quite simple because we will load the ip adapter here at the point I have made an extra video look in there that is a great part that analyzes a picture how it looks can describe that and based on the description that he has created then also tell the sampler ok I want to render such a picture now and that means we send the model from down here in here and then in the face detailer you can push it over a bit so by the way you have seen you just need to pull out here then you get the ip adapter apply then you pull from clip vision over here clip vision loader so you can stack it all together quite comfortably so all that is missing is what it should describe and then we take our picture what we have done with the upper sampler i always make a reroute so that you can see a little better where it comes from or you can do it that way that is better to see yes it works like that but I think you can get it out so and weight 1 is definitely too much I can already say that values between 0.5 and 0.6 work quite well here you can of course play around with it but that was so from my testing I really played around with it all week in the evening with the workflow and found it so best in between but we don't take the plus face model here but actually we take a normal ip adapter plus model you can also try it with the face model the face model but does not transfer one face to the other at the point but it only describes the face that we send in here and then try to do the other with this description that also works but I found that it works better with the ip adapter plus and now we are actually already at the point where we can just say we let the whole thing run what is she doing now now she is in the train or at the waiting station sharp radios crashed ok who doesn't want to get a different sharpening so I haven't even introduced the note to you yet I have to take a look if the workflow is missing or so let me know then I have to dig out the link again that was nothing that you could install via the comfio eye manager but download individually you can but the sharpening is optional you can also take it out again so we have arrived at the upscaling and now we go to the face detailer that then with the description of the ip adapter plus the picture again or the face again individually rendered and that is then the preview note down here we are waiting for it right now I have only increased it to 50 percent I could say here we want to raise the whole thing to 1080 pixels in height so and there it comes so and now we have transferred the face from here to there that is unfortunately a bit of a change at the point let it run again so that's a bit better and we see we have transferred it now I'll just leave it I'll do it at the back a safe note and that is the image safe exactly here I would like to have in the folder model I don't know anymore how my other model order is I'll start with 10 but I say he shouldn't include the workflow and he can do the rest quietly and now I'll just let five pictures run and we'll take a look at them after that five times q and off we go so I let five pictures run now even with errors or I let more pictures run because there were other errors but I also left some with errors we can talk about that this is our first picture and that's what I meant that happens more often so what does it mean more often maybe in 30 percent of cases 20 30 percent of cases that we get a shadow here that is I don't have a solution for that is rather stupid but yes then you just render a new picture and it fits again so here a problem has arisen that we now see that the face detailer has kicked in that is another problem but that concerns the face detailer that would have improved every picture here or every face but we are talking about the main face and I think that is already very consistent if we now move on to the next face here we also see such a slight shadow as I said then you probably have to render a little more pictures here in the back the face detailer has also jumped in here and that has become quite good because the rulers also came to us with the back but if you look at the face alone it is quite consistent in my opinion and I always find it a bit difficult to say yes they are the same people that's why I like to just put it to the discussion here suggestions from you what you might be able to do better or how you do it better we now definitely take another prompt 30 year old woman with blonde hair blonde short hair and green eyes it may be that the model or because of the model that the eyes become extremely green but that is another problem I switch the notes down here and we are now first of all a characteristic face up here yes I'm looking for really characteristic features like a wider nose or something like that I'm thinking right now we see that I mean with the eyes by the way they are exaggerated but somehow that makes the epic realism always when you put ice in it I don't know why you can probably control it a bit with prompting maybe we should say again here in front old 30 year old british woman maybe then a little better can trigger not quite so I don't want to say generically but I'm just looking for real characteristics the face could be quite good is a bit wider give me a seat up let's see we'll take that in front of it so if we switch on the whole glider radar again hours but before I want to edit the prompt again because in the jazz club he is bad there are too many people in the background what could they still do sitting on so now we do the whole note down here again because I got one too much so and I'll let a few pictures run the first one we'll look at again we'll get here again with underline person underline in about the same way the hair doesn't really fit now because the short interpreted a bit differently but it doesn't matter that you can prompt here is our face bob entered we can already see that the face was transferred from here to there compared to the two pictures that is the upscaling in general again people I hope the face detailer doesn't make any jokes he would have to run now yes and there he is done now we can take a look is that the lady we have up here and I think yes mother paint and so will unfortunately not always be transferred it is also in the neck he probably takes the face here but she is it the eyes are exaggerated you can maybe say over saturated ice that bothers me a bit but is a model question we take the second one again why has he why has he changed that now so because I changed the negative prompt yes ok so now it's on the meadow has the legs somehow in a rabbit hole or well it doesn't matter as I said I don't know what the good one is doing so basic picture with underline person underline face swap upscaling and here we have the whole thing again then with the face detailer afterwards and I think again and again yes it could really be the person I do it now like just now I create five pictures here and we'll take a look at them through so we're through here we'll take a look at it that's now our lady hopefully consistently on the photos here I would say if something like that still appears maybe a little post-processing and to melt it all a little better nothing nevertheless I think that it is about the same person yes as I said I am a bit difficult to assess here in the back the face detailer again that is ugly no matter what our main motif is and that's why I always look for characteristic faces like for example here she has a slightly more voluminous nose and she will also be shown through yes I think you get pretty good results out of it and the workflow is relatively easy to build up relatively easy to understand here in front of that with the replays you don't necessarily have to do that it's just more pleasant if you describe the person who you want to render that you don't have to do it in both prompts up and down here sharpening is optional I just like to have it at this point because I thought if we get such a pixel-like face-swapping image then we sharpen it before it goes to upscaling and yes the rest the rest is actually not very complicated I think you get good results but we can try the whole thing with a man 50 near 60 60 year old man with short gray hair and brown eyes here I would like to have a beard as a negative because it is difficult with beards because they can be long be short maybe a prompting thing but if our original image has a short beard and in the end it comes out for a long time the face detailer no longer recognizes that at the point and then yes it is not so nice so let's take a look is he already good he is already quite good I think we let him do the notes down here again and let the whole thing rattle through so yes that is not so consistent with the with the fleeing forehead now and the hair up here but as I said we can go there you can then go into the discussion or write what you would like to improve or so it was for me now that you don't have to reinstall any additional things and that the workflow is basically easy to build up is easy to understand you can still cancel it there you can still crawl in at different places and so on but that is what I have found out now to get a consistent face relatively easily so here you can still see the difference between the original edges and after face the upscaling makes it a bit broken again but the facebook ensures that we get the right proportions in the picture and now let's take a look at it here and here too I think yes that is the master the nose the mouth the eyes and the flying forehead no longer quite but it works and of course you can also jump between the models here so you can also just copy it here and hang it down here and then you could also take another model down here we take the absolute reality that means we create a face with the epic realism then create a picture with the absolute reality which is also very interesting here is the facebook that has worked so well in terms of size we are in the 128 pixel area on the way small face now let's just look at the end result and it is the nose again the mouth so i always think yes it is the people from the picture of course in other situations on other days but in principle it works yes please try it out yourself share your experiences with me and suggestions that you may have a small workflow that can create consistent characters but not too complex at the point you can also use open post by the way then do it here here in the open post area in between then the basic picture is generated with the pose and then you have your character in the pose if you want to create specific, consistent pictures yes have fun experimenting and playing around with it I wish you a good time until the next video bye

Channel: A Latent Place

Views: 1,378

Keywords: ComfyUI, Stable Diffusion, AI, Artificial Intelligence, KI, Künstliche Intelligenz, Image Generation, Bildgenerierung, LoRA, Textual Inversion, Control Net, Upscaling, Custom Nodes, Tutorial, How to, Img2Img, Image to Image, Model to Model, Model2Model, Model Merge, Face Swap

Id: MMght5dkMAc

Length: 34min 41sec (2081 seconds)

Published: Thu Oct 19 2023