Realistic Face Swap with Stable Diffusion | EasyPhoto sd webui A1111

easy photo is an easy way to swap faces in photographs train a face Laura pick a photo to Face Swap and then generate yes hello and welcome to More nerdy rodent geekery where today I explore the easy photo extension for the automatic 111 web interface see how to use it if it can do video what it does well and also what it does not quite as well this is indeed also available for comfy UI but I just like to switch things up a little as usual with these extensions you can install it via available from the extensions tab click load from to get the list click install for both easy photo and control net if you don't have both of those already and then apply and restart however be warned the first time you run this you'll be in for a little bit of a weight you'll also need to have a good chunk of disc dis space as this can take up to 60 gig for all of the downloads it will be less if you have some of the files already just so long as they are in exactly the place it expects with the file names it expects too things it downloads includes the chill out mix stable diffusion 1.5 checkpoint an sdxl checkpoint and a whole bunch of control net models also as usual with these extensions its requirements May clash with those of other extensions you have installed if you have issues where the easy photo tab won't appear you may need to disable some of your other extensions for this video the version I'm using is the one displayed there 9 c78 259f and in my tests the runs here need at least 10 gig of vram for training though of course your mileage may vary in addition it it needs at least three control nets for inference so be sure multicontrol net Max models is set to three or higher also take a look at the easy photo extension page on GitHub for any updates as well once installed hop on over to the easy photo tab where you'll need to start with training the defaults work well though there are Advanced options available as well should you wish to change those do exactly as it says on the screen upload your photo following that guide 5 to 20 half body photos or Head and Shoulder photos please don't make the proportion of your face to small training is going to take about 25 minutes and perhaps try to limit your photos to about 1 and a half Meg each okay so let's get this going click upload photos and then you will be able to upload a whole selection of photos there I'm just selecting them all you can upload them all at once and those will appear in your gallery this is also known as a data set because it's a set of data all right so there it is there's my data set 14 images following the guide provided apart from I made all my images for Meg because I'm extremely naughty sometimes you'll also notice that they do have a rather limited War warders here at rodent Studios we don't really have much of a budget and hats off to her she did really well being Barefoot that day anyway if you don't want to pay somebody to be your friend for the day and take photographs of them you could always try taking pictures of yourself or as I actually did here simply use my reposer workflow to generate a variety of poses instantly from a single image anyway with that data set up upload all you need to do now really is Click start training when you do that it is going to give you a little popup and ask for the name of your Laura it's going to save that as a Laura file so don't overwrite any of the ones you've got there it will pop up and say hey this name is already in use but before I click that let's have a look at these Advanced options as you can see there some fairly standard things it's also got a bit of information down the bottom there about the parameter paing Now by default you just get those two to start with those are the two models it downloads but if you click the refresh button then you'll get your usual list of models so pick the model you want to use there there you go that's the one I'm going to use and you got all these different settings across here resolution is good at 512 increasing that will of course need more vram for example if you put that up to 640 then you're going to need at least 16 gig validation and save steps default to 100 meaning with the max steps at 800 you'll get up to 8 checkpoints saved in the easy photo user ID infos user weights directory as well as the final model there as well so with each of those being 151 Meg I personally set that value a little bit higher and usually use around half the number of Max train steps so there in that example I set that to 400 and it saves fewer files along the way Max steps per photo defaults there to 200 which in this case would usually mean a total of 14 photos I've got there times that 200 Max steps per photo giving me 2,800 steps but as I have the max train steps there set to 800 that's the actual limit which is going to to be applied again if you have more vram you can change things like the train batch size and gradient accumulation as well but having played around with this for a while honestly those default settings are absolutely fine the same goes for the rank and network Alpha if you have experience with Laura training already then you know go ahead and change those but the defaults are absolutely fine it does have another option there for enable reinforcement learning that will take a little bit longer but can improve the quality of the result you've also got other options there for validation and skin retouching okay that about covers training so for the most part really all you want to do in the advanced options is change your base model perhaps the resolution if you have more vram but mostly those defaults are absolutely fine so all you really need to do is Click start training and then there where it says user ID it's not really a user ID it's just a name that you're giving for your training subject here now obviously the naming convention you use is entirely up to you but as the results do come out better when you use the same stable diffusion checkpoint that you trained with one thing I tend to do is put the a sort of summary of that checkpoint which checkpoint I used in this case analog Madness at the beginning of the user ID so I know which checkpoint I trained this model on all you need to do then is click okay and wait for around 25 minutes once you see the message the training has been completed then you're ready to go over to the inference tab there are four options here template upload batch upload and sdxl beta the template tab as you can see here simply has a variety of example images that you can select and then transfer like I've done there your newly trained face onto so all you have to do is pick an image whichever image you want and then down the bottom here once again you can select your base checkpoint the user ID that's the one that we just trained remember when that little popup box where you put the name once again the defaults are absolutely fine so all you really need to do is just click on start generation and that will transfer over if we crack open those advance aned options as you can see there are quite a few in there and it's got some more information at the bottom if you need to know more about what they do but essentially there you can see you got phas Fusion before so how much of the face do you want to transfer before and after number of diffusion steps by default 50 the den noising strength which can go all the way up to 0.6 or all the way down to 0.3 so they're limiting the ranges there for Fairly sensible things Second Step diffusion and also the D noising for that second step diffusion plus a whole bunch of options down here as well now as for the face ID there is meant to be something for multiple faces in settings but all I have is cache pre-process model in inference that's that's the only setting I've got there for easy photo now this isn't quite what it says in the read me I haven't got this select number of face ID so I don't know whether that's just the way I've installed it or there's a bug who knows but for now I can only select the one face so back over here we've got that number of face ID just leaving that at one okay onto these other settings basically I leave all of those alone crop face all of that super resolution you may or may not want one thing you can probably see missing from there is the actual resolution because it takes that from the input image so if you're dragging a really large image into the upload section which we'll look at in just a moment and you've got super resolution as the last step then that can result in a very large image indeed and will take quite some time to process one thing to note down here however is this makeup transfer option if you try to use this option chances are you'll get the same error I did which is related to numpy saying these aliases were deprecated in numpy 1.20 and uh that's a really old version of numpy which doesn't play well with python 3.10 so for now I just leave that option unticked these can take quite a while to generate so be prepared to wait much longer than usual as well I've also found that you actually need to do this template Gallery step first before you can use some of the other tabs which is a bit strange so always always just do one of those first okay the next tab is upload and this is where you can use your own images time to put things to the test what can we throw at it here all right let's put in this picture here now this person has glasses so what is it going to do with glasses let's just use all the defaults and start generation and it seems a bit touch and go there we've got sort of a bit of a glasses frame which is almost turned into her hair but it has removed her glasses which I guess is correct because there weren't glasses in the original photo how about if we are much kinder to it there we have a photograph without glasses okay that's a bit better it does seem to be quite nicely uncanny valley the eyes are weird but yes it's transferred her face across obviously it's kept the original hair and just changed the actual face portions there I know this is easy photo but what about if we use a painting excellent that has come out completely strange as you can see it's tried to make it more realistic if it prefers realistic how about if we use a photo but it's a photo of a statue looks to be equally as strange I like it it's still a little bit statuesque it's sort of done some blending in there but my is that weird how about the other way around though if you're training a cartoon face as it happens I trained one earlier using this data set so what will this one come out like pop over to inference and change that one to the cartoon yes that is certainly still just as strange got some really weird eyes going on there all right how about one final test here if we try to change the cartoon into a realistic photo uh okay well there's there's your answer I I guess I guess personally I think it does much better on photographic style images but do give it a go yourself and see all right the next tab is batch upload and I know what you're thinking can it make videos well judge for yourself and see uh yeah it's a little bit flickery isn't it for these I turned off super resolution as well there it is super resolution at last no thanks as processing already takes ages even for these 90 frames the sdxl beta tab is the last one here and I found this a little bit Touch and Go often it would generate the first image but never again hopefully this will work but maybe not uh this works slightly differently in that you can't upload an image instead it generates one using sdxl and then does all the technical face swapping stuff afterwards this also needs at least 16 gig of vram so make sure you've got enough being marked as experimental I'd expect some updates in the future to make this a little bit more stable all right you can see there obviously you've got loads of different options you can have upper body or head shot girl woman type of cloth he can have different colors all that sort of stuff so let's see if it generates or if it just crashes okay looks like we got lucky this time and you know it works quite nicely when it does so they're a completely new generated person but with that trained face transferred onto it just to finish off here it's will also create a Laura which you can use as normal so there I've put in the Laura uh we'll have it a strength one because why not see what this generates and there we go she looks a little bit raggedy there maybe perhaps slightly worse for wear but nevertheless quite a good output it looks like easy photo does appear to be easy and you can use it to swap faces in photos I guess it does what its name suggests a bit like reposer my workflow which not only generates a version of the face you provide but also the hair body and clothing too all in a few seconds without any training just from a single image

Channel: Nerdy Rodent

Views: 17,562

Keywords: nerdy rodent, ai, ai art, EasyPhoto, face swap, face clone, stable diffusion face swapping, stable diffusion face clone, automatic1111, a1111, automatic1111 Easy Photo

Id: K2VYtrpjyQU

Length: 15min 46sec (946 seconds)

Published: Fri Oct 20 2023