The Official ComfyUI PhotoMaker for Stable Diffusion

hello everyone how are you doing today I have a quick video for today following up on my last video on Photo Maker where I showed you Photo Maker by z z z which was an unofficial implementation of photomaker for comy y in today's video I'm going to talk about comy y photomaker by Shimizu and this one it's the official implementation of photomaker inside of confi it uses the confi way of doing things basically it has efficient memory it's fast and it's claiming that it will not break when there is a conf y update so the project came out about 5 days ago from the time of recording this video and uh there are quite a few updates happening in fact this is my second time recording this video I recorded the video on Sunday I was editing it yesterday when I checked the GitHub repository I found that there is a new node which basically break the previous version so I'm re-recording this one and I will show you how you can install it how you can avoid conflict in the name because if we go all the way to the top or the Shimizu version we can see that the name is confy UI daser and if I check for the zoo zoo zoo implementation the name is exactly the same confy ui- phica now in the previous video the one exploring the new Photo Maker project in the comment section A lot of you got the out of memory error hopefully when you try out the Shimizu version since it is memory epici the issue may be solved now I would like to give a shout out to Teo R 78 who was able to solve the out of memory issue by going into the Nvidia control panel manage 3D settings and making sure that the qdf system fallback is prefer system fallback I'm going to show you this really quickly before we get started so go in your taskar at the bottom or wherever your taskbar is then click on the little icon that says show hidden icons this will open your tree icons and then find the Nvidia settings right click on Nvidia settings and then click on Nvidia control panel alternatively you can go into the start menu and at the top where it says search you can type Nvidia control panel click on it to open it it may ask you for administrator wres click on okay if you get that popup then on the left side click on manage 3D settings and then from the settings list scroll down until you see Cuda system memory fullback policy and then for me driver default works because by default my GPU is going to fall back to system memory if I do not have enough VR round but for those of you who are having this out of memory issue try this out click on the driver default or whatever options is there and then select prefer system memory fallback and then at the bottom you can see a little bit what it's going to do so it's telling us that QAM may allow allocation made in GPU memory to fall back to system memory RAM and you can see the rest and then once you make change so once you make this change give it a few seconds and then on the bottom right side you will see an apply button click on the apply button and then you can close out of this window and then finally can open up of confy UI and try again however with this implementation of confy UI the Shimizu implementation it's using the confi way so it's supposed to have efficient model loading and the entire implementation is supposed to be efficient from my testing I was able to get around 3 minutes of rendering time using the Shibu implementation and if you watch my previous video you will know that using the zuuu implementation I was getting 15 minute for one image to render okay let's get started so for those of you who have the config manager already installed if I go into the GitHub repository for config manager and I search the custom node Photo Maker you can see that the custom node Photo Maker by Shimizu has already been added to the comp wi manager if when you are searching in comp y manager you do not get that option when you search for photomaker please make sure to update your comy UI first so start up comy UI as usual by running the Run Nvidia GPU or run Nvidia CPU and if you're using a different operating system you may have to do python go into config and then main.py and then you will have your arguments as you would usually do okay so confy one open up I already have a workflow here let me clear the workflow so that uh you do not get any confusion then on the right side click on the manager option and the first thing that you want to do is to update all this will update your comi as well as the confi manager then you may have to restart your confy do so and then go back into manager install custom node in the search bar at the top right type in go to maker click on search and then you will get the confy Y photomaker implementation by Shimizu then all you have to do is to click on install button however if you already have the previous zoo zoo zo implementation of Photo Maker so you may get an error when you install it using the comy wi manager and this is because both project is using the same name Cy ui- photomaker when Cy UI manager is looking at your confy UI custom nde folder it will see that there's already a folder Nam as confy y- Faker and it will give you that error so in that case if you are not able to successfully install put to maker the Shimizu version using the confy wide manager close out of the manager close out of this download box you can exit out of the web UI go into your custom nde folder and then at the top click on the address bar and type in CMD press enter this will open the terminal directly inside your custom node folder next go to the confy photomaker GitHub repository by Shimizu I will have a link in the description below and instead of clicking on the code here and then copying the GitHub repository link go down and copy this line where it says git glud the GitHub repository link and then we have a name at the end so what's going to happen is that we are going to clone the repository but it's going to be save inside this smz folder name to copy this line you can highlight it so click and drag press contrl C on your keyboard to copy you can also right click and copy then go into the terminal and you can press contrl V to paste if when you are pressing crl + V you get an error it doesn't paste you do not see the text as it's showing here in that case you may have to right click at the top where it says your path or the name usually it's under CMD when you're using the old CMD then you may have to right click and then you get an option to paste from there another way to Pace in case you're having difficulty is to press the control shift and V together in order to Pace once you have the line there press enter and it will start the cloning you can close out of the terminal go back into your file explorer and as I mentioned it will be under the name confy y- Faker dsmz then go into your confy y window portable where you have your rent NPD GPU double click it to open or you can do Python and then the command so once it's loaded and you have the compi web interface right click click on ADD node and you should have Photo Maker there now make sure that you're able to load all three of them can simply take them and open them if when you are trying to add any of these nodes you get a red nude inset it means that something went wrong and it's not able to load the nude so uninstall the the nude by simply going into your file explorer confy UI and go into custom nudes click on the file press delete on your keyboard to delete the project and then redo the step to get learn it now there's one more node that was added and this one is under the IP adapter category so IP adapter and we have the prepare image for click Vision if you already have IP adapter the original project you may have prepare image for clip Vision Noe but this one is prepare image for clip vision from pa okay now I'm going to show you how you can use these NES to make a workflow and then I will link to some workflows in the description below as well as a couple of test images I've done in photo for me in terms of speed the Shimizu one is better about 3 minute to generate one image compared to Z 15 minute for one image let me clear the canvas and let me load the default workflow if you do not have the default workflow or you've changed your default workflow by a custom node can pause the video and try to recreate it or you can just wait a little bit until I complete the entire workflow then you can go ahead and generate it alternatively you can go into the description and download the we from there now this one because it is using the confy Y way you can use the traditional load checkpoint to load your sdxl model you do not require a diffuser format model in order to use photomaker second we have the clip text and code for positive and then one for negative these two will be here but the text I will have to use the ad Noe photomaker and then will use the photomaker encode this is the one that is going to encode the positive PR and send it into the conditioning so we are going to remove this positive po one replace it with Photo Maker en code the clip will go into the clip the photomaker one will require add nde photomaker load Photo Maker and this photomaker will go into the photomaker for the encode node the image will take the image drag out and select load image this one it's okay if your image is already prepared if your image is not prepared you can take the load image image output drag out click on search and do crop image crop and then you can take the output of the image crop into the phot maker encode of course if you want to see a preview you can do a preview image there that's fine now this image crop nude is the default one that comes with comi so all of you should have it if you install custom nude and you have it better one then use that one now if you want to load multiple images then traditionally what we have to do is pick these two copy so I'm going to select these two by pressing control clicking on load image and then clicking on image crop this is going to select both then press contrl C to copy and to paste I'm going to press control shift and V to have these two links together now I need a way to combine these two images into one because I have only one input for the image so for that I'm going to take this image out click on search and then choose image batch okay this give me two images that I can combine into one image so I can take the these two okay so I have these two images here it goes into the image crop and then into the batch images and then this image can go here in order to have multiple images however since we have this new node add node in under IP adapter prepare images or click vision from path you can take the image output from this into the image input from the encode new and then the path you can go into your file explorer let's say right now I'm under custom node I'm going to go into the UI with your maker I will create a new folder you can create a new folder by right clicking on an empty space click on new folder and then I'm going to save photos go into it then at the top I'm going to click where I have the address bar this will give me the path and it will Auto Select it if it did not Auto Select it is like this for you you can press control e to highlight everything then press contrl C to copy go back into conf Pui and then you can paste the link by pressing contrl + V into the path field and this one you can choose to crop all the images from top bottom left right center or by padding now of course when you draw images wherever youve got them like let's say you got them from a smartphone you've taken some selfies so you can just copy all of your images into this folder or you can go online download a couple of images and then save them them in this folder now you do not have to use a photos folder you can have your images anywhere on your dis but we need to have the path to those folders and make sure images are all that same character the same person and the face is the main focus on those images that is why we have the crop here we are dropping the image so that the face is the main focus now regardless of which one you use whether you use the path one or the different load image and then go into bat images the image output will have to go into the photomaker en code node the trigger word IMG it's very important photomaker works when you have a class word the class word can be woman or man and then just next to it you need to have IMG as the trigger word if you do not have IMG it will actually give you an error when you try to run and queue a PR now this conditioning it goes into the positive input of a case amplock it can be a traditional simple Cas amplock or you can also have the advanced Cas sample the conditioning goes into the positive and it works exactly the same as you're used to the only step is that instead of using a clip text and code as your postive for you all ining the photom maker model sending it into the photomaker encode node by also passing the clip and images and then sending that conditioning over to the cas sample now we have one more note by going to add nde photomaker apply photomaker this one works slightly different compared to what we have so it tells us that we have a positive input a negative input and then a style input and then this node is going to Output positive and negative now at first plan you may be thinking that you just have your regular text string here passive to positive and then you take the positive over to the k sound block but that is not correct this positive that we have here it's actually a text and it's not a conditioning in order to pass it over to the cas sample we need a conditioning which we can then pass over to the key sample but this positive it's a string en code or you can also see TS so what we need to do is let me make some space I'm going to take the prep image a little bit on the left the preview image can be here I'm going to move this load photomaker over there now as you can see it's tell me undefined I'm going to show you where you can download the model now this photomaker it has a text field for for the positive FR I'm going to right click on it and then convert the text into input so now this text is going to receive the positive PRP I'm going to do the same thing for the negative flip text and code right click on it convert text to input and the negative is going to go into the negative clip text en code now okay so let me make some space so that you can see exactly what is happening all right so we have the photomaker enode it is taking the positive from this apply photomaker style and then then negative clip text and code is taking the negative output from the sty node so if you want to use that style then you will have to do it from this way now in order to type in the text you will need a text field so we can take the positive drag out click on the search and then where it says a strring click on it click on the top one it should be blank but you should be able to select it and then at the top most of you will have primitive nde at the top but if in case you have a lot of custom nude you cannot find it just simply type in primitive and you should be able to search for it this will give you a string field where you can type in your positive prompt and then you'll need a second one send that over to negative so you can do the same you can copy paste or you can press alt select it and drag out to get a copy whichever method you want this is going to be your positive and this is going to be your negative it will go through the apply photomaker style and then go into the photomaker incode but in case you do not want to use these three notes you can use the traditional clip text en Cod for your negative note and you can use the text field inside the fot maker and code as your positive field and then from here on every everything else is exactly the same so if after this you go into an advanced key sampler and then you return some noise and then maybe go into a second Advanced one you can continue doing the same if you were using traditional IP adapters the original one from the IP adapter plus group of custom node you can actually use those I do not have it install on this one because this is a fresh install but if you have that you can load it and since this version is using the confy Y way you can pretty much have this section into its own group so you can have all of these into a group and then anything else is whatever your imagination needs you say if you want to go into IP adapter into Aura into advanced case Handler you want to do upscale you can go ahead and do all of that I'm going to link to a post where I show all the different images I have some here so we can take a look around those earlier let me zoom in a little bit and the first one I only said a photo of a woman imj now and do start so if I go back into the workflow the way I did it in dep positive P I did a photo of a woman IMG like this and this goes into the apply but to make a style in the drop down I've selected new style and then the rest is pretty much the same and as you can see this is my input image and these two are the outputs it's close it was able to get some of the features like the hair the skin tool but um the face I'm not sure if it is exactly the same I run a second test and this was the result it's it's not bad but the hair is not exactly the same now if third one I want to try I just pass in a photo of a lenscap and just give me a random main image so like I mentioned in my last video this put make can do is not good at grabbing this style and then transferring the style over is just for identifying the different features of the base and then passing that over as a conditioning over to your casy sample now the difference with IB adapters is that IB adapter is going to add more information into the model in this case we are adding more information into the conditioning and sending that over to positive and negative okay so lastly let me show you where you can download the load photomaker model so go back into the GitHub regory from Shimizu and scroll a little bit down until you see installation and it will tell you to download the model from hooking face and place it into Photo Maker folder in your models so click on the link it will send you to 10cent art video maker project this is the original project then click on files and version and the one that you want to download is the photomaker dv1 dobin if in the future there's a new version let's say there's a V2 then you can try both and download those to download it you can click on the name and then click on download here or you can click on this little icon to download start the download next go inside your confy UI folder click on confy UI go into models and then I've already created a folder named photomaker but in your case you may not have this folder called photomaker then click on add new at the top folder and then rename the folder as Photo Maker old lower case go into the phot maker folder that you just created and then wherever you've downloaded the model copy it and paste it inside this folder go back into confy UI press on refresh and it should load up right here next you want to make sure that your load checkpoint you have checkpoint that are sdxl inversion now it doesn't matter whether is a base model realistic Vision sdxl all of those are okay and you can try them it will give you different styles some will be realistic some will be anime like but the main thing is that photomaker is not compatible with SD 1.5 models it is only for SD XL version so that is it hopefully you were able to get this version working let me know in the comments what are the results that you got when you use this version compared to your previous version so thank you thank you for watching and have a nice day ahead I will see you in the next [Music] one

Channel: Code Crafters Corner

Views: 5,182

Keywords: Photomaker, ComfyUI, Digital Art, AI Technology, GitHub, Open Source, Face Generation, Artistic AI, Hugging Face, Tech Tutorial, stable diffusion, IPAdapter, Shiimizu, Code Crafters Corner, CodeCraftersCorner

Id: CYx6I0OetF4

Length: 25min 14sec (1514 seconds)

Published: Tue Jan 23 2024