Create Consistent, Editable AI Characters & Backgrounds for your Projects! (ComfyUI Tutorial)

in this video I'm going to show you how you can create consistent characters POS them automatically integrate them into AI generated backgrounds and even control their emotions with simple prompts this workflow works with stable diffusion 1.5 and sdxl so any style is possible and you can use it to create children's books AI movies or one of these AI influencers that apparently will make you thousands of dollars a month one of the main tricks of this workflow to generate multiple views of the character in the same image and to be able to do that I created this post sheet which you can download for free on my patreon it depicts a character's Bones from different angles in the open pose format and using control net we can generate characters based on these bones if you want to use it in automatic 1111 make sure to set the pre-processor To None when using it with the open pose control net but I will use it with my custom workflow in comi and I prepared a free step-by-step guide on how to install it and set up my workflows it shows you where to get all the models that I used and where to put them in your comi folder structure but now let's create one of these AI influences so I open com UI and I drag and drop my workflow into the interface it consists of five parts but the first two are the most important ones as they will generate the sheet itself before we start we need to import the post sheet here and choose a model down here and I'm using the wildcard XL turbo model just for faster generation but you can use any model here just make sure that you match the K sampler settings to the recommended ones of the model and if you want to use an SD 1.5 model make sure to switch out the control net to an open post control net that actually works with SD 1.5 so now we can put in our prompt of course we could do like the typical influencer model beautiful woman let's click generate and see what it will create as you can see it works really well but that's just too easy right I need to find my own Niche and I think it's cheese let's create an influencer for cheese a cheese fluenc and let's just create a friendly German dude living in the Alps breathing the fresh air living among the cows so again I keep this part of the prompt and then add my character after that so let's try it out click Q prompt and after a few seconds you can see this preview here and this looks a bit creepy and weird but we will fix that I feel like he needs a mustache oh yeah that works really well I'm really happy that it's this type of mustach if you're having some bad luck with your characters generating in weird poses or looking inconsistent just stop the generation process by clicking view q and cancel and then try to add more descriptive prompts and play around with the seat in your sampler until you find something that you like so when you're happy with this preview result all you need to do is Click Q prompts and all the steps after that will run automatically so the first step after generating this preview image is to upscale it from 1K to 2K and you can see how much this already improved quality but the faces especially the small ones still look kind of broken so next I use the face detailer and this will automatically detect all the faces in your image and red diffuse them and you can see just how amazingly consistent the faces look now in the next part you have the option to save all these different poses as separate images so basically this just cuts out these poses and then they will just be saved out in the next step you can generate some expressions for that character and for for that I'm again using the face detailer and they look pretty bad face detailer always goes for realism unless you specify otherwise so the trick here is to also add Pixar character as a prompt in this box down here in these settings for the face detailers make sure to match the steps CFG sampler and scheduler to match the model that you're using I'm using wild card so this works pretty well finally the dooy strength will determine how strong the new generated expression will be I found 45% is a pretty good though kind of strong starting value if your character ends up looking too different you can reduce that value or also add elements that change to the prompt down here so for example for my character the mustache type changed so I also added handlebar mustache down here you can see that the faces now look much better and much closer to being a Pixar character finally this last part just adds all the different Expressions together and upscales them and also upscales the single image of the face if you don't need certain parts of this workflow you can just deactivate them by pressing contrl B and here's our full character sheet for hansal cheese fencer and presenter of the world's finest cheeses now we only have to put in in some nice locations and give him some cheese to present to his audience but before I show you how to do that let me show you what else you can do with this character sheet and this workflow you could for example train your own Laura for this character based on all the images we just created and to happy with that I added this save image node after the first face detailer here when you activate it and click Q prompt again it will save out all the different images of your character's faces you could also take these images to Mid journey and use the new character reference tool to put this character into different locations just upload all these images here and use a short prompt describing the action that you want to see in your scene and then put in the C parameter after that after that you just copy the links to your uploaded images and add the CW parameter set to 100 so that your whole character's appearance is taken into account but as always with mid Journey it's really hard to get specific poses and you might need to rerun a prompt a few times to get exactly what you want so now let me show you my free workflow for how we can post these characters and put them into different locations let's open up my controllable character workflow it consists of three steps first we pose our character and generate a fitting background then we integrate the character into this background and finally we can change the expression fix the face if it changed too much and upscale the image at the start here I choose the same model that I used to generate the character and make sure that all the settings in the K sampler match the recommended settings for that model I copy over the original prompt we used to generate the character but it's really important to cut out the multiple views and instead paste that into our negative prompt otherwise we would get really weird images like this one next we need to load in the correct images for our IP adapters an IP adapter basically takes the likeness of a character and transfers it into a sort of prompt that way all our generated characters will resemble the original one very closely until the first one you put the image of your character's face and into the second one we put the medium closeup or the full body pose depending on what pose you're generating but generally the medium closeup in the second one works really well for all types of of poses speaking of poses we need to create some and my favorite method to do that is to use openpose ai.com he can move this skeleton into the exact pose you want you can even move the individual fingers make sure to set the scene height and width to match the aspect ratio of your the image that you want to generate also play around with the focal length and the proportions of your character so they match the generated one when you're happy with a shot's composition simply click the play button down here next to the preview and click on the first and second image this will download the Open post bones and the hands as a depth map now you can drag and drop them into the correct control net inputs first the pose and the second one is the depth map if you want your character to follow this pose really closely leave the strength at one otherwise you can reduce it to give your character a bit more freedom when you're happy with your image you can generate a fitting background and I'm generating this cheesy flower field in the Alps let's move on to the next step here we composite the character onto the generated background and I know this stuff here looks complicated but most of it is happening automatically we only need to focus on this part up here first choose a model for removing the background for stylized characters anime illustrations works really well but general purpose is also really good next the image is added onto the background but it looks pretty ugly we have these ugly seams here so now we have three ways to fix this if you want to keep the character and background exactly the same and only fix these seams connect this second mask into the latent noise mask this one is automatically created around the edges of your character so only these parts get Den noised in the next sampler now this fixed it but we still have the issue of different focal planes and also the light is not really matching so to fix it we can connect the first mask to the latent noise mask this will denoise the full background depending on how high we set the de noising in our sampler this will generate a new background that better matches our character we can also come back to the background group and activate this blur note to help with the different focal planes and to create a more cinematic look finally you could use the crop note to move around your background and I did this in this wide shot for example so the character is not like a giant finally we could also completely deactivate this latent not noise Noe this will completely denoise the image changing not only the background to match the character but also the other way around notice how denoising the background again helps with integrating this character it even generated new Shadows if you now want to change the pose it's as simple as creating a new one in the open posst editor changing out the images in the control net and clicking Cube prompt but now let's finally give Huns some cheese I just add holding pieces of cheese into the positive prompt and reduce the weight of the control Nets a little bit to give Hans a little bit more freedom of how he exactly wants to hold these pieces of cheese and there we have him presenting the cheese in the pose that we created but we can also let stable diffusion create some poses for us instead of creating them manually for that I just deactivate the control Nets and generate an image now we don't have control but the post generally look pretty good you can also activate Auto que and set the seat in your Samplers to random and this way I can automatically generate hundreds of images of our character in different poses and locations I hope you'll play around with this workflow as there are endless possibilities to improve it and make it your own if you want access to exclusive example files and additional resources for this workflow and also access to our Discord Community consider supporting me on patreon making these videos takes a lot of time so thank you to my l l patreon supporters for making these videos Possible Oh and if you're in the big cheese industry and want to book hun schle to present your newest Creations feel free to reach out to me

Channel: Mickmumpitz

Views: 47,698

Keywords:

Id: 849xBkgpF3E

Length: 11min 8sec (668 seconds)

Published: Mon Apr 29 2024