2D Character Animation In ComfyUI

in the last video I explored how to take a 3D character from blender and turn it into a realistic one using stable diffusion through confy UI but a lot of you asked me if it was possible to use the same method to make an animated look so I went to work to solve the issues I was previously having trying to get this result so let's jump in a blender like the last time I'll be using the human generator but as I've said before you aren't limited to just using this you can use any model you like as per usual we'll delete the cube we can select a human from the drop down window we'll go with a female for this example once we've added the character let's give her some clothes so YouTube doesn't get the wrong idea we have a range of options here but I like to keep it a little bit simple in case I need to do any modifications later so jeans and t-shirt will do for this let's give her a hairstyle something that wouldn't excessively move since I don't plan on doing any hair simulations this time around I need to make some modifications to the body since I'm making her more of an anime like character she's going to need bigger eyes if we go to body special this stylized slider will make the character's eyes grow it also makes the character a little bit skinny so you may need to keep an eye on that while it looks like it maxes out at one we can actually enter any number we want manually within reason of course the effect will break quite fast if you go nuts with it then I'll go to face eyes and here I'll be able to increase the height of the eyes so they don't look too elongated to the sides feel free to play with all these settings and get the look that you're after this next step only goes for female characters though if you're making a male character you may want to paint eyeliner on him to get some of those eyelid outlines in the animated version go to skin makeup and then down the bottom you'll find the eyeliner it'll be set to Black already let's bump this up to 10 and then adjust it down so it doesn't look too overpowering we're going to change our camera lens by default blender uses a 50 mm lens but I found that when most people draw cartoons and Comics the shape of the face resembles more of a 100 mm length this keeps the face from looking too distorted at most angles but this is a personal preference you don't need to do this if you're after that more exaggerated perspective look you can also change the perspective camera in blender to 100 Mil so everything looks the same both in the camera and out of it I'm going to add an empty to the base of my character select the amateure press contrl and select the empty and then press contrl P to parent the amature to the empty this way we can use the empty to move the character around the scene instead of having to move the camera this me doesn't always work for every instance but for a 2d Caron I'm trying to think in the way the old cameras would in traditional animation it would be a static camera while the animator would move the character around the frame so that way if I give my character a walk cycle I can move her through the scene using the empty in saying that the other reason I don't move the camera is I want to make sure that the majority of the character is in the frame at all times when we get too close to the edge of the frame we risk something that I've been Haring on about for a while now when something passes over another object or it gets too close to the edge the AI tends to distort or change the image randomly we don't want that we want our character to stay the same let's add a plane rotate it on the x-axis by 90° and send it right back scale it up to cover the whole background in the camera view add an emission material and set it to 100% green this will act as our green screen so we can easily key our character out and post you'll notice that the green is a little bit pale head over to the render properties all the way to the bottom color management and set the view transform to standard this will give you a better idea of what your render will look like click on the shading tab select the eyes and then in materials pick the inner eye material we're going to plug the eye color into the emission of the principal Chader this way the lighting doesn't affect the eyes if we were going to see into the mouth I would do the exact same thing for those materials inside the mouth this way wouldn't end up with shadowed eyes and mouth cavities cartoons usually have the mouth be an actual color not just a black hole in the face I'm going to create another empty to drive the lights and I'll raise it up to the same height as the head in order to use an industry standard lighting setup we'll go with the three-point lights you'll need to go to the add-ons and activate this if it's the first time you've ever used it as you can see each light is designated a name you got the back light key light and fill light key light is our principal light fill light adds light to the shadowed side and the back light will give us our highlight Rim because we had the empty selected the lights are pointing towards it make sure to parent all three to that empty switch the render engine to Cycles don't stress it won't take hours to render this set the device to GPU and experimental feature set this is going to give us a feature that isn't available in Eevee but it's crucial for that Caron look that we're after beam shape is the feature that I'm referring to we need to tighten the lights Beam by more than half I'll set all the lights to 75° spread switch to rendered View and select the key light boost the light's power level to 75 select the fill light and set this one to 50 and then we'll select the backlight and send it to 150 you can see that I have a flat even look to the lighting in most cartoons you'll see that the lighting is flat with minimal Shadows let's say we wanted a nighttime scene instead of darkening the scenes and lowering the power of the lights we can change the color of the lights to something a little bit cooler and have the back light offset with a different color this technique gives the AI more information to work with and still has that look that we're going for you can also experiment with the lights by turning one or more of them off in this instance I kept only the key light and positioned it off to one side to give it a more Moody look I'll reset all my lights and render one frame you can see that it took a minute and 20 seconds this won't do we'll waste too much time rendering in blender head back to the render properties and change the noise thresholder 0.9 and the samples between 1024 and 2048 now let's render the same frame I'll leave this one going in real time the quality of the render is more than enough for what we need and it's down to almost 9 seconds you can drop that time even further if you go all the way down to 1024 the good thing about the human generator is that we can have it set up our riger fire rig automatically this is going to take a moment once this finishes we'll head to the object data properties under the bone Collections and make the new rig visible I'll create a small animation Now using the auto key frame function jumping back and forth in the timeline animation isn't the easiest thing to get a hand off and that brings me to the sponsor of this video when learning your skills it's easy to get lost in the sea of online tutorials I would know learning blender took me a long time I know what it feels like to not know where to start but most times you'll watch a 1hour tutorial just to walk away with one new tip with skill shares learning Pathways you have the option to go through carefully curated handpicked lessons that build upon each other skillshare offers a variety of different classes including but not limited to photography graphic design business marketing and music for instance the build 3D models and animation with blender classes start off by showing you the basics so you understand the user interface which let's be honest if it's the first time you're using blender it can be daunting no offense to the blender developers I know they've come a long way trying to make it more user friendly the awesome thing about this is that you can follow along with these lessons check out my render I may have gotten a little carried away and by the end of it you can submit your results and get feedback these lessons cover a range of experience levels from beginner all the way to advance it's never been a better time to invest in yourself by taking your skills to the next level the best thing is that the first 500 people to use the link in this video's description will get a one month free trial to skill share check it out and let me know how you go now we can render out our animation and if you guys want more in-depth tutorials on how I sculpted this t-shirt and made her muscles grow mid shot let me know in the comments before before I forget join my newsletter to get the companion PDF with the comy UI workflow that I'll be using the link will be in the description load the new workflow in the companion PDF if there's any red nodes use the comy UI manager to install them once you're ready add the directory of the image sequence you made make sure that the resolutions are correct in my case I'll be working with 1024x 1024 images do not do this if you don't have a powerful GPU I've got a 3090 with a lot of ram in the computer if you don't don't go there go 512 it works out just fine select your checkpoint I'll be using the ethernal dark golden Max with the moist mix vae model since I'm using a model that doesn't have a vae baked into it I set my vae switch to two I have my LCM Laura loaded I'll be updating the resolution to 2024 if you're using 512 go with 512 I'm going to be using the depth anime diff and open pose as my control net models you can change these if you want but I've been getting the best results with these ones you will see the previews come up in these windows so so you know what's going on the case sampler is set to LCM but you can change that to whatever you wish add your positive and negative prompts give the save imag name if you use a forward slash it will create a folder to store the resulting images do the same with the video combiner then click on the Q promp button remember depending on the length of the animation and the resolution it will influence how long this generation will take it's not a quick process this may take a while the first thing it'll do is it'll load all your images and run it through the def control net as you can see here it's perfectly separated her from the background the second one will be open pose you can check the command prompt to see how it's going as you can see here it's found one person in each frame this may take a while once the generation is complete we have both the individual frames and also the combined video in order to create the final video I ran the character through the conf UI workflow twice once with white skin and once with green skin and then I combined it in post in the next video I'll show you how to integrate your character into a scene using an AI generated background like this one in a 3D space if you have any questions about the pro let me know in the comments remember to subscribe to be notified when the next video comes out if you enjoyed this video check out this next one I pick for you

Channel: Sebastian Torres

Views: 9,783

Keywords: stable diffusion, ai animation, stable diffusion tutorial, stable diffusion img2img, stable diffusion controlnet, stable diffusion animation, ai animation generator, how to animate with ai, ai video editing, blender animation, stable video diffusion, stable diffusion anime to realistic, stable diffusion realistic vision, stable diffusion action scenes, ai filmmaking, ai film industry, ai filmmaker, comfyui tutorial, animatediff, blender tutorial

Id: MzfmoMcv1Bk

Length: 10min 15sec (615 seconds)

Published: Thu Jan 11 2024