Make your own ANIME with this new mind-blowing AI TOOL! (ComfyUI Tutorial + FREE Workflows!)

you want to create your own anime but don't know how well you've come to the right place I'm going to develop three workflows around a brand new AI tool that will help you achieve your childhood dream of creating your own animated TV show these workflows not only work for anime but for pretty much every other style as well and they are another big step towards completely controllable AI film making I'll start simple and show you how to use this tool for free together with your favorite image generator like mid Journey For example then I'm going to show you how to install it LO and combine it with my free consistent and posable character workflow that will allow you to control your characters like puppets add them on top of consistent backgrounds and even change their emotions with simple prompts and finally I want to show you a method that allows you to act out the scenes yourself and then turn them into anime this is so much fun and I'm going to use it to create an epic action sequence at the end of this video so if you like AI deep Dives like this make sure to subscribe and let's create the future of AI film making together all this is made possible by a new tool called toun crafter you can give tun crafter two or more frames and it will interpolate them filling in the gaps by generating the missing images tools that use AI to interpolate between frames are nothing new but this one is not only better than any other tool I've seen so far but it's also specifically trained on anime and 2D animation so it understands the aesthetic of this type of Animation look at this work cycle for for example toon crafter was able to generate that based on only two input frames it understands how wind moves through hair how to animate magic effects and you can also give it uncolored sketches of an animation and a colored reference image and it will automatically color the animation for you and that's why today with this powerful tool we will create something that looks like real anime and not like these AI style transfers that we created in one of my last videos let's now create some key frames for our animation and I'm going to start simple with mid Journey I'll start by generating a portrait of a character that I can use as a character reference image so that our protagonist looks consistent from shot to shot let's put our character in a futuristic cityscape with a vintage 2D animated look I'll add the CF parameter and copy in the link to the image of our character's face I like the look of this one let's upscale it now let's open the very region tool and change the prompt a little bit I want her to move by but the background should stay the same and this worked really well we now have a close-up shot and one that's a bit further away let's now go to the hugging face demo of toon crafter and upload our two images you can add a prompt and these ones are usually very simple as you can see here in the official examples they are usually just an anime scene so I just use that yep that's cool I like this one using mid Journey as your key frame generator can be a bit tedious you only have very limited control now you could of course use an additional image editor like Photoshop here for example I isolated the character and the falling leaves moved her closer to the camera scared the background a little bit and created my key frames this way I also was able to change her emotions with generator fill and you can see how well that worked if you're interested in this workflow make sure to check out my video on animating with generator fill but I know a lot of you don't have Photoshop or even mid Journey as these tools can get quite expensive so let's look at an option that is not only better but also runs on your own computer and is completely free it works inside of com UI a noe-based interface for stable diffusion and I created a free guide on installing it that tells you where to download and put all the models for this to work you can also watch my previous video on this workflow where I explain it step by step once you have everything set up you can just drag and drop the model sheet workflow into the comi interface and import the pose reference here so I want my character to look kind of like me so let's use this prompt here and that's all we need to do I click Q prompt and this work will then automatically generate views from different angles create different emotions and save out all the individual images and now we're ready to pose this character for this I go to openpose a.com and this is just a website that lets you play around with an open pose skeleton you can change the proportions of your character if you need to and even change the guestures of the hand when you're happy with your Poe don't forget to set your scene resolution and play around with your focal length and Camera position click this little play button here and click on the first and second image that appears to download the body pose and the hands as a depth map and by the way you can find all the poses that I Ed to test this character with on my patreon now let's just drag and drop my controllable character workflow into comi and I created a simplified version especially for this video so make sure to use the anime one I copy over the original prompt I used to generate this character from my character sheet workflow but I delete the multiple views from the positive prompt and copy it over to the negative one otherwise you would get some really broken results like this one next I put the closeup of the character that we generated with the model sheet workflow here and the full body pose here drag and drop the pose we just downloaded from open pose AI here and the depth map of the hands here below this we can create a background and let's create a code Sunny living room now you only need to click Q prompt and it will pose the character and merge it with the generative background if you want to change the emotion you can activate this part in the middle here using contrl B and put in your prompt for the emotion here I don't look too happy here so let's change that once you generated an image that you like you basically just need to switch out the pose and the depth map for the hands and click Q prompt again and this is exactly how I created all the key frames for my intro sequence if you want to create a moving camera for your scenes you could do this by changing the scale and position of your background for each frame down here but it's a bit tedious to do this in comu ey and that's why I also added file output noes that save the background and the character pose separately with the character having a transparent background now you can just use your favorite image or video editor and put them together and create your camera move there now let's make the character move I just drag and drop the toon crafter workflow into confi and this is based on one of the example workflows by kji who brought to crafter to comu you can see we now have two inputs but we could easily change this if we have more key frames just increase the input count and click update inputs and connect the additional images so I just drag and drop these key frames here and I add a simple prompt like anime scene again and you can leave the negative one empty just click Q prompt and when you first run it it might take a while because it's downloading all the models but once that's done it should run pretty fast not every result will be great though you will have a lot of weird broken shots so what I like to do is go to the extra options in confi here and click Auto queue so then you can just wait a few minutes and check out all the results and you can always mix and match them in the editing process since it's always generating from pose to pose and that way you can also add one of the original key frames in there if you want the character to hold a pose longer for example but in case of our intro sequence here it actually worked like it I think it was the third try and it looks pretty good right but when you go through it frame by frame you see that there are some in between frames that look really really broken and that's why I created another workflow that will not only clean up these Imes for you but also upscale them with this one you basically just have to import your video created with toun crafter mix and match your prompts that you used for generating the character and the background to create a prompt that describes the whole scene and then you can just click Q prompt and this will not only upscale the video to the resolution that you specify here in the beginning but it will also match your character with the background creating a much better shot for shots like this one of characters faces you can also activate this face fix group here it will detect the characters faces and clean them up even more and you can find it in my Advanced anime fix workflow I also ran the mid Journey shot that we created earlier through it and look at how much better it is while toon crafter is trained on animated can also work with any other style so here I tested this weird 3D clamation character and I also added another interpolation Noe here at the end for some smoother movement and this whole workflow even works well for more realistic types of shots like this one probably the most fun way to use this workflow is to shoot your own reference images and then transform them into an anime style and use them as your key frames so first I set up my phone camera and shoot a video of me performing all the movements of all my characters standing roughly where I want to have them in my shot I then save out the frames of the video with the main poses for my key frames and now we can use my image to anime workflow to transform myself into my anime version for this step you can also use any other AI style transfer tool you could even use a snapchat filter for this step but since we're already using comi why not stay here so I just import my reference image here into my workflow add the prompt for myself as an anime character and click Q prompt it will then transform me cut me out of the background and save the image as a PNG with a transparent background I repeated that process for all the images I needed for the sequence and I also tried if putting all the poses in one sheet and then generating them in one go would help with consistency which it did but I'm not sure if it's actually worth the extra amount of work finally I used Photoshop to put everything together and create my key frames and to help tun crafter understand what I wanted I quickly scribbled these action lines on top of these images I imported these images into the toon crafter workflow and it looks pretty cool there's one frame of the evil robot before it leaves the image before it leaves the frame and it's absolutely hilarious I love it there are still some broken frames in here but I really hope that my anime fix workflow and upscaling workflow will fix this but first let's create the other scenes I used the exact same method and it only took me a few minutes to generate these shots this one was a bit tougher though I wanted to create a rotating Matrix like bulle time camera but toon crafter had a really hard time interpolating these shots so I only used toon crafter for the lunging and the punch itself and for the movement around me I just put all the frames one after another which I think looks really good at 10 frames a second finally I ran this whole sequence through the anime upscaling setup and here is the final scene I really hope you're going to give this workflow a try I know it really seems like a lot but once you have everything figured out you will be generating anime scenes in no time and to further help you with this I also created an exclusive step-by-step guide on my patreon you'll also get access to our Discord Community where I try to help out everyone wherever I can so thank you very much for your support and making videos like this possible see you next time

Channel: Mickmumpitz

Views: 50,373

Keywords:

Id: mEn3CYU7s_A

Length: 11min 56sec (716 seconds)

Published: Wed Jun 19 2024