ComfyUI MagicAnimate + DensePose - Local Install

yo guys welcome do you want to see some cool stuff check out magic animate a new framework that can create realistic human animations from just a single photo and a dense pose video sounds awesome right this is the work of researchers from National University of Singapore and bedance yes the company behind Tik Tok they just made their code public a few days ago okay I'm going to walk you through the comfy UI workflow the local installation actually requires few extra steps but it's not too hard we will talk about it later just note that there is also a online graio version click demo and you can try here let's get back to the comfy UI so I have two groups top is image creation bottom is the magic animate you need to install the kuui magic animate custom node I did from comfy UI manager also comfy UI KJ nodes and use color match node to to correct the colors comparison example later so you can see I have already generated animation here not bad huh if you want to use your own video as the input you might run into some trouble it only takes the dense pose video however the official GitHub repo only has five demo videos to work with don't worry I'll show you how to install vid2 dense pose on your local machine and convert any video you want stay tuned till the end okay let's generate some pictures first activate the top group and deactivate the bottom the magic animate video size is 512x 512 sdxl turbo would be a perfect match simply add a face detailer for a good face for reference input of Animation click q and wait for the generation the face detailer did a good job here then deactivate the top and reactivate the bottom let's change a running video input here and drag the generated picture to the reference here run it loading magic animate models and running inference are relatively slow looks good but I would say the details are just okay at this resolution maybe can be improved by upscale if you don't use color rematch node the output color looks offset okay that's so much for the example then let's talk about the installation so first click manager and search magic you will see the magic animate node here just install it don't restart comfy UI yet go to the magic animate GitHub repo and download all the models here you can use git clone to download then copy the whole Magic animate folder to models folder in comfy UI the folder should look like this containing all these large model files let's go back to website again and download stable diffusion v1.5 and vae no need to download every model only bin files needed and you can use fp16 version and just make sure to rename it by removing that fp16 the stable diffusion me 1.5 and vae folders should contain these files then copy them to the previous magic animate folder in comfy UI now everything is ready here is the full list next for the vid two dense pose installation if you want to create your own dense pose video locally like these still use 512x 512 resolution then follow the steps here but need some modifications first first get clone the this repo then CD into the local vid to dense pose folder next create the python virtual environment for this repo in requirements.txt file make this change as suggested in the issues I installed before this suggestion and met with no module named torch issue even if I installed P torch which seems rooted from detectron 2 on Windows if you also met this error put this P project file in the root of detectron 2 folder and this to root of dense pose folder then activate the virtual environment and pip install the requirements finally get clone the detectron 2 to local vid 2 dense pose folder if everything goes through congratulations it takes me some time to figure out eventually in the virtual environment input command python app.py to run the gro interface just like this it's not easy but you did it time to have fun that's so much for today thanks for watching

Channel: JanRT

Views: 4,842

Keywords: stable diffusion, comfyui, magicanimate, magic animate, dense pose, local installation, work flow, vid2densepose, color match, animation, SDXL turbo, Detectron, Face detailer

Id: rm2UN2fYJ0s

Length: 5min 36sec (336 seconds)

Published: Sun Dec 10 2023