Ultimate Guide to Seamless AI Animations (Even on Low-End PCs!)

hello there my name is prompt Muse and today I'm  going to be showing you how to install comfy UI   not only that I'm going to be going through  and making sure you understand what the heck   you are doing and what is going on comfor UI is  just another user interface to be able to access   and control stable diffusion much like automatic  1111 with com UI being a node based layout this   allows for complete control of your workspace  which means you will only be using what you need   to use for your AI generation which in turns  means that you're going to be using less GPU   which hopefully would make your image Generations  a lot quicker to produce another neat thing about   comfor UI you can literally drag and drop an  image of a workflow into your comfor UI and it   will load up the actual node base workspace you  also can do that with image Generations you can   simply take an image that was created in comy  UI and drag that into your comy UI system and   it loads all the nodes that were using used  to create that image that is chef's kiss so   this makes it far more easier to share workflows  and as the community grows you will see a lot of   workflows on Discord and Reddit links in the  description there as well which will get you   up and started and I'll show you some resources  where to grab them from as we go along in this   tutorial in this Comfort UI tutorial you're going  to be learning how to set up a basic workflow and   then edit and customize that workflow add lwas to  that workflow and then do text to video animations   and then finally video to video animations the  goal of this video is to get you confident and   comfortable with comfy UI if you like this video  I would really appreciate it if you subscribe like   the notification Bell and hit the like button  so first thing we're going to go through system   requirements to make sure you have a PC capable  of running comfy UI now if you don't do not worry   I'm going to go through what I actually use  because I don't have a PC that is capable of   running compy UI so first off you can run this on  AMD cards as well as Max see description below if   you have those preferably you would have 12 GB  of vram because remember we're not just doing   static images here we are processing videos also  you would want about 50 to 80 GB of storage free   because we are creating quite large outputs we are  scaling images as well as using large models and   all this takes a lot of space I don't have any  of those things on my PC what I use is a system   called Shadow Tech and Shadow Tech actually  allows you to log in to a computer that is far   away somewhere I don't know and use that system  and I think I'm using an Nvidia 1080 and I've got   over 1 tbte of storage I'm using a 12 GB vram  I think it comes to about $40 a month you can   actually upgrade if you want to but that's what  I'm using on this tutorial you just log in and you   go to your designated computer which is the same  same computer every time so you can install all   sorts of stuff you don't have to reinstall all  the time you're logging into your own personal   PC from there you can install whatever you want  there's no limits on time or amount you can add   additional storage at any point if you want this  is not sponsored at all and I can say whatever I   want about Shadow Tech but to be honest they are  a great company and I've had no problems you just   got to make sure you have good Wi-Fi and it's not  interrupted otherwise you will lose connection to   your computer so with all that said let's get into  in inst in comfy UI so there's two programs that   you need to have to run comfy UI and the first one  is git and this just allows you to pull extensions   down from GitHub if you go to the git Link in the  description below and just install the windows   version for that you probably already have that  if you've been running automatic 1111 the second   is ffmpeg this will essentially encode videos and  allow you to make videos from your AI Generations   so if you're just doing images you might not need  that that I have made a dedicated video on how to   install FFM pig in the link below as well which  is very quick and easy to do so now finally we are   going to get into installing the comfy UI build so  if you head down into the description below and go   to the GitHub page which is the comfy Anonymous  comfy UI page and when you're on that page so you   scroll down until we see the heading installing so  under the windows heading you can see this button   called direct link to download we are downloading  the stand loone portable version which in my mind   is the easiest installation for comfor UI so  click on the hyperlink that says direct link   to download which is here give that a click and  that will download zip file so once your file has   downloaded go over to your download folder right  click on the file and you might notice I'm using   winr in all the documentation it says to use some  zip but it works with wi so I'm going to use that   and then click on extract files now I'm going to  look down this list here and select my D drive   usually I would install programs like this to  my C drive but in this case I actually have it   already installed on my C drive so I'm going to  now install it onto my D drive and just make sure   your destination path is your C drive or whatever  you're downloading it to and then click okay now   it's going to be extracting all the comy UI files  onto my D drive so let's head over to the D drive   over here and check it it out so here we go this  is the file structure for comy UI so if you go in   you can see it's pretty similar to automatic 1111  we need to download a checkpoint model and I've   gone over to civit AI the cardos anime checkpoint  comes recommended by cading the author of animate   diff evolve however you can download any model  you like just make sure it's stable diffusion   1.5 base model so if we go in and download this  checkpoint once the checkpoint it's downloaded   just cut and go to your drive where your comy UI  is go to comy UI models and then you got a folder   called checkpoints and just put your checkpoints  in there if you come back down to the vae and   we're going to be using for version 2 and this  is a good allrounder this is being downloaded   from the stability ai's hugging face website so  go to the download button and right click save   link as and then I'm just going to navigate to my  D drive where my comfy UI is is click on comy UI   models and then down here you'll find vae and  just save it in there so we're now going to go   ahead and install comy UI manager it's a really  simple user interface within comy UI that allows   you to install extensions and nodes similar to  the extension manager in automatic 1111 this just   means that you don't have to go to random GitHub  pages to download everything everything is indexed   really nicely in this manager so if we come to  the civit AI page we can can download The Comfy UI   manager and again the link is in the description  below and we just click on this download button   and download it so once it's downloaded head over  to your downloads folder and you've got your comfy   UI manager v2222 right click on that and again I'm  still using winwell whatever you got windzip or   whatever you're going to be using to extract and  I'm going to click on extract files I'm going to   go to this PC and choose my D drive cuz that's  where I've saved my Comey UI Windows portable   folder to and I'm just going to select the entire  folder and click okay now that's extracted to my   D drive so I'm going to head over to my D drive  and we extracted to and that has extracted this   install manager for portable version batch file  here so just double left click it and it will run   the script to install that finally my friends we  are ready to dive into comy UI so I'm just going   to go into the comy UI folder these two files here  are what you're going to double double left click   to run comy UI so whenever you start it up this  is where you're going to come to and click if you   have an Nvidia GPU click the Run Nvidia GPU so  we're going to double click the Run Nvidia GPU   and you can see here we get this little startup  screen if you have any errors it usually shows you   in this script here so it's important to just  keep an eye on the script as you can see here   this is running on your browser and down here on  the right hand corner is our comy UI installation   manager this is going to make it really easy to  update comy UI and install add-ons Etc without   having to do get pull requests and things like  that these are our nodes now please please you   might feel a bit intimidated at first but it is  super simple and I go through every single node   so you know exactly what they are doing so this  is a bit like a circuit board so you can see each   processing working at once was automatic 1111 was  like check this box tick this do this and then run   and you can't see how it's working and there are  all additional settings all compiled into one UI   whereas this is wi the wig so the first node we're  going to look at is the first node in the workflow   which is load checkpoints this is going to load  your checkpoints in and this is pulling from   your checkpoint folder in the folder structure and  I'm using cardus anime which we installed earlier   and then that runs into your positive prompt so  your positive prompt is in here so this is where   you can toite your prompt and then we've got the  negative prompt which is this one down here okay   and this then runs into the K sampler if you're  used to using automatic 1111 or any other image   generation uis you'll be familiar with terms such  as C setting steps CFG sulet names scheduled to   noise if not I do have a glossery on my website  down below where you can learn what each of these   things are so you can have more control over your  image generation so this is the latent image node   and this is just your width and height and batch  size so how many images are you creating and all   this then feeds into the vae dcode which we spoke  about earlier okay so let's run this workflow and   see what happens I'm actually just going to change  the checkpoint over here just by clicking it and   change it from the dropdown menu and then I'm  going to click on Q prompt which will run our   image generation so you can see the green outline  there shows you whereabouts it is in the processor   so it's going to spend the most time in the k  samp especially when you're making videos so   if we come over where it says save image you can  see it's created us the beautiful scenery nature   glass bottle landscape as requested in our prompt  so this image here is saved automatically in your   output folder so if we just go there and have  a look so we go to com UI and then in output   and you'll see all your output images in here  now a neat trick is that all these images were   created in comy UI and you can then take any of  these images like so and just drag and drop that   into comy UI and it loads up the node layout or  workflow so you can see here now it's loaded up   that images prompts and all the SE settings and  everything I set for that the width and the height   so it essentially embeds all the information in  the image which is really handy so if I click on   Q prompt now that will generate that image that we  just had in our output folder again like so so you   can go in and tweak that so this is as simple  as the node layout gets I'm gonna go ahead and   install animate diff we're also going to install  control Nets as well as motion models and we're   going to do this all through the comfy UI manager  and it makes it so much easier so we don't have to   go to various GitHub Pages it's all centralized  now within comfy UI so I want you to click on   manager here and then click on install custom  nodes the first thing we're going to install is   animate diff evolve by cinka Dink so if you go  to the top right of this dialogue where it says   search and type in cinka dink in the description  you can just copy his name from there and paste it   in and then search for cinka and then you see the  offer cining and we want these three items here we   want comy UI Advanced control net we want animate  diff evolve and comy UI video helper sweep we're   just going to check each one of these and we're  going to click on ins and that will install all   three at once okay once that is installed you'll  get a little message down here in red saying to   restart we've got a couple of models we want to  install before we restart now you'll notice in red   here it talks about motion models which we'll get  to in a minute but while we're here we just want   to type into the search box Biz noes and that's  another node that we want to install in order to   run batch scheduling so we're just going to click  on install Okay once that's done now we're going   to go back and install those motion models that  kinkad dink suggested so we're going to go to   close and go to install models and in the search  box we're going to type in mm and that's just   stands for motion models so we want the motion  model version 14 15 and version 15 uncore 2 these   high and mid ones are good ones as well so I would  download all of these so these might take a bit of   time because they are quite big files so give  it a second to download okay okay so once that   has finished you'll get the thread message at  the bottom so we're still in the model section   of the comy UI manager and what we're going to  now install is our control net so you don't need   to go to the GitHub page anymore you can install  everything directly from here so you're going to   just type in control net control net that we want  is stable diffusion 1.5 so we're going to have to   come down further down the list here so let's have  a look we want line artarts I'm going to click   click on install we also want to install open pose  you can install as many control Nets as you have   spaceo so just check that You' got enough storage  once it's done and I've got the red message at   the bottom I'm going to close everything down so  I'm going to close the manager and close my UI   and then I'm also going to close this batch file  down that's very important because you can't have   two instances of this running so I'm going to jump  back into my comfy UI and just run a comfy UI once   again you can see here in this window it's now  installing everything that we requested so it's   just get pulling all those nodes and extensions  down and installing it onto our computer so on   another tab on my browser I've opened up cinka  dink's GitHub page this is the author of animate   evolve and he's actually got some really nice  pre-made templates to get you started with animate   evolve and if you come down to where it says text  to image here we go got a workflow here which will   produce this animation at the bottom so if I just  take this off the browser here and move it over I   can literally drag and drop that into my comy  UI and it loads up that node layout and that's   a pretty cool feature of comy UI you can take an  image of a workflow and literally drag it and drop   if you dragged a workflow in and you've got this  message and missing notes which are showing as   bright red boxes do not panic it's really easy  just go to your manager and click on install   missing nodes and all the missing nodes will be  there just check once and install them remember   once you've installed them you need to close down  your comy UI browser and your batch file and then   restart again and those will install so that's  a super easy fix for that problem if you get the   red boxes okay so we just run this by clicking Q  prompt and we see that we should get an identical   image or animation to what kinkad dink had on his  website as you can see we've got exactly the same   animation that that cink had on his website  because we didn't change any of the settings   but I'm going to go through the new nodes that  we now have in our layout so you understand what   they do so we went through load checkpoint and  then we've got our clip setting which you can   change depending on what model you're using and  what kind of style of Animation you're trying   to create then we've got our positive Point here  and then our negative prompt and as before it goes   into the K sampler and then we have our width and  height of our AI generation which 512 x 512 will   give you the best looking result you obviously  can change that to 512 by 768 or whatever you   want to do and upscale it the batch size is the  amount of frames you are using so this is 16 and   again that gives the best result as it is at the  moment however you can increase that amount uh   but not in this workflow I'm going to show you  a third workflow where we will be increasing the   amount of frames we've got so up here we've got  the animate diff loader so this is loading in   that motion model so that's what's creating that  movement that subtle movement in the animation and   just helping with the overall fluidity of that  animation but you should kind of now be getting   the gist of the node layout and hopefully feeling  a little bit more comfortable with everything here   we're now going to start making some changes and  then introduce Aur this is the same workflow we   had a second ago but I've just changed a couple  of things to make it more bespoke so I've just   changed the checkpoint I've added tune U which you  can download from Civ Ai and then I've changed the   positive prompt as well as well as the negative  prompt I've upped the steps to 30 to hopefully   increase the quality and the samp is ddpm and  schul karus and if we go up here I've now switched   the version model to 15 version 2 and my width  and height I have increased as well we need to add   another node so we can increase the length of our  animations so to add add nodes all you simply do   is double left click and we're going to be using  something called the animate diff uniform context   options just click on that we're just going to pop  this guy over here or over there go on uh pop him   over here and then we're going to take the content  options and connect him to the content options in   the animate diff loader and now what that means  is when we increase that batch size we won't have   any ER whatsoever so let's run this prompt and  see what happens and this is what has happened   we've got a pretty picture now we now want to add  a Laura to this workflow and then we're going to   move on to video to video which is my favorite to  be honest so to add a Laura you just double left   in any empty space and in the search bar just type  Laura and we're going to use the Laur loader here   so I've already pre-loaded in some Laura files you  can just go to Civ Ai and grab any Laura you like   and then save it in the models Laura and then save  it in that folder there I'm going to go with more   details which should add more details to our final  output and the model on the left hand side I'm   going to click that and connect that to the model  in the load checkpoint the clip I'm going to then   connect that to the clip in the load checkpoint  just come over here so as you can see and then the   model on the right hand side I'm going to connect  that to the anime diff loader there and then the   clip I'm going to load that into the clip setting  here so you can see the load checkpoint runs into   the load Laura and then that runs into the rest  of the node layout the strength model is set at   one which is currently a bit high now this could  create artifacts in our output so I'm going to   just reduce that to 0.5 it's completely up to you  and it probably needs to be played around with a   couple of times and creating generations to test  this out so I'm ready to press this prompt and   see how much that changes the overall output and  hopefully adds more detail to it and that is our   final image with the Laura added on now the more  details Laura I believe gives way more detail to   the animation and it looks really nice something  I just want to say which I didn't say earlier you   don't actually need to add the Laura triggers  to your prompt anymore now I hopefully got you   comfortable with Comfy your are and you're feeling  a little bit more confident but if you're not and   you're having error after error please go down to  the description of this comments page and I have   pinned an FAQ where you can write your problems  and hopefully I can resolve them hopefully this   is a video to video workflow using one control  net and one Laura the original version of this   workflow came from inner Reflections you can go  over to civit Ai and downlo download any of his   workflows here so we're going to be using inner  Reflections guide as a template so if you come   to number one attachment which is basic video to  video with one control net this is always good   to start off with if you download that once it's  downloaded come to your comfort UI go to load and   just load in that Json file like so we're going to  go through each one of these notes and I'm going   to tell you what they're going to do and we're  going to adjust a couple of things here so first   we're going to go over to load images you'd use  images input if you were using frames as an input   but I'm actually using an MP4 video so I need to  change this to a video node so you want to delete   that and in any empty space just double left click  and we're going to add a video load node so just   type in video this is the node you're going to use  to load up your input video if you don't have that   node when you search remember you can download  it from the manager if you're missing anything   I'm just going to attach the image to the image so  we're flowing back into our workflow so there's a   couple of things you might want to change here  before we move on so we've got the frame load   cap so that limits how many frames it's going to  render of your animation so let's say you've got   a 700 framed animation you can cap it to 500 Fram  so it's not rendering the entire thing so we come   down to skip first frames so I've set mine to 60  so it's going to skip the first 60 frames and then   we come down to select every M frame so this is  set as one and means it's going to render every   single frame in the animation or input video  now if you to set it at 20 or 50 or something   like that that's great to preview an animation  before you do the final render just go to choose   file and upload the video input you want to use  so next we go to the upscale image node now you   don't have to upscale this image whatsoever and  for your first test render I would leave it as   the resolution that the input file is however I'm  going to show you how to increase any anyway you   just come to this width node here and increase  the width in proportion to what your input video   was anyway and then same with the height if you  don't want to upscale which I'm not going to do   here just set your width and height value to the  same as your input video and it will remain the   same if you go to load checkpoint here you can  select the checkpoint that you want to use I'm   going to use U here and then you got the vae  which is automatically loaded in and then we   come over to the middle animate diff nodes so  this is where your motion model is going to be   loaded in I've selected the motion model stable  diffusion version 15 version 2 you can select   whatever motion model you prefer I've left a list  Below in the description so you can find one that   suits your project okay so we come over to the  case sampler is it's on 25 steps I'm going to   leave that the same CFG is seven which is a  good medium remember the lower your CFG the   more you're allowing AI to be creative and take  control the higher the CFG the more it's going   to be listening to your prompt and what you want  it to do so the sample I'm going to stick to the   same that he has here and same with schedule and  Deno I'm going to keep the same as well we're not   really going to do much changes to that the seed  as well you can change the seed here if you wish   and then we've got the prompt so we've got the  positive prompt here and the negative prompt so   we're going to just adjust this so I've just put  a prompt in there it's very very basic because   we're just testing this out and we're going to  come up here and then last but not least we've   got our control net here so this is load in in  the line uh art here and then this will produce   the images here from the control net which will  be if I'm using line art lines to show you what   it is following okay so you got your strength  as well we don't need to change any of this just   leave it as it is for your first test run and we  are ready to run our workflow and that's it once   you get down to the bones of it it's actually  very simple so before we run this actually I   want to change this it says animate comined and  it says depreciated that means it's an old node   and we need to update it so I'm just going to  delete that node there and I'm going to double   left click on an empty space there to load up the  node search I'm going to type in VHS and I'm going   to select the video combin so this is the most up  to a date well according to the time and date of   this video that we're going to use so we're just  going to attach the image to the images there and   this is where our output will be so I'm going to  switch over to MP4 as well so hopefully it will be   a better quality and then all there is to do now  is hit Q prompt so it's been about 10 minutes and   this is what our final output looks like we've  definitely got the ears that we prompted in by   changing a few parameters the seed the checkpoint  the prompt you can get a whole different animation   isn't this cool another fantastic template for you  to try out is the prompt scheduling which allows   you to change your prompt over time and create  things like this and with all that said thank   you so much a written version of this tutorial  can be found at always on the prompt Muse website   and on my socials I am at prompt Muse on pretty  much everything thank you and that will do it

Channel: Prompt Muse

Views: 52,275

Keywords: ai animation, animate diff, comfyui, stable diffusion

Id: 1fWZNECdN9Y

Length: 26min 38sec (1598 seconds)

Published: Thu Oct 26 2023