ComfyUI Setup & AnimateDiff-Evolved Workflow + ControlNet OpenPose and QRcode Monster

Hello friends hopefully I'm live waiting to [Music] see ah there we go how's everybody doing today uh we're here to play with comfy UI uh welcome if you're first time here welcome if you're not welcome anyway um yeah you've probably seen some cool comfyi or sorry rather animate diff animations um all over the Internet lately and uh we're going to learn how to make them uh you need a Windows computer with um a 2060 or above uh RTX 2060 or above AMD I don't think works and Max I don't think work but I might be wrong about that I'm just not sure um it's also possible to do this stuff in Google collabs I think but I won't be covering that today we're just going to go through the windows version and uh play with that because that's all I have access to really but um yeah you might have luck just installing comfy in uh oh hey uh Leb no uh it's not uh well there is a uh animate diff um plugin for automatic 1111 but I haven't got anything I liked out of it yet um The Comfy UI um the one that's in comfy UI the anime diff evolved the one we'll be talking about today seems to be better for for my money which is free because all this stuff is free uh but yeah I think um I think the comfy UI one is a superior right now uh maybe at some point there'll be a better plugin for automatic 11-11 as far as anime diff goes but for now um comfyi is the best I've seen so yeah and also comfyi is really neat uh it's cool beyond all this anime diff stuff you can do lots of crazy things with it um for just regular image generation as well I'll show you there's a thousand plugins for it it's crazy it's kind of like automatic in that way uh so let's jump right into it uh where are we Chrome so this is the uh comfy UI GitHub um links in the description below um should also be in the chat in a second all right yeah so you can hop in there um read about it and see what it does and uh if you want to install it all you have to do is go to releases on the right here and then you want to download link this download link with stable P torch 2.0 Cuda 118 in next forers uh if you want to be really uh really adventurous you can play with the nightly builds but uh they'll be pretty pretty unstable cuz they're working on new code and stuff before it hits the uh stable version all right so you click this download link and download the file and it's going to put the file on your computer it's about a gig and a half just let that download uh you should probably install this on an SSD drive um it's up to you but I would recommend putting it on like your C drive or whatever your fastest hard drive is and you'll probably need about 30 to 40 gigabyt of space to be working with this stuff so uh clear that off before you start and uh yeah good to go so we're going to unpack comfy oh I think it's still writing the file still writing the file uh you might need a szip uh unpacking tool you can get one just type Windows 7zip into Google and uh you you can download this version here um it's like uh ZIP or raar or whatever it's uh archive tool I don't know why they chose seven zip WinRAR will open them so if you have WinRAR installed that'll work too but you will need uh WinRAR or uh seven zip as far as I know Windows doesn't understand what a 7zip file is yet okay uh I don't know why this is is just sitting here like this that's very strange let's go to my downloads folder yeah unconfirmed oh did it finish there it goes so you're going to right click on it and when you have your app installed you'll uh extract it so you can extract it with the seven zip extract it with WinRAR extract with whatever you want um or it might just open up if you click on it and unpack it and then when it's done you will have a portable installed version of comfy UI so you can uh put that wherever you want to keep it so for me personally I have all my AI stuff in a in a folder on my C drive called Ai and then I just put comfy inside here so it's here's my comfy UI Windows portable this is the unpacked directory that I I unpacked from here so I've already done this step so I'm going to stop it you just hit cancel and finish it all off and when it's done unpacking you go into the folder and you want to do the uh run Nvidia GPU dobat so you want to run this file here and then when you run that file it will uh do all of its startup stuff and all the checks and all the things it needs to do install the software it needs and you will be Off to the Races you may need to install git and uh python uh to get it to work but you might not I can't remember um I already had it installed so that's it so it'd be python 3.1.6 I think um and whatever version of git you get from git SCM and once you do that run uh run GPU it'll load up comfy UI and you uh won't see this it'll actually be probably either clear or something uh super simple uh comfyi looks scary but it's not that bad um once we hop into it all we have to do is go to uh well here's one thing I recommend doing right away is going to comfyi manager typing that inop Boop install this this is like the extensions tab an automatic 1111 it makes your life a lot easier and gives you the ability to update and download uh extensions and and custom nodes without having to um do it manually so uh go there and download the they tell you here how to install it um you can either do this if you have uh if you're comfortable with the command line you can go CMD CD Ai and then uh go into your comfy folder CD comfy uh CD custom nodes and then in there you can run this command here get clone com UI manager or you can just download the uh I believe you can download the zip oh there we go there's a script here so you can download this file and put it in your comf UI Windows portable uh directory that's I believe that's what I did and then you just run that batch file the instructions are right there on the site but basically you run that file and it'll in install all the stuff you need to run the manager but the manager is great because it adds this little thing down here and then you can just uh update comfy um automatically and then uh you can also um anytime you load a workflow and it has nodes in it that you don't have you can click this install missing custom nodes button and you'll go and check for the custom packages and just install them for you so it uh saves you a lot of headaches if you just want to try somebody's workflow um and then if you click this install custom nodes it brings up a um like a extensions list essentially of all the things that uh are available in comfy UI so uh in comfy UI what we want to do today is run animate diff so we type anime diff into the top and the one we want is the anime diff C cin CAD cinka dink version uh this is the one I've been using and it has um updated motion lauras which we can use to drive the motion of the clips like in Runway and Pika and stuff like that so download this hit install wait for it to finish and then it'll tell you when it's done that it needs you to restart comfy UI for it to take take uh take effect so you just close this and go back to the window or The Terminal window where where it's running comf UI and just close it yes this video will be available after the stream so I'm just trying to go through the installation process and then uh you can always just go back and check it I believe you can also scroll back in the live video too if you want to just uh roll back and start from the beginning and and follow along you're more than welcome to do so um so uh do comfy UI uh yeah so this uh we can just uh close this uh we want to close this window and this will kill comfy UI closing this window see you can't connect to it anymore because comfy UI is running on your computer and this is just talking to it like this is just the web browser that's talking to it so uh what we want to do is run that again so we got Ai comfyi and then uh run Nvidia GPU and it'll go blah blah blah blah blah blah blah blah blah blah and type a whole bunch of stuff and probably download some packages that it needs throw a bunch of Errors I guess all right we're back uh you can close the old window if you want the old tab it's less confusing that way uh and then the next thing you want to do is go to uh that second link in the description below um it's uh animate diff evolved uh I'll just drop the link in the chat as well all right so this is kinkad dink's animate diff for comfyi I hope I'm saying that right kin kin kadink um okay there's a lot of text here but we can get through it pretty easily all you need to know is that you need to install the animate diff comfy nodes which you've already done that's been done with the manager what we need to do now is make sure the models are in the right place so that it can do its thing so uh we might need these ones as well I might have already had these nodes um but we'll check that when we get to the control net part of it look for animate diff make sure yep we did that uh how to use the download motion models you will need at least one different modules produce different results so you click the hugging face link here this first hugging face link and uh go to files and versions and you want the uh mmsd version 15 version 2 checkpoint so it's this one here um so you click this little download button here and uh it'll download it and we'll move all these files out of our downloads folder when we're done because there's a lot of stuff to download these are the motion luras going to need these if we want to do the motion so go ahead and grab all those they're only 80 Megs each so just go like dink dink dink dink dink dink dink dink like that so just download all the all the motion models and I'll show you where to put them once they're all done downloading they're really small it's just these ones that are big all right we got all that stuff and then uh where was it let's go back back all right we're back on the animate diff link um and it's telling us we need to place the models in comfy UI custom nodes animate the evolved models so you would just pull that folder up in Windows AI comfy wherever you put it you can go any way you like Windows Explorer or command line uh and you want to go into what it says a comfy custom no comfy UI custom nodes com fui animate diff evolved models and then you drop that um file from your downloads folder the um uh this one here mmsd version 15 version two that goes into this folder here models and then in the motion Laura folder that's where the rest of them go so you just drag those in here you see I've already got them in mind but that's that's where they go so that'll um get those oh uh I think one more thing I also have to mention for comfy UI is you will need um a base model to work from uh if you don't already have them for automatic then you'll need one for comfy um there's a way in comfy to tell it what folder you are pulling your models from uh so uh you can tell it I believe it's in the [Music] uh yes inside uh the comfyi folder there's a file called folder paths. py if you right click that and edit it or open with a a text editor like a notepad or whatever um it'll give you all these uh folders and you can um replace your uh is it not this one maybe I'm walking you down the wrong path here give me a sec how does it know where my models are is it in the settings perhaps I did something at some point and told it where my models were so my point is uh let's Google that how to tell comfyi where you're stable Fusion models are uh okay let's what a better question is um comfy UI custom model folder okay uh I don't remember how I did it but it's in there it's in a config file somewhere anyway if you're if you already got checkpoints in like a automatic 1111 just point it to that folder or you I think you can just do like a Windows link like an Ln uh link uh directory I think it's Ln yeah symbolic link yeah so you can make a symbolic link from the um you can make a symbolic link from the output or not the output but the models directory um your stable diffusion models looks like I've already done oh yeah I already did that that's how I did it so you can see the I've got shortcuts to them and then it goes right to my uh my checkpoints folder anyway so yeah um that's where the models go into checkpoints you want to get like uh 1.5 or uh I don't think this stuff works with sdxl yet but it might I'm not sure um but you just download stable diffusion 1.5 model from hugging face uh you want this one here that'll go in your checkpoints folder uh mod files and versions uh I believe it's this one the EMA only safe tensors yeah uh or any anything you want any model you want from um uh Civ go grab a checkpoint drop it in there and and have fun um I would stick to the 1.5 stuff for anim make diff but yeah your mileage may vary so we get those files in place um we got the motion lauras we put them in place um there are more uh there are more models you can get these ones the stabilized models fine tunes higher resolution fine tunes um they're all here you put them in your um models directory inside the anime diff folder so uh you can play with those but we're just going to stick to the mmsd version 15 version two one because uh I think that's the only one the motion works with right now yeah V2 based motion models so if it doesn't say V2 it won't work um compatible so that's all we need um once those files are in the right place you scroll down a bit and the really really really cool thing about comfy UI is that workflows can be stored in images so um this explains how to use it but we don't actually really need to understand how to use it right away what we can do is click this first example download this image save it to your desktop or whatever open it up and then um all you have to do is go to uh comfy UI and just drag the file into comfy UI and let go and it rebuilds the workflow for you or you can um uh yeah I mean it's it's that easy and then each thing is just just you know it is what it is you just have to look through it and read it and and and figure it out but they're just nodes and they all plug into each other and they do stuff so once you get comfortable with the stable diffusion and what you're doing in automatic 1111 you can take it over to comfy because like now you can just plug stuff in where it's not supposed to be plugged in or uh reroute the way it works like a modular synth like it's really really fun so yeah highly recommend uh playing with comfy UI but uh right now we have a full set up ready to go so let's uh add a animated um animated uh checkpoint that I have um uh pick a vae uh if you need a vae they are uh right here so many links uh you put that vae file in the comfy y comfy models the AE folder you drop it in there and uh that will help fix faces and hands and stuff if you're using sdxl you need the sdxl one but uh yeah I haven't tried this with sdxl yet so who knows um for this we're using a fixed seed for the whole run and that tries to keep everything in the same you know location so you will need to change this number every time you want to do a new run with the same prompt so um yeah you can just make it whatever you want um so let's try a prompt for anime diff um let's try uh geometric shape unfolding paper origami yeah let's try that so this is a 16f frame animation at 512x 512 it's using the mmsd version 50 version 2 checkpoint which you select up here uh it's got a negative a positive and negative prompt these are your sampler settings they work just the same as automatic 1111 and uh this is your output um right away I'm noticing it's at 8 frames per second which is really really slow and gross so I'm going to switch it to 12 frames per second I'm also going to drag this noodle out and I'm going to let go out here and that's going to give me a search thing that I can hit search and I want to add another animate diff combin so let's drag this out click here or let go search animate we want this animate diff combined so I want another one of these but I also want to render at 24 frames per second and I want to change the um prefix file name to uh 24 FPS so we're going to change this to animate diff 24 FPS and this file name prefix to just animate diff 12 FPS we're going to make that one a gif we're going to make this one a MP4 um by default it makes GIF files but I don't like gift files because there only 256 colors in the pette so just keep doing this for all the versions you want I know this is a lot of manual labor sort of to get it set up but once it's set up it's really slick because you just change prompts and hit Q you'll see um I'm going to do also a MP4 version of the original at 12 so I've got a little gift I can share you know with Twitter or whatever then I've got um at 12 frame per second MP4 and a 24 frame per second MP4 what will likely end up happening is I'll take that 12 frame per second MP4 and I'll upscale it or interpolate it with like topaz or rif or film or whatever you like um to do your inter interpolation and upscaling uh to get it smoother and cleaner because it'll come out you know either really fast or really flickery because you're only getting 12 frames per second and you're only getting 16 frames so they're short animations um you can make them longer but then lots of crazy stuff starts happening and there are lots of rules to that but we can we can play with that in a bit uh for now this should just work um everything is set up everything is correct um so let's hit it and see what happens you just click this Q prompt button and that's going to run one cycle of everything that's here so we hit Q prompt and it'll go through the steps you can see it's doing the green bit like the the thing that lights up green is the thing that it's currently doing so just loaded the checkpoint into memory and now it's loaded the animate diff stuff into memory um we've just loaded the uh prompts into the sampler and now the sampler is rendering out the images and once that's done we'll have an animation this is only the first step of animate diff you can upscale with it you can uh run motion luras which will make it like pan left or pan right or rotate or zoom in or zoom out um what else can we do um you can do control net stuff so if you capture a pose of someone dancing you'll be able to uh you know try to enforce that on the animation yeah this is pretty cool so uh right away we already have something pretty dope um and that prompt was geometric shape unfolding paper origami so you want to follow along the same settings with the same models should get you the same results um yeah that is in a nutshell how to set it up and use it um so yeah anime diff comei this stuff uh again it's it's only scary the first time you look at it and once you start to get your head around it you're like oh yeah I know what that does and I know where that goes and that plugs into that and the really cool thing is everything is color coded like you can't plug a clip in anything that's not a clip you can't plug a Laten space in anything that's not a Laten space you can't plug a model into anything that's not a model so you won't like you won't rewire it wrong it just won't work so um it's really fun to play with uh you can use custom luras uh I think I'll have to check there's a whole bunch of more workflows in the um in the thing that we're going to go through so there probably some stuff for like adding extra luras to the mix I think you can literally just add them in line you can also use the VA that comes with the um checkpoint if you don't want to use the custom one you just drag it out of here and plug it into wherever the vae goes yeah right here VA code but yeah I mean I I like the 24 frame per second ones they're a little too fast but I like them I think they're cool and then if you go to your uh AI comfy UI Windows portable comy UI output all your stuff's in there um all the files are saved in this folder so you can just go back and uh you can see them and uh because we gave them uh prefixes oh I didn't give one to this derp 12 FPS and your 24 FPS right GIF I mean it already knows it's a gif these are the same oh yeah then I can just do this because this will make animate diff 12 fps. GI this will make animate diff 12 fps. MP4 and this will make animate diff 24 Fram per second. MP4 so you can just go back and grab whatever version you want for social or whatever so that's the basic basic basic workflow flow um you can increase the size I'll show you what happens so this whoops where were we so this here is the amount of frames that you're making um hey Kush um so batch size of 32 is going to make 32 frames uh clearly it'll take twice as long as 16 frames but um you know you get the animation is twice as long what will probably happen is after 16 frames we'll get some sort of like uh blip or a change in the animation but the animation will continue so as far as I know you can make these really really really really long um the only um drawback or the only uh thing the thing is stopping you being the motion luras can only go to 32 frames they're only trained up to 32 frames I I think that's their maximum so if you're using a motion Lura to to pan the camera left right up down whatever um those animations have to be 32 frames still um hopefully at some point they'll figure out a way to Loop those motion luras but I don't know how hard that is yeah see this is a 32 frame animation and it's lost its cohesion because it's not 16 so there's another node we're going to see in a minute that'll help you play with how long the segment of time it considers to be is so like right now it's doing 26s in a 32 by default but this guy here this context options has a node that um uh this one here where you can go in and say I actually want the length of the context to be 32 instead of 16 and that'll try to do a 32 frame animation as opposed to a 16 frame animation but I don't truly understand how context works so um I kind of just leave that stuff alone and uh sometimes you do like a 96 frame animation and it ends up looking great so who knows I don't understand it's magic just play um Okay so we got that we're going to go back to our this one so all these workflows text image text image 48 frame animation with 16 conents context link yeah as I said you just click on the image drag it onto your desktop or download it or whatever and then drag it into comfy UI and it Shaboom rebuilds your whole process with whatever he was working on you can also save your workflows you just hit save type a name and it'll save a a little J file and you just load up that Json file whenever you want to access your uh thing oh interesting I didn't know that that's cool yeah Kush this will be on playback and I think you can actually just go back while it's playing and and just follow along from the beginning if you want to install uh comfy along with me and because I did at the beginning I uh ran through how to install it and all the stuff you need to do have animated let's do spooky ghost in a forest cinematic lighting and let's try like a photon so this is a 48 frame animation with 16 context length it's using mm stabilized which is available on that site I was showing you um so let's see what it does I haven't actually run this one myself yet so interesting you can also uh click a node hit control C and then control+v to paste it uh so you can just do stuff like that where you're just rebuilding the uh the nodes that we were doing earlier so if you just want to quickly slap one of those out you can do that the other cool thing you can do is you could then select these three nodes once you have all the stuff set up animate diff 12 FPS you can grab these whoops grab these three nodes and right click uh how do you make a group I've done this before right click add group I think you can save this edit group I think you can save this as a as a node template and then you can just load it up so and then plug it in and make your life easier um and there's another cool node called the reroute so we can actually use that inside this node to reroute all these images so you only have to plug it into this reroute node so yeah we'll just grab this group Next time we uh switch out the thing and then I can just paste it back into the next one all right so let's try it out Q prompt this is the power of the nodes everything's modular and you can build build your own toolkits save them this one's going to take longer because it's a 48 frame animation but hopefully it'll be coherent some stuff that uh been making with animate diff [Music] already this stuff's been frame interpolated so um doesn't come out this smooth um normally but uh you can interpolate it with rif or um film or whatever you want it's got a very trippy motion to it I really like it reminds me of style gan oh it's done all right sweet it's definitely a spooky ghost in the forest the motion on that is great run another one of those well this is going let's try and figure out how to save this node as a node template whoops group let's edit the title and call it output uh call it animate diff outputs it was a manage maybe h I'm not sure how much RAM it takes honestly um it's possible it's not enough um let's see uh how much vram does anime diff require whoops spelled it wrong so I guess they're getting it in under six if you use x forers and sdp but I have no idea what that means uh maybe that's one of the models sdp support X formers and stdp what is sdp it must be [Music] some must be um like a setting or something oh this is the wrong um this isn't the one we're using that's the um Auto one this one doesn't mention I vram uh yeah I couldn't tell you yeah I think this one will look really good interpolated the speed seems right for it I kind of like the fast motion on this it's kind of creepy like a horror movie but yeah I mean the ground looks incredible uh I want to save this as a I want to save this as a node template I've done it before I just cannot remember how for the life of me uh you right click add group no oh oh save selected as template interesting ah animate diff outputs so if I'm right I can just go node templates animate diff outputs beautiful beautiful beautiful beautiful okay so now when we load up a new project all I have to do is just drop the uh drop the group on there right on okay um that's sick all right let's try uh the next one comy UI there we are where were we uh that's the one we just did 32 frame animation with 16 contact length pans left and zoom out so this one we'll be using the motion luras so let's try that one out we'll see if the motion motions oops make sure you drop it on comfy UI not a open tab with a screenshot you have of comfy UI uh let's go back to I like Photon made some cool stuff change your uh seed bloop uh we can just try this prompt here see what it does I think it's really good at drawing anime girls um it's just sort of what it was trained on for the most part um so here we have two luras and they're fed into each other so this is the cool thing that the motion luras can do um you can say I want this Laura fed into this Laura and then running out so you can actually stack all of your motion luras if you really want to so this one's going to try to zoom out at about a rate of you know 30% weight on the 33% weight on the on the prompt and this one's going to try to pan left at about you know 66% so it should try to pan left harder than it zooms out but it should be doing both motions as it generates the the animation context link is 16 batch size is 32 so we'll probably have one scene change is thing in the middle but maybe not I have given up on trying to predict what is going to happen with this stuff so we close this and right click node templates outputs paste or drag that into there and hit go your uh luras if you put them in the right place we'll just show up here so this stuff should all work if it doesn't just go Um here and click refresh and it'll uh refresh all the uh associations and then your luras will show up in the right place so this should zoom out and pan left and it hopefully will look something like this so it's panning left and I guess it's zooming out it's hard to tell really we shall see yeah I really like these modular node layouts for doing uh AI stuff it's really fun you also don't have to use their things you can just look at the screenshot and add the stuff that they added and put it in the spot like if you want to just keep building out your own personal template there's logic gates and stuff so you can set switches so you could add functions and then add a switch for that function and say you know only do it when this when the switch is on and and ignore it when it's off which is great for like if you have like an upscaling step you may not want to do it every single time because it takes twice as long as the original animation crazy you can see this animation is just Miles Ahead like not miles ahead but just different just super different than um the other AI animation stuff that we we've had access to so far um there's like a reality to the motion it's almost I friend of mine said it was like he thinks this is the way we actually dream you know which is possible something close to this but it's yeah super trippy let's do this with an animated uh do this with an animated uh model and see if we get like a more um animated uh version instead of because the model I used is um more for photo realistic stuff so I think that's why oh you know I should I'm on YouTube I should probably add uh NSFW in New to this negative prompt day last thing I need to be do is streaming Nudes on YouTube unintentionally yeah I know I hate to say it but Pika is my favorite too for the um like Runway is cool does some really great stuff and it's I think it's a different like you know different models or different algorithm or whatever but um Pika I get I think I get more consistently good results of Pika um more consistently usable results but I'm sure that'll just keep changing as they all just you know iterate and go back and forth and as a consumer we kind of win because we can just use whatever we want I think I got really lucky not streaming nudes to YouTube without having that in my negative prompt wow there definitely panting to the left yeah I guess it's zooming out a bit very interesting the fix seed makes a really big difference really keeps it in the same space for yeah I'm going to try the control net thing in a little bit just working through all the examples on their page um as far as I know it does have control net uh I think it's best used with pose right now um you sort of like generate a uh a pose video from your your video like a control Net series of control net images of the poses and then you use that as the guidance for the thing and it makes for some pretty trippy videos oh that's cool I like how they loop it's really cool how they Loop I don't know why they Loop but I'm glad they Loop okay let's try the uh next one uh which is text damage leaden upscale this one's really cool because uh it can actually upscale and get you a way higher quality version of the uh the image you you were building so let's uh let's dive into that one com UI got to remember to add an sfw nude and uh change it to the Rev animated and let's try uh actually I kind of like that prompt it makes the motion look really good uh 20 things so let's add uh outputs no out and then I want to do the same thing here outputs image out go oh uh model name mmd1 14 not in stabilized path oh because I don't have that model if you get that error just try to figure out where it's complaining it'll be red um just change it to something you do have like in my case I have this one so I'll use that one you can go download all the models if you want but like those ones are all like a gig and a half so I don't really want to hang have them hanging around if I don't use them that much okay so this is rendering the first 16 frame clip and then when it's done it's going to upscale that 16 frame clip change the file name for yeah H good point you should definitely do that it's probably too late for this run but yeah good call good call they will still show up in the previews but the file names wouldn't have saved this is where my stream probably dies from the upscaling we'll see gotta love that 3090 there's no way I could stream this stuff on my 3070 I mean obviously this step takes longer because it's you know doing twice the work or more four times the work I guess um but yeah let's see what it does these are um so these will be 512 x 512 by default so these should be 1024 x 1024 so we'll see it also does um it dreams it like it's not just a raw upscale like uh topaz does or whatever it's actually dreaming more detail into the images so you'll they'll be different like they yeah see how she's actually looking at the camera now and we got some cool light on her so it'll definitely be it'll be different in the upscale but you should have a lot more detail uh a lot more finer detail yeah big time so let's try um uh actually no we don't need to do this anymore because let's the upscaling it takes too long uh yeah text to image with Laten upscale pan left and zoomo motion Laura so same thing but with the motion Laura um longer animation same thing text damage relate an upscale full noise on upscale 48 frame animation so this would be like this one will take forever but it results in a long good animation wow that's cool text image with control net stabilized Laten up scale part D noise on upscales scill soft control net weights let's look at it and see if we can figure out what it's doing where's the control net stuff so it builds out the animate diff and then it uses control net only on the upscale interesting so it uses the line art control net from the first one to do a new generation with the line art from the first generation let's try it and see if it explodes I'm going to guess it's going to explode uh following graph the following node types are not found scaled soft control we control n advanc loader so because we installed comfy manager we should be able to go manager install missing custom nodes and it'll go oh hey this is the one and then you go install and it'll be done to apply the node please restart comfy UI you got it go back to here kill it and load it up again yeah and we're back okay I already have these files in the right place but if you don't uh you're going to need to pop those uh control net files into the control net place oh are those the sdxl control Nets no okay interesting uh that broke my brain a little bit one sec uh oh that's why the sd15 okay I only have the um sdxl one so I need to get these control Nets and put them in the right place so let's figure that out it's probably in comfyi yeah I just have to drop them into my models control net folder and I already have them in myh thing but if you want to get them they're um right here files yeah uh here drop it in the link and there we go and those are the control Nets they go into the control net folder and then refresh yeah so let me copy mine over from my stable diffusion uh AIS SD models no they're in extensions control net models yeah uh I need depth canny line art and uh scribble probably all right and put them in here for this test you'd only need line art you can get it right here lineart download pth file and put it in your comfy UI um models control net folder go back here hit refresh and then it should show up in this list so we'll go back to line art and YOLO all right so let's change this [Music] to uh uh flower blossoming in the sun sunrise uh beautiful Lush uh Countryside make sure we don't get anything weird that's good and let's add our outputs real quick no templates Boop and Boop and Boop we don't need you no more and everything's good what's it going to cry about now checkpoint worked stabilized version is working it is generating the first run so it should make an image here and we're going to see the um the one that it's going to upscale and then it should make one based on the um based on the line art control net of the this animation hopefully we'll have a subject in the animation I probably should have used the human face oh that one's this one might be too chill too stabilized all right let's try a person um a person standing in in an alley grinning holding a bouquet of flowers uh rainy afternoon Sunset here's the nice thing about um comfy is every time you change a setting you want to run another one you just add it to the queue and you can actually view your queue by hitting this view Q button and it says I'm running job one and job or job zero is running and job one is pending if you don't want to continue running job one which we don't because this is not going to be a good you know input for an animation because it's not moving we just hit cancel and it'll kill this first run that we're doing and clean itself up and then it'll start the next run which is the when I put a person standing in the alley grinning holding a bouquet of flowers rainy afternoon Sunset and let's see what it makes ideally I'd like to get a bit of motion so we can see what the control that's going to do with the motion and yeah I don't know how much vram this takes might take more than eight once you start adding control Nets and upscaling and other stuff okay they're walking backwards but they're in an alley which is cool so it did a really bad job on the face and I'm wondering why because we have the right vae for this model so that's very strange all right let's see what it makes probably don't need Steam and epic notifications while I'm playing because I'm sure some jerk has a terrible username on my list all right what did we get wow okay so I now now think control net is very cool for this and it did a fantastic job of maintaining the composition of the scene while just upscaling the pure data out of it um let me pull up the let me drag this one down here so we can see them side by side uh it's hard because this is eight frames per second but so this is the one the original that it made and then it used control art line control net line art to do this one with the outlines of the you know all the stuff I think it did a great job I mean she's got a few extra fingers and they kind of scale out a little weird like once you start once you start looking at the details you'll start seeing stuff you don't want to but uh like as an animation it's it's wild it's wild where we're at right now absolutely crazy okay so we got everything set up for all this I'm going to clean up these tabs a little bit because this is just ridiculous okay um yeah where were we first control net one that was first one we did it's this one text to image with control net stabilized lat and upscale partial T noise on UPS skill scaled soft control net weights 48 frame with 16 so I believe this is just a longer version of that last one we did uh image to image or text to image with initial control net input using normal linear pre-processor on the first text to image now what does that mean so that would let us that would let us impose our own control net animation on here but I think you might already have to have it oh no it does it itself so uh let's let's see uh uh we only get 16 images uh how I guess they're using another animation they're probably using another animation they made yeah they are this is the one I'm mostly interested in is this pose thing so you gener an open pose control net animation and then you can use that to drive the motion of the animation which is wild H okay have to find out where we get this uh wow okay that's a lot of nodes all right um R animated that's all fine so where do we get this open pose thing from courtesy of toy XYZ who's that toy XYZ is that a person toy XYZ control net pose H can I generate this animation from an existing video maybe should be able to or can I just download this and use it yeah open pose is great I don't know how to do this um can I convert this I wonder if I can convert this um this to an image sequence I think that's what I have to do uh FFM Peg convert GIF to image sequence yeah okay let's try that uh CMD desktop inut no it's uh 26 conversion failed make directory temp hey all right let's see stop temp all right okay uh let's try this this is really hacky uh desktop temp yeah uh go back to comfy image load cap is 16 uh we're going to do where's the length oh maybe it's just the length of however I'm gr frame it extracts uh Olo did I download open pose did I put open pose in the right place of course not AI models no AI [Music] SD models extensions control net models open pose you just download it from the same place that I showed you earlier um that goes into AI comfy uh model control net blop okay and then refresh and then select open pose okay that should work uh I think that's all the stuff that's all the [Music] models all right four four four four four four uniform 16 cortex length 16 okay okay so I think I think you could generate uh an image sequence of open pose images pretty easily uh by just running them through a control net pre processor and saving the outputs um I think you could do that by just simply running a deorum with open pose and it like with control and open pose it should make a folder of open pose images uh or does it just show the input I honestly can't remember may be a tough one but not unsolvable you could probably use footage of yourself too dancing or shaking or whatever awesome that is crazy let see how the upscale fares this a just a straight up upscale without using the open pose I wonder if we could Loop the open pose back into the upscale like how we used the um the other one to uh upscale like the line art use the line art to do uh a new run or I wonder if we just use that other system and use it the open pose that way no you want to generate the first one I think oh no because we can make an open pose from the generation interesting okay we're going to try something after this one builds we're going to try using open pose instead of lineart on that last sequence to see if we can convert the first animation we make into open pose and then use that to do a new animation might be total soup by the time it's done but it might not well I've been streaming for over an hour so I am going to stand up and walk around a little bit and I suggest you do the same uh save your knees some wear and tear and I'll be back in about 5 minutes all right let's get rid of our [Music] weirdos there we go okay so I want to try that thing I was just talking about um let's um let's close a lot of these windows because holy crap all right uh this one here so we did the line art control net a couple back here let's just try this first one because it's short where is it here we go 40 frame no where's the first control net one control net liner there we go so instead of a line art uh upscale I want to try the um I want to try this with an open pose into the upscale um oh shoot which one was it then delete all these delete all right uh boop boop boop boop comfy drop all right no templates animate diff bloop uh delete yeah and same deal here HD HD okay so uh we we have our line art but we don't want line art we want open pose we're flying control so instead of this normal line art do we have see if we can figure out what nodes these are a part of and I guess it's these ones nope uh normal line art what if I go here and go search open pose preprocessor YOLO let's see what happens all right uh okay and let's go um uh uh person uh dancing uh in an open field uh yeah that's fine person dancing in an open field uh and also Let's uh make sure we get some NSFW nude stuff in there in the in the negative and let's change this seed cuz you know we can okay this is going to definitely break but it should be funny I'm going to add a preview image to this to see what it see what it sees for the open pose because I have a feeling this is not going to work um let's change our height to 768 to get a uh I can do this 768 there we go uh hit it hom oh I don't have that I don't have that model do I yeah why is it running with that model oh it must be the first model that's why okay so we want to change this to our rev animated model Bae is fine are you still running what's going on here why are you purple yeah whatever oh it's purple because this is the one that complained that makes sense okay and we're building out our animation so now our animation should spit out a series of open pose uh uh for each frame is that how that works maybe it doesn't hey po what's up bud haven't seen you in a while I'm trying to break uh trying to break comfy right now I'm doing a pretty good job okay okay it's doing an open pose that's some pretty crazy dancing it seems to be doing the open pose recognition oh wow it worked we got an extra body here but it worked okay and now it's going to do an upscale run based on this uh based on this um open pose I'm surprised this worked I mean you could technically just go in and you know remove that from that frame I think I should save these little open pose animations into a folder Bas sampler control net model I'm really blowing Away by the lighting the way it follows is just crazy it's occurring to me that recording non-human animation may not be the best thing to emulate when you're trying to do the upscale but yeah you never know it might be better to take mixo dance things and make open poses from them I mean it worked maintain the silly dance that is crazy looking all right let's try uh penguin dancing on a glacier in the Arctic yeah I mean I really like automatic 11-11 for a lot of things like deorum and you know uh oh that's pretty much it now but um comfy UI is um you know for for a tinkerer like yourself I think you're going to love comfy because you can really just start rewiring how the diffusion works you know [Laughter] I can't wait to see what it thinks the pose for the dance looks like did it get oh it got nothing for the pose so I guess open pose only works when it detects a human so now I wonder what this is going to do to the upscale actually I was going to stop it because there's no image but I'm actually curious to see how much weight that has it might not do anything and it might just be an upscale of what we see or it might make the penguin stand still I'm not really sure we're learning together learning how far we can push comfy might save some time by sticking to square animations but I'm actually pretty blown away at how good this is at um uh odd aspects like uh the fact that I'm making tall videos and it seems to proportionately understand what's going on on a 1.5 model is pretty crazy I should have like two torsos and two heads and stuff like that but so this is what happens when we run a blank open pose through the upscaler I think it's basically the same right quality is better obviously so it looks like it would just impact the motion if it could but it doesn't so that's fine it would have also been interesting if it just stood still but I guess if you had an open pose thing where it was like just a still image 16 still image or whatever it would work uh well then if that's the case can we go longer okay let's try the anthropomorphic Fox working in a steampunk lab and let's try it as a 32 frame animation and see if it wigs out I think it will a I think it will upper okay okay so the model mm stab this is the thing about error messages don't freak out when you see an error message just read the first part it'll tell you what's wrong uh Arrow error occurred when you executing the sampler without a context window uh animate diff model mm stabilized High has an upper limit of 24 frames but it received 32 latent so we got 32 latent uh we uh I think that that means our upper limit batch size for this model is going to be 24 but if we switch to the 1.5 oh maybe that's why the motion's better sorry I'm coming to this realization while I'm working so this stabilized high is a fine tune of 1.5 that I think has uh better motion for for people so that's why this stuff's holding together better than the stuff we were doing with 1.5 and as a result has a 24 frame upper limit for the thing for the length of the animation but let's try a 24 frame and see if it switch which is um States after the first 16 frames cuz that's you know tends to be what happens with with these models but if not maybe we found some Secret Sauce well not so Secret Sauce but the fact that we're generating the open pose thing and using it is hilarious I I really dig that I'm delighted with myself render render render please put on some of your own music I don't play music on my stream anymore because uh all the royalty free stuff ends up getting put in someone's track at some point and then all your videos get copy stricken down the road so put on some tunes in the background and and hang out um feel free to just uh you know put on your own stuff and set it at your own volume okay I don't know how much work the fox is getting done but he's definitely at a he's not he's not at a steampunk office but he's at an office I think we're going to get the same thing with the open pose um where it can't figure out the body because it's a human or not a human use a different text prom for the upscaler use the girl for the initial dance but then slap it on the ooh interesting interesting maybe where's the upscale getting its positive and negative from it's just coming through the control net oh it's getting it from the same place why not uh yeah why not let's try that that's that's a Trippy idea I like that I like that so this is going to be our upscaler this is going to be our upscaler prompt and this is going to be our uh initial prompt so let's go back to the person dancing in an open field and read this and then we're going to go penguin dancing in an open field and we're going to see if it replaces the the the person with a penguin uh so the text encoder will go to the control net input here positive negative so now you will go through here become blah blah blah blah I believe that'll work do you have any yeah the images for the workflow are on this um they're on this site here uh where you download the um the comf UI stuff um the the plugin for this and all you do is you just download these workflow uh images and drag them into comfy and it it replaces the it replaces the um replaces the workflow so it just drops that workflow right on your your UI and then you can save them all locally by just saving it and you just save it and name it whatever it is so you can go in and save all these workflows yourself once you get them all you get them all kicking so person dancing in open field penguin dancing in an open field 20 make sure these are all right yep yep yep yep yep yep this one got three frames that's funny I had three points throughout the animation it thought there was a human in the frame I'm just curious what this upscale looks like I wonder if this like coffee but also soup thing turns into something else I don't love that the way it's phasing in and out of reality like that oh here we go Moment of Truth is it working is it broken that is cool I mean the control Net's doing nothing except maybe breaking those three frames where it gets all Wiggly these ones um but yeah uh so I want to try this person penguin thing first and then I'll run that uh run that prompt to cyber run the same prompt for both on that one and let's see what it do uh cool thing is it seems to be doing a 24 frame animation without wiggling out too much although maybe that's what's causing all this uh instability I am not sure we'll see if this next one is really unstable then I think think we'll know that you know more than 16 frames on these is potentially sketchy territory but I love potentially sketchy territory yes I think after the first 16 frames it's just wigging out I am curious if it turns into a penguin though all right let's uh turn this back to 16 and kill this one and run it again there we go uh I did I killed it and ran it again because it was at uh 24 frames so this one next one I'll run at 16 because they they Loop a lot better when they don't get all insane at the end okay we got a full we got a full set here that is so cool how Wild is controlling that seriously okay so now we are running the uh the prompts are gone but I had person dancing in a field and then uh Eric said maybe we can uh try running a separate prompt for the upscaling so after it runs through the uh open pose and grabs this thing it's now going to run the secondary prompt that we set up on the upscaler and if it works right it should change her into a penguin if it doesn't then all hell will break loose I don't see why it wouldn't unless the uh upscaler is uh like more like just like a refining run and if it's a refining run this should make like chaos just had penguin stickers everywhere or something you can see in the background it really starts to fall apart right there and I bet you that 16 frames in where it really starts to like you see it in the grass just the grass starts dancing around huh I wonder why I think I spelled that wrong uh I did oh no it's here think oh oh it's good to read the manual sometimes not all the time though just sometimes well most of the time I guess oh just realized my camera's not on okay we're decoding is it penguin time it is not it's seriously broker legs though okay I don't think passing a different prompt is the way to go I think the I think this final upscaling step is a lot more like a refiner pth than a than like a dream pass yeah I was very interested to see what it did it's a good idea I'm going to pass this um these ones back to uh original spot but that's a fun of comfy you can just try stuff like that without having to rewrite the whole extension you know Dapper monkey I like his shoes we're doing the open pose uh generation right now see what we get this uh body is pretty humanoid so I have a feeling oh it's got extra people in the background huh even got some faces oh right because it's shorter yeah less frames okay up scalers going okay so let's double check the other workflows here here oh I'm really just exploring the um workflows that are available on the on the page I I just literally started playing with this yesterday I'm just doing um uh generations and then uh yeah I thought I I was going to go through all these workflows today on the stream because I'm might as well uh show it while I figure it out because it's you know it's easier right now I think that's the case um unfortunately uh I haven't had much luck getting good results in automatic 11-11 with the anime diff implementation yet um it's probably me uh but this stuff seems to have a lot more [Music] um I don't know it just seems to make more sense so it's cool you can either start with the control net stuff from the front or the end so I wonder then so you could extract 16 frames from you know what be good uh is uh extracting uh 16 frames of an animation at eight frames per second so BL BL blop grabbing every eight frames um for 16 frames gobble those up put them in a folder and then use that as the control net uh starter for the animation then you could literally pull pull an open pose animation from any video um that being said we could slide over to chat GPT and make a little python gu for doing that for FFM Peg uh that might be super freaking handy I wonder if there's already a thing for doing that I've never seen anything that does that it's pretty unique use case um it can't be that hard it can't be that hard all right uh what would the FFM EG command B for uh extracting 16 frames um out of a video but skipping eight frames between each okay outp put pngs all right let's try this go to pixels yes yeah that's right Andy so we want somebody dancing not photos but uh videos going to stick to a tall one I wonder if there's one that isn't slow motion I need a full body with a relatively simple background hello oh that might work oh more slow motion uh the feet are hidden oh oh there we go oh no that's not going to work because he's twoo oh maybe from that point on though right here yeah yeah that might work the last little bit so I want to start at 5 seconds in so let's download this all right open that up open up resolve real quick we want to make a new project new project number 758,00 footage uh sure I want to start here all right that's probably enough where does it LO there all right cool uh actually I'm going to EXP for it all right uh dancer person uh 25 frames per second sure why not uh Daner personov videos sure danc personov okay and now let's try our little script that chat GPT spit out G to Go videos open that up in an editor real quick a look at it that's fine put it in a temp folder yeah and it's going to be dancer personov V frame 16 temp output yeah right gotta be okay okay it's not long enough that's okay that's okay we can make one it's damn close though let's see oh maybe we can use a a [Music] shorter a shorter um maybe like every six six frames yeah shorter inter interval no we need 16 so three yes okay so this if we go have a look at it wait maybe it doesn't need to be maybe it doesn't need an interval maybe we just need 16 clean frames like maybe having cleaner motion is better [Laughter] uh yeah why not I like the head wobble too all right let's uh run this through our thing and see what the hell happens cuz uh interesting okay um that was a different workflow uh let me find it real quick down here this workflow here grab that and we're going to save this one because uh oh whoa that came out cool open pose helped neat very cool yeah improvised cooking show yeah that one's really neat I like the hand and the fire it's cool the way it like flips around interesting all right uh okay so we're going to save this workflow because it's pretty cool so this is uh animate if uh open pose control net uh generated by input uh by uh first prom we'll save that workflow and we're going to drop the workflow I just grabbed which does the folder thing we're going to go to the folder of images I just created which is in my videos folder uh temp yeah okay so let's pop that in here okay our image loow cap is uh 16 oh maybe we should set it to 48 interesting uh give me one sec I have an idea uh 48 frames 48 do it yeah yeah okay okay so let's change you to 48 so now we have a long animation so hopefully um yeah okay I'm actually going to use the fire dancer prompt for the um for this because I think it'll be cool on the uh the dancing and asfw nude for sure uh okay contact link is 16 that's normal the image Lo cap is 48 let go to rev animated one uh let's change the seed blah blah blah you're good you're good you're good you're good uh we're using open pose to generate from the video we just made in resolve uh and yeah I just want to swap out my node groups here real quick yep and we'll pop in these anime diff outputs that we made plug that into the reroute and these are good to go and then we want to make the same ones down here yope and then we want to change these to HD HD HD and we're going to save this workflow as animate diff control net um input video uh open pose yeah and then I know what it does okay so that's all good that's all good we're trying temporal diff I guess temporal diff can do 48 frames because that was the default we're learning all right this is good this is all good oh oh oh oh hold on we got to oh no no we're open posing them applying the control net wait does it do the control net in here I want to control net these on the way in uh yes yeah it can make GIF and MP4 it can make uh h264 and it can make uh h265 uh webm's web PS all that stuff yeah the only thing I can make is PNG sequences but make it rad showed me how to do that with the uh PNG saver thing in the W Suite I haven't set it up yet but you can use something in here to save um thingies I don't think this is going to work uh because I think it wants control net images uh but we can generate them I just need to use my brain here for a minute uh we generated them before because we have the images so I just need to run yeah I just need to run the control net on this first I think I think I think so we want to add a node um open pose going to image out to open pose search for open pose preprocessor yes yes yes this is what we want preview image and then that goes in here not that yeah you're fine I think why does it need this control net oh yeah of course yeah yeah and it'll use open pose on this open pose yes brain make work yes okay so now we can load any video pre-process it into open pose use that video the open pose thing as a driver to drive the animation of the animate diff and if this works right we'll be able to make like 48 frame animations from animations because we'll have lots of input data and we're using the control net to keep the motion doing stuff so you know things might Flicker and change but the motion should stay coherent throughout the all 48 frames hopefully hopefully this is taking a while because it is a 48 frame animation oh that's right I forgot Jonathan fishof came up with a new anime diff model um I'm G to try that out okay I don't think it followed the control net oh no it didn't because this one I should I should cancel this one this one's broken yeah cancel cancel yeah so you see how every like 16 frames it sort of loses its cohesion changes into a different animation um I it's my hope that the control net will help resolve that by um well we'll see doing the open pose recognition now it's going to generate a a sequence of images that [Music] um oh the screenshots with the nodes that's on the page for um animate diff evolved so any of the workflows that are on the page you can just uh click on it uh put it like uh save it and then just drag it on to comfy UI and it will rebuild this node structure in your in your window uh it only works with these particular workflow files because he injected the workflows into them but every image you make with comfy UI like PNG file it contains this stuff so you can drag any image you've made with comfy back into comfy and it will rebuild the node structure that it took to actually build that image in the first place which is like super handy it's a really really cool way of sharing workflows with people [Music] is it done I don't know because we're looking at a screenshot H der H there we go is it done oh no it's still doing the open pose recognition is it okay oh yeah it did it okay nice so we got our animation from the the video we grabbed from pexels now it should be using that to do the motion for the for this so when it's done we'll see that's super cool open pose is so cool so I didn't even need uh it turns out I didn't even need ffmpeg script all you would have to do is just take any file you want into the um uh into a video editor and just grab uh 48 frames or like uh two seconds at 24 frames per second of any video and then you can um just export it as a PNG sequence and drop it in there or you can use that FFM Peg thing that we did um pretty handy actually let me find that that little chunk there it is you can use that to extract 48 frames of a video um you just have to have FFM Peg installed on your computer um that set that one where it says n n slash1 uh that's how many frames to skip so if you want to skip every frame you change it to two you want to skip two frames you change that to three so you could get you know more movement with less frames cuz this is going to kind of interpolate the motion anyway as it moves so I think you could get up to maybe two or three four frame skip without it freaking out too hard if in your animation the limbs go like you know 45 degrees within you know one frame you probably got too much uh frame [Music] skipping render render render now I need a 4090 somebody send me a 4090 and the renders will go faster I'm just kidding I'm super happy with the 3090 so nice to be able to just do this stuff and share it couldn't share it on my 3070 there's something really cool about the motion even when it's wrong in this decoding hey y it's doing it it's doing it it did the thing oh that is so cool holy cow that's powerful so any animation you want you can just take the motion capture from now and use it as an anime diff input wow they already loop I don't know how they did it but they they do kind of loop like the the change is abrupt but it's uh you know it's pretty close to a loop um getting actual seamless Loops I think is pretty far away um there's a pingpong checkbox but what that'll do is play the video out and then reverse it and play it out and reverse it so it it will feel like it's you know seamless but it's literally just going you know sign wave back and forth I'm curious what the upskill looks like Z fire looks really awesome this one especially is like I think Loop count just like makes two Loops in your video or three Loops in your video like right now they're just looping like constantly but like in the video it saves instead of being one Loop it'll be two Loops or three Loops or four Loops or five Loops so yeah extract 48 frames of any animation you want um try to keep it the same aspect ratio as your output footage and then um yeah you can run it through the thing this might take a while might skip the upscaling step for this uh 48 frame animations but like when I'm live but you know when you're at home by yourself just let it run go make coffee and hang out in the other room chill out uh this is not one of the example workflows I kind of hack this one together um hello um this one is the same as the uh the same as this text image with initial control net the it's the same as the last one except uh I'm just running control net open pose pose uh recognition on the input video instead of having it already made so in the example the person had one already made but um I figured we can just do that on the input so make your input video 48 frames or 16 frames or 32 frames and then um tell it where the PNG sequence is tell it how many frames it is and run it through the open pose and then run everything back out through that and it'll generate this instead of having you to have to already make it so that's the only change made to the thing and I added some output options um I can drop this um I can drop an image that this made on my Discord uh like the PNG file and you can just uh grab it from there and drop it on and and and play with this if you want I am curious to see what the upscaler does but this is taking a while oh let me pull up the [Music] um uh VLC files videos uh dancer person so this is the uh input footage here for the motion so you can see it did a pretty good job of um did a pretty good job of capturing what was happening and and transferring it into the animation so if my Loop my input Loop was more looped the animation would be more looped like if he was in the same position going in and going out um that would really affect the loop of this like the ones that are just from raw text prompts they they Loop pretty good um oh this yeah this is local and it's on a 3090 uh with like a an i5 uh 12 gen I'm not sure if you need more than 8 gigabytes of RAM to do anime diff with control Nets on top but I think it's possible to do it with as little as six gigs if you optimize a hell out of it but I don't know I'm not entirely sure I move my mouth away from the mic to breathe you have to be pretty old to get that reference it just occurred to me that's a very old person thing to say my computer is so hot right now I had a PS4 controller sitting on top of it and it's like cooking I got that one from what I heard tand is quite a cool dude I was talking to somebody who hung out with him a bit and said he was super chill I get chocolate raining in my head all the time need to get a portal gun let's put a little portal to the Antarctic behind my computer let it suck in air from the from the South Pole I guess I'd have to go there to open the other portal wouldn't I it's a whole thing I do have a portal gun though might be easier to get there I guess you just put your portal on an airplane right and just wait for it to land and then step off into the new place we need Portal guns you think chat GPT can make portal guns maybe chat jpt 5 I just saw that imod retweeted this stream that's amazing thanks thanks Imad welcome if you're watching yeah ask it come on jat GPT hook us up do we really want chat GPT to be like Aperture Science though probably not well the refining run is weird the quality is really wild though like really wild that is really cool 24 14 yes to 12 yeah all right cool cool cool cool um that takes forever so let's um let's slide back to um one of the faster workflows so we can get you know better results and you guys can drop prompts and we can see what this thing could do um but this is a really cool workflow and I will drop this um let me save this uh workflow oh I already did save this workflow so um I'm going to drop this on my Discord you guys can grab it there if you want um General uh I'll just put that uh where do those download to uh net input so here's the one I'll drop that on there so that's uh control net input video that's what if you have uh an input video you can use that to generate it and then um this is the workflow where the open pose is uh generated by the first prompt and then done in the refining step so yeah both ways so you don't you do you don't need input footage if you don't have it you can do it from a prompt um and by if you need access to my Discord the link is at the top of my website uh it's the one that looks like a little video game controller um oops I to go back to YouTube I want to go to pers. XYZ uh Discord invite is right here on my website just click this little uh link right here on the social menu and that'll bring up the Discord invite and you'll come in in as always if if you have questions or you're trying to set stuff up and it doesn't make sense just pop in and start chatting and either me or somebody else will help you um we don't bite it's pretty chill in there okay let's go back to a simpler workflow let's do I'm going to can the ups scalers for now cuz we know they look cool but the upscaling takes four forever um so let's just go back to that super simple motion Lura workflow and try and get some cool stuff out of that boop boop boop you get used to doing this little uh this little dance all right uh we don't need you we need you bloop bloop good to go okay where's that prompt melted candle wax blah blah blah all right right and then NSFW nude and uh rev animated we got our zoom out and pan left Laura uh contest of 16 32 frame animation this should be nice and fast let's change the seed too uh yeah the I'm running the one right now the melted wax but uh I didn't see the other one let's pop that in there the workflow where you generate the first clip and then use that to make a control net for the next clip is probably the easiest workflow if you want to do control net stuff [Music] um that's with open pose uh the line art one works really cool too the line art example is on the website but if you want the open pose one it's on my Discord but you can build it too you just delete the delete though let me make sure that's safe for work yeah I think we're okay never know all right done let's increase the height actually let's increase the width we had we were doing Tall ones but let's try uh let's try wide ones 768 by 512 d oh that's cool like she like sneakily smashes an espresso real quick I want to do another one but wide of that and then I'll try this promp here look at the the the coffee spilling in the cup when she moves like what the hell that's crazy bloop yeah yeah that happened to me when I was watching um somebody else play with comfy a couple months ago I was like yeah today's the day so it's zooming out and panning to the left I mean kind of is let's go back to 512 these wide ones take too long I like how consistent it's saying over the 32 frames I mean there's definitely like a scene change like her shirt like changes like hard yeah the the the the best thing to do with comfy is just get that comfy manager installed ASAP then you can use a manager to install and uninstall like uh extra nodes cuz like some of the sweets are really great I have another video from about two weeks go where uh I go through using the image processing chains in comfy to like make uh glitch machines that'll like give you like glitchy alternatives to your outputs that you can mess with oh that one's really cool oh man that is cool [Music] d actually this one can [Music] go there we go now we can see it all on the same screen it's nice to have the gifts but gifts are 256 color palet so you get a lot of color banding and stuff so it's a good idea to save them as uh MP4s as well um and like I said make it rad showed me how to do a PNG sequence export as well to make sure that you get top quality for all the frames so that you could like interpolate it later uh in whatever you choose to use like topaz r or uh because like this 24 frame animation is a little too fast right and this one's a little too choppy but if you interpolate the 12 frame animation that you make at 24 frames then it'll smooth it out and you'll get like U you know happy medium so I just use topaz but uh you can actually use R and uh film right in deorum so if you open up the Forum in automatic 1111 and you go to the video output there's actually a place in there where you can do the uh upscaling uh an interpolation uh with um uh come on brain with fil and R um and just spit those out uh into your deform folder kind of Harley Quinn it's weird [Music] there's so many nodes for comfy it's wild hey pancakes what's up buddy just making anime girls on YouTube you know the usual so watching you let's see oh that's cool I love how it loops [Music] I like how like morphs that's cool yeah automatic 111's anime diff implementation is extremely broken highly recommend doing comfy to be honest bro um nice let's try that oh yeah I make tons of um tons of deform stuff I do everything I really like mixing this stuff together and I what I'll probably end up doing is making a bunch of animated diff stuff and using that to drive deform animations and all kinds of other stuff I really like mixing all this stuff together so yeah um right now my big Obsession into forum is the QR code control net ooh oh there's no node for it huh I was just thinking I could use the um QR code control net but um oh hold on yeah I think if I uh if I pre-process it just like threshold or using like the black and white masks that I already have and use the QR code control net in place of the other one we could do animate diffs that have the uh the uh uh control net stuff uh no yeah no that that's cool but it's not what I'm trying to do this is cool yeah QR code control net with the Forum there's I did a video couple days ago on it you should check it out it's a really cool it's really really cool um this is really neat uh now I want to now I want to try that control that process all right give me five seconds I want to try something uh let's load up the workflow we were just working with the uh control nut input video uh yep and then we don't need to process it and we just copy it in so C AI uh stable diffusion oh no it's AI extensions control net uh models um QR code monster I [Music] comy and we want to go to models control net all right cool and refresh Beauty and then QR code monster and then I want to use one of my masks as input let's try this one and we're just going to do 16 frames of it and oh it's got a start index Ah that's dope uh let's do 24 frames in so QR code monster okay and we're going to do uh a wave of Sakura Blossom Flowers U moving through the ocean [Music] yeah all right so we got QR codes good monster okay uh what do you say B okay okay might have to frame skip these a bit to get uh more fluid animation but let's see what it do oh no the wrong height see what happens I think I'm like the you're like the fourth person I've got what what oh my God what the oh my God oh this is so cool oh my God it's working exactly like I thought it would oh this is [Music] cool oh man with this driving the thing you could probably just let it sit and do chug away at like a 280 frame animation too oh it's like it's it's like deorum but style Gan yeah uh yeah I don't know how you wire luras in but I think you can like I think there's a Laura node and it goes through so you like send the model into the Laura and then Laura out to the like the model input of the sampler so you can like just stack them with a weight there's a little weight setting on it I feel like I just came across some pretty Secret Sauce here I love the way this animates I can't wait to see what the upskill looks like uh this checkpoint is um rev animated which is crazy because this is like doesn't look super animated yeah yeah they're all on um if you look in the description of the video the second link to the the uh animate diff evolved uh sdxl stuff you got to install comfy manager and then um just follow all the steps on that thing um yeah and then install it with comfy manager and uh then when you load it up it'll complain about the stuff that it needs then you can use comfy manager to uh install the missing custom nodes and it'll just fix all the depths then you restart it and it's all good to go and just put the models from the things where it says in the right folders and you're Off to the Races if it's confusing at all just go back to the beginning of this video I I did the whole setup process at the beginning geez I think I like this one better honestly but wow all right I'm going to kill the upscaler for now just unplug it [Music] uh and I think that should stop this process and we're going to try a different uh mask a mask that moves more uh let's try this one oh I don't have pngs where are the ones with all the PNG files there no that one yeah that one all right 1624 blah blah blah oh that's weird oh no yeah 16 okay so let's try uh 48 excuse me 48 512 by 512 kick it yeah I don't it's okay we don't need that yeah okay still working I'm curious what it does uh with uh 48 frames of input if it'll change every 16 frames or if it'll stay coherent because it's got so much uh information for input this is chuing frames pretty fast yeah it's pretty easy man you can just do it from the the there's instructions on the thing like you'll have no problem you just put the right models in the right folders and you it's easy just gonna go step back in time for come on oh what oh this is so good okay [Music] so does this mean I can make a Loop by doing all 280 frames of this animation I mean come on no slow con but maybe I'll make one holy this is wild look at that animation look at that all right I'm going to try a 280 frame one and see if my computer explodes fake cooking show improv cooking show all right 280 frames wish me luck it's going to be like nah that's way too big [Music] H or is it no is working I wouldn't want to upscale this one uh latently but uh it' be a good candidate for topaz I think I can't believe this is this is working so it says going to take five minutes to do I mean that's about as long as a deorum animation takes it's working 16 seconds per iteration this may be uh speed up by more gpus is a 3090 I mean but when it says one iteration it's doing 280 frames right I think or is it doing 12 12th of 280 frames I have no idea I guess it does the whole animation at once uh yes as far as I know I can um uh I posted a link to one earlier on my [Music] Twitter uh somebody asked if it could be run uh someone asked oh wait uh it's um comfy UI uh collab it's by cam uh yeah cam Cam's got one here you go as far as I know you just blowed that up like uh like in the comfy and or in the collab and then when you're in comfy yeah install the manager and uh like you can just do everything one to one like you would on a Windows install um yeah might be a bit of a pain in the ass to get the models into the right folders but uh should work he's already got a bunch of different uh setups for comfy as well if you just want one to to play with that isn't it I wonder if he's got an animate he doesn't have an animate diff one yet but uh I imagine it won't be long Cam's really fast with it rer render render render render render render render render render render oh yeah computer's cooking now holy cow clean up some of these damn Windows [Music] eh yeah Andy yeah a big blender nerd too this is this is like heaven for blender nerds like once you figure out because it's just like blender right they're colorcoded like you can't you know you can't plug a a float into a vector and stuff so it's like well you can but you know what I mean just tells you what you know what goes in and what comes out and it also does that thing where you drag out and let go and you it gives you like you know the most common examples of what you'd want or you can SE for the you know the node you're trying to go I wonder if the vae is going to be able to handle decoding 280 frames without just exploding I can't believe it's rendering 280 frames right now maybe it's not maybe it's got an upper limit cuz CU this doesn't look like 280 to me this looks like 1 2 3 4 5 6 7 8 9 10 11 12 13 14 17 * 17 * 6 + 1 2 3 four five7 * 6 plus 14 116 frames nah weird uh this is a 3090 I don't know I I've never used ue5 Beyond just kind of messing around with the the default project um but I know it does have a node system for like materials and and blueprints and stuff like that so yeah I mean maybe it is like this I don't know whoa yeah so we got I would say maybe like aund and some odd frames 116 frames oh that is so cool uh it's in the description uh it's um this one and it matters the other one's not maintained as [Music] well oh oh but I told it to start at 24 start at zero and let's try uh one of the other masks I wish I had made more with PNG sequences uh masks those are the movs uh because the uh I mean I think it might actually work with D XL but the uh all the dumb control net stuff I'm doing right now yeah just kill command line um all the control net stuff I'm doing right now only works with 1.5 um there is sdxl control net nodes for uh depth canny uh scribble and reference I think um but I don't think they work with this workflow yet uh I need some PNG sequences please oh man when did I start making PNG sequences wait was that a 140 frame animation it was okay okay all right let's jump over to blender and just make one real quick because I'm got an idea this comfy UI tutorial has just turned into a blender tutorial it's not really a tutorial we're just messing around trying stuff together the beginning was a tutorial I suppose on setting it up but then it just got a little free chat has no actually I think Runway and uh I think Runway and um and maybe Pika I don't know they they feel more like model scope or zeroscope or like their own trained thing to me uh I mean I don't know like CU nobody knows stuff's blackbox um but uh yeah I have a feeling it's um it's more in the model scope then cuz uh as far as I know Runway just hired the guy who makes model scope uh so I don't know if that means that they're using it or if it means that they want to use it uh could be some combination of all the Styles I I really don't know uh 1024 will be fine uh I'm listening to 80 stuff uh I usually listen to 80 stuff while I'm doing this stuff uh temp we'll call this wave uh YouTube [Music] standard all right let's make a let's make an animated mask that was 140 frame video so I think we can make a 280 frame animation ah I believe in it I Believe in Us I have misplaced faith and our abilities color ramp constant these work best with constant uh lines uh gradients and ease and stuff like that tends to make blurry but I don't know what'll do with uh huh might do different stuff with animate Tiff something to explore not right now uh texture wave add a wave texture so what we're going to do is we're going to animate the wave texture uh on a course of 280 frames we're going to start it at zero and on the frame past the final frame we are going to uh animate it to the value of negative to T so moving the phase offset from 0 to -2 Pi will make it Loop perfectly no matter what we do so we hit space bar we've got a perfect Loop here bring the scale down boom and we got a scanning line in this case because it's bands X we want to change that we go Rings Z or Zed and we add a mapping node and the mapping node we need to provide it some texture coordinates we're going to feed it texture coordinates from the object so it knows where the center is if it doesn't know where the center is it thinks the center of the image is the bottom left but that's not the case we actually want the center of the image to be the center of the plane so if we use the object texture coordinates as the mapping and plug that into the wave texture now it knows to come out from the center so we've got a an expand in wave right now so it's similar to the animation we just made but it's easier um so let's go uh yeah let's sck go Z so one cool thing we can do with textures is add a new texture into it so we add a secondary texture use the same mapping now we've got bubbles so now that we got these bubbles we think they look pretty cool but they are a little too slow for our animation so what can I do um the animation is running from negative 0 to -2 Pi so why don't we go 0 to -2 Pi * 2 and then it'll be twice as fast because it's going twice as far but it's maintaining the same phase offset so two to negative to times two so now the animation will be twice as fast that's probably good I'm just trying to imagine what that'll look like yeah I think that'll be interesting I Mayan if we do like clouds or uh you know stormy sky or something cool I think that's a good speed let's leave that alone um go here export it as a wave texture and make sure it's a PNG sequence because that's what we need for uh bringing them in and we're doing a 512 x 512 animation so I'm actually going to render this at 512 x 512 and that might save us a bit of memory on the overhead uh that's all good and it's a PNG sequence we can stop the animation render animation and it should spit out our little PNG sequence into that folder right on that's our image sequence so let's go back over to comfy and have a look at our temp folder that's where I save my renders wave YouTube this is our thing so we plop it here and we know that it's a 280 frame animation and we want to start on frame zero so let's go storm clouds um uh terrible storm humon Nimbus thunder lightning let's try this now it should render two 280 frames of this and feed it into the QR code control net uh hope that's 280 yeah it starts where it ends beautiful beautiful beautiful I bet this one takes a little bit longer I can't believe it's doing chunks of 280 frames what is that about how does that even make sense [Laughter] computer's heating up now we are cooking so this one says it's going to take 12 minutes so that's maybe about twice as long is a deform animation of this size well maybe not with control net and other stuff going on it's not bad that's 20 full steps I can't believe how cool these animations are I really love the way everything oh so cool yeah dude you just drag any of the screenshots of the workflows into into comfy and it builds the the the table for you yeah so just like grab any of uh it's confusing at first but um pretty much everything that you do with comfy you can just drag the images in so these workflows here like like this you just drag this into comfy like you know save it and drag it into comfy and you're Off to the Races then you just got to make sure the um make sure the models are in the right folders and you've got The Right Stuff selected and just works the link in the description that I've told you seven [Laughter] times the link to the anime diff workflow that's in the description below I linked it up there too the kadink kadink cad dink kinkad dink this thing is cooking it's working though yeah yeah you literally drag the picture into the app and let go and it [Music] like it's such a great workflow and anything you make in com you can just drag it in and it rebuilds the nose structure with the prompt and the seed and all that crap uh no uh just add more of these anime diff combines plug the uh image into them and then in format there's more options uh if there aren't you might have to go to manager and install missing custom nodes and then maybe restart comfy um might just be something that's missing or maybe you need it's possible you need FFM Peg installed in your system to make those work I'm not actually sure they might rely on that on the back end but they're in the anime diff combine uh [Music] option yeah I mean I didn't know either when I first started I just uh uh I just started rebuilding the node structures from the screenshots and then I was like it says right on the page it's like you can just drag the screenshots and get the workflows and I I just missed it I wasn't even paying attention I had rebuilt three or four of those node trees by myself before I realized you you could just drag the image in I mean it's a good way to learn what's happening though to rebuild the node structures on your own but you don't have to [Music] 50% done not quite almost I know there's nothing more captivating to watch than someone's render window I need to get like a little mini game to play in the corner or something while this is happening for just like uh like on Tik Tok half the screen it'll just be like uh Subway Surfers or something Mario 64 speedruns in the corner it's not a bad idea actually all right while this finishes I'm going to stand up and walk around and I suggest you do the same take a break we've been at this for a while um I'll be back in about 5 minutes stretch your legs or don't I'm not your mom see you soon render render render all I do is render feels like that sometimes oh I guess we're back I don't need to have that little text up there anymore I stably diffused my apartment do you like it it's very fake I've got a TV on the wall and some pictures and a really ugly couch I'm going try to remember to make a new apartment for every stream but I forgot this time slide I got from Olivia yeah next time I'll live in the Shire would be cool to have a little hobbit house we're getting close I can't believe this thing's just chunking 200 80 frames every go is that what's happening or is it like doing like so many frames per step maybe what is happening anybody smart here that knows what anime diff is and how it works I'm curious it would make sense to me that I'm doing chunks of the animation like Step One is the first whatever you know 280 divided by 20 frames would like the times kind of make sense for that but I don't know does that mean long animations are going to look like crap it's hard to say well we're decoding what was the prompt oh yeah storm clouds just no preview at all till it's done that's hilarious assembling oh yeah it might be the same seed I don't think I changed it it's different prompt though so the thing is if you try to run the same thing with the same seed it just doesn't run like it's it knows not to just waste time and you know run the same prompt with the same seed [Music] whoa what the oh my God that is unreal hey Wendy what's up this is crazy holy cow yeah yeah I think tornado's a way to go I can't believe how cool this looks okay uh yeah guess we should try a tornado you know what I'm going to try a uh I'm gonna just like move half of the frames out so that it's uh you know it takes five minutes to render instead of 10 just because like that's a long time on stream um so let me do that real quick let's just do the first like 140 frames which is I think yeah 140 so let's just grab these ones and put them in uh second half hopefully it won't recursively grab the whole folder so let's uh replace this okay oh I don't even need to do that I could just load 140 images nodes all right uh oh no go back to Oiler okay so let's try um tornado um high winds uh Devastation uh flying debris uh lightning and we don't our we don't want our tornado to be NSFW or nude ideally Okay C I think this one I think this one frame interpolated to 24 frames per second is going to look wild like I love the way this one looks but it's a little too fast with the some of the motion maybe not I don't know this is really really smooth I love the lightning in the the way the clouds or the the waves Splash oh my God that is so cool yeah did 140 frames nice it should take less time that on there pop that on there and then we can see everything hey pixel yeah QR codes with uh animate diff the QR code monster with animate diff works really good works really really good how do you get the previews uh that's the animate diff combine uh so you pull that out of the vae decode at the end sorry I missed your question there um you just pull out and then you want to add a uh animate diff combine and then uh you can select the format you want and then you can select the frame rate you want and just make one for U make one for all the different ones you want like I just made three one that does an eight frame gift 8 frame per second GIF 12 frame per second M4 and 24 frame per second M4 and then I just made that a node template so I can drop it on the end of any of these um any of these workflows just like making a group node and [Music] blender oh I love the way the motion moves it's so cool like that wave that comes over the bottom uh right uh right here oh man let's make a new mask all right I bet that'll make a pretty freaking cool animation let's try that one I will say it's nice having the vram to be able to do this and run blender and stream and do Nvidia broadcast did you install the comfy man manager and do all the install missing nodes and set it all up the with the the instructions and stuff or did you like install it manually I think there's some stuff that happens when you install The Comfy Through the batch file that maybe you missed a dependency or something not sure but in the manager if you go to install missing custom nodes and then restart comfy it might fix it uh on my Discord there's two uh there's two workflows you can try one of my workflows and see if that's a problem although they're control net workflows I should drop this uh well the simpler workflows are up there it should work I don't know why it's not working comes out of the a decode into the uh animate diff combine uh I used the um FTM 8400 EMA pruned one the uh yeah this one what did that game Jam fry your brain did you guys get the game done you'll have to send me a build no you didn't get it done or no you won't send me a build a sorry to hear that man a lot of work though yeah send me the broken game okay here we go vae decoding let's check out the tornado oh it's crazy how similar it is oh yeah same seed duh actually we talked about it I forgot to change the seed that is cool though man those are cool I think that one Loops almost loops and enough God that's cool okay let's um let's try that new one I made do 140 frames and we'll change the seed this one let's try uh uh cellular growth microscopic life um nanop punk uh electron microscope uh living organisms see if we can get some cool science it's crazy there's absolutely no preview until it's done a I just love the Motions so much in this it's so trippy 6 minutes for 140 frames not horrible considering whatever crazy stuff is going on here yeah I'm curious what it does with a human when the uh when the QR code stuff is so weird let's try that prompt I'll wait till this one's done to uh run it the Curr code control net works best with black and white the more gray you go the less um it doesn't like make a difference until you hit a certain um point in the gray uh basically everything that's gray is blurry everything that's black and white is solid so you could use little like bands of Gray to add blur in between white and black lines but uh yeah QR code just doesn't react very well to to Grays and and gray scale and and colors yeah it is control no it is control code QR net or QR code control net see that's why it's so exciting yeah I am doing QR code control net that's why it looks like this let's go I think it's closer to like seven or eight minutes by the time it's actually [Music] [Music] done [Music] oh I wonder if I move the strength down to 0 eight like I do in deorum if we'll get some uh some cool stuff going on cuz this is at 100% right now strength on the weight uh of the QR code control net so I wonder I wonder I wonder I wonder let to try that on the next run oh we're still on the animated one too wonder what this would look like with um like a super photo realistic uh uh checkpoint I wish I worked with SXL I would totally Loop in the uh the new Ikea Laura that is amazing it's so funny uh uh what did I make that with oh I made it with automatic you puts text to image images the day yeah it's so good how do you make this go away go away Windows you suck why would that preview show up all the time I hate windows so much but yeah the Ikea Laura is amazing uh Laura is a um uh Laura is a like uh it's like a layer you can drop on top of your uh model to uh drive it into a new place so like allur will like like say you have just a regular image you want to make it look like Studio GBL you can run a studio GBL Laura on top of your able diffusion thing and put it in the prompt and then it'll uh it'll incorporate that at the end to guide the result towards what the Laura was trained on so you can use luras to train on like really specific things so if you go to citi.com and you search for lauras they're on the um in the little thingy under models Laura all these are like specialized training sets and they're smaller than a checkpoint they're like you know anywhere from like 50 to 200 Megs and they have like very specific functions so you can use them to and you can mix them together too so you can you know um really extend the capabilities of uh stable diffusion like I trained Aur on my face I trained Aura on some of my art you know whoa oh this is so cool now I want to see what it does with a humanoid so let's try this uh this prompt here 52 52 yeah everything's cool same uh same input I can't can't believe how cool that is huh man the people who ignored this stream on Twitter are going to be really sad they missed this I wonder what the top limit is I wonder like I wonder how many frames like I wonder how far we can push this can I render for an hour and do like a thousand frames will it work will my GPU catch on fire find out next week on the lone diffuser I like how they turn into snakes a little bit there at the uh yeah yeah that one I think I'll try offline in the winter yeah just let me do this on the cluster just give me cluster access I'll do anime diff research for you so we got control on that access so we can use any control net the POS one works really good this one works really good this one works really good now what if we can stack control Nets like we do in everything else so I got the pose and I've got the thresholded silhouette of the thing that's happening uh I've got a depth map can I do a three layer control net run with animate diff how do we stack control Nets in this setup uh can we stack control Nets yep no yeah [Laughter] no yes no yes no uh no no is it like a control net sequencer or pass through huh well I don't know question for a smarter person how the hell long would a upscale take of 140 frames twice was four times 20 minutes yeah maybe like 25 minutes to upscale one of these latently might not be worth it considering the way these upscales look I mean the flowers look good but they kind of look like deviled eggs up here almost done I'm wondering if the humans will just be like the human face will just get like separated and turned into new faces and CU like this stuff's all very abstract and gooey it be even faster on yours bud yeah like right now just go back to the beginning of the video and set it all [Music] up right on I got to try that too so it definitely works with a gigs of vram then take me I wasn't sure [Music] here we go [Laughter] whoa oh my God oh looks cool as hell super sick this is wild underwater symmetrical octopus was that a prompt whb are you just describing what you're seeing it's crazy okay cool cool I'm going to run this other prompt hey webn uh we're using anime diff incorrectly to Great results I passed the uh QR code control net in in um to use like a mask one of my masks to drive the diffusion for anime diff results are pretty [Music] [Music] wild [Music] [Music] let try to load all the videos we made there we go whoa big ones are big it's all the crazy stuff we've made tonight a lot of this was before we were doing the control net that's control net render render render uh it's like uh it can run open pose on a folder of images that's what I was using to make the open pose loader that would load up uh an open pose from a image sequence so I like just grabbed a video of a dancer from pixels and Ed the the little thing to make an open pose uh sequence to run the dancing so the actually is coming up right now that one right there that that's from uh open pose that we pulled in from aexel video so much stuff going on here yeah it's it's on my Discord actually if you go to um uh p. XYZ um the link to my Discord is just uh let me hide this video Bo it's this guy right here just click on that and uh jump on over and I dropped all the control net stuff on my uh in the general chat just grab it there I don't know why do you [Music] hello I have handbrake what do you want me to use handbreak [Music] for crazy [Music] no absolutely not sorry it's trade secret literally figured it out on stream C uh where's my chat with [Music] you there we go yeah that's wicked I saw that that's awesome yeah you can just like interpolate that to you know make it smoother or whatever [Music] well thanks for hanging out yeah absolutely have a great night oh this came out cool wavy lines and spirals it's still like um it's still changing um sort of like uh context every 16 frames um because of the context length but I don't know if I can actually turn that up or not so just leave that alone I I kind of like it honestly it's it's sort of like jumping the strength and deorum having it like click into some new stuff very fun okay let's try uh yeah let's try a different mask sure we can try just a simple like uh just bands maybe yeah Rings we go Z out yeah and we'll go Z in Tower times two go in there we go we'll try that with the uh oh actually I want to switch it yeah yeah see temp wave Distortion render that [Music] and wave Distortion back to [Music] comfy try this the frogs with the uh waves sure why not wavy frogs that's good that's good that's good that's all set 20 okay I really like this one this one's cool all right cool let's try that [Music] prompt we got our wave Distortion yeah it's working all right cool because we're doing to times 2 when I do these animations at half like half the frames they still work because we're going through it twice this one should be cool though whoops how do I unselect that yeah there we go oh ah I forgot to change the seed how far in are we uh yeah I'm going to try a different seed let's try the best number on Earth it's been a marathon stream I think we're coming up on four hours yeah geez who streams for four hours can you say hyperfocus hey what's up how you doing bud good to see you love your stuff so much yeah it is fun don't you love watching progress bars me too whoops I keep grabbing the wrong window I'm trying to get everything on the screen at the same time come on those ones are so cool oh right on yeah no comfy UI is definitely super fun this is animate diff using um blender masks as a control net QR code inputs um similar to the deorum stuff I've been doing lately but uh wanted to try it in anime diff and it turns out it's freaking wild I didn't actually come out planning to do this it just sort of came to me while we were working on the working through the workflow and scoo here we are yeah it's crazy especially when you're using um the control net mask like these ones that are playing right now they they're so smooth um this is um animate diff so so this is uh this this node set for um uh automat or comfy definitely recommend uh installing the comfy manager to to handle all this stuff holy cow lots of new friends on the Discord render render render render render render render render render I need to put like clickbait porn titles for these YouTube videos like 40-year-old man uh rendering on a hot GPU I don't even remember what the prompt is for this one roses and teeny frogs puddles raindrops waves streamers water Smoke by artist Carol back all right let's see what we got and a bathtub everybody keeps their GPU in a bathtub with of vegetable oil right oh I'm the weird one okay vae decoding here we go comine whoa what the does he hop from here to here oh come on that is crazy okay let's change the seed and we'll do the thing that I forgot to do which is reduce the weight from 1 to 0.8 because that's the weight I use in deorum so maybe we'll get a little bit more outside ideas going on but this is so cool I can't even tell where the loop point is I I don't see the loop super cool all right what was the large pile of whipped cream and you wanted to see a spiral so let's see that in a spiral for this particular thing I think fixed is better but um yeah I was doing ones with randomized earlier and they're cool too so I don't know more research is needed let's try that's FX nodes by EOD love them oh yeah all right let's try some spiral strawberries not to brag or anything but oh he got rid of it LOL yeah he made I made the um I made the demo uh I made the demo for effects [Music] nodes everybody else listens to Madonna all they do this right just me oh let's get the well that's a cool texture now how to animate that almost done interesting so sign smoother and then reverse it H maybe the other way around if we do that like Tangled vines or something thing growing Tangled Vines let's try it temp tret all right oh we're rendering let's see so this was a large pile of whip cream and strawberries melting and running over the table with a spiral animation nice ah it's trippy I want to eat that yum Quantum dessert I keep saying I should try the realistic model and then I keep forgetting to enable it let's try um realistic Vision I want to actually run that same prompt but with this weird uh weird video I just made tret YouTube tret oh it's so weird the way it like Globs out I just want to run this prompt again on realistic vision sh don't like it I don't think I have the latest version of realistic [Music] I'm curious what it does with all this detail I have a feeling it's just going to be goo but you never know it's actually made a pretty cool image in itself all right I need to stretch my knees you should too you don't have to I'm not your mom but I've been streaming for four hours so I'm going to stand up and walk around a bit and I'll be back in five minutes oh thanks for coming out n bid art really appreciate it have a good night night that mold doesn't look healthy [Music] H yeah gross okay so too much detail is probably a bad thing but at least we know that now that is disgusting all right oh something crashed wonder why it crashed let's see if it crash crashed where's that last prompt I thought you sent another prompt where am I not seeing it oh there it is prompt Dre there we go Dre cat dog tulip all right let's go back to blender real quick and make a mask that isn't insane I'm not going to save this one it sucks we've got our wave out uh let's do wave in instead uh Tow Times [Music] Two get good no I don't know let's uh debug it why do you think that is that's kind of cool that might work let's try adding in a uh I wonder if smooth tiles will look cool folded into that kinda or the other way maybe where trying to get a nice 5050 mix of uh no not yet uh trying to get a nice 50/50 mix of black and white too many Jagged edges and weirdness um actually maybe two oh I know why that's happening yeah that makes sense oh that's interesting that's probably crazy enough all right let's get that going see temp uh I don't even know what to call this uh let's do smooth tile wave texture tube render go that make be your Laura temp the YouTube smooth to all wave texture yeah there you go Jam that in there 40 God I hate this so much all right let's see what the Drew cat dog tulip Tom Bagshaw does let go blow all right uh what are you saying okay so it didn't crash crash it just oh maybe it did it's rendering let's Let It Go God that's awful can't stop watching The Moldy moldy strawberries w we accidentally made moldy strawberries cuz my uh anim was too detailed we're learning especially a 512 by 512 I don't know what the hell I was thinking oh also I set my strength to 08 maybe that's why it's less coherent and white we'll know when this one's done oh that's even worse gross for render render render render oh hey webn it's uh um I5 uh 12 gen and uh 390 390 is the only way I'm able to actually stream this stuff trying to stream this on my 3070 would have been nightmare [Music] decoding so if this is all effed up um I'm going to assume it's because the strength is at 08 instead of one so we'll try swapping that out for the next run because I think that'll uh solve um if that's the problem doesn't seem to be the problem though Jesus Christ well that that's certainly a thing that's happening I'm not high enough for this that is Jesus that is something okay so 0.8 maybe not a bad thing definitely getting a little more like wiggle room in the dream I wonder if I crank that back up to one if it's like uh you know super serious about sticking to the the QR code uh temp nope wrong temp temp um I'm going to try the jail cell one I just made curious about that [Music] one all right 140 that's fine did I go to Tower or tow times two times [Music] two all right uh would be a good prompt for the [Music] the thing I just made I wonder um let's go with uh come on the um the UI is pretty good but the fact that you can't uh move the comf youi Thing by holding oh you can now oh they fixed it oh my god when did they add this this is so much better okay so for those who didn't know comfy you never used to be able to press down in the middle Mouse button to uh move the stuff around you always had to like click outside of a node to grab it and move it which is like in blender you just you have a button like middle Mouse button or whatever and no matter where you are you can press it and drag the thing this has a couple caveats like you can't do it in the dialogue windows and you can't do it inside the images but you can do it inside the nodes oh my God that is a game changer this my biggest pet peeve about this app all right uh where were we melting liquid flowing to the bottom of the drop uh I'm going to go back to one I just wanted on one uh and then yeah cool I'll try one at 0 five triangle there we go uh what I pretend I know how to use that later h well I think that's a winner all right uh YouTube uh Cloud growth I don't know oh my chat was stuck I missed everything you guys said yeah my last Mouse the middle Mouse button was totally screwed on it it was super annoying like it was really inconsistent the button press it down I ended up just getting a little half decent Logitech mouse and I'm much happier now no more 8 Microsoft mice this one is so good all right render render render space bar and mouse movement ah nice I wish my space bar was less sketchy that works right on images too that's cool I like that we everybody's nauseous now yeah I like that a lot gross put the uh lament configuration from Hellraiser there's a Laura for that should pop that in there for the ornate box oh I got latest version of realistic Vision let's see if it shows up in the [Music] thing refresh hey try that for the next prompt although I think it was it's art so let's try the animated one and or an eight box and [Music] then uh what was he doing I was going to change it to rev and contest link 16's good maybe we'll try the other oh oh it's done wow well that is pretty cool jeez I think I like anime diff guys I think I like it the White's definitely Whiting so maybe thin whites are best let's see what was that last one I just made yeah let's try that one with the spiders rev animated yeah yeah it's really cool all right all right well I'm coming up on 5 hours so I think at 5 hours I'm to cut the stream um so I I'll I'll jump off at 10 p.m. so got about 8 12 minutes left so I'll run this prompt uh maybe I'll do I'll do two more prompts and then I'll uh I'll jump off this has been really fun and really eye openening and uh yeah hopefully uh everybody tries this this stuff out and we get some crazy stuff wasn't expecting to come across this tonight but uh most of this stuff is Happy accidents right this video will be available um like it'll just be up if you just go to live on my channel it'll be there you know till a time so if you miss something we can run back and have a look at it and hopefully uh the chat will be up like in a day or two so it won't sound like I'm just talking to nobody um the live chat will show up like in a couple days so it'll be able to see in context what we were talking about so if you miss stuff um and you need links or whatever and I dropped it in the chat it will show up uh just might not show up tonight YouTube's really slow for that and uh yeah this is awesome anime diff is really cool I do a lot of deorum but I'm going to be doing a lot more anime diff also I think tonight I'm going to try and run a you know 480 frame one you know see uh see where the upper limits are I'm curious these ones we're doing now they're all 40 frame animations which is pretty cool and it's a fun way to sort of extend animate diff's um I [Music] mean maybe I don't know if there's a hybrid most of the comfy nodes are not for animation there's a thing called AI nodes uh for deorum nodes and I haven't tried it yet um I don't know if you can even wire it into this because the way it works but you could then just Optical flow your stuff later I guess can't really do it to a mask like the swirly donut that appears yeah this is C monster with masks made in blender try yeah it'd be a good one to try yes no I can um the only reason I'm doing 512 x 512 is for brevity on the stream um uh I just want the I want the renders to take a less you know shorter amount of time so I'm just rendering them small but you can render at different uh whatever resolution you want I was doing stuff at uh 768 by 512 to test it and it was fine um I think I I did one earlier that was well I mean even the some of them have upscaler runs too so you'll run the thing and then yeah I got to run one of these through an upscaler that reminds me anyway um so yeah the you can render It Whatever resolution you want it's just right here in the um the empty Laten image that you create when you start you just change the size of it that's sort of like the canvas for what it dreams well not sort of it is yeah I know I know exactly what you mean whoa this is great honestly I might just go stretch my legs go for a little walk and just be off the computer for a while and I might just actually jump right back on YouTube in about an hour or two to uh continue making this stuff cuz um yeah pretty Blown Away um but uh thanks for everybody for hanging out this is um this has been really cool um I don't know how to do that I think you can if you like right click on it there I might there might be a way to run it from there but might not because of the way that the nodes work they're kind of nonlinear so I don't know I think they have like start order based on when the queue starts versus the actual chain of events so I'm not sure you'd have to ask somebody or check a tutorial I have haven't been able to figure out how to like get it to run from a point yet this one's super trippy all right thanks everybody I really appreciate everybody coming out and hanging out and sticking it out with me a lot of you been here the whole time H that's amazing um have a great night and uh yeah I think I'll start a patreon for Hangouts like this because like uh yeah it'd be really cool to just you know hang out and and make some cool stuff um yeah I'm going to um cut a uh uh I'm going to cut a video together and drop it on Twitter and uh all that other stuff so you guys be able to see all the generations I just going to upscale them and stuff and make them look good so yeah thanks for hanging out everybody and I'll make sure that I'll tweet that out tonight so you can see all the crazy stuff and uh I'll make a Youtube short and stuff like that but yeah thanks for the inspiration and

Channel: Purz

Views: 10,853

Keywords:

Id: GV_syPyGSDY

Length: 297min 55sec (17875 seconds)

Published: Sat Sep 30 2023