Blender Animated Masks - Shader/Geometry Nodes for AnimateDiff Animation in ComfyUI

Hello friends I think I'm live feels live how's everybody doing tonight here we are back again on YouTube as usual uh tonight I am going to stick around and blender for quite a while we're going to make some stuff and then uh we're going to take that stuff and try to use it as input for AI animations so it's sort of like a workflow uh I've been showing this workflow quite a bit so probably at some point I'll stop introducing the workflow and just start working but you know for now uh it's probably best to explain what I'm doing so it makes sense um this is uh I just made a bunch of uh little shapes with some procedural modeling with geometry nodes uh and then uh just did some some fun um displacement stuff uh instancing on points to make them float around and uh you know uh uh make them Loop so that uh when we make Clips they Loop and then hopefully we can cut Loop and clips together and make uh music videos out of it and stuff like that so yeah uh you can make this audio reactive you could uh just make loops like this like I'm doing and then cut it all together later um there's also other things like uh you could generate a bunch of footage with AI and then take it into like touch designer and uh make an active playhead that reacts to the you know amplitude of the tracks you're working on or whatever to drag the video along more when there's more sound and stuff as my friend tenia has been really working on that a lot lately if you check out his Instagram he's got some really amazing stuff on there uh just see if I can find that link super quick it's awesome it's a really good way to use this footage that we keep making uh tenia yeah uh I'll just link the post because um I can't play the audio because I don't want to get copy stricken but yeah uh check that out when you get a chance that is basically using touch designer to make it drag along the video based on different things in the music maybe it's the amplitude of the bass or the you know the intensity of the music or you know certain frequency ranges and things like that and I think he's got other effects going on as well on other parts of the track so you can see where this stuff can go you know we're just we're making the building blocks and then we're going to sample it and remix it into new stuff so I think that's the most fun part of AI is making stuff and then cutting it up and twisting it and making new stuff with it using it as samples and loops and and and and remixing just like we do with audio so yeah uh let's get started I'll uh zoom in on this you guys can see what these what's going on here but we're basically got these cool little things we just find a place where they're not bumping into the camera and uh yeah these Loops are all perfect Loops when they get to uh frame 60 they loop back on themselves so uh you can record it from a bunch of different angles and then have yourself a bunch of Loops um but yeah I'm going to rebuild this whole thing from scratch so you can see what I'm doing but yeah basic idea is we're just going to uh have fun with blender and make some cool masks and uh yeah if you have questions if you've never used blender uh and you don't know what the hell I'm doing go ahead and ask and I'll try and answer all right let's get started we'll save this as weird Helix thing all right let's have a new blender project so I'm using blender 4.1 which is the uh or 4 yeah 4.1 Alpha which is the uh daily build of blender uh and the reason I'm using that is that it has EV next in it which is the render engine the render engine EV next has uh support for ambient occlusion which is the Shader node that I'm using to make it look like all of the shapes are lit from every angle uh I'll explain this when I pull the Shader in so you can see it but uh yeah if you're trying to follow along you're going to need the 4.1.0 daily build on blender uh.org uh so uh that's uh if you go to the download page you need to go to builds and then uh I think it's just this one right here the 4.1.0 alpha so download that uh and then when you run it you'll have access to uh EV uh next which is the one that doesn't say Legacy so that's e next and yeah I'll I'll show you the difference in a minute and when we get a shape in there actually I'll show you the difference now uh we'll just throw a quick Shader on this [Music] Cube we already have a Shader on the cube so let's have a look at our Shader editor switch this to rendered mode all right so right now it is just uh uh principled material which means it reflects light with roughness and um uh metallic so you can you know decide on your your materials and and that kind of thing otherwise we I I'm going to use an emissive texture which doesn't pay attention to what light is hitting it it just emits light so uh you can go any way you want about this and it's all different ways of lighting things and stuff and like if you're doing um uh depth masks and stuff it's good to have gradients but if you want to do uh um like QR code masks and stuff like that you're going to want to use like emissive shaders because they're like very uh stiff like black and white like you can set constant colors and have them react accordingly and and be black and white things as opposed to stuff with gradients but yeah uh so let's uh kill this and add an emissive Shader so that's an emission Shader you can also just click this and switch it to an emission Shader um and what we need to punch in here is a color ramp all right and the other thing we need is an ambient occlusion node and we're going to go on the AO out of the ambient inclusion node and I'm going to have a quick wonder why this isn't working feeling pretty dumb oh I think it is working there's just not enough stuff going on uh let me change this to an ecosphere it might need some uh bear with me here sorry I haven't come at it this way yet I want to let me inset these faces oh sorry friends let's get this going basically I'm just got to give this some depth so I can see if the AO is working so we need to extrude faces along normals yeah I'm just basically trying to yeah see this is what the ambient occlusion does it adds this sort of shadow where there should be shadows and it lights it from all directions so it's like this very static way of doing lighting um but it works well for depth maps because you know these things look like depth maps [Music] um so yeah uh if we're using this stuff to animate then this is kind of a really cool way to to make fake depth maps for using control net um which we can see how that works later but yeah I'm not going to uh model it like I did here I'm going to actually use procedural modeling um so let's add a cube and uh go over to ometry nodes and we want to add a geometry nodes to the cube and first things first we're going to set material to that uh material that we made again this has no edges so there's nothing for it to light that's why it's all white all right in Geometry nodes uh we have all kinds of fun stuff but we can add where are the meshes we can add a mesh primitive uh ecosphere so this is going to give us a little ecosphere um this is hard to let me switch it to wireframe mode yeah solid there we go so this is giving us a little ecosphere we can use whatever nodes we want here this is where we do instead of the shaders we have access to the geometry so we can do stuff like uh extrusions and selections and all kinds of fun math stuff to do uh crazy models so uh yeah check this out let's add a dual mesh it's going to turn it into a soccer ball type situation once we have a soccer ball type situation we add an extrude and then we got our little poke bits after our extrude we add a scale elements plug the top into the selection that means it's going to select the top of each um each point and then it'll scale it down so you have the ability to scale it up scale it down so as you can imagine this is going to give you a lot of procedural control um because you can you know decide what these look like and then if you like them oh that's weird oh cool that's just totally broken nice control shift d there we go uh if you have node Wrangler installed uh which is in add-ons you just type node Wrangler just click that turn it on um you can duplicate nodes and create uh mapping nodes and stuff gives you a ton of node shortcuts highly recommend it um so that was control shift d and I duplicated these nodes with their connections this one we don't have to connect here we're actually going to do this twice right so now we've got two so we can get some really cool patterns going here switch back over to our thing and you can see we've got a all the things in the Shader all things are cool we can make a bunch of puffy balls so let's do that let's add a subdivide surface do how's that look not too bad let's do a set shade [Music] smooth three not bad you always have to decide you know how much subdivision is worth it because every time you subdivide it you make it uh more and more um like uh uh resource intensive on the machine so you got to decide how smooth you want it based on you know how much your computer can handle so uh if you have a pretty good video card you can actually probably do three for all of them and then keep moving them around but you'll notice when you we start making more and more of these objects if they're if they're subdivided too much they you know a little much yeah you can see by just playing with some of the just playing with some of the variables you can get all kinds of really cool stuff so you imagine you can tie any of these values to audio so right and then you can make an audio reactive mask and you're Off to the Races uh this is kind of cool very accidental very cool I wonder if this would be cool to just like go 60 frames we're going to do short animations so that I can do quick renders later um but as a result it might be too fast so let me think about how to handle that why don't I do 240 frame animations and make it sort of loop twice over the course that's pretty long let's do 60s all right I'll just make it work okay um I'm going to use a node from uh Arendale uh arendelle's node tool kit but uh actually I'm trying to think of how I would do that without his node I'm just trying to get this back to sanity oh that's cool I like that yeah as you imagine you can get absolutely lost in this stuff for hours because it looks so cool um you know so go ahead get lost who cares crazy trying to find the right yeah that's pretty cool all right cool so um uh we could just animate this or we can make a whole animation with a bunch of these uh so let's make a bunch of these and then make one of those morphing animations that I was doing before so uh after you have one made that you like uh go ahead and click this little Shield up here and then click the copy button next to it and it's going to make a new uh geometry node group uh actually you don't need to click the she the copy yet but you do click the shield then uh click on the object you want to duplicate and shift d to duplicate it just click to get rid of the selection and then hit uh Y3 sorry gy3 got to select it gy3 why is that not working what's going on here there we go gy3 and that's going to move it 3 meters on the y axis G for grab Y for the y axis 3 for 3 m we're just going to do that over and over again to make a bunch of different shapes so for this second shape you want to click that little copy button and that's going to make a new geometry nodes uh patch for this one so now you can make changes on this one and it won't affect this one so we can have a whole bunch of different spheres or whatever so uh this is an ecosphere uh we don't need to be be an ecosphere we can use any of the mesh Primitives so let's go back to mesh and Primitives and let's try a cube see what that looks like that's kind of cool and then let's play with these a bit let's try and get them around the same size so yeah as you can see just by selecting a few oh that's pretty cool interesting and then uh I'm noticing the Shader could use some love well hit home on your keyboard to find the uh find the stuff once it's selected there we go there we go cool so yeah just switch between Shader node and geometry node editor when you want to work on stuff um and yeah we'll try and find a happy medium so that uh they always look good with the default Shader settings so you don't have to have multiple shaders that's pretty cool spiky little thing same deal control D select that new one gy3 um probably make sense to name him let's go with hex virus what's this H spiky flower thing and we don't have a name for this one yet so let's let's play uh oh yeah right we have to do the shield and copy on the new one and then I can edit it without messing up the other ones just remove the dual mesh see if we can get something cool whoa crazy what if we go backwards on these that's kind of cool whoa don't crank those up too much I guess oo I like that that's very cool yeah just slide them around so they look cool right uh the other thing I need to do is if they're not scaled up correctly uh you can just add a uh I'm just trying to think of where to add it I think right before the subdivide let's add a uh scale elements like just another one of these and then instead of plugging a selection into it it'll just scale the whole thing so let's have a look at it on the x- axis and we'll just try and make them all around the same size so that they don't uh you know they look cool all side by side and whatnot same deal with this one just add one of these oh that's weird the copy and paste is just not working right now scale elements Boop and then bring up a little bit yeah cool so there's three let's add another one control d gy3 maybe four makes more sense let's just move those out a bit so that'll be four eight 12 yeah little easier to see all right let's do the shield and the new one okay let's get rid of the cube and let's try something else uh mesh primitive let's try cylinder let's turn down the vertices to like four six two side three side elements oh that's kind of [Music] cool crazy you gotta love procedural modeling you don't even have to know how to model you just have to know how to slide sliders these are all pretty cool so for the radius and depth we'll be able to control oh that's neat whoa that's cool very cool all right let's try that again control D gy4 this time whoopsy doodle oh control Z is not working either oh we're having a rough one with blender right now mesh Primitives [Music] cylinder top oh jeez four was it h try to see if I can remember what we had here four or three I think it was three depth was that there six verts yeah yeah yeah six verts cool think that feels right does it feel rer more rer okay uh this one we're going to copy and paste yep and then yeah get rid of the cylinder uh let's do a let's do another ecosphere I like them oh that's trippy already all right let's uh set all this stuff to what what did I break that is interesting well that's pretty cool okay all right well these are pretty neat um this is probably enough to get a cool thing going so yeah uh if you want to make them yourself just uh here this is the take a second and screenshot this uh this setup and rebuild it I'll zoom in on it in a second just get it all close together so you can see it Scooby booop doop doop doop there we go I'll just leave that up for a few minutes or whatever so you can make a good screenshot of that um that's the basic setup for uh the mesh stuff so you can just plug whatever mesh you like into the front uh play with the settings play with that final scale at the end there to make it fit uh decide how many subdivisions you want on the surface like how smooth it is and that the more you increase that the more it's going to affect your uh uh like performance uh when it comes time to like make these things move and stuff but for now uh yeah just this is the setup uh use it um to make stuffff and then I'll just show you now we got all this stuff so let's select them all so we're going to hit M and we to M to move to collection and we're going to make a new collection called shapes and we're going to hide it all right so now we have all those shapes in a collection in The Ether and they're they're safe and they're ready to go so what we're going to do is now make a thing and then instance those uh meshes all over the thing so let's create a cube this cube is just a placeholder for the the geometry nodes so let's add geometry nodes and then uh all right uh we want to uh I'm trying to think of what the like easiest easiest animation will look like let's do a grid so we can visualize this so you can understand what's happening okay uh you know I'm going to also make a join geometry so we can always see the grid then I can turn it off later once we've built it so you can see what what it is we're doing but right now we have this grid right and we want to take that grid and then on all of the vertices which are uh of course they don't show up here anyway this is a verticy this is a verticy this is a verticy this is a verticy um on those points it's going to generate stuff the more we add the more points we're going to get the more stuff we're going to have all over the thing so I'll show you what I mean okay we're going to instance on points our instance is going to be a collection info node and then that collection info node we're going to select shapes we're going to separate and reset the children that basically makes it so that it only selects one per instance pick instance that's make it choose an instance all right and then we're just going to scale them [Music] down okay so does this make sense we've put we've put a thing on each point so if we start adding more vertices we get more things yes we increase our size right 4 16 so let me get rid of the uh the plane because we only need the yeah there we go we only actually need the the things so yeah you can see now we have a grid and we've instanced all these things onto the grid so one of the really fun things we can do with the grid now is uh I use Arendale toolkit for this but you could probably build your own uh texture displacement thing uh I just use erands because it's awesome and aon's pack is great and I don't get a kickback or anything but uh I think if you want uh Aon toolkit gumroad yeah highly recommend these uh tools um it's just like a huge pack of like awesome tools and uh I don't mean to brag or anything but there is a pers node I'm just saying I'm just saying um yeah uh awesome stuff there's a whole video here on what they do I think yeah every etk node what they do so when you're ready to get more crazy with uh uh geometry nodes I this is really fun lots of really fun tools here oh yeah and the oh yeah pasted it cool all right so I'm going to use a tool from Arendale toolkit called my brain will find it displace texture all right all right so we're going to displace the grid with a texture if you've seen any of my other streams you'll understand why we're going to do this because we can Loop a wave texture and then have control over what it does so watch this we'll take our displace texture we're going to put it in between the grid and the points the instance on points okay and we're going to plug in a wave texture and our wave texture if we remember from other stuff goes from zero to to which is pi * 2 when you go from zero to to it perfectly Loops so what do we do on frame one we go phase offset of zero press I to set the key frame then we go to frame 61 one frame after our animation ends and we change it to in my case I'm using negative TOA but you can use to it's fine or Pi * 2 it's the same it's the same calculation to so negative to negative 6.2 is cu it's 3.141592654 blah blah um yeah so hit I to save it and now it's going to be really fast because our scale is high but it will Loop so this is going to actually perfectly Loop so you can just set up a camera and you know you've got a a looping mask already there's you know way more we can do with this go away steam all right let me get rid of my uh overlay because it's ver annoying okay so what can we do with this let's turn the scale down look at that it's a wave look at that it's really a wave so what looks cool as hell is just a simple wave with a little bit of distortion look at that you got a perfectly flapping blanket of cool stuff what's another cool thing we can do well maybe we can make all these little uh meshes rotate the whole time so what do we do there we only want them to rotate 180° CU they're all like symmetrical so when they hit 180 they'll be they will look the same as when they at zero so we get a full rotation uh each Loop by saving half the time which means they get to Loop a little slower which means they don't look insane spinning around really fast CU we only have 60 frames right so let's try 180° so same process say start at zero uh let's pick a direction we want them to spin I think I want them to spin that way so on the x- axis so we'll start at zero and we'll go to the final frame + one 61 frames and then 180° and press I to save the key frame so now they should Spin and spin pretty cool eh and so a grid is just uh literally a plane um so like things get a lot more interesting when you don't use a grid so this is pretty cool too um you know nothing wrong with it but uh and you know if you crank up the Distortion more you get even more interesting uh motions uh crank up the Distortion and uh turn the scale down even more get some really cool ropey transitions it's always to look at them dead on too but yeah you know once you kill the background of the world and like that already right there is probably a pretty cool would make a pretty cool animation mask um but we can go deeper um so let's uh get the Distortion back to 0.1 let's get a nice wave going all right so we got our wave motion so we can kind of visualize what it's doing I'm actually going to bring this down to eight and change it to uh three there we go a little more diff distance between them and stuff okay so let's try a uh uh Cube now this is interesting because our Cube needs more vertices now the wav is pushing through all the vertices of the cube in a very organic way oh don't do that my bad that would be a pretty cool mask too I think man that's trippy let's put a little plane down so you can see what the hell it looks like yeah as you can imagine just moving a couple things you get some really wicked animations out of this whoa yeah okay that's pretty cool so yeah cube is cool let's try cylinder I was getting some really cool stuff with cylinders [Music] yeah well these are cool so yeah I mean you can only imagine what the AI is going to dream over this stuff so let's uh decide on some stuff and make some masks let's do grid start with grid again got our cool little grid that's I think it was three six no seven eight I think oh that's interesting I think yeah there is a point where they all line up yeah yeah what do we think does it look better symmetrical or asymmetrical with like different set ups I don't know I can't tell it's hard to say I think I kind of like that I'm curious what that'll make Let's uh so another thing you can do is increase the strength of the texture displacement and that's actually going to make them move more you know so you can try to find your happy medium of what you want to move I want them to move like quite a bit so that like there's stuff happening but I want to try to keep it within the confines of the thing I think that does a pretty good job one thing I'm going to do is just pull the camera up a little bit to maybe 11 yeah so let's save this mask uh we'll save it as uh temp I put in my temp folder it's going to save a PNG sequence so let's do uh uh depth uh icons and render oh uh one other thing I forgot to do [Music] is in color Management in this uh render tab it set to agx by default which is great if your scene is realistic but this is not really a realistic scene it's going to be a depth map so I want it to go from black to actual white which means I need it to be standard see how it's like actually white now agx is sort of gray reasoning for that is when you're using cameras in the real world there is no real like like white white white white white light like that so agx and filmic they're sort of designed to mimic the real world whereas we don't want to mimic the real world we want like full zero to one black to white you know uh contrast in our image because that's what the control net needs to work so render animation and we're going to save this it'll just render this out this is pretty cool I like this one I like the weird CL H shell ones those things oh might be cool to do a really close up one of these too like just like uh nine of them or something we'll do that uh as well here and then oh it's done okay so let's uh save that make a new one and uh size one three let's do four yeah and then we'll just and then we just bring the strength down a bit Yeah and then camera oh let's go back to 35 millimeter but then Zoom it down yeah and then let's set it to like 0.25 five that's cool let's move it down a bit camera Mo it down we're just trying to get it into the frame without it cliping out that's cool all right this might be a good mask too but we shall see depth icons too render uh what do I offer on my patreon oh uh I uh patreon is really more of a a tip jar right now um uh the uh uh supporter status uh you get supporter status on my Discord which means you can hang out in a special Channel and uh you know if you have questions and stuff I try to answer those first um and then um uh you'll also get access to uh a comprehensive um mask pack which I'm working on right now of all the blender masks I've been making um so like all the subscribers get uh all the supporters get uh access to that you only have to support once I'll send it out to everyone whoever supported so you don't feel like you have to keep supporting if you want that mask pack it just take me too long to build it so um yeah it's coming I swear okay uh I'm actually going to import the other ones from that other project I was using because they're cool so how can I do that append from weird Helix thing objects cool all right you you you you and you go in here cool all right and then we'll go eight and then cool I wonder if we set this to 360 and have it spinning oh that's kind of trippy well that's just vomit inducing vitous so I think it's still in a pattern so if I instance index random does it like random random yeah and I think it's yeah 0 to one 0 to zero yeah if you wanted to make them all the same you can use that to do that or if you just wanted to select a range of them you know what I mean it's kind of cool too so I think there's I don't have to think there's 1 two three four five six seven 1 two 3 four five six seven8 nine 10 objects in the shapes group so zero to nine that should be everything maybe it is0 to 10 yeah that definitely stops it from being like a sequence and you could randomize a seed I think as long as you have more than the number it's you know it's going to get everything cool I think I actually like him sorted though oh that's kind of trippy for some reason they just seem like mutated pineapple now that's kind of trippy let's try a big wobbly one big old wobbly one all right let's try that death icons three all right I promise to get to the AI stuff oh thanks JC might not be my stuff uh the other thing you can generate with these is normal Maps uh you don't necessarily have to make depth maps um you can just apply the the normal uh to the shaders and I think that will just make normal Maps which you can use with normal map uh control net that's a nice thing about blenders you get real [Music] normals cool I'm actually want to try that uh let's uh bring the strength back down to like one bring these down to like four yeah that's cool which kind of like those pillars those are neat be nice if they all fit though oh that's cool all right so let's um I'm G to also kill this rotation I don't love this one clear single key frame nice uh and now instead of the ambient occlusion the Shader editor edit our material all right let's kill the Ami inclusion for now oh why are some of them not oh it's a different material interesting to do this twice normal what does this look like oh uhoh W whoa uh I've completely forgotten how to get the normal those aren't not normals interesting yeah we're just going to delete that and uh that seems right strength is fine screw it uh let's do the same thing on the other one uh the other Shader is this one not you want the bump node Bonk oh that's actually interesting for depth maps too cool good to remember that normal into a normal into a color ramp is an interesting death map okay these should make pretty cool normals so let's save this one as well normal one [Music] cool okay let's make some stuff come on comfy I know you got 400 packages to look through but you know do you think okay uh [Music] Chrome we want pers 127 all right all righty oh yeah oh hell yeah okay oh all right all right so what have we generated we we've generated this normal map we can try the normal map first I'm actually super interested in what this does looks good looks loopy oh crap those aren't the same at 180 degrees whatever anime just going to chunk that anyway who cares all right uh don't tell anyone I screwed up as he broadcasts it on YouTube so we'll load that into the path uh we're going to select every image because it's only 60 frames and we're actually only going to do 16 to start no we're not skip start with 16 all right uh if you need this workflow it's on my Discord uh just dropped it last night uh go ahead and grab it uh okay uh do I have the normal control net is there an SXL normal control net uh oh aw really really bummer oh man cany lonard open pose depth always catch no way you don't have a normal nobody he train to normal really why does it keep going to that well that sucks somebody who isn't me should get on that because I don't want to I don't know how bomber all right uh screw it back to depth so what's this line art and depth all right let's do let's do the big one first okay yep yep yep yep uh it's a little high five five line art uh depth uh blah blah blah blah blah I'm going to unplug the upscaler for now cuz we don't need it yeah hey exad what's up so I do a 16 frame run to see if it looks good if it looks good we will continue I just realized my prompt is still using the balloons Laura and it's about a ocean of balloons so should make for an interesting animation because these don't look like that but they might look cool as balloons hard to say hey oh hell yeah why are you ripply okay let's crank up the control net strength a little bit to 08 for both of them and see what that makes uh yeah it's animate diff plus Hot Shot plus uh control net uh plus uh IP adapter uh plus mask we just made in blender Plus plus plus plus plus anything is possible with animate diff the power of comfy UI well not everything but some things oh those are trippy feel like the line arts may be a little too in a little too a little too much let's put that at zero keep the depth at 0 eight when I'm happy I'll just do a full run and we'll get all the stuff oh I think I know what's happening here I know why it looks like crap I'm aware of why I think we can fix it too yeah all right we need an image crop or is it upscale no crop uh actually width and height we're going to switch to inputs we're going to grab them from the front width and height all right uh pull this into here we don't need this and this plugs into our vae encoder no it doesn't yes it does yeah I think that's right we'll see and then you go in here you go in here and that does that and up there should do so if I'm right it should crop yeah why did you wig out oh because you're not plugged in so I was rendering at the wrong resolution I was running at 432 by 432 which is because it's it was Square so I've just just uh cropped the the video to the aspect that we were rendering at so this should work take a little longer but it should look a lot less like ass because this is bad I don't like it but it's the resolution for sure see last night I was doing stuff that was was wide so it didn't remember even noticed it yeah lineart in depth right now but the line art is disabled and I'm not pre-processing the depth I'm just using my mask the mask I made uh that kind of already looks like depth map [Music] B this will also work with a QR code control net which we can try after this run [Music] wider that looks better that looks better already still a little low like we could uh probably I am curious what it looks like at like proper [Music] res oh don't tell my video card I've done this we're going to do 1344 by 768 I am curious what this takes vram wise actually so we haven't hit 16 gigs of vram never mind just hit it spoke too soon but again I think we noticed yesterday my Baseline is like 3 gigs 2 2.5 gigs of vram for like streaming and stuff so uh chances are uh if you were doing this at the resolution I was working at which is 768 by 432 you should be able to render this on a 12 gig card uh what card do I have I have a 390 so uh I have 24 gigs of vram uh to play with I actually need that to be able to stream and diffuse stuff cuz like I tried to stream diffusion stuff on my 3070 and did not go well three and a half minutes four minutes for 16 frames that's not so bad right that's like old school render times this better looks so good oh I don't think it will [Music] actually oh I screwed up uh okay yeah so my input footage is is 1080 uh so I just cropped it at I think probably 1080 by 768 that's another weird resolution for sdxl so chances are this is also going to look like ass but we are too far down the whoops no we are too far down the rabbit hole uh for for that I can't get you off so yeah we've done this at the wrong res but it's a higher res see how sdxl holds up all I do think I've also noticed this like Bottom bar of drip um when using hot shot and uh I don't know if it's the control net I don't know what it is but it's been showing up on all my renders with this method so um if it bothers you crop your videos man I don't know that looks cool 60 frames would take I could do it double time though 30 frames oh let's try it a but before we do that I screwed up image resize no now let's try uh image resize I think it's resize plus yeah that nearest maybe it's weird it didn't work uh so let's resize it to this do we need the crops still oh we can tell by here I guess immediately that is not right uh cancel okay well that's not right either so the resize just isn't doing what it's supposed to why what did I what did I do before with and height it's not that is it no cancel it should be cropping but it's squishing it's so weird okay screw it uh 768 by 432 we're back to where we [Music] were quite a bit yeah I like warp fusion it's a little slow um it's a little slow even on my 3090 but it's great ah see this one's funky too what oh image here image into here that's tall still this is crazy this is it was working to fine yesterday all right screw imagery size then we'll have to just crop it and it'll just have to be bigger than the crop size uh width height right 768 by 432 you and you and also canel oh jeez wrong crop tool that's so annoying there it is he W Hees where are you going uh no be this guy you all right and then Center and you you and uh excuse me you okay why are you a square oh jeez s 68 by 432 hey y we did it see one to one just cropped in on the image that's all we needed so your Source image has to be larger than the crop size you know which makes sense um I guess if you're uh really concerned what you could then do is just [Music] um upscale your input footage on the way in and then crop it um yeah crop it back down use like a two times upscaler crop it back down if it's if it's too small but uh I'm not going to do that in the interest of time this uh K sampler is unplugged because that's my upscaler I'm not going to use it uh for these runs but when I want to use it I got this reroute here we just plug that back in and you can also mute it I just find the muting doesn't always mute things right basically if we get a run we like because the seeds are fixed I can just plug this in and it'll upscale it uh as the next run so if we get a really good one we can then decide to upscale it 1.5 and that's just another K sampler with all the same inputs and then the up lateen up scale by 1.5 yeah no worries yes good ey it's definitely screaming for I don't play music on my streams so that you can play your own music and set it to whatever volume like uh if you want to listen along to what I am listening to uh you can click on my name on uh Discord and then just click listen along uh on my Spotify uh I usually have music playing in my headphones but it's not playing out because I find streams with music kind of annoying so listen whatever the hell you want to pop two s on the bus so this should be a perfect Loop uh full a full render UH 60 well 30 frames at Double speed uh which is 60 frame animation will make it just like a really fast version of the animation but uh should be a perfect Loop thank you animate Tiff well that's pretty cool I'm pretty happy with that kind of want to run an upscale on that so we just plug the model back in and then because it's all the same seed it'll just run the upscale run now yeah it looks cool balloon flowers wiggling in the wind hey Simone after this upscale I'll switch to a different mask that we just made uh so for the upscale uh it's 1.5 so uh I don't even know what's that 768 it's 1152 by 648 see what it looks like it's using like 16 gigs of vram so I guess that's not bad for like two control Nets I have all the IP adapter stuff loaded it's just off but we could use that to infuse style too this might look bad because it's only eight steps I haven't really I haven't really refined this uh upscaling process yet but that's kind of cool lose some of the detail but gain some Fidelity it's cool very cool I like how it's like a fabric in the background like a curtain see all the little wrinkles appearing yeah super cool all right let's try a different [Music] mask actually I made this one yesterday but it's very long but it may be cool let's try [Music] it let uh double speed it what's that two 280 I'm trying to think if I I think I made it so it's divisible by four which would be 70 frames there's only 10 more frames than we had so uh for now let's just do 16 frames how it looks and we'll do double speed because it's a really slow one uh what would be a cool Laura let's try the clation one uh what does the mask look like again uh is bubbling and stuff while bubes and Bubbles so let's go uh oh let's go uh volcanic eruption clay and uh magma uh clay and magma a volcanic rock um black glass stone try that with the claymation thing with the claymation Laura ideally would get like a claymation volcanic eruption thing coming out of the center but who [Music] knows well actually yeah let's make it happen I'll show you what that IP adapters is for um H [Music] Json so if we generate the image we want uh with the same uh model and the same Laura that we're going to be animating with and we use it as an IP adapter uh input it really helps to define the style so if we are able to define the style with the Laura before we even run an animation grab that image use it as the uh uh use it as the input then we can drive the result to where we started which is really cool have you done hand masks for control Nets uh no not yet but there is a new hand masking uh thing I haven't played with it yet but uh I think it it looks like it would work those are cool what if I use the what if I use the first frame curious what if I use the first frame of the input what folder was that in uh this one we use that as the uh VA [Music] encoder here's a little trick I learned so I want to load this image I want to repeat the image batch four times so I get four at 1024 by 1024 I think if I'm right actually I should probably have a look at this preview image make sure it looks right on the way out yep we got four Images so now it should make four versions that actually looks kind of cool like that for weird the first one didn't work whatever um I don't like any of these uh let's try uh flowing magma eruption I gave it way too much stuff about rocks I'll turn into little [Music] diaramas yeah it's fixed seeds so it's going to keep going from this seed and doing [Music] four there we go I that's a little better I think that's the one right there save image Boop and go back here and we're going to load this in [Music] here Boop and we're going to set this weight back to 08 so now this is going to inform our animation uh we got our new mask loaded in we're going to do 16 frames it's going to do them at 6 768 by 432 let's try it uh Lon art control n is still at zero uh this one's at 08 so we'll [Music] see minute and a half for 16 frames it's not too bad considering you're getting sdxl plus allur plus two control Nets plus IP adapters uh because when you have a fixed seed and a batch input it'll do so say you had seed four number four the four numbers in the batch it would do seed four seed five seed six and Seed seven um it just incrementally goes up uh seed by seed uh for each image in the batch so in that case I had four images in the batch and it was on seed 42069 so those four Images were seeds 42069 70 71 and 72 respectively otherwise it would just generate four of the same image and uh it' be a complete waste of GPU power but if you keep doing that over and over again you're going to get the first damage is always going to be same you knowless you change something else that's cool yeah I think we're running this one uh this is neat this is neat all right let's run it uh sdxl turbo is cool but it is not as diverse um I don't know if it works with luras and I don't know if it works with all this stuff it might uh the main the reason I don't use sdxl turbo right now is because I don't have a commercial license for it and I don't know if using it like on YouTube which I'm monetizing is like considered commercial usage of the models so I'm just kind of steering clear of um that for now um but uh eventually I I will probably become a member of the uh the stability like model crew uh like the membership thing because uh once SVD gets a little longer I'll probably want to use it you know commercially so be more viable for me but right now uh the 1 second or two second Generations just aren't quite aren quite worth it for me they look awesome though they look really good with like realistic prompts and stuff so you know it's all it's all growing it's all getting better okay uh so we're going to hit zero no how many frames we going to load half half of the animation that's uh 100 140 frames it's a lot of frames uh let's let's cap it at 70 and see if it Loops I actually we'll be able to tell really quickly if it Loops because the this will update oh I think they loot sick oh the upskill worked forgot I had it plugged in oh the up scale looks like crap all right we're going to unplug that for this run I think it's too late now I think it's already running but got to remember and don't need to run the upscaler run unless I'm going to do more frames because Jesus I think that's because it's only eight steps uh yeah so that should be 70 right yeah 70 nice [Music] I am G to make a coffee while this renders um I've been at this for what almost two hours now so if your knees are sore get up and stretch but uh you know if you don't want to don't I'm not your mom I'll be back enjoy uh stretch or don't you know whatever well this looks cool well this is really cool so that's with eight 80% on the uh have scalers running so that's with 80% on this but no nothing on the thing so let's try 0 five on the line art and we'll just go back to 16 frames to do like a real quick one because I just I'm curious what kind of difference that makes but these uh I really like this one this one came out awesome don't you love waiting don't you love watching things render me too nothing more fun than watching a progress bar go by very slowly every time I talk about that I lose viewers I should stop talking about that uh so looping you do 50% of the total frames uh yeah it's just skipping every other frame to make it twice as fast so it saves me yeah that's what that um on the load video load images path all the video helper suet nodes you can select every nth which means like frame skip so you can frame Skip by two frame Skip by three frame Skip by four so it renders every other frame or every you know third frame every fourth frame uh so the video is faster but less frames but you still get a perfect Loop if you let the whole video run oh that's cool with the line art actually H I got to run one of these too because that looks cool as [Music] hell I would also like to do some tests where we don't use the um come on brain don't use the vae encoder uh to get the frames actually use empty uh frames and just let the control net do its thing um yeah I don't know if that would be better or worse but it is definitely worth a crack I could rewire that after this run see what it looks like excuse me got the Yan itis again I don't know why this be another another seven minute run I think 17 minutes that seems long oh yeah of course because it's 140 frames I need to set that to [Music] 70 uh sorry the reason this video Loops uh Loops at um 70 is because when I did my tow thing like the the animation that Loop actually Loops four times over the course of the video I think so you can actually chop it into four sections and it'll Loop you know what I mean like uh so you can run a long one a long slow one or a short one and it'll Loop twice o twice over the course of the animation twice over the course of the animation because it Loops twice over the course of the animation excuse me when I double the speed by skipping a frame I'm able to then do half of those frames to make a perfect Loop because yeah twice per so when you make your input masks it might be a good idea to make them Loop twice and then you don't have to render the whole thing to get a perfect Loop oh thank you thank you for the donation I really appreciate it I need to get one of those cool alarm things that can't be hard to set up right okay this is working eight minutes all right we got this patreon Discord YouTube uh Instagram Twitter pers. XYZ all the links are there subscribe uh join the Discord get the workflows get the masks get all the fun stuff uh patreon gets you uh inter sanctum Discord access to the special Channel and special voice channel uh and there'll be lots of cool stuff coming soon for the subscribers I just don't don't know what it is yet uh once I figure out what people want I will make it uh and then um yeah Twitter to see all my experiments and to see when I'm going live Instagram for Less frequent updates and uh yeah all the links that's all the links I think check me out on giy too I got like 30 million views on giy things are [Music] happening back when Discord had a giy tab you could just type pers in there and all my gifs would come up but they only have ten or now yeah I just got to learn to set that stuff up I also have to learn how to do like different scenes better I have and like this but I need to set up other ones for the other views like when I jump to another browser window or Discord or blender Etc Etc it's just that I like to share my main monitors because then I can just when I jump between stuff you just it just shows up easy peasy all right well screw these normal Maps although they do look cool one thing I wanted to do was mess with the Distortion a bit on the wave texture because uh this is boring there we go trippy whoa oh you're not connected that's why we need less and then maybe closer together oh that might be a cool one let's zoom in on that whoa this is cool I like this I like this I wonder if I can randomize the rotation of each of them uh yeah wait can I do this that's creepy okay so they spin like that but then yeah can I just pop like a wait what that's weird okay oh cool all right right right right right because they're all that's cool all right so let's go zero to 180 now that's trippy I don't like the way they clip in and out of each other still for Wiggly was in the ch oh man I just revealed my age I guess the gray hair does that okay that's pretty cool wiggly biggly biggly wiggly biggly all right let's get rid of the narmal maps as cool as they are let's add uh ambian occlusion again and beond contusion and color ramp dark it up that's pretty cool should be a cool depth map now actually I'm curious if we have something else in the background like spinning like what about another ecosphere maybe I'm believe believe this is the problem with this stuff you can really go down the rabbit hole adjusting sliders until you go absolutely crazy well that's freaking cool why is this so big oh maybe way too much it's cool but this one's way too distracting go away never be afraid to erase what you did temp y noids I don't even know what to call these things ah [Music] whoa this is awesome it's like a top down view of a volcano exploding erupting let's try this one well I mean that might be a cool one totally by accident so uh you know let's render it it's cool like the mesh inter sections and stuff don't really matter because the AI is going to dream weird stuff on top of it anyway so you can kind of go crazy with the blender stuff and not really worry about like fantastic topology and all the stuff you have to worry about normally with blender you can just kind of go crazy make weird procedural stuff and render it oh yeah we got to render that whole one 70 I don't know if this one will work hey yeah it's a loot let's go oh just occurred to me I should do versions of my stream where it goes all chipmunk mode while it's rendering I bet we could cut the 4our streams down to like 45 minutes of actual Talking well we're just sitting here waiting to render it's a lot of editing though I have to start saving my streams locally it's getting late here so I think I'll cut this off in about 10 minutes whenever this render is finished we'll have a look at it and uh yeah about eight minutes or so um yeah make masks of blender um as long as your phase offset on your wave texture goes from zero to TOA over the course of your timeline and you just make that last frame one frame after your frame uh everything you make a loop you can just add other textures into the excuse me into the wave texture and get yourself all kinds of fun stuff like it doesn't necessarily need to be um uh this like uh let's go uh 01 no Distortion uh bring you guys all back to one squish them down all right what do we got here these lumpy lumpy thumbs okay let's get rid of the uh rotation so we can see them and we're going to get rid of this random rotation as well so they're all facing the same direction okay so we got a wave texture here actually we can add some more so we can get a better idea of what's going on here all right beautiful so because our wave texture is perfectly looping we can actually add a math node and just like add any other texture you want in here A voro texture now here's the cool thing if you do this correctly you can get some really interesting patterns by adding two patterns together uh try to find a good example here it's hard to visualize this texture through this one um let's add the position and uh randomize seems to help let's increase the strength well that's a lot maybe just two this is basically like introducing noise into your into your animation but no matter what you add here uh VI a random or unrandomized uh whatever pattern you add to it it's going to always Loop perfectly because that phase off set's just bouncing from zero to to so yeah have fun with that you know you can make some really cool looking because remember when you when you look at this stuff it's or orthographic but you can get really cool stuff by doing perspective so you hit X and it's automatically orthographic but when you set your camera your camera is going to be set to whatever your uh your thing is so just zero off the two you're not using back it off to the location you want zoom it up and zoom it in get yourself some cool patterns going half of them still got normals I don't remember which ones do and which ones don't oh so we're look at ones that have material we want to switch material [Music] o1 is that all them yeah can hi that collection again sometimes it's fun to plug the wrong math in but sometimes it goes [Music] Haywire snap is pretty interesting it makes things sort of look like stop motion wrap can wrap's kind of weird well that's trippy that's actually kind of cool oh I like that all right well I'm just going to render that real quick because that's cool as hell boom be boom be all right save this project and let's see two minutes left one and a half minutes left this should be a perfect Loop because our input footage is a perfect Loop and kind of cool if I don't say so myself it's a refraction Shader I was working on yesterday um I was just stuff I was working on on my own time I wasn't streaming when I made it but uh if anyone's interested in how this Shader is made I can make this next time um it's pretty cool I like it render yeah I'll be announcing a proper streaming schedule soon um and try and take this more seriously uh three times a week uh they say you should try to stream for four hours but I think four hours is a bit of a stretch goal right now uh I'm going to try and do at least two hours uh for stream and then uh we'll see where it goes from there if we were're on to something something we'll just keep streaming longer but uh yeah I don't want to artificially inflate the length of the streams just to you know try to match some crazy metric so yeah we'll have fun uh we'll do our thing and uh when we get tired we'll stop and uh yeah it'll be cool this is so cool this is so freaking cool wow awesome all right well thanks everybody for hanging out and uh yeah if you want to go back through the blender stuff just go back to the beginning of the video and uh check it out and I got a bunch of other videos on here of me building shaders and stuff and blenders so if you want to do those go for it too because uh yeah making shaders is the way to make masks and making masks is the way to make good uh anime diff animations so it all feeds into each other uh AI as workflow augmentation not replacement we don't want to replace people we want to replace uh nothing we want to uh get some new tools and get some new workflows and and use our skills and uh push them forward so thank you again everyone who came out uh really appreciate you all and we will see you very soon uh swing by the Discord come hang out post your art uh post your questions pers loves you have a great night we'll see you all very very soon

Channel: Purz

Views: 886

Keywords:

Id: FPWmfoxXZKI

Length: 143min 35sec (8615 seconds)

Published: Fri Jan 12 2024