How to use Runway's Motion Brush

Runway just released motion brush and if you've ever wanted more precise control over image to video animation you're probably just as intrigued as I am we're going to see if it'll help me fix an image to video problem I had before motion brush existed we'll try motion brush and a text prompt we'll combine motion brush and a camera movement and we'll look at an example where sometimes you might just want to start with image to video and only use motion brush if you need more precise control and there'll be a few more things but let's get started if you log in you can just go here here to Gen 2 and here we see motion brush beta I'm going to go into image here so I did a tutorial a few months ago and one of the things that bugged me about this video is that her hair was moving she was moving but I just wanted the scenery outside the bus window to be moving so now I think I should be able to do that let's try that so I'll tap here so I'll just take this brush this brush comes here it looks like I can change the direction of the video as I'm going through so let's just uh um I'm just going to draw this window here I try to not do this because it's her hair and it'll make her hair move see can I adjust the size of the is that the size no that's proximity oh that so that must be how much it affects the things around it so I want it to be really I just want it to be this stuff right and then let's have the motion go horizontal well let's just let's stick with the defaults and see what happens all right let's click save so it should just be the the only the windows should have motion there let's generate that all right moment of truth let's see how this goes all right this is super impressive because you get the sunlight here it changes on her face because the sun is changing on her face she blinks a little bit this is totally the shot that I wanted to do a few months ago that I couldn't do and now I can do it this is very cool I'm already impressed based on one video know all right let's stay on public transportation but we're going to do something a little bit different here I want to have two different versions of this so on one I want the aliens to be the ones that are in motion and the woman is sitting there absolutely still and in the other one I want the woman to be reacting to the aliens that are absolutely still so I'm able to do that I think with this motion brush so let's go here I'm going to click in let's just start with her so we're just going to make her move and I'm probably doing too much I probably just want to do just her upper body but that's okay let's see how it does so it's just her all right and then because I don't really want to go up or down like I don't want camera movement I just want her moving right all right so then I'll click save um and go okay while that one's going I'm going to try again so I it it's keeps the same image down here which is cool and I want to do a new one so I'm going to click this and now I'm I'm going to clear it and now I want these people I just want that let's just do just the face here and just the hands and then the same thing here just the hands um and just the face maybe for this one let's do a little bit more of the shoulders there could be some like uncomfortable um moving around maybe and then we'll kind of compare if it's this one just the head and the hands and this this on the whole body and then she shouldn't be moving at all and then we'll just change the proximity a little bit and click save all right so we've got two different ones going now all right here's the first one so interesting that the people in the background are moving this person's moving a little bit she's not really moving a lot and it maybe it's just because her posture doesn't have a lot of movement in it but it doesn't look like she's doing much her hair is moving a little bit all right here's the one where I motion brushed the aliens they're moving a little I guess the brush only lets us choose horizontal or vertical motion so maybe we need to use text prompting to get the motion that isn't directional let's give it a shot we're going to motion brush her and then let's have her kind of look that way maybe that's what she'll do so we'll just do her we'll say the woman turns her head to look out the window and we'll generate all right let's see how it goes yeah oh this is ex see this is what I wanted this to me is the best example of the brush I think a lot of people can relate to this I I didn't get her newspaper I missed that but like where you just want one thing to happen in the frame and you just want to be able to say she's moving away from them or she's moving toward them you have that control now this is really cool I think this gives creators a lot more control you're able to generate an image that you want and you have so much more control over it than just using word prompts you can actually paint on there and say this is it this is what she should be doing and to me it works great all right let's try this this is just a Sci-Fi image I did in I think Dolly um let's say we just want this thing to be flaming electricity or whatever it looks like it might do something there too let's just have that go what do we want I think we want the direction to go vertical let's make it go up a little bit I'm imagining that it's making the electricity go up so we'll have it just go up a little bit oh wait I see oh I see if I go this way then it goes down so there's like this zero here um so you can slide it or you can hit this if I go up it's like just resets it let's just have it go up a little bit and keep the proximity there let's have to go up a little bit more maybe like three and then we'll click save and then we'll we'll click generate all right let's see how this sword goes nice added some little extra things there too the other thing that we can do is we can combine this with a camera motion I think it keeps the motion brush yeah so here it's a little still in purple here so we still have our motion brush same thing there and then we'll do the camera motion and we'll just zoom out a little bit and click save and then so we've got our camera motion zoom out and we've got our painted motion zoom up like this and let's click generate now you will notice there there are some things I I feel like the proximity I should have changed let's see what is the proximity setting so the proximity is zero on this so um it's still picking up some little bits of red there and animating them as you can see um so I'm not sure why it's doing that if the proximity is low maybe you have to really drag the the proximity down well let's try it so let's say proximity let's let's just drag that down to like 10 and save and see what happens there so we reduce the proximity here hopefully Only The Sword and the camera move movement is there so let's see how this one turned out oh that's cool huh a little bit different sword like it didn't keep oh there's fire there though that's smart very contextual like yeah there's probably fire in this scene so it adds fire so I definitely like this sword more like it would have been nice if it could have just kept this animation on the zoom out but still very cool that you get to keep that kind of motion all right and here's our third one let's check out this guy better sword I like the fire last time so the second one is probably my favorite because as much as I don't like how it turns into like a lightsaber there his whole hand moves movement in the little fire that's kind of nice but if you're just looking for this like this is like a cool effect for this motion brush one I want to try this thing like where let's do this waterfall right let's do this waterfall I'm wondering if it's just going to know like hey there's waterfalls here um let's see so let's have the vertical go down cuz waterfalls go down right and the proximity I'm going to I don't know let's keep it here I wonder if the proximity knows this stuff or if it's just related to what is adjacent to this mask so I don't know if this will answer that question but let's see let's check out these waterfalls yeah I did all of them that's very and I was whoa a lot of extra Splash there but even this one down here which I hadn't masked out it knows like this is a waterfall so just out of curiosity with this one let's not do the motion brush there's no camera movement all right so no camera movement no motion brush same image let's see how it Compares all right this is the the one with no God that's actually a pretty pretty awesome shot too I actually like that one better if I'm being totally honest amazing Runway nice maybe you don't always need motion brush maybe you can just stick with regular image to video first and then use motion brush if you need it all right so I did this um these like little fireplace scenes with some mythical creatures I had to do a lot of this masking in a video editor um because I wanted just this fire to move and nothing else um so let's see how this works um let's have the uh have it go up because it's fire so we wanted to go up and proximity we'll bring it down we'll save and oh let's try this fire crackles in the fireplace like it should figure that out right so here's our fir crackling I don't know that flame I've seen better flames and it kind of dimmed it I think because it's trying to react to the fire um so yeah I don't know I think I'd still have to do it the old way all right go have fun with motion brush and remember make things weird they don't always have to be like beautiful and perfect and pretty go make weird things that you can only do with generative AI

Channel: AI Video School

Views: 12,247

Keywords:

Id: soZbh9O25ig

Length: 10min 28sec (628 seconds)

Published: Fri Nov 24 2023