Real-Time VFX Demo | Rotoscoping 3 Shots in 10 Minutes Using Machine Learning

[Music] hey everybody luke thompson from actionvfx.com and today i'm going to show you a really cool tool that i've been playing around with a lot lately it's called runway ml runway ml is a web-based application that really brings together and bridges the gap between artificial intelligence the complexities of machine learning but the creative application for visual effects has a really nice ui so you can really do a lot of powerful stuff inside of this application it's really important to me that the rest of this demo be in real time so you can actually see how the platform works and how fast i can roto so what i'm gonna do is i've got three shots loaded here so we're just gonna jump right into the app again it is web-based so there's not a need for any huge gpus or fancy computers or anything like that because it's all being handled on their machines so i have these three different plates uploaded we're just going to hop in and start rotoscoping and creating mats for these so i'm going to click the home tab up here the green screen model is going to be the one that we're using today for this demo there are quite a few other things that they have like removing backgrounds quickly from images videos adding color to old photos using ai and machine learning and i mean just about a million other things so we're just going to click start editing it's going to go ahead and throw us in the platform with the timeline here and you can see my assets over here on the left and i have there's a couple other like stock media assets that are included here but i'm going to start off with this flame torch shot by default all my settings are going to be ready to go so i'm just going to click to add points where i'm wanting my mask to be and as you'll see it's just going to generate and build that out for me so i'm on the first frame here since i placed those there the computer is going to start processing forward and backwards and i'm going to jump to the four second mark so you'll see it's splitting up the way that it's supposed to or how i would expect it to at that four second mark and still loading a little bit there so it looks like in the darkness his hat is being left out a little bit so i just put one point there it added it there and it's immediately as i said gonna work backwards and fill in those adjustments that i made you can also notice that it's dropping little keyframes on the timeline similar to how it would operate inside of a program like after effects and so that's pretty much it for this shot i'm going to go ahead and hit export up here it's going to give me a little demo so i can see right there there's a gap so i'm going to click exclude i'm going to punch in on this and just put one marker right there so that's kind of going to help even that out a little bit i'm going to hop back up to export i'll just leave the title the same and i am on a paid version of runway ml it is very very affordable considering the amount of time that it saves but there is a free version that you can play around and test out just to see how powerful this is and on the paid version there's an only export alpha channel option so i can just see the map and that's all i'm going to be needing right now for this test so i'm going to hit export only export alpha channel hd that's going to be 2048 by 1080 and i'm going to hit export video while this is cooking over here i'm going to go ahead and hop into our second shot which is these two soldiers running in the water we have a couple different explosions kind of splashing down behind them so this is a real shot that i actually wrote i'm going to clear this composition drag in a running soldiers clip i'm going to start about halfway where they're split up here and again i'm just dropping those keyframes in and it's gonna start doing its thing so see it even has already kept them apart right there if they overlap it overlaps very very powerful tool so down here at the end i'm just going to drop a few more points to make sure it gets everything and for this shot i mean i really only needed the roto up until about this point but you'll see it's still hanging in there dropping a few more keyframes here but all things considered that's pretty close all right i'm going to go ahead and hit export on this do the same settings as the last so only export the alpha it's 1920x1080 hit export video and i'm just gonna gonna hop right in and do that last i'm gonna clear this composition window and do our last shot here which is a demo shot we used for falling dust so there's a lot of shaky camera movement this was kind of a nightmare to track and then you know roto so that the falling dust could be coming down uh behind me as i'm kind of running through this hallway um so let's see how this one does i'm just gonna go about midway yeah that's a good that's a that's a good face right there so one point and it is already already man that's that's pretty impressive considering two clicks let's see how well it keeps up i'm gonna back to the four second mark two second mark so one more click and it filled all of that in and even just the arms as they start to walk in wow that's that's pretty oh man i love this this thing is so cool it's got a little bit of motion blur uh you'll see there the mask is a little too high so i'm gonna click exclude put that point right there and this mask should give us more than enough for what we're needing just add a couple a couple more there if i wanted to kind of hop in on the rim of these glasses this might be a little trickier because it's out of focus but we'll see i'll take that for the time being i mean obviously you know if this is like a hero shot you've got something really detailed um you'd be cleaning that up anyway but as far as just quick result um getting a mask that we could at least throw some elements in behind um i mean this would be more than enough for what we need ooh got a nice big green spot right there all right going back over to export and it is possible in this editor to i'm just going to hit space bar and it can generate kind of like a ram preview as if it were inside of after effects it is pretty quick considering that you know it's not having to actually track or do any other complex processes on my local system and so this would kind of let us see okay some of the holes here so if i wanted to just kind of watch this through with the green background and it'll start to autofill because i added those points it'll just keep doing its thing and keep adjusting as it goes so see i'm not even adding new keyframes here it's just picking up from back here the ones that i added as it's reprocessing this through just make sure it gets the rest of this it's a little harder because the edge is so soft right here and so out of focus cool all right well we'll let this thing cook so i'll hit export only export alpha channel leave the title 2048 by 1080. click export video and now let's hop over and review our three mattes that we have so all of the masking creating the masks is done for these three shots in just a matter of minutes as opposed to i mean i can attest i'm not good at rotoscoping um but i've definitely definitely spent hours and hours trying to get a mask that fits the actors but is close enough for me to put something behind them i like that water shot for example i mean it took me forever and then i was having problems with like program crashing and everything else so a lot of mental turmoil and if you've ever rotod you probably know the feeling but the fact that this is a result in just a matter of minutes and obviously there's some gaps in this because i didn't do a ram preview and actually click through but for the purposes of this demo i think you can kind of see the the power of the application itself even before refining anything so at this point i would just download this mat bring it into after effects if i needed to add any additional masks obviously i would touch it up as best as i can in runway ml for you know the best result i would kind of get rid of some of this flickering right in there but i mean yeah not a bad result for a couple seconds let's hop over to the flame torch see what this one looks like he's got a little flicker down here around his hand um but really i mean his hat is pretty spot on the torch itself and even how it passes over and has those small gaps that kind of peek through between his arm and stuff are very impressive and our last one is done processing so we'll see how this one came out yeah man such cool applications in a matter of minutes we've rotoscoped three shots um albeit these aren't perfect mats but for most applications these will at least get you where you need to go and in the in the same ballpark that you need and these may work for your application depending on what you're doing um but yeah so that is the end of this live demo we've rotoscoped three shots in a matter of minutes uh using runway ml it's platform as i said that utilizes machine learning and artificial intelligence but kind of packages it together and has a really great interface for creatives to utilize those high-powered tools without having to know all of the technical coding and programming knowledge of artificial intelligence and machine learning thanks so much for watching this video i know it was really different than the normal format that we take but as soon as i saw this tool i knew it was something that i needed to make a video and show everyone in the vfx community about because of the amount of time that it saves we at action vfx are all about taking really powerful tools like this and democratizing them so everyone can have access to make better stories to work faster there's no reason why this level of technology should be kept away in a closet and hidden somewhere but unfortunately that's what a lot of people do thank you so much for watching this video if you liked it please drop a comment and consider subscribing to our youtube channel and let us know what you want to see more of as always i'm luke thompson for actionvfx.com you

Channel: ActionVFX

Views: 33,107

Keywords: Tutorial, VFX, VFX Artist, VFX Tutorial, Visual Effects, Visual Effects Tutorial, VFX Stock Footage, Stock Footage, Stock, Footage, stock video, stock video footage, action essentials, 2k action essentials, 4k action essentials, 4k stock footage, 4k action vfx, actionvfx, video copilot, Production, filmmaking, vfx quick tips, visual effects quick tips, How to rotoscope, rotoscoping, rotoscope, how to roto, rotoscope faster, best rotoscope tool, rotoscope plugin, runway ml

Id: Jo9a73fECXY

Length: 15min 5sec (905 seconds)

Published: Thu Feb 04 2021