VFX Artists Compete Against Artificial Intelligence

the world of visual effects is about to change forever actually the whole world is changing right now because machine learning and artificial intelligence is revolutionizing how we do things now in the world of visual effects there's a triforce of techniques that you have to apply to almost every shot you'll be able to separate your shot into layers aka rotoscoping you need to be able to fill in those holes that are left behind when you separate the shots of the layers aka in painting and then finally you need to be able to track your camera so you can replicate that world in the computer to put new things into the shot ai has solved potentially two of those problems and in this video we are going to do a test to see if that holds true if it works it's going to change how we do things at corridor it's going to change how the industry is doing things we're going to be able to do projects in weeks that previously took us months all right let's go see what i mean all right guys i got a challenge for you we're gonna have a race today and the race is a rotoscope race we have to cut out clint from this little shot i took of him dancing trust me it's a very important shot it's essential that i have them separated from the background now i need to figure out the best technique we have for rose scoping what's faster so we got doing it by hand painting style that's griffin we'll take that over we have old fashioned basic after effects masking clint will take that over peter here he's got more cutting edge tools he's going to roller brush 2.0 it oh matt here always trying to come up with a curveball he's going to use mocha and i'm going to use a secret weapon it's done when there's an explosion behind you when you're doing your cartwheel there's an explosion going off behind you and it looks good and clean it's done and whoever finishes rotoscoping the shot first ding your bell you win a candy bar may the best roto artist win their favorite candy bar so stressful today i'm gonna be rotoscoping by hand each individual frame sometimes the traditional way to do things is the way to go i'm feeling pretty confident that i'm gonna win this how many friends in are you um well i'm almost done with the first frame and uh let's see how many frames are i don't think there are too many there's one two three four oh oh the trick with manual masking manual rotoscoping is to do it limb by limb i'm gonna go like as fast as possible i'm gonna start with my head and then i'll probably do like my torso you need to separate your arm from the body kind of roll through both the arm and the body at the same time you start with your keyframe at frame one and you and you end with a keyframe on frame zero so no matter what you have the first and last frame rotoscope anything in between it starts out real crappy and it gets better and better and better and better five and a half minutes guys what's taking so long huh i will be doing mocha pro i'm using mocha pro today um and it has this awesome magnetic spline tool uh so i can take his silhouette and it basically sticks to the edges of him that is my secret weapon so we'll see we'll see we can do here dude i'm freaking chilling here look at look at all these frames i got so far this i've barely done any work so i'm going to be rotoscoping with the after effects beta rotobrush 2. the original brotobrush was like fine it was it was a little janky but it worked in some situations but the second one is using some kind of ai magic and it's amazing it works super fast and you barely have to tweak it the only problem i'm having is like it's eating up clint's crotch here do you think sorry it's eating up clint's crotch i think i'm gonna i think you know i'm gonna be right behind nico with his fancy ai i'm i think i'm going to come in second here what's going on nico how's your progress here i mean i'm pretty much there it's pretty good whoa it's pretty dang good six minutes it's been six minutes so i'm using runway ml's new green screen tool which is actually a rotoscoping tool uh it's actually a site that you access through your browser and they have this ai model this machine learning model they trained specifically for cutting things out you know i was a little worried like clint being upside down you know is that going to throw off things like this orientation matter to an ai because it matters to us you know we look for things to be oriented a certain way i think i need more explosion you know do you have to render it finish it are you are you recording right now it's in my render queue there's no chance for me then let's see whose renders faster i need to set up my quicktime settings [Applause] oh yeah there's like a ton of artifacting i totally phoned it in a little bit what is this don't worry about it what is that don't worry about it what's all this background i you know i i was there and then i heard you picking out your explosions and i was like i need to just finish this well hold on i mean you want to see mine yeah oh my god that's like dude better dumb it's definitely better than roto brush that's clean as well because usually with ai you know you're trading speed for detail yeah but with this you're getting both how far did you guys get i got like a head dude i got really far i got about halfway done with my first friend yeah yeah you actually feel bad it's just my torso in my head yeah yeah i thought you said you're gonna have the first frame and the last frame for my torso in my head i was actually not expecting rotor brush 2 to be as fast as it was it was super fast a little dirty but still super fast rotor brush 2 which is ai enhanced and then runway ml which is even more ai enhanced i literally had that shot rotoscoped in 30 seconds and i spent too much time getting my explosions comp i got you there if i did have three so yeah ai is pretty crazy let's just explore how crazy [Music] if there's one thing i hope that you're already learning from this video it's that ai is already all around us especially online and now nothing is more representative of just how prevalent ai is and when you can go online and using a text to image ai generator generate all kinds of images so i've got this pulled up here and courtesy of today's sponsor squarespace i'm going to be showing you guys exactly how crazy some of these images can be let's get started everybody knows that squarespace has beautiful award-winning templates so let's start there beautiful award-winning templates hey oh my god a beautiful dog of course okay squarespace has member areas meaning that you can create a membership area for the people that are on your site to give out exclusive rewards have a little bit of fun communicate whatever you want to do with your site that's your choice hey no judgment from me let's see what it gives us what what is that son it looks like a shark had its head cut in half while still obtaining its mouth while eating a diver guys it's 20 21. and for those of you who don't know squarespace has video blocks meaning that you can take your videos from all over the internet say you have them on youtube or vimeo or whatever and put them on your squarespace site that's right you can keep videos from all over the internet on your squarespace okay all right we're getting into nightmare fuel territory here we go we've got some sort of lime green like snake thing that also kind of looks like a hummingbird it has nothing to do with video blocks so let's keep going if there's any image that's almost guaranteed to give me a great representation it's going to be this one now if you don't know multiple contributors allow small teams of people to contribute with their respective strengths to a squarespace website teamwork makes the dream work okay okay it's a it's this i i feel like there's a lot of contribution happening in this image and uh i don't want to be any part of it so let's move on traffic overviews meaning that you can see who's coming to your site which allows you to create a better experience for your members users what have you let's give it a shot yeah yeah nothing reads cozy and human like a stop sign that's kind of met with a an airplane one thing we can be sure of nothing with ai really gives you that sense of coziness and and comfort like you get from a beautiful award-winning template you can only get that with squarespace so remember go to squarespace.com corridor crew to get started and you'll get 10 off your first purchase of a website or domain you know you won't have nightmare fuel like this okay great all right back to the studio you guys might remember our boss town dynamics video well that video took us months and months of work to do partially because we had to erase clint from every single shot now this ai is proving to have some potential for actually letting us erase things easily from the frame so now i've done some tests i'm going to try it on an actual shot from our old boss town video we have a shadow to think about and we have me passing in front of clint we have ram passing in front of clint so there's a challenge there so imagine we're gonna do eight to ten hours of rotoscoping for just this shot there's just a day of drawing an outline around a person the second stage here is the paint out so to speak where do you put in behind the gap you've created now to do this we use basically 3d cards effectively you track the camera and then you're basically constantly freezing and isolating parts of the background where the characters are not and you're sliding that into the tracked shot to basically fill in that data so i took all this and i generated a mask yeah look at that so i'm running this thing called flow guided video completion on here excuse me flow guided video completion so it's going to follow is to follow all the pixels all the motion of the pixels which looks really trippy this is a way to do like the the super mario effect when you get the star so the idea is that because it knows where things are moving it can then predict what should be in the hole that's been created by the mask now if we could remove him automatically using ai that would save us weeks of time on this project well we already can after effects has content aware fill a response just a laugh i mean we used it on boss town and it was okay for parts and we definitely had to clean up a lot you want to see the results of my experiment yep oh wow oh that's it i'll never see the world the same way again pretty dang good though all right here's here's the challenge i'm gonna walk in front of him oh it gets weird yeah because it's thinking it's treating your pixels as part of the background right so what i i think i need to do is when it comes to overlaps i now mask all of you out you and clinton mast out there and then you just use the same roto thing to bring me back in yeah yeah yeah and that's you know the quirk of learning this stuff but all the way up until that point that's this is totally passable it's really good that's yeah thank you thank you now i think what makes this work is that there's clean plate like generated as clint's moving through a shot so we're going to paint somebody out i think we need to make sure you start the shot with people walking into the shot and then you do the scene or just making sure the computer at least gets a moment to see the background without a person there all right so that's cool and all right yeah that's pretty cool but you guys want to just get a little freaked out by ai so the guys that made the gpt3 thing which basically just lets the computer write things it gets a little weird and surreal but the computer just makes cohesive writing they did it for pictures and it's insane an illustration of a baby daikon radish in a tutu walking a dog this is an ai drawing that picture but guys let's play with it oh um shrimp shrimp shrimp shrimp in a tutu walking a dog trip in a wizard hat playing a guitar yeah look at that there's no way that there's there's no way there's literally no way this is possible yo i want a shrimp with a mustache sipping a latte animal this is pikachu playing a grand piano like it did where does this that is where does this come from how does it know what playing a piano looks like versus just a piano dude this is probably just the result of literally looking at billions of images and i think there's another ai tool that literally estimates motion capture and poses from 2d video and so it can see like you're like this and so all right so you got like you got your shoulders you got your spine you got your arms here and it's not like one to one accurate it's not nearly as good as like say an xs or rococo suit we're not there yet we are quickly approaching that my biggest takeaway right now is we are on the precipice of revolutionary new techniques for visual effects so if you think back it's like you had the blue screen revolution where people could cut things out and put the put new backgrounds behind them and then boom 3d animation hits and suddenly cg is big you can create worlds you can create characters you can create anything you can imagine but the next big thing is ai and visual effects you saw with our luke skywalker deep fake we'd never touched a 3d model there but we have a total new character being created what about other things what about paint out what about rotoscope what about color correction what about green screening all these things that we have to grind through as vfx artists if suddenly all those challenges disappear i mean not only is it huge for the vfx industry and how it's going to change all the jobs that are here ideally it's just going to streamline your creativity you come with an idea oh i want to have this character in my shot oh i need to put an explosion in the background or whatever and then boom it happens i think a lot of this is going to happen this year already and you're seeing us explore it on the corridor crew channel we are we are standing on the edge of a giant cliff and it's not just visual effects it's everything machine learning is such a crazy tool and it's a tool like we're not at the point where computers are sentient and they're fighting robot wars yet but this is literally the cutting edge of computer technology and you can run it on you know kind of have to be good machines but you can run it on a machine that you can have access to it's gonna make it so that anybody who wants to go out and make a film is going going to be able to do it at the speed of thought and not at how fast they can click their mice and tap their keyboards it's pretty crazy and where does the newborn go from here the net is vast and infinite

Channel: Corridor Crew

Views: 1,299,759

Rating: 4.9661131 out of 5

Keywords: VFX Artists React, CGi, Visual Effects, deeplearning, computers, virtual, future, AI, digital, gan, deepfake

Id: fmJ74774RO8

Length: 14min 55sec (895 seconds)

Published: Sun Jan 31 2021

Reddit Comments

[u/AutoModerator]

Hello, everyone!

Our debate with /r/collapse is happening ~RIGHT NOW~ and will continue for the next few days!

Keep in mind! Even though we have designated “representatives” for each community, you’re still welcome to participate, too!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.