Planar Tracking with Mocha AE

in this After Effects tutorial we're going to do some planar tracking with mocha AE there are a number of different ways that we can track data in Adobe After Effects we've talked about a couple of them in class and I want to show you some of the disadvantages or some of the negatives in trying to track stuff using some of the native applications and After Effects and then show you an alternative to track information which coincidentally also happens to come with After Effects as a third-party install so the first thing that I'm going to do is I am going to take this clip and create a sequence and because we're being good we'll go ahead and put our sequence into the right cup folder and we can see that we have just your traditional phone screen replacement it is a 1920 by 1080 project it is twenty nine nine seven and it matches the size and the duration of our original clip now we can see that there is a hand reaching in and the phone's moving around and we want to replace the green screen with this intro okay and I think I have this pretty much lined up to trigger when the hand presses the screen but we'll come back to this layer in just a moment I'm just going to turn off its visibility and one of the first things that we've talked about in the past when it comes to tracking data is the track motion option inside of After Effects and of course we want to make sure that we are on the correct layer when we click on that so I'm going to click on track motion and you can see we get the single point tracker and again it's tracking a point and we can even turn on rotation and scale if we need to and we can get two boxes that we can track but that's not really going to help us when we're trying to attract this entire phone so what we need to do is we need to go under the transform property and we need to click on that and we need to select the perspective corner pin which is what we're going to use and you can see now that we have four tracking options and four tracking markers so what we could do in this example is we could move our little tracking point up here to the corner and I'm gonna remember these little points these little crosshairs represent where the new data is going to be placed or the new information is going to be placed or corner pinned to the screen and I'm not gonna spend a whole lot of time on this because this is technically not going to work and then we have our search region and our tracking region options and what I'm gonna do is I'm actually going to move these actually gonna move everything over here to the big X's and then I'm going to move the point data where the tracking data to each of these corners I might be able to do it just by there we go nope yeah so I'm gonna move these tracking and search areas around a little bit and again come up here to the top and do the same thing right on that to where it turns white and then I can move the entire search region and the search box and so I'm tracking these squares I'm sorry these crosses on our screen and if I reposition everything if I if I fit it onto the screen and we decide that we are going to start to track our motion let's see how good of a track that I have set up you can see as this is moving through yeah we've got some pretty good data the tracking markers on the phone are following along quite nicely our little corner pin information is there it's staying around it is jiggling around a little bit this doesn't look like a bad solution although we have seen already that one of our tracking points has already come loose so we could back this up and as we've talked about before zoom in reposition this information as needed it looks like it's pretty good up until until about right there so you know we'll just move things around everything looks good continue tracking I mean we could do that all we want but notice what happens the minute that the hand reaches into the shot it covers up so much of the tracking data that our track points and everything just get blown all out of whack and because the hand is covering the screen we really don't know where that corner is and so we're gonna end up with some horrible tracking data so while this is nice and it does work anytime that you have an uninterrupted view of what you're trying to replace it really doesn't do a great job unfortunately now another option that we could try of course is track camera so let's go ahead and track camera and see what happens okay well let's go over to our composition window there we go it's already tracking in the background okay tracking is complete it only took about a minute to track this 13 second clip and as we scrub through it well we see a couple of points here and there and we some see some stuff on the floor a lot of this though has to do with the noise in this very low shot but we don't really have anything on our phone in fact I was really surprised that this even attracted all this should have created a big old no-no and said no there's very little information that we can track in this shot and that's because this shot has very little depth in that the camera is not moving it's a locked-off shot and so After Effects doesn't know whether the hand that moves into the shot is closer to the camera or the phone or the couch in the back where they are in 3d space it doesn't have that ability it doesn't have light are built into it so it is a failed track it really can't track anything if the camera and objects are not moving in relation to one another so what do we do well I want to show you I'm gonna go ahead and again make sure that our phone layer is selected I'm going to go up here to animation I'm gonna go do track in borås FX mocha now you may have mocha AE you may find this also in the effects under under Boris affects mocha mocha AE okay I'm gonna use the mocha AE if you don't you can use the Boris FX mocha in fact we'll just go ahead and use this one it's going to go ahead and click on the mocha icon and that will launch mocha AE and that will open up the mocha interface and I'm gonna be working in the essential mode even though there are classic modes big-picture roto and a bunch of other options that you can use and if you have mocha Pro you're gonna have even more options that you can do with this application now mocha is a little bit different in that it is tracking groups of points or tracking planes in this case in this example of information that we can then use to track or lock other layers on to do our corner pinning etc fairly easily so one of the first things I'm going to do is I'm going to use the X spline tool which is located right here and I like the X blowing tool first of all let's make sure that we're at the beginning of our timeline I like the X blowing tool because it very quickly allows us to create a spline around our object now I've already laid down the four points I'm going to go ahead and right click to close that off and then I can go in and reposition these you don't have to have a really tight tracking area but I'm just going to tighten this up just a little bit and if we want we can pull on these handles and move them from very rounded edges to very hard edges just by dragging on these corners now then what we've laid down with the X plane is kind of like our search region what mocha is going to do is it's going to search everything within this region and look at all the pixels and see how they shift and move and change over time so for this example we probably want to use transform scale rotate skew and perspective since this loan is gonna rock back and forth and do a lot of different things but this area this red area where our spline is does not define where the lair will be attached to it's simply our search area so I'm going to go over here to layer one and let us just rename this phone BG in order to see where elements will be placed to or attach to within this area we need to turn on the show planar surface and that is this blue box right here might be a little hard to see depending on compression but I'm going to go ahead and take these elements and I'm going to move them to the very specific corners of our frame where I want that final phone screen to be and I can use the magnify tool and I can zoom in and I can use the hand tool to just kind of move around and then go back to the pen tool and I'm just going to move these around until I get these corner parts to align both along the vertical and horizontal edge of the phone so maybe something like that go back to my hand tool and do the same over here and move this around just a little bit maybe something like that and you can see we're way off down here I'm just going to pull this back in something like this and then our final corner right down here and we should be good to go so I can just zoom out and reframe our area and the only thing that I have left to do and I guess I should show you before we click on that is the show planar grid this is going to show us a grid that kind of gives us an idea of how well our tracking is attached to this surface so I'm going to turn all three of these on for just a moment just so we can kind of see how well the tracking is going and when I'm comfortable with everything the way it is I'm going to go ahead and click on the track forward button until we run into a problem and right here once again we run into our problem with the hand reaching in and potentially throwing off our entire track so I'm just going to go ahead and stop right there and what I'm going to do is I want you to look down here in our time line this little green dot represents a keyframe for our tracking information I'm just going to go back to right about here before the hand comes in and I'm just going to click on add new keyframe now I was very surprised that this track held as well as it did as the hand started to move in you can see from the planar information the grid how locked on this layer is in fact if we go over to the layer properties we can look at inserting a logo and we can see how does that logo work within this space and I think it works pretty well it doesn't drift around very much but I am concerned that when that hand moves in that it's going to throw off all of my tracking data so here's what I want to do I am going to deselect everything by just clicking in an empty area I'm going to turn these two elements off for a moment and I'm going to scrub down to the point right where the hand is and let's go back to this layer right here and change this back to none and then click off I'm going to go right to the point where the hand is fully touching the phone right there and I'm gonna grab my X spline again and I'm gonna draw a real quick spline it doesn't again it doesn't have to be tight although Mokka does do some really good amazing things with rotoscoping and so I'm just going to draw a really quick spline around this hand I'm gonna rename this and and the other amazing thing that's that's great about mocha especially when it comes to planar tracking is that it tracks top to bottom so the hand for example is going to be its own separate track but it knows that it should not use that data or that information that's contained in that that lair area for the phone tracking the other thing we should probably do is turn off this little tracking indicator for phone background at the moment and what I'm going to do is I'm going to now track this hand backwards and forwards through our shot until it's completely out the out of the out of the scene and one of the things that is going to happen is that very quickly this tracking data should fail so I'm just going to track this backwards and you can see that it's already starting to slip so I'm just gonna go back a little bit and use my tool to move this finger move these points around the finger just so that everything looks good and then I will continue to track backwards it's doing a pretty good job except for right there so now we can drag all of these points and I'm just going to move them back there's really what we're more concerned about is not the stuff on the back side of this track but more these stuff that's crossing over at the front of the phone I'm gonna track backwards some more and then of course I'm gonna have to move all these points around and you can see it's creating a number of different keyframes down here in this little timeline and I'm gonna go ahead and just track backwards just a little bit further until everything is well off off of the screen of the phone but I'm just gonna jump ahead to this end keyframe and I'm clicking this button down here to advance to the previous and next keyframes and now I'm going to track forward and we run into and we're just gonna stop it now depending on how well your tracking is set up mokume just stop and say hey I can't I can't track any information I'll just go ahead and continue to track forward again you can see right here tracking terminated prematurely that's because it's lost the ability to track I'm just gonna go in and move all of these points themselves whoops too much too many points selected let's just grab the points individually let's find no big deal we've done this before with tracking where sometimes you do have to stop and track a frame by frame so that this information can get out so again I'm just gonna go forward one frame and I think I will now just go ahead and start to track this as a group move forward another frame move it again Trek forward another frame and now this is enough enough tracking data where this is out of the picture maybe we need to go here do that and maybe one more time right there and we should be good so what's happening now is mocha is saying okay we've tracked to this hand layer and now we're gonna go back and track this phone layer I'm gonna go back I'm right at my previous keyframe I can track forward now and now when the hand moves into the shot the surface information and the grid information should not be disrupted by the hand so let's track this forward to the end of the shot and there we go everything looks really good I'm gonna just move this one off a little bit and because there is not really any key point data associated with the little blue surface area I can move that and refine it through this project but now if we go back to our layer properties for the phone layer and I turn on like a grid we can see and we can play through and we can see how well our tracking has occurred and that's a really solid track that looks really good so backwards and forwards this I don't see any chatter I don't see any jumping around I don't see any drifting it looks pretty solid so if we're satisfied with our track we do need to save the information I'm just saving it into a temp file right and then we can close mocha and now we're back into After Effects so big important thing remember anytime we're pasting data into our scene let's make sure that our timeline indicator is back at the beginning of our project I'm gonna turn on the visibility for our phone layer the replacement screen layer and then with the background plate still selected I'm gonna go up to tracking data and I'm going to say create track data and we don't want to use the tracking data from the hand we want to use the tracking data from the phone so we want to make sure that the little gear is turned on for the phone background we're gonna go ahead and hit OK and then we are going to say hey we want to use this data and apply it to a corner pan or a transform or a corner pin that supports motion blur and you know kind of a I kind of wish this was the default but for us we always want to try to choose corner pin that supports a motion blur then we get a pick which layer we're going to apply this data to in this case it's our phone intro and then we can apply the effect and boom well that didn't work did it I mean if we scrub through our timeline oops we scrub through the timeline it looks like everything is aligned but it's off-track by quite a bit and there's a reason for this I'm just gonna undo this and that is because what mocha is doing is it looked at the entire screen of information the 1920 by 1080 screen and said okay let's take all that tracking data that we've done and let's apply it to a layer that matches this exact same size but we don't have a screen that's 1920 by 1080 for our phone in fact if we look at our phone layer the phone is quite the opposite of that it is 498 by 1080 so how do we get these things to line up well the easy way to do this and the correct way to do this is to create a pre comp by pressing command Shift + C or control shift or M what is it on the PC Alt shift + C whatever it is on the PC I don't know I use a Mac but I'm using command shift + C to create a pre comp and for those of you that don't know what a pre comp is it's kind of like nesting layers like you might find in your nonlinear editing system we can just leave this as is but I'll just go ahead and rename this phone intro pre comp and I want to select move all attributes into the new comp and adjust the composition duration to the timespan of the selected layer maybe yes maybe no that you want to do that you don't always have to do that but I'm gonna go ahead and select that now and I'm gonna also say open new composition so that when I click OK it's going to open up a new composition called the phone intro pre comp and we'll notice that it is a another composition that's 1920 by 1080 but that it has our layer right here what I want to do is I want to stretch and manipulate this layer so that it is 1920 by 1080 and the easy way to do it is go to layer transform that to come then when I go back to our main comp that we're working with again you can see that our phone has been stretched to size so now when we go back to our background layer and we go back to our effects tab and we say corner pin that supports motion blur we want to make sure that it's still selecting the foam layer and this time it's saying the pre comp now when we apply or apply the exported data boom it lands right there on our phone and if we make a Ram preview this should track perfectly and it does a pretty good job I don't see any drifting I don't see any jittering now the problem that we have obviously is the hand reaching in and it's below the the foam layer or below the phone replacement layer so we need to fix that how are we gonna fix it well we don't we're not gonna do any rotoscoping remember this screen is green I'm just going to drag this phone pre-comp layer down below and on our background layer I'm just going to do key light with the key cleaner and the advanced spill suppression I'm just going to drag it right on there I'm gonna pick our green color and boom that does a pretty good job right there we can go in and look at the combined Matt we can see that there's a lot of chatter around here so we're just going to go to our screen mat and we're going to let's see clip our whites until that's all gone away and we can clip our blacks so that's all gone away and for this example you can go in and you can continue to tweak this Matt and refine this Matt as much as you want but I think that since this is a tracking project and not really a keen project go back and see the previous lectures on keying what I just want to do right now is get in here and look at my intermediate result and my spill suppressor which will get rid of a lot of the green that's on my thumb in my hand and basically we're done this moves by so quickly that we have a nice little composite of our new screen on this phone alright that looks pretty good I do want to point out a couple of things about the spill suppressor and what its removing in the green I just want to point out that there was a gray card kind of right in front of this phone and the phone I probably should have turned down the brightness because that green was really bouncing forward and then bouncing backwards and you can see that there is a lot of green spill back here on the back of the couch and on this pillow notice what happens when I turn this on and off you can see that there's a lot of green spill that's going on in this image fortunately the advanced spill suppressor does a pretty good job but if you see a bunch of chatter and junk you might want to go in and turn on reduce the chatter and crank that amount up quite a bit if you have problems with noise or other information in your in your in your key the one final thing that I want to do and this is just really quick and dirty is I want to try to match the color of this CG layer with the rest of my scene as we can see this screen looks really sharp for what we're looking at lips don't move that layer around it looks really sharp even though this is video even though we're have some some CG elements here it just looks a little bit too sharp so two things that I'm going to do first thing that I'm going to do to this layer is I'm going to put a Gaussian blur I guess you could put a I guess you could put a fast blur on here but I'm always been a big fan of the Gaussian blur and I'm gonna apply this to the phone pre-comp layer I'm going to turn that up to about three to a three just depending on taste and you can see it doesn't do a whole lot but it does just spose those edges out just a little bit and if I take that down to two it just softens those edges a little bit to help it look a little bit more believable in this shot now obviously we are dealing with a lit-up screen so color doesn't necessarily and intensity and brightness doesn't necessarily have to match the surroundings but if we wanted to make this match a little bit more with the surroundings here's a really cool quick down and dirty way to get your CG elements to match with the rest of your scene and that's the ply the tint effect onto this foam layer and what I'm going to do obviously right now it turns it completely to black and white but what I'm going to do is I'm gonna use the black eyedropper and select something that should be black in this scene maybe like my watch or the couch I'll probably pick the couch right there we could see if there's any difference with my watch up here not really a whole lot and then I'm gonna use the white color picker to pick something that should be white like this baseboard over here in the shot and you can see it kind of makes it a kind of a brownish color which is kind of what the whites are in this scene they're kind of brownish and muddy obviously this doesn't look good for our video element because we don't need this to be black and white or muddy and white but if I reduce this down to say 20% or even 30% it does take the edge off of the whites and the rest of the colors in our scene and does help blend these elements together it's still projected we can still do a brightness contrast if we wanted and turn the brightness values down but this I think creates a little bit personally no you may not agree creates a little bit better blending of what that phone might look like in this real environment if we were playing this video from the phone in the scene for real final thing that we need to do is we need to turn on motion blur for this layer and turn on motion blur overall and then take a look at our final result so there you go planar tracking in a mocha ae you can find the link to the project files below and if you have any questions please ask hopefully you've learned a little something and this will help you in your neck screen replacement thanks for watching [Music]

Channel: Stephen Schleicher

Views: 21,267

Keywords: After Effects, Tutorial, Mocha AE, Mocha, Planar Tracking, Screen Replacement

Id: jeNR3nmZBJw

Length: 28min 37sec (1717 seconds)

Published: Sat Apr 11 2020