After Effects tutorial - 3D camera tracking problems

welcome back to digital visual effects we're back in After Effects here and I wanted to talk about some specific issues with tracking namely 3d tracking using the camera tracker in After Effects now there are really a lot of good videos out there especially on YouTube that have some great tutorials on how to use this tool but there are a lot of issues that you couldn't come across and potential problems there's a good bit of information out there but it's kind of in a whole bunch of separate areas and separate videos I'd like to see if I could compile all the issues I've come across and techniques and so on into one video here so let's start off we're doing the live tracking I've got some footage here I've imported now I'll just take that and drag it down and make a new composition so let's have some rulers zoom in a bit here there we go that'll do adjust the screen a bit okay now I just kind of playing through this let me see what's happening there's a lot going on here and there's a lot of movement you're not really going to be able to at least my experience do with some successful tracking well there's this much movement of the camera so look at all this you want to kind of reduce it down a bit all the stuff that the tracker has to work with german-speaking special effects shots are done in little segments and this might be too much to work with at least for After Effects tools now I've done some test tracking kind of right at this area to try and you know pull a 3d track off of here and also some kind of right here at the beginning as well so I want to show you what you might want to do we've got you know a decent amount of footage here it's not not a whole lot if I go in here and I interpret the footage and take a look at this I interpret of course at 24 frames a second I'm gonna override the start time code maybe tells B zero and say okay and anyway if you've got too much to work with here and especially though it's a long bit of footage it might take a long time to track you might get errors what I would suggest is that you pull us up some here there we go that I'm gonna double click this and actually go right into the footage and trim the footage here so if I'm pulling the playhead and going along here I'm looking for some consistent areas in the back so as I pull this along kind of like that and it moves along will go to safe out there and right here in the footage itself I can trim it you notice how I grabbed it some it's duration here I'm just pulling it in to set this end I set this end mark notice how my composition it gets trimmed down as well let's go back into the composition and we'll go back here to the start there we go and I'll need my trackers so here recourse we have the the tracking tools and we're gonna do track camera and this is a 3d tracking as opposed to some of the other ones here so I'll just click here a track camera and it should start doing this now this will take a while so I apologize for that but you'll see this right here analyzing in background and we can see that it's doing some stuff up here I ever go it's finally moving so it'll give you a kind of percentage here as it's going along pretty nice and once it's done that it kind of solves the camera and you'll see these these points right along here and that shows you that it's some that's tracked so as I move along here there we go it's of course looking for areas that well of course looking for areas where it sees pixels of do print color and following their movement now what you might not know right away here is that there's a problem with this all these little X's where it's tracking final tracking markers or trying to make up its own they're not really doing a whole lot as I'm moving my cursor along I'm wanting to see this target changing in size and angle as it's going along the background what's happened here especially if you look there's a little message on this it says the depth no depth from a tripod pan solve what that means is that the 3d tracker and that's what it is a 3d camera tracker it hasn't been able to pull any 3d information out of this it hasn't seen enough parallax to be able to calculate depth and that's why all these little X's here are the same size and this would actually if you see this it's kind of a problem you want to see different size X's one that ones that are closer to the camera are going to be bigger ones that are farther away should be smaller and that's what you'll see in a successful 3d track I'm not seeing this here it has tracked it but it's it hasn't given me any 3d information and that's part of the problem so if you ever see this no depth from a tripod pan solve that's what it means that After Effects hasn't found enough parallax to give you any 3d stuff you know so I can't for example effectively track anything and and stick it along this door or maybe along the ground here because there's no 3d information now you could of course click something and you could pin something to it you could do a 2d track with this but it's about the same as doing track motion so this I would say is not a successful track especially for a 3d 3d camera track which is what we're trying to get here now you can do a few things here with this if we look at shot type that says fixed angle of view and you can change a few things here and we look at advanced solve method the height says auto detect and so that's what what your your default is but if I tell it these other methods it might try and figure out a bit more 3d information so let me just show you there are some examples here also my spirit has been that tracked the same footage twice you get two different solutions a few things that we're looking for average error 1.75 pixels that's a bit of a high rate of error which means that we're not going to get a really good solid track pins to this background here so that's you want to look at your error there but I'm going to change this here to mostly flat scene as I do that it's going to resolve the camera and so as I'm looking at this it's I'm actually getting some some targets here and it has its course looking for three different points to triangulate and you can actually click and shift click to choose a point to triangulate on so it's getting me some information there it still doesn't quite look right and so as I go through this footage here as I'm looking for for stuff this is um and not not really a great tracking setup here and like I said this might be something where you have more of a 2d 2d kind of tracking situation but I just want to let you know that the kinds of problems you could come across back to look right here I've got using these these particular targets right there I've got something that that might work that target looks like it's kind of flat there on the ground so it's given me some better tracking information and that was with the same footage and changing nothing here the same tracking tools but just changing the solve method so that may be something you want to look at if you're not getting good tracking information right there and you see I'm not getting that same error message some of these really aren't that accurate but like I say right there looks like that could be a pretty a pretty good one I also notice look at my error average error is 0.30 pixels now that's actually really good anything under one pixel is a pretty good error to have just so just by changing this solve method I do have what could be a much better track so if I want to put something on the ground here yeah that could be a good way to go see right about there and as I'm looking along here here's this sign won't put something on the sign you know that right there could be a good track to kind of match up with with the sign and so we'll just uh littles kind of go through the footage here a little bit looking for places to track you can see there's still not a whole lot of parallax going on so there's not a whole lot of 3d stuff here if I want to put something on the ground here you know there's not a lot of good information along there now keep in mind that you can let's try this you know click a point and then I'm gonna shift-click that point and that point and you see now it's actually being able to triangulate it's giving me a target right there that's still not quite right I can see that doesn't quite fit the angle of the ground but you can see sort of what what you can do here as it looks like this might be more of a 2d scene to track or maybe something that after-effects just you know it doesn't like that much but you can see how you can do a little bit here to give you a better chance of tracking and you also want to check some these controls especially advanced that average error you always want to check so let's try something else here as well I'm going to make a whole new new project and import some footage and let's see here I'll just I've got something ready here and open up this QuickTime movie there we go then drag it down there to make a composition okay so I'm gonna show my rulers command ours I always wanna show the rulers so you got a good size here in 1920 by 1080 and remember when you're gonna track don't just grab piece of footage and start tracking what you really want to do is you want to play through with the footage first and look for a good bit to track notice all the camera movement here as the camera comes in here and it swings in close on the subject and goes around like that so now we've lost a lot of that data so this extreme camera movement where you're losing your initial points to track it's not really gonna be very helpful again we've got some good stuff here that might track as it's coming here now we've lost all that and going along here you know so I want to narrow this footage down to find you know some area I can track I said look at this it's a good chunk of footage as well it's um looks like it's about 50 seconds now that's a lot and so I'm gonna just trim this down to track it like the previous one so I'll double click on the footage going to the footage itself out of the composition and then just pull its its end time here to about 15 seconds so we'll just track this this bit right here so go back into the composition and I'll hit home to go to the very first frame okay so what I'm hoping here is is gonna find some things to track now of course the best way to do this is to do it right during the production when it's actually being shot so if this scene had been made for tracking you have tracking markers you know a little bits of brightly colored tape right here on the wall and that's what the tracker would see very easily to to to pick up on so this doesn't have that we're just using this as an example what I'm hoping is that there's enough contrast of some of these parts here that it will kind of you know make it make it work so here's my track camera I'm at the first frame and I'll just click that and I guess I could have been explaining all these things here while I was tracking to save some time but remember you want to work in short little chunks for the most part even working on big productions VFX studios when you give it a shot it's usually only a few seconds a bit at a time and of course lakh people working together are making making that happen but very common - to cut work on a little segments and in my VFX classes when I make assignments I'm gently telling students to give me you know 10 seconds of a shot or less so between 5 and 10 seconds depending upon the technique that they're that they're showing me here so of course it's gonna do the same sort of thing notice how I haven't changed anything here using all the default settings it's coming down the percentage done there so I'm you know there's some time remaining and there's the there shows you how many frames it's tracked so I'm doing three and sixty frames and it's you know seventy one percent so you know working on long okay I like that countdown there that shows you the the status look at the advanced here and there's that auto detect which you know we'll see how that works for us and it's kind of a good thing to do with - you know just do that I'll attack right there and see what it gives you so now it's solving the camera and we'll see the track points and it's going to come up with okay so this looks like it's a lot more successful now notice that the greatest contrast here is of course on on the model and it's really picked those out now we're not gonna pin something on her and that's not usually the part of you planar tracking like this but I want to show you something how we have these large large tracking X's here and we have small ones here in the background this is actually the mark of a successful track this is what you want to see what this means then is that After Effects has figured out based on the movement here in the model and the movement in the background it's actually calculated depth so these small tracks here in the background are actually shown as being farther away so if I look through the footage here I can see these tracking markers and the variety of sizes here there's a big one as a medium-sized one there's a really tiny one that's kind of what I want to see so that shows me that it has calculated some 3d specifics and depth to the scene so good stuff right there so if you have a good track and that's really what it's all about and this is of course a technique that's referred to as match moving if you're doing you know heavy-duty visual effects work when you have a good track first now you gonna start with some with some markers here and try and you know put stuff on the wall as I'm looking here and looking putting my cursor along here I'm getting what looks like to be I think a really very accurate a very accurate track there imagine if you had that round target on the wall what would it look like you know I think see if I can find a bad one there it's got a little bit angled see that's no good but something like that looks pretty good so that's kind of what I'm looking to see here so I'm gonna go back to the first frame here find one that looks good and again notice how it's triangulating it gets at least three track points and it's triangulating and I'm gonna tell take what is getting here cuz that looks good to me of course as I mentioned you can shift click and pick three points to have it you know do a sort of manual triangulation but I like this right here and so well if I like that and I say we're gonna start with this bit I'm gonna right-click on it and choose create solid and camera now of course I could do a text but you know I'm gonna do a solid which I could put anything into so create solid and camera okay great so there's my solid so let's put it right there on the wall and if I open up my my layers here a little bit more come on there we go you can see that I've got a 3d tracker camera and I've got the track solid so click on that you know there's my there's my solid and there's a tracking camera and just to show you this is a 3d space if I click here where it says active camera I can take a look at it in various sort of dimensions around so if I go to the top what it's showing me here is there we go there's the camera see that there's the the camera moving through the scene all right really interesting there and so that's if you are in the very top of it looking down if I click on this track solid you know solids in there usually going to want to look at this you know active camera right there so we're back in it with that okay so I want to click on this and notice that if you open up the properties and look at transform and you look at things like anchor point position scale and orientation you'll see instead of three excuse me instead of two numbers of coordinates you now have three and that's because this is in 3d space you have x y and z so if you're familiar with 3d programs that lets you know that After Effects is it's thinking in in three dimensions now three axes and that's why you'd have these it's gonna make it really helpful for you so it's gonna see what's happened here now I want to click on this and I want to adjust this because it looks like it's on the wall it's pretty good but if I'm using these concrete kind of blocks I guess as a guide maybe I want to replace one it doesn't quite fit in there so you know it's kind of turned a little bit well that's easy enough to fix but I want to show something here and if there's something that you're kind of missing out on I think I'm missing out on something I'm uh let's see click off of this there we go can't weird have to click off a layer I'm going to go to view options under view right here there we are and for some reason my after-effects keeps turning this off what I really want here is handles so I'll click on that and I'll say okay now when I click on this solid look what I get very handy what I can see is I've got three handles X Y & Z and this is so useful because X Y & Z evening 3d programs they're generally displayed as specific colors so Y is green and the X is red and the Z is blue such really helpful to let you know what's going on now with we've got this solid here and it's just kind of a square really and there are a couple things you can do obviously this is just a placeholder and if I want to see through this I could just hit T for opacity and take the opacity down and now I can see you know some of the texture through it you know that could be very handy but it's just a solid sitting there in 3d space so let me see here with the handles active if I go back to look at my top if i zoom out here let's see there's the camera I guess the solid is right about there it kind of throws you off he has this a little bit here you think is the image it's not it's kind of a weird little thing there so I'm going to go back to my active camera which is probably gonna do most of your work here so a couple things here I want to show you is obviously you're gonna want to replace this and there a couple of ways you can do this obviously you would come in here and you would import something so you're going to import artwork whatever this is where you could do matte painting and then you know attach the matte painting to the to track with the background here you could actually kind of import something and then option drag it and drop it right on top of this but what I like to do with these which is a really good thing to do is I like to pre comp this solid it's a really nice thing to do so if I right-click on this and I do pre-compose so we're going to are going to make it sold as a composition here and I'm gonna keep all of its attributes right out here and so the attributes would be any changes or transformations or stuff going on so right now I'm going to leave all that here you do have the option you can cast ik it and move attributes in to the new composition well I'm not going to do that and we'll leave it out here and hopefully that'll make some sense in a moment so now we'll say okay now we've got this this little composition here sitting right here on the wall and of course it's it's not moving but it's the camera see the camera is moving and it should be moving along here with the footage and I didn't check this let's go here notice that I should have checked that that the tracking to see what my what my attributes are us click on this again and we'll go to the effects controls and looking at my average error that's 1.8 pixels yeah that's not too cool it's a little too high more than I would like but and we'll see what happens there we'll see kind of how this goes again that was the auto detect I could try another one here again mostly flat scene from this but you know we'll see maybe we'll see how that auto tech works but just let you know that average error is a little higher than I would like okay so we've got our comm parents going to show you what happens so I'm going to double click on this comp and now we're in the comp and of course the cool thing here is the comp is completely flat so that's really handy it's really cool because in the artwork you're gonna create or even say like a video image or animation whatever to pop in there are you going to do it flat and then because it's in 3d space as the camera moves it'll fit in and move with the scene but notice what's done is it's getting me a composition and if I go here and look at its settings there we go okay it's a thousand by a thousand pixels and probably you're not really going to want a composition that is you know square a thousand by a thousand pixels so what I'm gonna do here is I'm going to just take it's wet or even oh it's ten eighty nine thousand excuse me should you have 1080 so I'm gonna take its width and just maybe you know double that so maybe to one sixty right and I'll say okay so you now you can see I've doubled that composition now this is the solid sitting right there so I would need to I'll hit s for scale and I'm going to uncheck those attributes there and for the for the X I'm going to do two hundred percent and see now I've scaled up that that's solid so it fills that range there let me show you something else what's often done for tracking as you're putting on placeholders to check the movement a very common thing to do is to do a grid so what I might do here on this solid here is go up to effect and generate and do a checkerboard so if I do that there's my checkerboard what does it look good sighs okay so so there's the checkerboard right there and I'm just going to do a new solid here so layer new and solid and I'll make that well those kind of keep this traditional usually it's a black and white checkerboard so I'll say okay and okay and of course pop that underneath so now you have got a checkerboard going and you know there we have it so if you go back into your main composition let's go back here model track there we go so there's the checkerboard sitting on the on the wall there and often a match move artist is going to do something like this and just kind of the job of Matthieu of artists is to get the data and fitting in there right so that it's um the track is good and smooth and that work it's enough to someone else so it's very common to do this with a checker board so let me show you a couple things we can do here first of all I can see that there's it's it's not quite fitting in with the the rotation here I want to I want to fit this in say to one of these one of these blocks and make that you know kind of fill this area here so here's my my checker board here so if I go to transform I can do a few things here I'm going to go to orientation oddly enough it's not rotation orientation seems to work better and see I've got these three coordinates it's actually X Y and C and I want to rotate this around I think it looks like I need to rotate around 2 X so if I come over here and I rotate that see what's happening I'm trying to get that so that it matches vertically this this line right here and then how about position so I can move it x y&z so seeing that if Z is along this way right so remember um just have to remember what these are x y&z if I do this I'm going to kind of slide it along there there we go and so I'm sliding it along like that ok that's good and what if I scale it so let's take the scale notice there's an ad that's that you've got three ways you can scale it three dimensions but this really is kind of a 2d object so scaling in Z isn't going to help but I'm gonna grab there and scale it up maybe 200% so see I'll scale it up to fit in there and now I'm going to take the position and move it so again scaling along that X right along there err excuse me sliding and now I'll scale along the or move along the Y see I'm trying to move it along here move it along that Y so that it fits right in there and what I'm noticing here is that it's kind of lined up here but see how it's not quite fitting in there it's almost like it's sort of tilted forward and it actually is so I need to change its orientation again I think I'm going to rotate around the Z so if I come to orientation there's XY and Z I'm going to rotate it a little bit this way I didn't like that orientation let's try this one okay I'll wait X so let's try that that's why wait I think this one now I might to do a little bit of rotation so let's see here ah there we go so I'm gonna rotate that see that right there so I've rotated that and now looks like it's matching up you know fairly well with that side so if I take a nice scale it a bit more scale it up scale it up and I'll change the position let's move it up a bit right about there alright so now it counts like it fits in place and then of course I could do just take the opacity down a little bit so you can kind of see through it and that helps you to see how it's moving along here okay so let's go to the the first frame here and so notice as I'm moving this along you know for a good bit of time it's actually moving quite well I would say that this is really at this point here it looks like it sits nicely tracked and so I'm wondering you know it looks like it's kinda I see track notice how here it goes out of focus as the camera goes out of focus so if you're putting something here as you replace it in that composition out here in this composition the main comp you would need to animate say a camera lens blur to try and match this and that's as well as little subtle things as it's a kind of a compositing issue you said make sure it really looks good so as I'm coming along here you know the track is looking pretty good I'm just kinda going through this frame by frame and I can sort of look through here see as I took the opacity down right in this little dark black I'll check the checkerboard I can see a little spot of the concrete there and that's gonna make it a little reference point to see so all that is looking pretty good right now as it moves along here which you might see because of that error up it's starting to slide I can see that that that's it's starting to slide a bit and then I'm losing it where it's supposed to be sticking right here so if get all the way out to here well oh I see if we'd have that move there so there it is it's starting to kind of slip a little bit and I think it's you know slipped a bit more so I think that line was right about right about here and of course if you were putting something in here he wants you to make sure for that layer you check the option to have motion blur happening you know for that layer as the camera moves really quickly it'll add motion blur to it but overall this is pretty good you could try and kind of cheat this a little bit by saying okay I know it's supposed to end up about here like for example that's the edge of that concrete block it should be in and it hasn't really fit so you could actually in the position you could turn on the stopwatch to keyframe and kind of keyframe and sort of maybe slide it along the wall to try and make up for the sliding that the lack of an accurate track is given so I mean not a great idea but a little cheat to sort of keep in mind what do I show you is that right about in this area here it's pretty good track that looks pretty solid Meghan you could you know pop anything you want it in there again double-click and there's your composition sitting right there and it's of course you know what 2160 by 1080 you know that's that's the size but you can adjust it you know right here you know change any change you make here with these composition settings will adjust its size out here in the main composition so for example if I want to do something that was you know long like this block right here I would think you know what that's probably twice that length again I would double click on this come to this composition come to these composition settings and then double this number again right there so you know so it'd be four thousand you know 320 right anyway so these are some some basics on how to try and get this to work like I say from here to here I think is a really good track you know I'm liking how that's looking it's just as we get the extreme camera motion safe starting around here that's when it starts sliding off now I've got one item in here and remember anything can be put in here so it could be it could be matte painting it could be any kind of image creating Photoshop a poster of graffiti graphic whatever you put on the wall it could be a movie you could have animated frames it could be like some type of screen here with some animated messaging or something it can even be motion graphics you create in After Effects remember when you go into this composition you've got this complete composition to work with with its own timeline its own layers and you're working all nice and flat you could pretty much do anything you wanted to right in here pretty cool stuff I want to show you just one more thing here if I if I click back here on the model track movie here they can actually add more add more more items to to track in so I could come over here and add something else and just kind of think about this this error there's that auto detect and you know this sum this piece is already created here and remember it's not moving it's the camera that's moving around here which is why it's a 3d camera tracker makes that 3d animated camera to to make up what's happening here so if this auto detect isn't working I could try typical and let's just see what it does here when it solves you know this camera okay look at that that average air is 1.71 so that's a little bit better not much notice how it still has these smaller faraway points here and these larger ones here so that's all good I wonder if I go to mostly flat scene and see what that does so again if you're not getting a really high level a good bit of error I guess if you want you're not getting a low error if your error is too high you can try these different options here okay so this one has given me one point six three and notice how here when I say mostly flat scene it's actually given more of the the points right here on this wall and they're actually fewer here on the model so I have more points here to choose from the error is down it's a little more than I would like but it is it is a little bit better so notice that even though I've got that that camera and this you know 3d composition already there you know that's going to update and move along with with things here so you know just try and see what what you can get here and it's it's at times it's a little bit of a trial and error and of course this is not what After Effects is really made to do a hundred percent but it's you can do a decent job which work with it well how is that so I'm not sure that I like this mostly flat scene here just for fun I'll try the tripod pan and see what that gives me okay so again there's that no depth from a tripod pan song so that's not gonna be very helpful at all no I'm gonna go back to it see typical and I want to see that well I see the points that I have there okay now I've I've got some points here along this um you know kind of along here right and oh wait oh yeah there we go okay I kind of like that so if I was gonna put something along that wall I'll I'll look for something that'd be fairly flat so it's not as flat as I would like but it's about maybe as flat as I can get so I would come right here and there we go right click and then I'm going to you know again create that's um you know saw in the camera now ideally when it's all done because I've already created a camera it would just say you know create a solid because the cameras already is already there so there's my 3d camera Joran speaking I don't want two cameras there I don't want to tell to generate another one so if I go back to auto detect I've been kind of messing around the scene you know doing a whole bunch of stuff and let's just see what we get here again go back to 180 but let's see okay there we go see I kind of like that right there so I would you know create that create a new solid and usually the the best thing to do is to kind of create these solids right away and then ideally you want just that one camera and then you can have a whole bunch of solids and they're all going to be using that 3d information from the one camera but I hope that this has been helpful to kind of show you some of the pitfalls of a 3d camera tracker and how you can get beyond them you know what you're trying to see and look for to tell you that your track is successful but any questions let me know we've been dealing with this in my class currently and this is probably one of the more complex ways to track and after a fair and that also means it's the one that has the most possibility for error and you got to kind of really fine-tune it and make it happen so hopefully this is helpful if you ever having problems with this particular tool as it's kind of finicky maybe these will show you some points where you can you can figure it out and make it work better for you until next time

Channel: VFX Tuts with Dan

Views: 22,224

Keywords: After Effects, camera tracking, 3D, matchmoving, problems, 3D camera, no depth, tripod pan solve

Id: 2SiFcfIy5Ro

Length: 40min 3sec (2403 seconds)

Published: Mon Apr 13 2020