Camera Tracker In depth - Davinci Resolve 17 / Blackmagic Fusion

hi this is brian with profitless medium post and today we're going to talk about the camera tracker in davinci resolve fusion this is going to be an in-depth tutorial about all the ins and outs of camera tracking inside fusion if you want to follow along i'll put a link in the description for the video clip that i'm using so i'm going to try to do something a little different with this video and that i'm setting it up more like a reference it's going to be a little bit more luxury so this tutorial is going to be a long tutorial so i will be putting timestamps to break it up into as many sections as i can if you're looking for specific areas you can skip around if you don't want to listen to the whole thing i will also put some links below to other camera tracking tutorials i've found on youtube that i think are worth checking out if you want to learn more about camera tracking or you are looking for something a lot quicker so before we start actually camera tracking the first thing we should do is analyze a shot and understand what kind of effect we need to do things we should be looking for are is there lens distortion and do we need to remove it for the effect we need to do do we need to alter the image before tracking like removing noise or grain or do we need to color correct the image before we start tracking to add more contrast to help with tracking the clip another thing we might need to do is do we need to add mass for objects that are moving in the scene that we don't want tracked these are all things to keep aware of before we even start to track our shot so let's bring in a camera tracker and when we bring in the camera tracker we can see we have two inputs we have an orange input and this is where our image that we're going to track goes on the top we have the white input which is for spline masks then we have on the right hand side we have a image output which is a 2d output and at the bottom we have a 3d output which is our scene output so i'm just going to hold down shift and let's bring this camera tracker into our scene so now we have our main footage loaded in and this is what we'll be tracking so let's bring the camera tracker to the viewer and the first thing we'll take a look at is the mask input so to visualize this a little easier we're going to go up into the camera tracker and let's turn on preview auto track locations and this will give us a rough idea of what is going to be tracked within the scene now let's add in a mask so i'm just going to grab a polygon mask and say for example we don't want to track say this chair so we would take this polygon we would put a little shape around it and then we'll add this in to our camera tracker now you notice what happened as soon as i did that all the points are inside the mask so for if we want to occlude that we're just going to have to invert our polygon so open up the polygon click invert and now we can see that the tracking points are everywhere but there's nothing on the chair and so we can add as many masks as we want to include whatever we need from the scene so as far as i can tell the input only allows for spline masks so image-based alphas won't work luckily most rotoscoping software can be imported as splines into fusion if you're working with a team that uses something like mocha or silhouette if you have image based alphas you can add them to the original image using a boolean node but the camera tracker won't recognize the edge of the alpha like it does with the splines which will cause some sliding along the edge so i'm just going to give you an example of how that might come in i'm going to set up a basic so i'm going to grab a background i'm just going to grab a an ellipse put that in here and then if we look at our image we have our alpha and then we'd use a channel boolean right here bring that in and we're going to add our new mask into the boolean and this is just to show you how you could do it if you need to and let's take a look at that okay we need to swap the inputs and now let's look at this inside the channel boolean all you have to do is we keep the operation to copy and what we want to do is this is the background we want to add the alpha from the background in and now you can see we have our alpha one another thing another step you'll have to do is we will have to multiply this alpha against the image so that it cuts everything out if we take a look at the camera tracker right now let's put that up you can see we have tracking markers here on the inside and the outside of the alpha and that's because we haven't multiplied it yet we'll add in an alpha multiply and now you can see that this is cut out and now if we go in here you can see we have all the tracking markers inside the mask now this is what i wanted to show you let's take a closer look here what's happening with the image base alpha is actually taking a look at this edge as though it's part of the image so as we go through you can you'll start to see that some of the tracking markers like right here are sliding along which will definitely give us a bad track one solution you can use and i'll show you here is uh in the mask and you can use either in this situation we have the ellipse but you could use a mac control to blur the edges of your mat and that seems to help out with that so if we blur that edge now let's take a look at the camera tracker now you can see that we're not getting that sliding along the edge that's just one way to approach image based alphas if you have to work with them so let's delete that i'm going to bring our our let's bring our images back into the camera tracker and i'm going to delete that polygon okay so the next thing we'll talk about is noisy or grainy images this can add a lot of shakiness in the camera solve and can really make it problematic when you get to compositing things in the scene here's a couple different approaches to solving this problem the first thing i typically take a look at is the color channels a lot of times one channel is cleaner than the others and may even have good contrast which is also helpful in tracking and we'll be able to do this right inside the camera tracker now i'm going to go over this in a lot more detail later we can take a look right here in the track channel and we have an option if you click on the luma we have an option to go red green blue or the luma and let me show you how this could be helpful so if we come up here the top of the screen we can look through each channel and kind of analyze what they look like so right now let's see we have this is the red channel and if we zoom in a little bit we can see we have a little bit of noise but it's actually not too bad let's take a look at the green okay the green is very contrasty and that's actually pretty clean and then we have our blue blue is definitely really noisy and in general the blue is the noisiest but depending on your camera maybe the blue will be cleaner so we would go through each one of these channels and pick the best one so in this case i'd probably use green if i was having problems with my camera track and it was shaky because that's one way to approach dealing with noisy or grainy footage so if you're inside fusion one option is to use the remove noise so let's take a look at that okay i'm gonna oops now we want that before the camera tracker so in the move noise let's take a look at that bring it come in closer and if you want to get really detailed you should go through each one separately because as you've noticed when we went through the each channel you could see that some channels have more noise than others but for this example i'm just going to hit lock and that's just only going to be dealing with all the channels at the same time and you would just adjust these between and try to find a happy medium between softness and sharpness and try to pull out that that graininess let me just play with those and you can see the difference you have that kind of graininess and then now it's a little bit softer so this is another option another common approach to dealing with grainy or noisy footage is actually just to blur the image so we could just add a blur and something like that just like a one maybe not even that much and that will help smooth out some of the jitter within the grain but not enough to remove the natural shakiness of the camera because you don't want to blur too much generally speaking you don't want to do something like that that's not going to give us a good track so these are just a couple different options you'll have to play with the settings in each one but i want to get you in the right direction as to how to deal with some of these pretty common problems another option if you're using davinci resolve studio is you can use the noise reduction tools in the color tab and can get excellent results for removing grain and noise in that case if you have the studio you can come over here to the color tab and go to the motion effects tab in the color tab and then you have your temporal and spatial noise reduction and the noise reduction inside davinci resolve studio is actually really good i've used it many times to clean up my footage before i start tracking so if you do this route you're going to have to render out your clip in its entirety and then you're going to re-input that footage into fusion for tracking if you want to know more about how to do the noise removal in davinci resolve studio there's a lot of great tutorials out there on youtube one of the ones i really like was done by someone named quasi i'll put a link up right now and i'll also put a link in the description and he goes into some pretty good detail about how to do the noise reduction so the next thing i wanted to talk about was not having enough contrast on the image for the trackers to pick up details so there are a couple approaches you can use if you have this issue like we did in dealing with the noisy images taking a look at the different color channels is a great place to start so we already looked at these just by going through them you can tell that this green channel would might be a really good choice and that's if you're not getting good results with the luma most shots the luma is going to work out fine but again you have some choices to pick different channels and that might help another way that you can deal with adding contrast is just to add a color correct before you start tracking so you just bring in a color correct and then you can add more contrast and just change alter the image before you start to track so that's another another approach to adding contrast and the last thing we're going to talk about before we get into camera tracking is lens distortion if we do need to remove lens distortion we can either do it beforehand using something like the lens distortion node or let the camera tracker figure it out for us so inside the camera tracker and i'm going to point this out and we're going to get into this in more detail later on is if we go into the solve under solve options let's just click that open we have an option here to enable lens parameters and if we turn that on then when the camera tracker gets ready to solve it's going to try to figure out the lens distortion best works for the solve we'll just turn that off for now but i just wanted to point it out as to where it is another approach is to do it either by eye or manually so let me show you how you can do that let's add in a lens oops let's add in a lens distortion i'm just going to add that in okay we don't want it there we want it over here and we would take our media and bring it into the orange input and let's put that up and then in the lens distortion let's click down this lens distortion model for most things the classic ld model is going to work out great that's what i usually use and we can sit here and we can just take the value and change it and make it look try to straighten out the lines within the image a helpful hint when you're trying to do this one way that i found to that's helpful is to add something like maybe the rectangle and we'll take the rectangle we'll pin it and we'll pin our lens distortion and then that way we can click on the rectangle so you can see this line and we can use that as a reference and we can just bring this around and try to find edges so maybe the inside edge of here there's not much distortion so but you could use this line as a reference and then you would just kind of move the image you can distort the image and then you can just you can go through and just keep on moving this around to different parts of the image where you have straight lines or where there should be straight lines and try to remove the lens distortion so if you have to do it manually that's one way you could do it okay let's delete those okay so let's start getting into the camera tracker now so the first tab is the track tab and at the top we have our auto track and this is the button you'll press when you want to start tracking then we have our reset and that will reset all the trackers then we've already talked about this this is our preview auto track location so we can turn this on and off if we want to see what we're going to track now typically i'll have this on just to as i change my settings and before i track i'll turn this off uh it used to be and i guess it's a habit now it used to be that if you left that on and you started tracking uh in certain versions of fusion everything would crash so i just got into the habit of turning that off before i start tracking next we have the detection threshold which is basically how sensitive the tracker is to the contrast in the image as we bring this down it'll start adding more tracking points because it needs less contrast to find edges and if we bring this up it's going to com it's only going to find the most contrasting edges so it'll be a lot less points so that's how the detection threshold works the minimum feature separation is similar as it works to separate out the trackers more over the image so if i bring this down we're going to add a lot more trackers and of course this is at the cost of processing time so if we were to add in this many trackers it's going to take longer to process but you might get better results and the same if we go up you'll have less then we have our tracking channel which we talked about a little bit so if we decided uh specifically in this shot that maybe the green channel is the best one on track we can put it in here um so we can do that but typically the luma will work just fine then underneath that we have our tracking range which is pretty self-explanatory we have our render range which is our timeline then we have our global range which is the full duration of the timeline then we have valid which would be the raw footage or the source media and then we have custom where we can set our own in and out points typically you leave it on render then underneath that we have the bi-directional tracking and i use this quite often and what this does is it will track through the entire timeline and if you have this turned on the bi-directional tracking it's going to start going backwards and extend all the tracks which is helpful in getting longer tracks the longer the tracks and more solid they are the better the results you'll get okay and before below this we have the gutter size and what that is is the pattern that is being tracked as the pattern gets closer to the edge what percentage of the image if occluded will be turned off the gutter size is set to 1 which means once the pattern size even touches the edge it's going to shut off the tracker and let me visualize that let's see if we can visualize that i'm going to pop this you know get a lot of trackers on there but if you look along this edge you'll see what happens when we lower the percentage so if i lower this down to like 0.5 that means that the tracker will stay on you can see it right here actually even better at the bottom you can see how there's no tracking markers here along this edge so once i lower this gutter size it's going to allow for half of the pattern to be off screen before it shuts off so i'm going to bring this down you can see you'll start to see that the tracking markers are creeping closer to the edge so that's what it is obviously turning this down lower will get your tracks to last a little longer but at the cost of potentially not being as accurate at the edge so i don't typically change that okay so let's go down in the next section we have the new track defaults and here we can set a few options for the way the camera tracker will track and each one will give you a few more options so here's the optical flow this is the typical workflow and from my understanding that all of these actually work on optical flow the optical flow and the fusion tracker are very similar the planar has a few more options so let's go over that we have our optical flow if we go to the fusion tracker you can see we have an option to change the pattern size so we can make it smaller or larger this would be similar to the gutter size then below that we have the planar tracker we have the same options here with the pattern size and below that are the options that make the planar tracker different from both the optical flow and the fusion tracker if your shot has changing pattern over time and the other two options haven't worked trying the planar tracker might be an option the planar tracker is helpful when the track pattern is warping or turning through the shot you have several options on how the planar tracker will track the default option is usually the best this will take into consideration translation rotation and scale then we have the options for just translation or translation and rotation if you're still not getting good results after trying all other options then you could take a look at the affine in perspective modes but understand that these two modes need larger patterns to be accurate okay so let's switch this back to optical flow then underneath that we have the closed track when tracking error exceeds so if we raise this we will get longer tracks but at the cost of a higher error value then underneath that we have the solve weight with a solid weight what that does is how much of an influence the tracker will have with the camera solve now typically one is what you need but there's going to be situations and we're going to get into this a little bit later where you might want to change this but we can also do this in the solve tab so we'll talk about this in more detail later under that we have the auto track defaults here we can set the name for the new trackers and the colors uh useful if you're going to track a scene multiple times with different settings again i don't usually use these because you have similar options in the solve tab but there are occasions where you'll want to use this and later on in the tutorial i will be talking about that okay so now that we've gone through all of the different settings in the track tab let's uh set everything back to default and so i want to put these back to default we have optical flow everything on let's turn off the preview auto tracks and now we can track our scene okay so our camera tracking is done so let's head over to the next tab which is the camera tab and let's discuss some of that so here we can input any of the camera information that we know so if we know the focal length you can enter it in here but you also need to know the camera sensor size so we have a bunch of presets here for a lot of the common cameras that are out there so you can go through these if your camera is on here you can plug that in below that we have our aperture width and aperture height which is the sensor size and then underneath that we have our resolution gate and we can set this to a bunch of different options typically the height is fine height and width are the most common then we have the center point which is the alignment of the lens to the sensor i also don't usually change this a reason that you might want to change this setting is if you're working with film scans that have the audio on one side in this scenario chances are the lens is off center but a lot of times you will not need to change this then we have the last two settings the pixel aspect and the camera planes i usually keep these on for example the pixel aspect if you're working with the old footage that's ntsc you might have to change this to 0.9 and then the auto camera planes is for setting up the near and far distance on the camera so if we click this off you can manually set your near and far i usually don't need to change this but on occasion if you're not getting a big enough scene or something like that you can come in here and change your near and far planes but typically you don't need to because once you start to set up your scene and you scale it it will automatically set the scale of the near and far planes with the camera information i also did want to point out that with the cameras today that can shoot so many different formats like hd 2k 4k and 6k that every shooting mode can use a different part of the sensor so even if the specs of your camera are something like full frame but you're shooting hd you might only be using a fraction of the sensor and that will make a big difference when you're tracking your shot so a suggestion is to go to a place like vfx camera database and there are several sites like this where they have a lot of information on the different cameras different shooting modes and the size of the sensor used so very helpful information for doing 3d camera tracking and i'll put a link below in the description so now let's head over to the solve tab and so in the soft tab at the top we have our solve button so when we're ready to start solving the scene we can click this beside that we have delete which will delete the solve not the tracks so if you get a bad solve you can just hit delete and we can start over again next is the solve options first we have the clean tracks by filter which will use a settings in the track filtering which is right here to select and delete tracks then we have the clean foreground tracks which is using the foreground threshold and the foreground threshold is used for objects that are moving faster than the rest of the scene a good example of that would be if we were doing a dolly shot down the street and we had things like these poles or a sign that moves in front of the camera really fast but the rest of the scene doesn't move as fast if we up this foreground threshold that'll give it a better chance of potentially adding trackers to those things that go past really fast and so the these two options here auto delete tracks by filter and auto delete foreground tracks are similar to these two buttons and i don't ever turn these on but you can use these if you want it'll just auto delete them below that we have accept solve error and this is a threshold for solving the camera where when the error value goes above 3 it's going to start taking a look at our camera and trying to manipulate the focal length to adjust the scene for our solve so below that we have the auto select seed frames a lot of times you won't need to change this but this is a good place to start if it looks like your trackers are all good in the scene but the solve is not giving you the right solution by turning this off let's turn this off we get to come in here and actually pick the the seed frames ourselves and now the seed frames are the starting point for solving the reconstruction of the scene so this is a really important part of 3d tracking so the biggest thing to take into consideration with choosing the seed frames is perspective change in the amount of similar trackers on the seed frames as a basic rule of thumb about 30 percent change in perspective between the seed frames is a good place to start so if you're not getting any reconstruction that you want this might be another place to look for answers i can tell you that i've had many shots where i had good tracks and i wasn't getting good results and just changing the seed frames by even one frame will give me better results so this can be really really really helpful if you're having problems solving a scene so let's just turn that back on and we'll go with the auto select because a lot of times you don't need to do this but if you do that's a great place to start then we have refined focal length which if you know exactly what your camera is just turn that off if you put in all the camera information accurately but if you don't know let's just keep the refined focal length on now below that is the enable lens parameters and we talked about this a little bit earlier let's turn that on because we don't know what the lens distortion is so now we're enabling the camera tracker to figure out our lens distortion below that we have the refined center point again don't usually use this unless you're using something like film scans this would be important find lens distortion parameters i typically leave this on until i'm done with my solve and what that will do is every time you hit solve it's going to keep on refining that lens distortion to give you the best results it can then underneath that we have our enabled parameters and these are all the different options you have for the lens distortion models and there's quite a few in most cases you'll be using the radio quadratic in the division quadratic which will solve for either pin cushion or barrel distortion values i usually leave it at its default which is division quadratic the division and quadratic methods are very similar as they are both solving for radial distortion the division method is technically more accurate the radial and division quartic will solve for both high and low distortion values this would help with getting an even more accurate lens distortion model but it would really depend on the effects that you're trying to do in a lot of cases the default division quadratic will work well the other choice we have is the radial and tangential which is what you would use with something like off-center film scans below that we have our radial distortion so when our when our camera tracker figures out the lens distortion we're going to have that value there okay so at this point we're ready to solve the scene uh we'll get into the other part of the solve after we solve so let's hit let's see we got everything on we got refined uh yeah everything looks good so let's just hit solve okay great so our solve is done and we can see this information right here under the solve tab and this is going to give us all the information that we need to see how we're doing so far so we have our average solved error is 0.69 which is actually pretty good uh but i think we can refine that a little bit we have a focal length it's figured out to be 26.39 and we have our lens distortion is a value of 0.003 it tells you here what seed frames it used so 440 to 488 so this is basically all the information about the solve so now we can take a peek at what our camera solve looks like and that's where we can use this scene output so let's grab a merge 3d i'll bring that down and we're going to connect the output from the scene output into the merge 3d and then if we bring that merge 3d up into the viewer we should be able to let's see if we can zoom in here this looks like the scene is pretty small right now but we'll get it bigger but you can see here we have it's hard to see you can see we're starting to get the structure of the alley and it's starting to look pretty good uh one way to do it actually is to go into camera and look through our camera and if we do that you can see oh it's kind of a mess one thing you can do about this and we're going to get into this in more detail but i'll show you right now if you go over to options in the camera tracker we can go to locator size and just change that to make it smaller it's just to help you visualize so now we can step if we start playing you can see we're going through the scene you can kind of see what we're getting so we're starting to get a pretty good shape for the scene but we're going to refine this a little bit further i'm just going to set that back to default and let's go back to the camera tracker and back to the solve tab okay so now let's get into the track filtering so here we have some settings for the minimum track length this is set to six frames so that means any tracker that doesn't track at least six frames is going to be shut off and not be part of the solve in this case with this scene we have quite a few trackers and i'm going to actually crank this up to something like 15 so now only trackers that have at least 15 frames will be on screen or will be left under that we have the maximum track error and this is used for selecting tracks based on the average track error a good example of how this works would be something like a drone shot on a windy day over a grassy field where you would have some tracks that are moving more erratic than the others this slider will help identify the highest error trackers based on the average error typically the default is good but you may find situations where you need to turn this up to let a higher error value come through just so you can get a basic camera path like tracking an ocean or turning it down if you have a lot of trackers on a grassy field and just want the most accurate ones um the maximum solved error is the error value for each individual track so any tracker that is on the screen that has an error value above 2.15 will be shut off and this is one of the ones that i usually play with when i'm trying to refine my solve and since again since we have so many trackers i'm going to try to lower this down to get just the best trackers so for example i might go down to 1.5 now any any tracker that's above 1.5 is going to be selected so you can see as i've changed these settings we have trackers here that are yellow and these are all fitting in this track filtering so now that we have those selected and you can see let's go in if we go further into the shot you can see it a little bit more clearly you can see all these trackers that have been selected and these are selected because we have the auto select tracks while dragging already clicked on we could turn that off if we wanted to and then also use the select track satisfying filters we can go down to the operations on selected trackers and the first thing we'll do is hit delete so now we've deleted all of the the trackers that don't fit within the tracking filter another thing i want to point out is that you can also select trackers manually so we can just go into the 2d view or the 3d view and just select the trackers you can add to your selection by holding down shift and dragging over the trackers you can remove trackers from the selection by holding down command and just dragging over the trackers to deselect the ones you don't want so i just wanted to point that out i also wanted to at this point point out that over here on the side toolbar inside the viewer and if we look up here on the top left a lot of these options here are just quick shortcuts to some of the different tabs within the inspector so you can see here we have the delete selected trackers and we're about to get into some of these other settings soon but i just wanted to point that out so now that we've deleted the trackers let's go up to the top of the solve and let's hit solve okay great so our solve is done and we can see now our average solve ever is down to 0.35 which is very good our focal length changed a little bit to 30 and it looks like our lens distortion also changed and they changed the seed frames from 20 to 25. let's take a look at this in the 3d view and see what we're getting switch this back to perspective so we can see from the outside okay okay so let's take a look okay that's looking like the alleyway for sure we're looking like we're getting pretty good pretty good looking scene you can see that the orientation is not correct because oops let me get over here to the side if we come from the side you can see that the ground of the alleyway is below the origin of the scene so we're going to get into that soon but let's go through a few more settings let's go back to the camera tracker and i want to go through the rest of these operation on selected trackers so let's zoom in here i want to show you some of this stuff in the operation on selected trackers we have a whole bunch of different options here we have trim previous trim next and so the way that works is let's select one of these trackers i think it's pretty self-explanatory it will trim the the tracking points before or after so if we see like an error right here we can just like hit trim previous and they can see that removed there and if we did trim next that will trim everything towards the end now we're left with one frame of course we don't want that so i'm just gonna undo undo these two options are also up here in the viewer toolbar so you have your trim previous and next below the trim previous and next we have our rename and set color for example we may want to let me just fit this back for example we may want to have a model for this cage on the side and so to make it easier to select we can change the name of those trackers and the colors so this is a good place to talk about something i was talking about earlier about solving weight so let's use this cage as an example and what we want to do is add a bunch more trackers to this cage but we don't want to change our current solve so let's go back to the tracking tab and now we're going to add a rotoshape around the cage which i've already done so i'm going to bring that over here and bring this rectangle in and bring it into our camera tracker and now let's take a look at the camera tracker let's turn on the preview auto location auto track locations and i'm going to bring down the separation sum let's get a little bit more get some more trackers on that so something like that maybe yeah something like that turn off that preview and let's change the name so this is going to be for the new tracks so the ones that we're doing right now so i'm going to change that to cage so i know where these trackers are and i'm also going to change this color let's change that to a dark blue from okay and the other thing we want to do is right here with that solve weight so it's at one we want to set this all the way to zero okay with that set let's just hit auto track and now it's going to go through and track our new points okay great so now we have our a whole bunch more tracking points let's go back over to the solve now this is early enough in that we could probably solve and leave those points on but i want to show you that you can add these points without changing the solve before we do the solve though some of the things we need to make sure that are turned off is any of the things that are refining the camera even though these won't be added in we don't want anything to change at all so things that we can turn off are the refined focal length and refine lens parameters the auto select seeds i think i'm going to turn that off because i want to keep this exactly the same and the reason you might want to do this is because you've already started compositing a shot and you decided oh geez or the client said hey you know what we want to do something to the cage now and you need more points i want to show you how you can go through it without changing your camera solve but adding more points so you may want to change the seed frame and lock in these two frames so we'll just put in 20 and 25 because these are the seed frames that it used now with those things set you shouldn't do any kind of refinement and it should just add these points to the scene so let's hit solve and see what happens okay so now we have a whole bunch more points well nothing else changed but it looks like the error changed a little bit the scene should look exactly the same yes okay so let's continue on with the the operations on selected trackers so below that we have our export flag which we can select which points we want exported by default everything will be exported i'm gonna let's go put the camera tracker up so for example if we want to adjust the point cloud from the cage this is an example of what you could do so we could come down to selected trackers and hit all and if we come over to export we could hit clear and that means none of the point cloud will be exported and then if we come down to search and this is the advantage of naming the trackers we go to search type in cage and once we do that you can see that it's selected all of the points that have cage so if you scroll over these points you can see name cage 163 so all the cage points are set and then we could set hit set and only this point cloud will be exported okay so i just wanted to show you that so i'm going to select all again and then set because i want all of these to come out below that we have the solve weight with the solve weight again the solve weight is how much influence the tracker will have on the solve i'll just show you something real quick uh for example in the maximum solver error we have it at 1.5 we could do something like this just bring it way down which is going to select almost everything maybe like 0.1 okay so you can see that it picked up 2709 of 2897 of the trackers so the only tracker the trackers who have not been picked up you can see right here if we scroll over one of them their error is like extremely low which means they're probably really accurate so one of the things we could do is we could take you know with that setting set since it selected everything above that error value we'd come down to selected trackers hit invert and now we only have the highest quality trackers and we can increase this solve weight maybe something like 1.5 and we'll set that at 1.5 and then we can come back up here and we can hit solve again to see if it will help re refine the scene even more so i'm going to turn back on refine focal and refine lens go back to clicking on the auto select and let's see what this does for our error okay so you can see now we have a solve error of 0.34 it doesn't look like the distortion has really changed much and it's still using the same seed frames uh so looking pretty good uh again you're probably not going to need to do that but i just wanted to show you how you could use solve weights okay and so below that we have a whole bunch of different options for showing names frame range and solver and this is for the trackers right now if you hit toggle only the selected ones will show names so if i hit toggle you can see the names of the different trackers this is good for locating stuff or you can do all obviously and then none and the same for each one of these and this these these can be helpful if you're searching through if you're having a difficult track i want to see where the errors are you can see the different error values you do have these color coding so you can find okay there's some bad ones up there below that we have the selected trackers and we have a bunch of options here we can clear the input invert visible will grab all of the all trackers that are are live on the current frame that's what visible does then of course all and then you have the option to search for trackers so that's all of the options with inside the solve tab so before we get into the export tab i wanted to take some time to take a look at the options tab we've jumped in here a couple times but i want to point out some of these options because it can be these uh these options can be helpful in your tracking and stuff like that probably should have talked about this a little bit earlier bring this up okay so the first thing we have is the trail length which we have set to five look in the viewer you can see the more we turn this up the longer it will show the path of the trackers this can be very helpful in finding trackers that are not working well i can see one right here that that probably should be deleted because it's inside the reflection so i know that's probably not right it's a great way to visualize the path of the trackers okay let's put that down then we have the locator size which we talked about earlier so if we change the locator size you can see in the 3d we can change the size of that to make it more reasonable okay let's go back up here then we have the track locator and export colors and these are fairly self-explanatory we have our user assigned so if i change this to user assigned you should see let's deselect everything we had our solve error if we went to user assigned you can see now we have the color that we had put on the cage track below that we have the solve error which is the default and the reason that just to point out that we have the solve error gradient from good to bad that's what it is by default just so you know where the colors are coming from take from image now we can't see anything within the scene because it's using the colors from the 2d image which can be pretty helpful in visualizing once you get into the 3d scene and then we have white it might be easier to see so i'm going to put this back to solve error and then we have the same option for the locators and in the export so if we were in the locators you can change that from take from image and now if we zoom in you can see that might be a little bit easier to kind of see what's going on in the 3d world okay so i'm just going to leave these at the defaults and the same thing for export sometimes the people like to export out their point cloud and use the colors from the image and that can be pretty helpful in your 3d solves below that we have the darken image and this is really great for let's bring this way back down here it's really hard to see some of these green trackers right now so if i hit darken it's a lot easier to see the trackers in the scene this is another one a lot of these controls are up here on this the viewer toolbar this one right here is the darken image now just so you know so you necessarily have to open up the camera tracker just to darken it i'm going to turn that off and then we come into visibility we can turn the tracker markers on and off reprojected locators which is taking our 3d locations from our scene and projecting them back in 2d space to see how they line up so you can click on this and you can see that it added a little x so things are looking pretty good if we have some high error ones you might be able to okay see this yellow one you can see the reprojection of this point is just a little off now that that i wouldn't get too caught up on that because a lot of times that's not it may not matter but if you really want to get in there and get detailed you can come in here and delete some of these there's another way to check things then we have the option to turn on and off the trails so you can turn them off turn it back on then we have our patterns so you can turn this on and this shows you the pattern size what what it's actually tracking and that could be useful for figuring out the gutter size we talked about earlier and then we have options here to turn on and off the tool tips and the 2d and 3d views below that we have our colors so obviously the selection color so when we select a item and we can change that if we wanted to we can make it red green blue whatever we want typically don't change these preview of the new trackers this is the green we talked about the solver gradient so if we wanted to change the colors of how these look we can do it in here and then at the very bottom we have the reporting and this is just for sending information to the console if you want to do that so those are all the different options let's head back over to the export we have the export and then we have update previous export which is pretty self-explanatory it'll update the scene that we've already exported below that we have the auto update export after solves i typically turn that off i'll update the previous export when i'm ready to okay so i'll turn that off then we come down to the 3d scene transform and this is where we're going to orient and scale the 3d scene by default it's aligned which will export the scene as it appears in our merge 3d so if we were to export right now our scene is going to look just like this let's start little lining up the scene so let's move to the end of the shot so that our origin can be located at the end of the camera move but you can set this up whichever way you like okay so to set up our scale and orientation we're going to switch this to unaligned and then we need to pick a point for the origin i'm going to just pick this one in the center go over to the origin and set from selection then we need to orient the ground plane and that's typically what you would do but in different scenarios if you click on this you have a bunch of different options for how your selection will be used to orient the scene but typically you just need the ground plane so now we're going to go through and pick a whole bunch of points that we think are on the ground i'm holding shift and i'm just going to grab some of these ground trackers i'm going to go forward grab some of these so i'm going to get like kind of a nice spread of trackers that i think are on the ground now i know this ground is very uneven so it's not going to be exactly right but this should be close let me grab that one okay so that's i think that's a decent enough selection so let's hit set from selection so that selected all those is the ground and then now we have our scale let's go to the end here so what i'm going to do is i'm going to grab my origin and maybe that point over here and i'm going to set the scale between these two so set from selection so now it's asking the the real world distance between the two points i'm going to start off with just putting this at one and now that all of that is set we can take a look in the 3d view i'm just going to kind of zoom out here so you can see what happens and now from unaligned we're going to go to aligned and now you can see what's happened everything's got scaled and you can see now our point cloud is on the ground it might be kind of hard to change this locator size so you can see it a little bit easier you can see we're now getting a really nice orientation for our scene okay now set that back up so one of the things that is pretty common is that you would be working with a 3d artist and you your 3d artist will really need a real world scale so i was going to show you a quick tip on how i deal with real world scale and that's what i've done is i have a human model then i have scaled to the average height of a human that i can bring into any of my scenes just kind of judge the scale and we're going to do that right now i'm going to bring in this mesh right here and let's take a look at our 3d scene and i'm going to drop it into the merge now you can see where our scale really is so this model right here was scaled properly to a human height so our scene is actually very very very small if you're going to do everything inside fusion this is going to be fine but if you're going to be exporting out your 3d camera to help out you know you have someone working in blender or my or a set of a 4d you might want the scale to be correct so now we're going to have to go in and change the scale again so let's go back over to our scale and i want to go back here we still have our two points selected so let's hit unaligned set from selection and now i'm going to set this to 100 and see if that helps and now let's go back to aligned and now let's look at our scene okay now we're getting a lot closer one way we can check this is inside whoops let's look through the camera so let's go through the camera tracker okay it's still a little large um let me turn down these locator sizes again so we're still a little large but we can go back and we can fix that so let's go back to export unaligned we know that those trackers are still selected oh oh nope they're not okay um let's go back to the 2d so it was like i think it was this one and that one let's set from selection what would you put in 100 the last time it's probably let's do 130. okay and then let's hit aligned again um and so this is oh there we go that's much better so that looks a lot closer to the real world scale okay so that's just a little tip on how you can set up your scene for real world okay i'm going to put that aside and we'll get rid of that rectangle okay okay so now that we have our scene aligned let's go over a couple more options we have our ground plane options so we can show our ground plane let's just put this up so we're looking through the 3d view now and we can change the color we can change the size of it you know how many subdivisions so a bunch of different options here whether it's wireframe or solid you can change your line thickness which might be helpful if you're trying to visualize your camera solve and then offset from origin and this is this is going to offset in the y axis so i don't usually touch this but i can see that being useful if you're trying to set your origin you know on on the ground and say there was a whole set of steps here and you really were doing most of your visual effects at the top step maybe you want to offset that's just one one example of why you might want to do that then below that we have our export options so let's go down here so we have our camera point cloud ground plane renderer and image plane usually i keep all of those on at one point in the past if i remember correctly there was an option for your lens distortion node but maybe i was dreaming about it i have no idea but currently the lens distortion will not export out from our camera tracker but we'll get into that in a little bit under animation we have a couple options we have our anime camera which is the typical way you would use this the other option is animate point cloud and if you'd like to i'm not going to go over animate point cloud because last video i did was all about how you can animate the point cloud and do object tracks i'll put a link up right now where you can check that out okay and then we have our previous export which gives us the option so you can see here we have our camera point cloud ground playing lens distort and renderer they do have the lens lens distort node here in the past when i have after we export you'll see that there will be no lens distortion but you can add atlantis distort and then bring that value in here and i haven't been able to get that to work really well if someone else knows anything about that you know let me know in the comments below now that we have all that set up we're ready to export so let's hit the export now we have our 3d scene i'm just going to spread this out a little bit okay great so let's look through our merge scene and let's zoom out here now you can see uh if it's hard to see we can go into our point cloud and we have a bunch of different options within the point cloud you got your cross or you can switch it to point and obviously that's a little bit easier to see and we can change that size if we want make it yeah so now you can see that that's looking really nice okay so let's take a look at the 3d scene i'm going to look through the camera so we'll go down to right click go to camera camera 3d and now we can kind of see what's going on with our scene so let's take a look first off the ground plane looks pretty good but as we can see here the ground is very uneven so for some general effects this may be good enough but if we need a very detailed ground with all the dips and unevenness we'll have to do some more work in this situation if i needed a detailed ground that is uneven i would do it in a dedicated 3d tracking software that has tools to reproduce the ground using the tracking points i've seen this done in fusion and if you're interested i would take a look at a tutorial by static vfx which i'll put up right now and it'll also be linked in the description below and it's a great tutorial on using the camera tracker infusion so now let's take a look at the distortion if we go into the camera tracker to our solve you can see we do have this distortion value so anyway let's add in a lens distortion i already have it added and we're going to hold shift and bring it in before our camera and in that lens distortion node we're going to come up to the lens distortion model the classic ld model is perfect we will go to back to our camera tracker and in lens distortion we want to double click and copy that number and then just come over to our lens distortion and we're going to paste it right here in the distortion and we set it to undistort now if we look through our final camera you can see that there was a shift now you can see the difference with the distortion without so these are in the right space now okay okay so now that we have our lens distortion set up we need to set it up for a proper composite the way that a normal composite was done is it would be you would render your elements on top of the original plate so we're going to take let's take this original plate move it down here holding down option you can add these little points to help organize it okay we're going to copy it let's copy that lens distortion and we're going to bring it over here holding shift we'll just add it into our line and in this lens distortion we want to click on distort and now we have a couple options we don't want to alter the original image so we're going to either have to inside the camera we can go to the image and disable the image plane or we can just disconnect it here okay and so now when we look at our image if we um we're going to switch on the merge sorry swapped there we go um okay so now you can see this is the proper way to set up your your scene we have our camera we have our 3d scene with our ground plane and point cloud and we render it out distort it and then merge it back over our original clip okay so now we're ready to start adding objects to our scene and begin compositing so one way we to get our objects placed in the scene is to use the point clouds to help us align the shapes so let's go to the 3d view and let's oh let's plug in our undistorted image again so the way we can add objects obviously we can just grab an object and drop it into the scene and let's see if we go out to the perspective view and we got this shape here let's make that bigger it's going to have to be a lot bigger 150 50. okay so that's a plan maybe we'll set up a cube with a hundred okay so now we got this box now the the pivot is in the center so let's see we want to bring it so that the bottom of the box is on the ground plane so this is one way you can add in let's go to you know let's go from the left or something like that and change our perspective view and hit f to frame it and then you can bring this right down to the ground plane that's the ground plane and now we know that this is sitting right smack on the ground now we can go back to perspective or actually yeah perspective and let's see we want this box to be um somewhere down in the middle of the alley or something like that okay that's looking pretty good so this is one way you can set it up now if we look through we have our box uh obviously we're gonna have to put lights on and all that kind of stuff actually maybe it's a little bit easier to look through go to the 3d and look through the camera 3d sometimes this is easier to do and you can see now and now you can see if we start playing we have our box and it's sitting nicely on the ground okay so that's one way you can do it another thing and a pretty common task is going to be something like um is going to be for example let me make these point clouds a little the points a little bit smaller bring it down okay something like that another common thing is we want to add an object to certain points we can use several points to align the objects and this works okay but you're going to have to you will have to do some tweaking so for example say we wanted to have like a card here because we want to add a poster to the wall or something like that we would go in and grab the points that you want to use as a reference you know something like maybe at least three if you're going to do a plane okay so i have these three points selected and then if we right click in the 3d viewer come down to our point cloud 3d and then at the bottom we have a couple options here we have a create shape create a line shape and if we do this it's going to so it's going to use the points that we selected as the orientation for the 3d shape which means you can change it to whatever you want uh then we have a create image plane which is what i'm going to use now and then create a locator which will just give you a location in 3d space that you can copy and paste to other objects so let's hit the create image plane and you can see it's using those three points so it's actually on the planes if we can just use the axises to position this then it should line up perfectly so let me grab the let's see what happened when we added it added this merge to the point cloud you can actually just take the image drop it into that merge 3d and get rid of this one if you want so let's go to the image plane and let's start by rotating and we want to rotate on this like this you can see i'm just using the controls for the axis okay and then we want to scale it so let's go over to our scale and we want it to be wider lower and then we can bring this down something like that okay now i have my have a hard time grabbing that through this view so maybe i can go to the perspective bring it down probably somewhere like that now let's go back to the camera view okay that's pretty nicely set up if you want to check it you can go over to our controls for the image plane hit wireframe and then we can kind of take a look at this turn down the subdivisions okay and then we can take a look at how well it's tracking let's look at the rent oh maybe i'll just do it from the 3d view might be easier and we can just kind of roll through and make sure that it's all lined up but you can see how that's lining up well so there's a few different options there on how you can line up your objects within the scene okay so the last thing i want to talk about is the over scan sometimes when we're doing lens distortion when we undistort the image the image will extend out past the size of the actual image and we'll lose some of that information when we redistort the image back over our original image so let's take a look at what's happening here let's go so you can see here if i just leave in the image you can see that we're missing some information on the edge so i'm gonna let's take off the background from the camera and then let's look through the merge and if we come closer to this edge let's take a look here okay so right here is probably a good example there should be that other tracking marker there so we're just cutting off just a little bit of an edge and the way to fix that is go inside the camera renderer and let's go over to image and then you have an option here for domain over scan and if we turn that up you'll see that you can see that marker turned back on so that's giving us enough information so you can bring that up a little bit and that's how you can add back in the information that would be missing outside of the image frame size okay i think that's it for this video this was a really long video if you have any questions or if i left anything out leave a comment below and i will try to get to you as quick as i can thanks again for watching and i'll see you in the next video

Channel: Prophetless

Views: 5,073

Keywords: Davinci Resolve 17, Blackmagic Fusion, Prophetless, Fusion Camera Tracker, Fusion 3d tracker, blackmagic fusion camera tracker, blackmagic fusion 17, camera tracker davinci resolve 17, blackmagic fusion 3d camera tracking, camera tracker in depth, 3d tracker in davinci resolve 17, prophetless davinci resolve, camera tracker blackmagic fusion, blackmagic design, camera tracker davinci resolve fusion, davinci resolve 17 tutorial, davinci resolve 17 tutorial for beginners, fusion

Id: Nin66hrSy9c

Length: 69min 43sec (4183 seconds)

Published: Thu Jul 01 2021