Blender 2.8 Markerless 3D tracking tutorial

whenever your 3d camera tracking in blender it's gonna require you to have at least eight active trackers throughout their shot this is going to give blender enough data to at least try to get a camera solve and if you don't have enough trackers it's gonna throw this error now this isn't really ideal because we'd like to get away with four trackers or sometimes even just three and the basic idea is we're gonna use our three or four trackers to generate the rest of them and even though it seems like we're adding dependent tracker data this is actually enough to fool blender and ends up giving us pretty good results so if we can find any plane in our shot we can actually use this technique even if that plane is featureless so in this example we're going to track this computer screen using nothing except the white area we're not gonna cheat using any extra data now because this tutorial is obviously gonna have a lot to do with camera tracking I highly recommend watching my two part camera tracking series if you haven't already I may even go as far as to say that it is a prerequisite for this tutorial so from this point onwards I'm just gonna assume that you have some initial knowledge of camera tracking so let's hop into blender and get started the first thing we're gonna do is convert our video into an image sequence because that's nicer to deal with when we're tracking and there are tons of ways to do this but I'm just gonna go to the video editing workspace and once we're there just add in your movie and it will import in as this strip and when we play through it everything looks correct but it's definitely a bit too long and I'm just gonna trim it at frame 206 there's no need for a super long shot now something we need to make sure to do is in our render settings under color management change our look to default this is going to give our movie clip its original color back and not change anything now in the output tab I'm just gonna have this export out as a JPEG sequence with maximum quality and just create a folder to store our sequence and at this point if we hit ctrl f12 to render out this animation it's gonna work but you may or may not get some glitchy frames you see that in this case I definitely did have this happen and we want to make sure to fix this before continuing onwards so an easy solution that works for me is using proxies and time codes so I'm just going to enable this checkbox and choose the 100% proxy at maximum quality and change our timecode index to record run and then all you have to do is hit rebuild proxy and timecode indices and what this does is it forces blender to look at our movie as going one frame at a time in the order that we'd expect and when we render out this animation again you see that this fixes the issue and now that we've converted to an image sequence I'm just gonna open up a new blender project the next step is to begin tracking our white screen so I'm just gonna head over to the movie clip editor and import in our image sequence now because we're using an image sequence it's gonna be frame rate independent that means we can choose any frame rate we want and in the output tab I'm just gonna choose 30 frames per second and now we can just hit set scene frames to match our project length and then hit prefetch to load the sequence into memory and when we play this everything is working perfectly and again before we begin adding our trackers just make sure that in color management we have our looks set to default now in our tracker settings I'm going to enable normalize and set our correlation to point 9 this is going to make our tracker and normalize light intensities and only continue tracking to the next frame if it's 90 percent confident in a match so now we can drop in our tracker by ctrl-clicking and unhide our search area with alt s and in this tracking panel you see that our feature is kind of blurry and has this blue haze around the screen and this isn't really going to be ideal for tracking so to get rid of this an easy fix is to disable the blue Channel in our tracking this means we're only tracking based off the red and green channels which are overall much cleaner and now that we're using a cleaner pattern we can just track forwards with alt and right arrow so there are a couple things to keep in mind while we're tracking these four corners probably the most important thing is that these four trackers need to be extremely accurate since the rest of our trackers are going to be generated from these the quality of these trackers is going to be directly dependent on how good we do here so we do have the downside that any small error here will be amplified in our final camera solve but once you're done with these we should have four trackers that are outlining our planar surface and since we've done all this work it probably makes sense to save our project we don't want to lose all our progress and at this point we want to bundle our four trackers together inside a plain track and to do that just select all the trackers and then in to solve panel we're going to add a new plain track and now I'm just gonna rename this to something like screen and clearly we can see that this is tracking on but we do want our plane match our screen so to align these we can just right click any of the corners and then just drag it to where we want it to be and if we actually hold down shift while doing this it will slow down the movement so we can get precise positioning and now our plane is both tracked on and in the right position and we can see that it's locking on fairly well so at this point we want to add some extra features we can track so what we need to do is make an image with lots of little tracking dots that we'll put on top of our screen and we can create this insert in the image editor so create a new image and name it something convenient and in this case I'm going to choose a resolution that is in a 16 by 9 aspect ratio and that's just because my laptop screen is in a 16 by 9 aspect ratio so now we want to change over to paint mode and enable the toolbar with T and the properties with N and I'm gonna make sure that the brush strength is set to 1 using shift F and using F we can get a small brush size and if we want our dots to be very crisp we can do that by changing our brush fall-off to constant and now just paint a grid of white dots all over this image and we don't need it to be perfect or anything just make sure that these dots are gonna be well distributed and again this is the image we want to overlay on top of our screen so we need to do that overlaying using some compositing so go over to the compositing workspace and enable auto render and use nodes we can delete the render layers node because we're not going to be using it and instead we're gonna add in a movie clip node and load in the image sequence from before and by ctrl shift clicking this node of course we can hook this up to the viewer and see what's going on on each frame we have our viewer updating and everything is working perfectly so let's add in our dots image using an image texture node and again if we try to overlay these using an alpha over node you see that this is what we get which is not what we want so to get this image to be on our screen what we have to do is add in a plane track deform node to our image and set it to use our plane track from before and now we have our image going with our screen and this is what we want to end up rendering so finally hook this up to the composite node and then in the output tab I'm going to set this to export out as a JPEG sequence and maximum quality and save it to the same folder as before and now we can just hit ctrl f12 to render out this animation and again the point of this is we used for trackers to generate many more trackers so we can actually use this method on any scene that has a planar surface to generate more features and now that we have this rendered out we can get rid of our notes set up and go into the movie clip editor so we can begin 3d camera tracking and of course we want to track our new image sequence so just import that in and just like last time for our trackers I'm gonna enable normalize and set the correlation to 0.9 and actually when I recorded this I accidentally set the weight to 0.9 instead of the correlation which isn't a big deal but no need for you to make that same mistake as well so now just choose a good pattern size for your trackers that cover the whole dot and begin control clicking to add in all our trackers and once we have them all placed we can just select them all and ctrl T to track forwards and because our trackers really pop out from the black background this is going to track perfectly and we can lock all our trackers with ctrl L and now finally let's get our camera solved so in the solve panel I'm gonna enable keyframe and then in the refine option I'm gonna choose focal length and K 1 and K 2 distortions and solve our camera motion and you see we actually get a pretty reasonable saw there and we could have even gone a lower solve error if we were more careful with our initial four tracks but overall this is a pretty good camera solve and let's actually go through the process of setting up our scene just so we know that it's a good track so make sure your lower window is set to the 3d viewport so we can see what's going on and then just hit set up tracking scene and when we look through our camera view we can definitely see that something's going on but it does not look like it's tracked correctly and of course this is just an issue of orientation so to fix that just pick any three trackers and set them as our floor and we'll use two trackers to set our scale to something more reasonable I'm then going to take this tracker in the middle and set it as our origin and choose a tracker to the right of it to make our x-axis and now our scene has the proper orientation and we can see that this is tracking onto the screen perfectly and at this point of course we can add a bunch of objects to our scene and make a more interesting but instead I think it would be more valuable to go over the three-point approach now so going back to where we had four trackers let's say that we deleted one of them so we no longer have our plane track and we need to use a different approach well what we're gonna do is in the 3d viewport I'm gonna delete everything except our camera and then position it more nicely now we just select all our trackers and go to reconstruction and then link empty to track and now you see that we get these three empties moving around corresponding to our trackers from before and what we want to do is have a triangle with the vertices linked to these empties so it looks like the triangle is attached to the screen so to do that just add in a plane and delete one of the vertices and then with the remaining vertices we can click F to fill the face and then by using vertex snapping we can very easily move our vertices to the position of these empties and to link these all we have to do select a vertex and it's corresponding empty by ctrl-clicking and then just ctrl H to hook to selected object and we're gonna do this for all three of the vertices and now in object mode we can see that our triangle is properly deforming and to make this more obvious we can add in our image sequence as the background for our camera and we can see that this triangle is moving with the screen so now we need to texture this with extra tracking dots to do this go over to the UV editing workspace and we're gonna unwrap our triangle with you and we can definitely optimize our UV space a bit better so just move around these vertices until we're using up more of the UV space and now to create our texture go over to the image editor and add in a new image with a convenient name and then in paint mode just like before we can add in some white dots and to make sure that this texture is applied to our triangle go over to the shading workspace and assign a material and we just want an image texture node with our dots image feeding directly into the surface lot of the material output and now when we go back into the layout workspace you see that we have generated tracking features on our screen and then of course you would just render this out and do a 3d camera solve on that sequence but this process is pretty much the same and that's pretty much all I got for you so again this technique lets us generate extra tracking features that we can use to help us get a camera solved so whenever you have any planar surface and your shot this is something that you can use but I hope you guys found this tutorial helpful and I'll see you guys in the next one

Channel: CGMatter

Views: 69,658

Keywords: blender, 2.8, camera, tracking, markerless, features, tutorial, how to, marker, tracker

Id: PGwRpbYK45Y

Length: 10min 43sec (643 seconds)

Published: Sat May 18 2019