3D camera tracking with Syntheyes Blender tutorial

here's how you can use synthis to make your 3D camera tracking go so much faster than using blenders built-in tracker synth eyes is now owned by Boris effects and there is a link for 15% off in the description below so check it out I'm going to quickly walk through the process of tracking footage and synthis and bringing that information into blender so that you can do 3D set extensions like this this obviously is not a modeling tutorial just a tracking tutorial so here I am in synthis which is a Boris effects piece of software now which is great um we are going to import shot which I had converted to an image sequence open that up and it's going to ask about frame rate and stuff uh go to 23976 um it is 4K and um 360 frames hit okay and here we go down here it's going to start caching your footage um to get started with syth eyes which is one of the really nice things about it is that uh you start with just clicking Auto and it does an initial kind of like setup of everything for you so uh let's get started there the shot itself um is pulling back in a kitchen and what I'm going to do is add like a window and a wall here so as the camera moves back it like passes outside and we see the outside of a building um and this is a cool like way to do a set extension in blender if like obviously a camera move through a window from the inside to the outside would be incredibly expensive but you can actually film it all just inside and then uh add a lot of that production value later so to get started uh we're just going to hit Auto and let it uh do an initial kind of setup so here's our initial solve um here are all the trackers that it put in and then up here we can see our camera move from different angles so it kind of pulls back tilts up and kind of wobbles around there so uh the other thing to look at here is our error so we want this error to be below one errors in pixels so how many pixels off is the actual track from the footage uh our goal would be zero for it to be a perfect track um but we're definitely going to have something there and we just want it to be as small as possible my goal is usually three uh but depending on what your needs are um anything under one is fine um and for this track for example because we're going to be building a wall and like pulling out from the wall um it doesn't have to be as accurate as let's say we wanted to place a 3D object in the scene on the counter we want that um to be extremely accurate so um we're going to do some refinement but first while we have a ton of tracker markers um you'll see that there are uh a bunch of them with a Blue Cross and that is what it is decided is the floor plane and we don't actually have uh much of the floor visible and so what we would ideally do is choose three points on the floor and say no this is the floor plane uh because we don't have that um I'm going to use this tabletop uh and so first I'm just going to select everything coord Ates on the floor select it again coordinates on the floor again to turn it all off there's a faster way somebody please tell me because I cannot find any information on that um but I'm going to take these three um points that are on the tblop and uh say that this is and let's do this one too actually and say that those are on the floor so now we have a new floor plane here um as we refine the tracks we might even lose these points um but this will set us up for Success down the road so anyway now we need to get this error down so to do that we are going to choose refine from this drop down um choose slow but sure and then now we're going to make some you know get rid of some of the trackers that have the most errors and then click go again to keep refining it down but first some things that we can do manually are to find any trackers in reflection and just delete them here this one is reflecting from the blinds but that one looks like it's on top of the [Music] counter there we go um and now we're going to go to track cleanup trackers the shortcut here is shift c um we're going to pull this up a few times so just remember shift C is that shortcut and we're going to go down here to high error trackers we want this error to be much much lower um so let's say something like four we have 18 of them let's hit fix they all disappeared hit go and here we are under one so we are already in a good spot we shift C again um we can see that now there are 21 trackers with an era 4 um that's because with the new refined track it's looking at that data and going oh actually there is some more error here so if we hit fix again Pio that's going to drop again so we're just going to keep doing this until uh we get somewhere closer to. 3 and we can even drop this to three six go now we're at 0 4 we're getting real close this last one should probably do it here we go 36 is great um here we can see uh we're all in the green here with our track so this is a pretty accurate track and you can see we only have one of the points left that was on our ground plane um but now we will uh get everything lined up even better and uh you'll see where that comes into play so if we go into the 3D view choose quad perspective and hit lock that's going to show our footage in there too um we can see that our floor plane is like on the correct axis but it is like a little off it's kind of rotated and we want it to be you know the X Y and Z coordinates to line up as well as it can with this footage so to do that we can like rotate all of these elements um you can choose anything and like move it rotate it or scale it but if you choose whole it'll you can move everything in one go so we want this white line to line up with that counter uh we can rotate here from this top view until that is the case right now it's looking like the floor plane is on top of the counter which actually is is you know fine really um because we're not actually using the floor you know itself so um but if we wanted to like Drop it lower we what we could do is find that one point turn off hole click here we can move everything up so that is on the floor here um that is going to make the floor plane closer to the table overall this track looks pretty good um you can obviously see like the in the distance uh it's we really have nothing to reference whether how accurate that is um so we can actually add geometry into the scene to see how close our track is so um here we're already set to box if we hit uh this little wand um we can click and drag and add an object to our scene so what I want to do is find a place in the shot where like the wall would be somewhere like here um and then I'm just going to draw a a shape and then let's move it a little bit this way move it down so our camera kind of crosses over it um we can also add you know another one above it and then let's kind of move these over so like that the wall itself kind of is just past where that actual wall is get everything lined up let's add another also one here and we're going to scale this one up so this is just going to give us some rough geometry so um let's see what this looks like as if we kind of came out of a window and then this is sort of the exterior of the building that looks pretty good so now um let's take uh it looks like we have a great track um this is all done in 12 minutes uh this would take me a lot longer to get to this point in blender um now we can just import this into blender and then start modeling and adding stuff to our scene from there so the first thing we're going to do is export it out of synth eyes um there is a blender python option every time I've done that and try to run the python code in blender I get uh crashes and whatever um but you can also export it as an almic file we'll just save that and then we're going to hop into blender so here we are in blender let's delete everything um and then we're going to import our almic our ABC going choose that and open it and here we have our scene so we have a camera move and um we have all of our points and that reference geometry is in here as well so a couple things you will notice is that unlike using blender tracker um our tracking points are actual objects themselves so one of the first things we want to do is let's select that hierarchy M new collection trackers put everything in here so that we can make sure it doesn't render in our scene um the other thing is that the scene is gigantic so if you hit create a new Cube that's kind of like our base size um this scene is quite large um we also Al have to go in and change our resolution 3840 by 2160 um and sometimes your camera will come in with just like this white in the background um which is this camera screen that is this giant thing way back there um if you just delete it um that solves a ton of problems with the scale of your scene um but now let's uh add our footage as a background image on this camera so our camera selected choose background images open navigate to your footage image sequence select everything and now we see that we have our footage and also this 3D geometry and then one thing you might need to do here is you might notice there's like a little bit of sliding um on your tracker markers and your footage and that is because blender usually starts at frame one but synthis and editing software and everything else starts at frame zero um so you can see here that olymic file opened up um a project file starting at zero so if you go into your camera um over here in the constraints and time if you offset by one frame um that is is going to lock right on to your to your footage so here is here's the before where you can see that sliding where the track is close but something's off and here it is um adjusted one frame um I made these by just doing uh an export um as EG video changing coding to Quick time then if you go to view uh viewport render animation and you're over here in this viewport it'll render out what's in this Frame so instead of doing like a full render um you can render stuff out with this stuff intact and that's how I uh did this real quickly just so I can get realtime playback and also confirm that um this all works now let's scale the scene down a little bit we create an empty scale it up pretty big let's grab our uh select everything then make sure you're empty is the last thing hit contrl p and then we can scale everything down to a normal size here that's our Cube size let's get our scene kind of more uh at the scale so that's how you can get your tracking data from synth eyes into blender I found the process to be so much smoother than using blender's camera tracker I went ahead and did a quick little like mockup of a model here kind of doing a set extension as if we were pulling out of the kitchen to see the outside of the building you could obviously spend all the time in the world to make this look good uh but this was just a tracking tutorial so yeah there's a link in the description below for 15% off if you're interested in synthis uh let me know if you find it [Music] helpful

Channel: Alden Peters

Views: 11,149

Keywords: alden peters, filmmaking, bts, how to make a film, film youtube, indie filmmaking, short film, blender, syntheyes, 3d tracking, camera tracking, visual effects, how to, blender vfx

Id: VqlokaIsCsM

Length: 15min 49sec (949 seconds)

Published: Wed Jan 10 2024