3D Set Extension with Camera Tracking in DaVinci Resolve & Fusion

[Music] welcome to another set extension tutorial for DaVinci Resolve or for fusion studio this time I'm using footage with a moving camera and I'm placing two photographic elements into the moving shot so the techniques covered in this tutorial I'm using the 3d camera track to get the extract the camera movement and then I go into fusions 3d environment and use a card projection place two photographs into that 3d environment in order to come up with this set extension as always you can download the footage from my website together with the solution files if you want to have a look at it I should mention that for this tutorial the 3d camera tracker is only available in the studio version either the studio version of DaVinci Resolve or you can use fusions to do whatever you prefer in both ways get the footage to follow along and when you're ready let's get started [Music] okay so let's start directly from the fusion page or from fusion studio I have loaded my source footage so I have here on the left side this forest clip and I have this bound picture which was already cut out in Photoshop and a signpost here which is just a graphic element also with an alpha Channel already prepared for me so starting from there what do we do first I want to get the camera track so I get my virtual camera which emulates this movement and then I go into the 3d space where I can place the elements let me bring in the camera tracker from the Tools menu and attach it here to this forest clips or control spacebar for the tools camera camera tracker this is the 3d tracker and this tracker even though it's just one note here in the floor it actually represents a whole workflow of tracking techniques that you apply here so if you look into this tool there are multiple tabs and basically you work your way from left to right so if you're not familiar with the camera tracker I just give a brief overview for this track it won't be too difficult so camera tracking can be anywhere from super easy to completely impossible depending on what kind of shot you have so the principle of this tracker is that first you select or the tool selects tracking points on the on distinct features of the image throughout the timeline then with the selection of these tracking points the actual tracking is happening so points or features in the image are being tracked in 2d on the image then afterwards you can also give some camera information with that camera information there is a solver the solver uses the tracks in order to calculate what the actual camera position would have been like right and then once you have that you can export this into into the flow and can get actually a camera object in the 3d scene with the certain reference points can get those paste it into the fusion flow so let me just do this I go here into the camera tap and the first thing you can enable here this preview Auto track locations and then you see if you go through the frame and let me bring this into the this we don't need let me bring this into the view you see here the tiny little green dots all over the frame and these are the points where fusion believes that there are features that can be tracked and this is selected automatically but you do have some control you have a threshold which if you bring it down you will see more points so this threshold says how much contrast should there be for example let's just zoom in on one so here's one automatic point and I guess fusion recognized here the contrast this kind of feature which can then be followed over multiple frames right so wherever these contrasts are recognized fusion puts the points and if you lower the threshold and more points will appear just as a general rule you need definitely at the minimum eight points this is the mathematical minimum in order to be able to calculate for a camera position at the end but typically you should have much more you are in a good place if the points are distributed over the image if you have a good amount of points in the foreground and in the mid-ground and maybe a few in the background if you have lots of points in the background like on the sky or so and not much in the foreground and the perspective calculation will not work so you want a good distribution of the points both over the width of the image and from a depth perspective it's possible to attach a mask here if you have certain areas of the image which are unusable or if there's an actor moving through the image or something like this then can do this here we are in a relatively simple scenario so we should be pretty much fine with the preset so let me now just do the click here on auto track which is the next step it runs through and does the tracking and then let's see what we are getting here so now we have the color changed of the points and if i zoom in you see these little paths here motion paths for each point so each point now was tracked over as long as it was possible for the solution so if you now go through the timeline you will see that some points might drop out of the frame either on the side or they can no longer be tracked and the tracker automatically dismisses them after some time and new points are being added towards the end and that's all fine as long as you have in each frame a good amount of tracking points you should be on the safe side one thing to improve sometimes you can do be directional tracking if you enable this it will track once forward and runs backward in time and this can give an additional improvement in some cases over the tracking locations for this I think this should be fine let me go to the camera settings now the more you know about the actual film camera that was used the better in this case I actually don't really know what was used because this is coming from stock footage but I have seen on the side that the cinematographer mostly uses DSLRs with an aps-c sensor which is the like I think this might be a Canon 60d or something like this so you see here a few presets there are not many cameras listed here but a few are listed the point is if you select the preset it will set the aperture width and height here these two parameters are being set automatically so which is just the the sensor data so the Canon EOS c500 I believe an aps-c sensor I'm not too familiar with all kinds of camera technology so I won't go into the depth but basically if you don't find the camera here either find one which has the same type of sensor or start looking up in the manual or online what the width and height is and just set this manually here it's set in an inch you might find in millimeter but I guess you can convert okay so the rest usually is default focal lengths my logic here I don't know what lens was used I don't know what focal length was used but I see that the image is not really distorted I see even the trees on the side are straight so I think that something in the normal range which for this kind of sensor would be like 25 28 so I just leave it here I just leave it at the default if you have fish eye or extreme zoom or something like this then it would be good to adjust this the better you get with these settings the better for the solver but if you don't know the settings you can only guess and try to get as close as as possible then I go to the soft tab and here there again different presets most of those we will not need at least not for this setting there's one thing here you see a setting we find focal length so the solver itself tries to also calculate the focal length again and and get closer to my initial setting so if this was not correct its trying to improve upon this in the solution and once I click solve it starts calculating and depending on your machine and lengths and settings and blah blah blah it will take maybe one two minutes or longer or shorter in my case so this was pretty quick and now I have my solution calculated and I can see it in different ways first of all there is a error calculation and here it says average solve error 0.3 pixels and this is damn good actually so the manual suggests that everything below 1 is already considered good and point 5 point 4 or so is considered excellent so apparently if I go by this this should be a very good track and this is because my image here has lots of great features and lots of depth and you know the tracking points are pretty good in this in this image if this is not good and if you have to improve upon things then you can start looking at which of these points are better or worse and you find here in this menu above the viewer there are different settings and I always have to look into which is which but you can you have here a solve error in this coloring menu so different tracking points can be colored in different ways you can manually color them for to use them later for reference but it automatically can color here by solve error and I don't see much because of the contrast but if I enable this I darken the image and now you see some points which are yellow and some which are green so green are the ones where the solver figured out these points are pretty good and gave a good result yellow there was a little bit of a problem and do we have any red ones let's see yeah here's something red so here the solver had some some issue and if I look at where this point is right now I don't even see any feature I don't know what was tracked here maybe it tracked it tried to track this side of the tree here somehow not sure so if you have a lot of red points you know that there are problems you can also enable the arrow itself you can turn on all and then you see numerical values where you have the biggest errors in your solution due to which point what you can do as you can delete individual points then you can go back refine this change the threshold if you don't have enough points all in all so there are different things you can tweak and typically you might have to run the solver multiple times until you are there with the final solution so you might delete some points just mark them for example if I want this red one want it deleted I can just mark it and press Delete key and it will just be removed and then I can solve again if I if I want to yeah so I can run the solver after adjusting removing some of these tracking points I can can run it again changing some of the other parameters maybe even change the camera parameters and then run it again until I get an error which is as good as it can get the second point you can do is this camera tracker so I can see here the points let me quickly turn off the these values again you can see the tracking points on 2d but after you have solved you can already look into a 3d scene as well and how you can do this you can just add a 3d tool like a 3d merge or transform doesn't matter and connect the one of these outputs here to this merge and there are two outputs on this tool actually and one output is a 2d output and the other one is a 3d output and the scene output here the lower one at the moment scene output this should be a 3d output and if I connect this into my 3d merge node and put this on the second viewer I already get a 3d point cloud here so even before exporting I can see here the camera this is what was calculated with a camera movement that was calculated by the solver and the points that were here on the frame put into a 3d space as a point cloud now this point cloudy I can later use for reference adding elements and I can use the camera for my 3d scene once I'm happy I can go and go to the export tab and can here just click on on export and this will give me a few tools in the flow let me increase the flow area now to bring this up a bit and let me move them somewhere down here maybe so what do I have here I have my camera here in 3d space then I have my point cloud I have a ground plane which is quite nice so this is a plane which was just inserted in a default place and I can adjust it and then I have a renderer automatically here added to the end you also see the image again projected into the scene this can be nice for orientation in the beginning if you don't want that you can always disconnect this here yeah so for now let's leave it in so the next thing I do is I like to adjust the ground plane and let me bring the 2d image in here so you'll see it here as well and in the ground plane for now just disable the wireframe for a second so I see a full plane here just so I can see it better and let me move this a bit and I oops and I want to move this towards where all the the points are from the ground so if I look into my original camera track you see that these points here they were all tracked from the ground so presumably these points the ground plane should somehow go through these points and again the ground plane is mostly for reference when adding 3d elements onto it so that you know that that they're really on the ground and not floating above and so on so let me bring this to the lowest through the lowest points here and it up a bit and I don't think I need to change the orientation sometimes you might have to tilt it a bit depending you know if the camera is tilted during filming it's possible that you get something here which is not flat but here I think the orientation is quite good yeah so maybe that's not the ground is also bit wobbly so it won't be hundred percent but I think this is a fair approximation if I go into the view again I can now let's bring this again to wireframe now then okay so I can at the end I can just disable it before before rendering but for now I just use it for orientation now let's bring some elements into the scene so I already have them here I'll have my signpost and my barn picture and I can use now the image planes to bring them in so let's use an image plane for the signpost and one for the barn and see where we can add them in starting with this signpost and now I need definitely a bit more space so let's see where it is okay and bring this in full screen so it's floating here at the moment and I definitely want to bring it down onto the ground plane so and maybe maybe I make it again fully opaque like this or I can even go into transparency here and just give it a bit of an alpha channel maybe that's the right way to navigate here oops wrong control let me do it from the image plane move this up a little bit yeah this should be yeah I think no it's almost perfect okay now it's standing on the plane and of course you can also go by the points here so especially since the ground here is not completely even that may be even better and let me bring it a bit forward it should be in oops too much way too much and let me turn off some of the stuff here just so I can move a bit faster okay let me bring this a bit forward maybe here and let's look again at this points here okay they're a little bit lower in this area because this is a bit lower so let's bring it tiny little bit down okay now when I bring in the other part if I bring in this bond and let me bring this image plane in here and place it somewhere where do we have it by default somewhere here let me bring it here to the side and this definitely needs to be much larger let's go here into the scale bring it up a good portion and I wanted to appear at at the end here on the standing here on the side like this so let's bring it there I think I need to bring it much make it much larger let's see maybe that's too large we'll find out and rotate it and move it again and [Music] let me check compared to the ground plane and maybe I should make this ground plane much bigger as well just so that it's everywhere so let me make this a huge grant plane again the plane is only for reference we can remove it at the end and I need to bring this so somewhere above mm okay so the points got to here we are more or less on the ground I'm okay with the orientation maybe I'm doing it a bit front and let's check again okay okay now that I have the elements in place let me disable the ground plane and then we can have a look at how things are looking here actually I would say terrible I would say this needs to go further to the front I don't want this bright patches here this looks not very good so further to the front so I will start somewhere I will start somewhere here and then do some improvements okay let's have a quick look at the scene in motion so the signpost here I think looks pretty okay this looks solid and then as the house comes in it looks pretty stable here and yeah I don't see any at least from a first glance I don't see any sliding or so so I think the track and the positioning should be relatively solid so next step I need to do is work a little bit on the integration and get this the color correct and so on so first let me do quickly something on the signpost if you want maybe you want to add a text to the sign that's quite simple let me just do this directly infusion and a text note here a lot of 3ds text sorry I can use a 2d text here it is and a merge and just merge a text note over the signpost and bring it up what do we want the text to say I don't know 3d set extension and give it some I don't know brown color and a bit larger and now maybe I want to in the merge maybe I want to multiply instead of put it normal let's see how that looks like yeah so it's getting darker into the wood and I keep the structure of the wood okay I can play more with this but that's not really the point of this tutorial just to have something on this board the next point I can do some color correction here let me do this overall after this merge just add the color corrector and perhaps in this case reduce the saturation a little bit well that's up to you what do you think is this working best here maybe I even leave it like this now one element that definitely needs some correction is this house obviously so let's look at the end and this definitely needs color correction it doesn't match at all enter the scene so again let me add this and first I might even colorize this a little bit and give it a bit of a brown tone just a little bit like this I think it's good that it's flat overall but I can perhaps reduce the brightness a little bit and maybe we work with brightness and contrast I'll leave this so definitely some bit darker and now the point is I think not so much the brightness but more the biggest problem is here the ground and let me see what I can do there and I think I will first of all remove this grass here so I left it in when cutting it out from Photoshop to see maybe sometimes you can can use it or can use some of this but this all in all it looks not very good I will fix it in 2d and we'll fix it with a paint node let me add a paint node just quickly and I can use here on the color nodes I can bring this down to completely black and alpha zero so now I have a paintbrush here which paints with a which basically paints with transparency so my paintbrush will cause transparency and it should do this over the whole duration so I will and the stroke duration I will just make this maximum so this whole thing here is 500 frames what everyone likes 1000 so I just want to make sure that everything I paint in is permanent and now let me bring the paint into the view here and quickly I can remove here and I'm using the paint tool rather than a polygon mask because I can make a less perfect more natural kind of mask here so I can just go around here and maybe I leave some of this stuff hoping that it fits together with the other grass and make sure that otherwise I'm not too perfect and not just have a straight line because the new ground will also be a bit wobbly so I think this looks better and let's have a look how it looks like in the final image now I definitely need to bring this down again a little just little bit and the image plane here y-coordinate just so that we are covering this okay this is of course they're not perfect and it won't get perfect here to be be quite honest but a few things you might think about first of all if I had a choice how to shoot this and how to integrate this I would say let's not film the ground here at all let's just frame this so that the horse comes in somewhat like this without the ground being film because then you have don't have to worry about the grass and it will be much easier to integrate this element now if you do want the grass the next best thing would probably be to go into a photoshop and try to clone some of the real grass I mean from the from the footage and try to paint it a bit on top so that there's a little bit of grass here on top of the house that could work you could theoretically you know create real 3d grass or something and put that into the scene but there we are getting really into more way to complicated stuff for a simple fusion compositing tutorial and out of this domain here what I want to do is I want to give a little bit of appearance of grass by painting out some some grass I just go into the paint note here let me go again to frame zero I quickly take my drawing tablet and just make some very fine let's bring the size of the paint brush down and just do very quickly here some some transparent things that look a bit like like grass at least from a distance and I just do very quickly some something like this and let me accelerate this okay this will not lead to the most realistic grass ever but if I look at the end I imagine that this looks a little bit better than before and we are kind of faking faking it a bit here yeah not perfect but if I that's on high quality for a second so this is how it now looks like on the ground and I think that's at least better than what we had before even if it's not perfect okay final thing let's look at the light behind here this has very nice lighting the scene and if I look at some reference to see if I look at the trees you see that there's actually light blowing blowing over so everything gets a bit blurred by this light so what can we do here we can do something like a light wrap and where the light from the background is going a bit over the edges spilling a bit over the edges of the foreground and I think in this particular scene this may be the right thing to do yeah so yeah I'd like to look at this tree here so you see that there's light kind of wrapping around till here and then only here the tree gets darker and so the you have this kind of diffused glowy filter edge and if I look at the house we don't really have that so what can I do first of all I think it's time to go from 3d space into 2d at the moment the reason why I see everything here is if you remember that the camera here is projecting the scene so there's this back image here which is coming from the camera we don't really need this so let me disable this so now I have only my 3d elements rendered instead I go here from the forest scene and merge this over just with a simple 2d merge and now I can use whatever tools I want in 2d so here I will now build the light rap or something similar to a light wrap and a tool that is pretty nice is this matte control tool matte control from the tool menu and this can basically extend and shrink the Alpha channel and do manipulations like this let me show you what I mean so if I blow a bit and then extend or shrink you see that on the edges I'm manipulating the edges bringing the edges in or out here and I actually want only the edges now so what I will do is add a channel boolean and subtract my manipulated image from the original so I add a channel boolean connect my original 3d output attach subtract the manipulated one select here subtract and you see now I have the edge here only and I can decide at overdo it for the beginning can contract this and blur it then you see I get a blurred and contracted edge probably I don't want this in the lower area we'll see for now let's get started like this and now I can use a go back to this image what I want to do is I want to blur the background and merge it carefully over the foreground so I will use a blur tool and from my background image get a very blurred version of this background maybe like this but I don't recognize the features too much then I can merge this blurred version over the foreground and I do this only as per the boundary areas which I got here from this Matt control so let me put this into the mask of the merge and now you see what happened I merged a blurry background over the foreground on the edges of this image and of course I can now modify this maybe I want to only screen it oh maybe we leave it normal but I can reduce the blend of course and I can further adjust these boundaries here so now let's dial it maybe relieve the blur but dial this back a little bit so I can give this a very subtle glow here just for comparison if I disable it enable it so you see what is happening okay so this was a very simple kind of light wrap you might want to adjust this further maybe mask it a bit check doesn't make sense at the bottom doesn't make sense that the roof doesn't make sense on this signpost and then do all kinds of fine-tuning and then you come to a result like this one in this tutorial [Music] okay this was again a full basket of different techniques in this tutorial starting with a 3d camera track going over the 3d projections onto the image planes in the 3d space and then doing some 2d post manipulations afterwards in this case with this light wrap I hope you enjoyed this this overview if you have any particular thing you want to have more in focus let me know in the comments I'm thinking maybe the 3d camera tracker could use a dedicated tutorial sometime later let's see just to get some more advanced pictures into this otherwise again let me know what you think and what you would like to see and see you around next time thanks for watching [Music] [Music]

Channel: VFXstudy

Views: 39,438

Keywords: Set Extension, 3D tracking, camera track, camera tracker, card projection, compositing, light wrap, VFX, VFXstudy, Fusion, visual effects, Blackmagicdesign, Blackmagic, DaVinci Resolve, Fusion Studio, Resolve 16, Fusion 16

Id: 4I4C8gDOQS4

Length: 35min 45sec (2145 seconds)

Published: Sun Jun 23 2019