Replacing a Window - Set Extension in DaVinci Resolve & Fusion

[Music] welcome to another set extension tutorial for DaVinci Resolve and Fusion today I'm using this footage from a model photo shoot and I'm replacing the white window in the background like this I have done a few tutorials on set extension recently this one here where I focused especially on color correction for atmospheric background integration then there was one where I extended the frame itself using paint and clone techniques one where I focused especially on the 3d camera tracker this one here is again focused on tracking but this time I'm not using a 3d camera tracker but I'm using a point tracker and I attached a mask and make a kind of fake parallax movement just using the point tracker so as always you can download the footage from my website and when you're ready let's get started this tutorial worked again in the free version of DaVinci Resolve or infusion studio whatever you prefer just load your the footage into the fusion tab and some background picture that we will place behind her here into the window and I will start now first by extracting the window and then I do some tracking and we'll do some match moving and see how we get the parallax required for this window so let me start with the keying for the window I add the luma keyer here luma keyer which just can distinguish between the brighter and darker parts here and you can see already so the window is white everything else transparent so maybe you want the opposite and can invert it I can crank this up a bit and see that I get not too much transparency otherwise at least I get this fully opaque the curtain on this side here I won't get perfect so I will also use a mask but let me use to bring this down a bit so we get a bit rid of these areas in the middle and maybe I want to bring the gamma a bit down to have not to have transparency in the final mask here now this doesn't work perfectly fine here you see I have some some problems here I still get some of the curtains here but I will not fine tune this to the very end instead I will do like a 80% solution and then merge it over the image and then see how good it actually has to be how the background is in this area and then maybe go back and adjust it a little bit afterwards so for now I start like this on this side I'm okay with this curtain this curtain doesn't work at all and that makes sense because if you look into it the contrast here is is very low it's extremely white so instead I will just track this with a mask so let me attach here a polygon mask on me go to the beginning and just mask this curtain once very quickly let me do this masking in slightly accelerated speed so you don't get bored yeah and while I'm asking you see how low the contrast is so keying something like this is not very promising I don't intend to rotoscope this frame-by-frame I just want to make a mask on the first frame and then hopefully I can use tracking in order to attract that mask to the background and here where the hand is and the basket I will just go go through and just estimate and maybe I have to adjust this later okay of course if you want to get really good with this mask you could also mask out the chair and then maybe do different keys in order to get this pushier a little bit better and get rid of these problems but I'm not doing too much now I just start like this and let me add this to the what do we need we need a solid mat here so everything that is masked is solid including this line and then the key should work inside and I get this curtain from the mask now I need to do some tracking and let me bring a tracker node in so from the menu control space bug tracker and not the camera or planar tracker just a simple point tracker and I attach it and here I will bring the tracking point over a feature close to the curtain and this clip here looks really good so this clip is I mean throughout the image this is not hidden she is not moving in front of it and it gives relatively good contrast in both directions so and I just make sure that the whole clip is in my pattern and my search area doesn't really matter just the default a bit around so this pattern here will now be tracked over the whole time period and I just use all the default settings I hope this will already do just click track forward and this looks already doing the tracking it looks very stable okay so this worked you also see here this flip book here you see a green bar okay for some reason I don't see a full preview of this pattern anyway so the green also means low tracking error so it gets this pattern really really well and that's also what we just saw now how do I use this you can use the tracker tool itself has these match moving operations here and you can use this but I often don't use it because I might use the same tracker for multiple things so in this case I want to use it to modify the position of this mask and then I want to use the same tracker later for the background and piping things through the tracker like connecting here to the foreground and then using this match movement can sometimes be confusing and that always of course it doesn't work for multiple elements and so on so often I just keep the tracker on the side and then use the tracking information wherever in the flow I need to use it so in this case I want to use it here on the polygon and for this I can go to the polygon go to the center position right click connect to tracker one and I want the unsteady position so unsteady is like the unsteady path that the tracker point followed steady is like stabilization or perturbation so unsteady position and now I can see what my lumic here with the mask does and if I play it you see that this mask looks pretty solid on the curtain I should look at the lower area because I was guessing a little bit with this basket here but I think this looks this looks pretty good here so yeah I think I got the curtain with this track so wonderful let's leave it at this next thing let's do the match moving so I have here my Empire State Building and obviously she is in an apartment in a building that is at least as tall never mind just for the fun you can of course use a different picture and let me just merge this over and okay I can actually what I can do is well either I can change this I can put it in the background then the frame size may not work and I have to crop this to get or merge it over a plain background whatever to get the background size from my model or what I can actually do I will just you know I will just connect not the keyer itself but the model and the key I will just use this for the Alpha Channel and just connect this to the Alpha Channel so I'm only using the Alpha channel from the luma keyer now and do the merge in with the merge tool itself and not with the blue mark here so here I get the Alpha Channel and right now I get the opposite of want but in the merge settings here I can do apply mask inverted okay so I have the Alpha channel from the luma keyer and I have my building here merged over where is it here merged over now I can do this in the merge I think I will do at least one transform here or maybe two so first let me I will separate this from the merge just for logical reasons and also for the for another match movement so first I want to position this in the right place and maybe even make it a bit larger and now so far this is just a static background which is not moving and if you play this then it kind of looks like a fixed background and the apartment is like spinning in front of it which obviously is incorrect so let me now do some match movement and I just two separate things I will again use a second transform so the first transform was just to get the initial position right and now I will attach another one where I do the match movement and I do it the same way I just did with the polyline I go to the center position we will right click connect to tracker one unsteady position and now let's see what we get so this transform does the match movement and maybe you even rename it match for example so this way I can see directly in the flow what tool is doing what now I have accurate match movement but now I have kind of the opposite problem now it looks like this is like a poster in the window right so it moves exactly with the window and there is no parallax so this is because it's now exactly attached to this point and it just keeps in the window like like a poster so the real perspective that I want is actually something in between right so first I had without much movement I have the apartment like moving in front of the background with the perfect match movement I have it like a poster I want something in between and I can kind of fake this with another transform if after the match movement I do some match movement then I scale afterwards and then the Centrepointe that I'm moving will no longer fit the scale and I get something that is like too large or too small and just due to the scale difference my match movement won't fit anymore and I get something that hopefully looks a bit like perspective and if that explanation was not perfect I apologize but I just demonstrated what I mean instead so I put another transform here after my match move and here I'm now changing the size and scaling a bit out and now this will no longer attached perfectly but you see now that here the distance between the curtain for example if you look for the perspective it's closing in here and it's wider here so I'm still not in the initial situation where it's just completely detached I'm still doing some movement but due to the scale I'm kind of not doing a hundred percent correct movement and I might bring this even down further to make this make this even clearer and now I have some problems here with the at the top so let me go back to my initial frame maybe bring this up a bit and okay so and perhaps even I might have to zoom a bit in in the beginning otherwise my frame size is just not enough but now I do have some kind of parallax movement now this is not accurate parallax because the background is still like a straight background it doesn't have any perspective change even though the camera is changing a bit but I'm hoping that it's not very visible as long as you see that movement compared to the curtains now you actually have the visual reference one more thing I want to do now let's say I'm more or less satisfied with the fake parallax what I can do this law looks a bit like there's no glass and no window at all and at the same time we still have these skiing problems here so I think what I would do in this case is not remove the curtain completely but perhaps leave a bit of the curtain and just make it like a transparent curtain instead so I think this might look more realistic also it might make more sense because there's also curtain here which is the reflection of the curtain here that had just been moved and now I could go even further and try to put a mirror image in here in which case I would have to key her out and so on and then it's getting more and more difficult but I would say for this tutorial I just make it a bit easier and perhaps easier is even better I go back into the merge node and I just reduced the blend a bit to bring here I can dial in how much see through my curtain should be and I just dial it in a bit also maybe I don't want the normal apply mode if you just if you want to have the contrast from the curtain maybe you can multiply in which case it only gets darker so you still get a bit more of this curtain and still get the background so perhaps that's a nice mode so this way I have kind of a shine through curtain in front my keying problems here now okay this is still not not perfect and would probably require a bit more treatment but all in all I think I'm I'm getting much closer with this okay so this is how far I can get with the simple point tracker now the complete accurate version would require again a 3d camera track but let me quickly show you some things why I didn't do that here you can try it if you have the studio version but the problems here are first of all you have not many features here that can be now this clip here was easy to track you could track here on this tree some points but otherwise for a 3d camera tracking needs usually lots of tracking points and they're just not many features here so there are some straight lines but not many points which are really solid recognizable here that's one issue another issue is that she is moving around in the foreground and you want for to get 3d camera movement you need to track a solid background not a moving actor so you need to exclude her and third problem this mirror that's also a problem because mirror images confused the tracker that the if you have track points on the mirror you don't get the right perspective right and the 3d camera tracker doesn't know what a mirror is so that's another problem so you would basically have to exclude all of this area and this area here will be pretty useless so it's really difficult to get enough good points for calculating 3d camera movement and working in 3d space here it's possible I think but very challenging so I'm happy that such a 2d solution was actually very simple just one tracking point a little bit of keying here and we can already improve the view from this apartment so I hope you enjoyed this as always please like and subscribe let me know in the comments what you want to see next and see you around next week thanks for watching [Music]

Channel: VFXstudy

Views: 15,986

Keywords: Set Extension, Window Replacement, Tracking, camera track, camera tracker, card projection, compositing, luma keyer, VFX, VFXstudy, Fusion, visual effects, Blackmagicdesign, Blackmagic, DaVinci Resolve, Fusion Studio, Resolve 16, Fusion 16

Id: FsxdguZo6fw

Length: 18min 4sec (1084 seconds)

Published: Sun Jun 30 2019