Paint and Track - Essential DaVinci Resolve Tools to "Fix it in Post"

today I want to show you how you can use paint and planar tracking in combination in Fusion for some simple post-production fixes this can be super useful for a variety of things like doing sign replacements removing logos that shouldn't be there removing some unwanted objects far in the background sometimes even stuff like removing power lines sometimes more advanced things like doing some even digital make up in some cases you can do just some simple paint techniques and then make sure that you track that paint frame-by-frame properly so that it matches the movement in the scene or the movement of the camera tracking in this case can sometimes mean a simple point tracker sometimes it can be very advanced and can mean I need a 3d camera track but very often a planar tracker can just do the job and this is about the easiest scenario I believe for many cases so this is what I want to do today I did an extensive tutorial on cleaner tracking last week so if you have just basic questions about planar tracking you may want to have a look at this tutorial as well today I show it to you again but now in combination with some paint and I'll show you how I do a simple logo replacement in this case okay so I have the exercise 5 again on my website for download if you want to follow along and let's jump in I have in front of me this river ferry here and I want to replace this logo now let's look at the clip so the ferry comes in and the logo comes in so it it comes from out of the frame into the frame and then the logo is in full view it moves it changes perspective sort the ferry turns comes towards us and he had also moves out of the frame again so I need to attract this area and then do some paint to remove it and replace it in a realistic way with my own my own ferry line so let's get started first of all I need a trick and then I will show you what the options are to to work on that trick let me start with the planar track I just add a planar tracker from the control spacebar selection planar tracker and I select a frame where it's nicely in view where I can see the whole area and I select an area which looks like it's good to track so for tracking you want a lot of features which are all in one plane so in this case I can maybe even go like along here and take this is like all in one plane right so I have some contrast I get these areas here these points which looks good I get some of these pipes so perhaps there are some contrast points which the tracker can latch onto so it's maybe good to take a bit more than just the logo just to get good tracking data then from the frame here I click set to select the current frame from where I selected my tracking area and this tracking area I can now set this I can set it to a hybrid point area that should work perspective is correct this should all be fine and I can track from here once to the end and once to the beginning let me start track to end and there are quite a few frames here so I might accelerate this a little bit okay here let my ferry went out of out of the frame and I can go again click on go to get to my initial frame and see how I can track to the beginning and at some point I may lose the track but that's okay okay um I think the last tracks here were not reasonable but till here my tracking area looks fine so I will delete the tracks before that so I will delete the tracks because there's something strange is happening so you see it's jumping so I take the last frame which looks reasonable and then I delete the I trimmed to start delete before let me check again at the end did something similar happen yeah something was jumping here so I take the frame here this looks still okay so this looks fine this looks fine I think yeah and then something strange happens so I think until here it looks fine and then I delete the remaining frames to the end and I can either try and improve my tracking here maybe change from perspective to translation and so on and still try to track with a few more frames but instead I will just go to the spline editor here and I will try to extrapolate the rest of the movement and hope that this is good enough so if I go to the spline editor fit this to the view you see the tracking points these are all the points the points from this tracking path so these are not like coordinate points but keyframes basically and what you can do here right click and do a gradient extrapolation and that says just continue the movement from the last frame continue it in a linear way continue it forward and this means that yeah here it's it's also shrinking apparently the last frame was also shrinking it but it doesn't matter because it's going out of frame I just need to make sure that the last area here is really moving out of frame and I think that should probably be okay and at the beginning the same so here this is all still track still tracked and then it's well it's moving out of frame so probably that's accurate enough for this example with my track in place I have two different options to do my paint work now option one is I use a steel frame I take one frame from the sequence do my paint work on that steel frame and then use the tracker to track it on top of the moving image that's one option option two is I take the Moving Image use the tracking tool to stabilize the area where I want to do the paint work this planar region stabilize that work on the stabilized region and then undo that stabilization again so so there are two different ways both of them have advantages and disadvantages and I will show both of them to you of course so let me start with the method one creating a still frame I can create a still frame from the time time speed time stretcher both of them worked I would just use the time speed tool of course you can export a frame and there different ways let me start from here so time speed I just set the speed to zero meaning here I'm I now get a still frame but I don't want this still frame this is the first one here at 327 but I actually want the frame that the planar track I used as reference time because this is the frame that a planar transform will use again to transform on top of the moving image you will see in a second so first of all I need that frame here at 700 so I get this via the delay and the delay from the beginning of the clip is to 700 is 700 - the beginning frame which here is 327 so in DaVinci Resolve I have edited this clip so I have cut a rate 327 frames so this is the clip handle in resolve and I just need this for the delay here so that's why I and others and thereby I have now a still frame here so this frame is frozen at frame 327 which corresponds to if I go here in toward this viewer corresponds to frame 700 let me do that side-by-side so this is frame 700 and this is the sorry frame 1 and 700 yep it's the still frame I can actually enter it here or you can go into the tracker and click on go to get there okay so these are now the same so here I have a still frame and in this frame I will do some paint so let me already add the paint node and let me just set up the flow first so you can imagine I start now doing some work then I want to merge this over the final frame and I want to reduce this to this area I don't want to use the full frame but only this area so I will mask it out with a simple polygon mask so something like the area where I'm maybe I give it a little bit of space so I I can be generous with my painting and I will mask this area out now the different ways to apply a mask I want to apply it after the paint node so in the paint node I might still clone from wherever I want to in the image so I don't want to mask the paint node in any way I want to be fully flexible here but afterwards I want to apply that mask one way is to add any node I just used the channel boolean for example attach the mask go into the settings and select multiply by mask what does that do it does this it multiplies by the mask there are a hundred different ways of doing this this is just one way um and I might be there's a little bit of a soft edge just so that it is easier to composite it back into the original image and now I will use this still to merge it back on top of the original image here actually I no longer need this trekker I'm done with the tracking if you want you can keep it connected but the tracking is done but what I need is I need the transform from the tracking and this I get from the planar tracker create planar transform it was created here so you get a transform node by clicking this and I bring this into the flow here to transform that segment which is part of the still image transform it back onto the merge and this transform uses reference time 700 so on frame 700 the transform doesn't do anything and for all the other frames it matches the plane that was tracked to those frames so reference times 700 and now I should have my still image here on transformed back on top of the original image and basically you shouldn't see a difference for now because well I'm on the same frame so you will see a difference once I start painting so let me start here I see the final composite on the right side here I see my still frame so let's see how that goes I go into the paint node and now I painting on the still frame but I should select you still have to take care of the duration of the paint node because you don't want your paint to work only at a certain time here even though it's a still frame you want it to be valid everywhere so a quick way to do that is here to use the single stroke instead of this multi strokes or the multi stroke has a duration here single stroke by default works on all frames by default so I used the single stroke for the paint let me bring the paint in the viewer and then I use the clone tool here clone tool you can use alt to select an area and then apply to clone if you happen to have a drawing tablets now is a good time to start using it I will probably also want to increase the view here decrease the brush size maybe a bit to make sure that you're getting these lines here and then the principle is always alt click to select a reference point and then draw to clone the image from there and I will do this quickly in accelerated speed okay that's it for my paintwork I hope I did a reasonably good job here you can use of course all the other paint tools as well you can draw with color as well adjusted cloning but if you need to draw these lines and maybe get them at the better you could also use here the color bucket after all it's just a still image right so you can really do whatever image changes you want on it you could also go into an external application if you prefer to do this kind of stuff in Photoshop or any other paint application you can just right click here on the viewer actually on the freeze-frame if I go back here on that time speed I have my freeze-frame I right click save image and I can just export that say a PNG file I can just save a PNG file take that into any image application and editing application do the changes I need to do but the paint note here is is also quite sophisticated and getting something simple like this done directly and fusion is quite feasible let me take this one little step further and add a logo as well you can add any text or logo you want I think I'll make a little bit of advertisement for my wonderful fusion courses in case you don't know about them they are on VFX Telecom so I will just add my own logo and I think I should advertise my courses on the side of a fairy ship right I mean that should be the minimum or okay maybe that's a bit over the top but I want to have some fun as well so I will just add my logo here way too large of course and actually I will first do a corner positioning here to map out a rectangle area where I want to position this to get the perspective right so I can use a corner positioner and you can use the corner positioner in different ways but I like sometimes I like to just match it to an area which I know should be a rectangle even though it's not the final area but where I see like straight lines which sometimes getting the perspective right so in this case I know that this is probably a straight line on the ferry the backside and here the ground and I can follow up here the blue line here on top and make sure that this is reasonably straight and thereby get the perspective even though I don't get the final size and I can always do a separate transform to get the size right so even an x and y separately like make it small and then get the aspect ratio right again but at the same time I know that the final perspective is if there is any strong perspective now here there isn't any strong perspective but you know sometimes it's it's hard to judge how the perspective should be and this is a simple way to get it accurately matching to the perspective of the ship in this case okay so let's say I want it in this size and so this is the first step next I probably want to wrap it like around these changing geometry of the surface here so the unevenness of this the surface I want to consider this and there is a tool in fusion called this place and this tool is often helpful for like light reflections reflections shadows that kind of stuff and it can take illuminance and then based on a luminance input can wrap images but you have to pay attention the input that you want to displace should match the size of the reference so my reference will be my painted out image so this will be my displace input so the green one is the input that I used to displace the orange background but the two should actually have the same matching dimensions which right now they don't so I will add I will just fix this by quickly adding just a simple plain transparent background note and I will before I go into the displace just merge this over just so that sorry let me add a merge like this I haven't changed anything except for the dimension of this image because here from my corner position I had the size four dimensions from my logo but now I have the dimensions from my background just met merged it over the background everything else we made the same and now if I use my displace no it already looks better and now I can use this displace especially this refraction of strength here now it also moves it so this is a bit like refraction from glass which is trying to assimilate based on the luminance of this image but I can always move it back at this and now it's like a bit hard and fuzzy here but you can spread it a little bit to get this done and go a bit back and forth you see that it's kind of wrapping around a bit yeah so maybe something like this to get a little bit of this wrapping here so the displace can work quite well for this if you need to go even further just another quick alternative or additional tip you can use a grid warp to manually add some some of these distortions so in this case I go to select it I don't want this magnet tool and go to a sauce and let me just reduce the size here and reduce it further actually I can edit grid you can go to edit rectangular which gives you these corners so I just position some rectangle corners rather here just to reduce it by going to the rectangle and then I can go back to grid now I have a grid over my text and you can see change that grid and get a certain number of grid lines I'm working on the sauce grid and what the grid verb does it transforms the image from the sauce grid to the destination grid and I will just let's put some clothes lines here along this fold like this and maybe two here as well and then I will use them on the destination grid to do some transform there as well so I this doesn't transform anything this is just setting up the grid then copy source to destination going to destination now I have the same grid on the destination I still haven't transformed anything but now I used the grid that I have set up from the destination to make changes and now the image is being distorted so that the image from the source is being moved towards the grid from the destination so now these two are different source is what I set up first I copied it to the destination and destination grid is the one I'm changing now and I can make such such changes here and maybe bring this a bit in just to squeeze things a bit to make this step here more more obvious okay so okay that was probably too much and you know you can dial it down a bit but that was the the idea okay maybe I tuned that a bit later but that's not really the point the point here is the tracking so let me go back to this before that only one last thing the merge node is set to normal so maybe I want to multiply in this case multiply makes things darker but I still have like these highlights coming through a little bit and I can reduce the gain a bit to make it a bit fader and perhaps it's a bit better integrate it this way but now I should definitely go back to the real topic here which was the transformation with the planar transform see if that all works together so I have mapped out this area and I see now that my polygon here a bit too small after I added the logo so I will change the polygon to get the full patch and give some space here as well so let's have a look at the final image and well this is rendering a bit slowly so I might put the render it out once and show you the final result [Music] [Music] and here in the render you see that it's working nicely in the middle section but in the beginning you see an issue of the weather logo isn't tracked properly and if I go to the end you also see an issue here with the color matching so the logo works this patch that I'm adding this still works fine in the middle but towards the end the lighting situation changes and then my patch stands out because the way I caught it from the still in the beginning is just too bright for the the end and here at the end it gets darker and it gets more blueish because of the blue water being reflected from the the side of the ship so the color isn't matching anymore what I need to do is in this case get to the last frame where it's working maybe here and try to do an animated color correction for this and then I can manually match it and this process is kind of the downside of this still image procedure that I showed you so on the still image painting is very simple but now matching it into the scene can be difficult if the lighting situation changes still if you if you do want to do the matching I would just add for example a curves tool which I can set two animated color correct this able D alpha and because I have transparency you want pretty right post multiplied on and animated so there's now a keyframe here and you can change the curve on every single frame so I will just go to where it's like very obvious towards the end and I can make a keyframe here just by animating this curve and you see it needs to be darker probably yeah definitely and then it should be more blueish so I probably have to remove some red for example and now it's getting a bit greenish so I might need to remove some some green something like this oh that that seems to work quite well at least on this frame and now I have automatically set another keyframe just by changing the curves and I have animated it between those keyframes so hopefully yeah this looks already quite good in the frames in between let's look further down here towards the end I definitely need to do even more so maybe I need to go to a frame here and and bring those down even more let's try to bring everything down a bit more no not yet so it will go in it will go in maybe bring red again down and you get you get the idea I have done tutorials on color matching before I don't want to go too deep into this instead I want to show you the second way of how I can achieve this paint and track trick and there I won't have that particular issue but I will have different issues but nonetheless there are two different ways of achieving this and the second way I want to show you now is to stabilize then paint and then invert the stabilization actually a stabilization may not be the correct term here actually the stabilized but actually I want steady so I want to make the image steady so I click on this steady button in the planar tracker for my second approach now and you can go on any frame now and I think just for to make it simple I will again go on on frame 700 where I worked before let me enter that 700 but you can use any frame and then click set to make that frame stable or steady in the tracked area so it's now taking that tracked plane holds that plane steady and wraps the whole image around so that that plane remains steady in one place so how does that look like so let me just go back to a single view for now and enlarge this a bit so you see now here I clicked on steady on 700 if I go before you see the ferry was moving into the frame like this and then here it's moving out of the frame and it's transforming the rest of the image but it's keeping that rectangle that I tracked the or the plane that I tracked with the planar tracker it's keeping that plane steady in one place and that can help me to do the paint of course because now I can paint on every single frame I can use a clone which works on all frames I don't need to work under still anymore but I can just use that paint stroke in one steady area and then apply that on all the frames so that's the idea now so just the workflow so this is my steady image so if I have the steady image I need to invert that operation at the end let me copy the tracker and just paste it and here in the pasted I need to click on invert steady transform so this brings it back let me maybe I should go on to viewers so this was making it steady this is bringing it back this looks good so I'm back to where I started here so from here I can output and in between I need to do my paint let me see if I can just copy that paint notes inside it only cloning here maybe I'm I'm lucky yeah so I paint it out here and now if you look at that steady image even if I go further to the beginning or to the end you see okay so it doesn't look like perfect here however it does use you see the the color at least is is correct so maybe my paint job was a bit too rough for these darker areas it worked in the in the brighter region but I would have to do this this paint a bit more carefully and I could work on a different frame now right I could work different frames to get that paint adjusted properly now but you see that the color is not sticking out so my paint which was using the clone brush is now cloning from every single frame of the image that's the idea I could do the paint job again and make sure that it's nice towards the end but I think I will leave the paint demonstration and just show you a bit of what the issues are now with this approach first of all after I added the paint node you may have noticed that something went wrong here on the right viewer with the second planar transformation so before I had if I don't have the paint node nothing happens if I turn it off but if I take it out I take it out I have inverted correctly if I bring it in you see something happened something gets chopped off and the reason here is that the paint node chops off the frame so this planer trekker transforms the image and transforms part of the image out of the frame and the paint node doesn't care about anything outside the frame the paint node only keeps the frame itself and that causes the issue here you can actually see that if I show you the domain of definition so this is just a little bit of expert advice added into this tutorial so if I go to region and click show domain of definition in the viewer if I enable this you see so first my media in hat you see this dotted line which shows you 0 0 and here 1920 1080 so there's a domain of definition where computation takes place in fusion by default that's just the frame then here we do a transformation and now you see that domain of definition gets larger so it's automatically extended to allow for additional information to be computed here and if I now go to the paint note the paint note just removes it again and there's nothing in the paint node to prevent it from doing that there's no function that I have seen for that so instead I'll just do a simple trick I just merge the paint on top of the original image so the merge retains the domain of definition and if I attach the merge now to the transform now you see I'm again doing the correct transform here so I'm keeping that however far it's transformed out of the image it's being now transformed back and the paint not only worked in between so this is the workflow already if you want to do the paint on the steady image between those trekker nodes keep in mind if you now want to do different techniques other than cloning so with the cloning your cloning from each frame if you want to apply other techniques if you want to add the logo then this logo should again be animated to match the different lighting situations right so in those cases if you want to use the paint bucket for example instead of the clone tool if you want to add additional elements and so on I feel in those cases working on the still image may be easier because you have to adjust all those elements anyways so the final to the final scene but if you do only clone strokes and you want to clone on every single frame then you can have an advantage with this because potentially you are cloning also the different lighting situations however it also means you have to check every frame of course if your clone works so in this case it seems to work in most frames but towards the end I'm not very happy so there were some some things at the end which I feel didn't look very good here so that's one thing which where I have to revisit my paint and cloning and see what I did here um but there are other things that can go wrong especially if you clone from areas which get moved out of the frame so because I'm cloning on every frame pay attention to what happens at the beginning and at the end if your area moves in and out of the frame like here now here at the end that still looks that still seems okay let me look - what's the beginning and bring this bring the final image in and here you see something is wrong and if I look in the paint note you see my cloning is now cloning from transparent pixels so these cloning strokes that I have in the paint node these cloning strokes they go you know all over the place basically and they also cloned from pixels which now are not available in the first few frames because the Fairy is only moving in so in this scenario I would need to do something extra to fix those frames fundraise to fix it one way would be to go into the paint strokes and do separate treatment in the beginning maybe fade it in and out maybe use two different paint nodes one for the beginning one for the end third one for the middle maybe you know and kind of fix it that way there is also if you work with the clone brush here there is a time offset which is normally set to zero so you with zero means you are cloning from the current frame you can also clone from previous or from future frames could be a solution or if I want to stick to what I have done so far I would need a way to get additional frames here to get additional extension to the ferry before it actually comes into the view and there is actually a way to to do that there's probably more than one way to do it one tool which could help here is called frame average frame average I don't know if you have used it before it's a bit so this two can use frames average multiple frames together and there's also the trails tool which can accumulate frames so there are some some tools which can work across a range of frames so if I bring this frame average tool in here and I'm just looking now at the beginning sorry I didn't want to disconnect here just here so I bring the frame average tool in and just at the beginning I want to go forward sample forward and want to add some frames and if I add if I look at my image here from the ferry and then select some frames here you see I'm taking additional frames which are from the future basically and I'm averaging them together with the current frame that's the idea of this frame average so I could use here even a larger number although computational time will increase so these tools are of not super efficient but you see this way without the frame average with the frame average and the frame average is averaging them so so everything gets a bit blurry as well because it's taking the average of of different frames that are coming here so I'm kind of getting this area you may want to still use this and and merge the original on top so that I still have where I have the original frame that I don't use the average but use a clearer picture here so if I look at this merge without so you see something so you get changed so some things are being averaged here which is maybe not a good idea always but to the end where there's just transparency and now getting some pixels extended and what does that mean for the paint do I still have it in the view oh yeah I think it's already getting fixed here so it's now sampling from this area now this area says semi-transparent so if I want to drive this even further I might still have to do more fixes here I might need to get rid of this transparency here of these average frames replace the alpha channel with a solid image and and do it that way to get to get here full opaque pixels so that which can then be cloned from and so so these are the the tricks I can go into I don't think I want to drive this to completion here again it's they're different ways how you can do this you just have to watch out because here it's it can get it can get tricky okay I think for this example I don't even need to go that far it seems like I'm I'm kind of fixing it here there still may be a little bit of tuning that I I might need to do but the basic principle should be there so this frame average it's just one tool I don't want to go too far into that direction it's just what you have to watch out for so if you are using the second technique where you are cloning on the stabilized steady image and then you have to make sure that your whatever paint work you are doing is actually working on every single frame so if you are cloning you need to make sure it works on the first frame it works on the last frame it works in the middle if you're cloning from outside the frame obviously it doesn't work so you have to do some some tricks and think about how to to improve that on the other hand if you are just taking a still image or completely on the still image then match the still image into the moving image I think in many cases that can be easier however again then you have the issue that you have to make sure that this still image matches the scene in every frame and matches the lighting situation and so on so each of these scenarios have their pros and cons it's good to know both and that's it for this tutorial I hope you enjoyed it let me know click you know like and thumbs up whatever let me know what you think and what you need to know and what you want to see next thanks for watching my name is dan see you next time [Music]

Channel: VFXstudy

Views: 23,778

Keywords: Blackmagicdesign, Blackmagic, DaVinci Resolve 16, DaVinci Resolve, Fusion 16, Fusion Studio, VFX, Fusion, visual effects, compositing, planar tracking, planar tracker, paint, clone, object removal, sign replacement, logo replacement

Id: 9PHtpkGwPHI

Length: 40min 45sec (2445 seconds)

Published: Sun May 10 2020