Cleanplate Projections | Nuke Compositing Guide

foreign plates are the Unseen visual effects in the visual effects world where artists paint out and remove objects so you'll never see them in the film if they've been done correctly there are different levels of detail for different Productions so television episodic and films have different requirements my background is working to feature films so we'll be talking about the techniques used for this level work professionally by the end of this video you'll be able to look at a scene and break down the steps necessary to approach a clean plate with modern difficulty as well as understanding different methods of 3D reconstruction that we can use so this is the scene we're going to be looking at and we have a camera that kind of pushes in here the the first thing that we actually need to do is establish 3D for basically everything that is not the object that we're going to be removing so we're going to be removing this object that's in the center and also flattening the grass just a tiny bit around it so we need to have 3D for the area behind and the rocks that are a bit vertical the three primary techniques we can use are cards Point Cloud meshes or a lighter scan so we need to have a ground plane and some kind of vertical geometry or proper 3D geometry for the rocks and an area that's behind the Rocks if we do it like this instead this would be the alternative and this would be kind of a mistake so if you project this whole image onto just a flat card you're going to actually lose the three dimensionality of these rocks and it's going to feel like you have a stretch projection so beginners might make this mistake sometimes and you'll notice in the projections they're looking stretched and that's because they're not actually building out the geometry properly and layering it together so using the first method it looks like this we get a basic Point cloud from the camera track and then after that we just Place some geometry flat and a little bit vertical where the rocks are the second method is trying to achieve the same result except with the point Cloud generator so if we go to vertex selection and we select the vertices like this and then we go to groups and we say create group it will create this group here that we can click on and say bake select to group the mesh and when you do that you can get this geometry that represents across your point Cloud it's a little bit better for organic scenes as well whereas cards sometimes could be better for flat surfaces so this is kind of what that would look like our last method here is using a lighter scan which is more accurate than the point Cloud mesh and this can be captured using an iPhone Pro which has a lighter scanner built in and we can reconstruct 3D scene using this depth data that comes from the laser so this is showing where the lighter scanner is on an iPhone and this is me using polycam just shooting some lidar and getting a capture so it's really easy you just basically film and it uses the lighter scanner in the back you can also use the photogrammetry mode if you don't have the scanner and that will also give you a high quality model out of the application so polycam is a sponsor on this video but I thought it'd be a good fit for this channel because I've been using their software for a few years and I think it's really convenient to have a lighter scanner in your pocket being able to capture large-scale environments either for lining 3D up or doing projections directly the photogrammetry side is really good for assets as well to get detailed assets and not have to worry about the processing time or dealing with that on the computer now I did use a lighter scan for this specific project even though we could have done it with either of the methods there but basically one thing you need to do is delete the object that you're trying to clean plate out because we don't want to project onto this so basically what I did was bringing a blender and delete the faces and you get a geometry that looks like this but after that you can go to the vertices and just merge them all to the center so you can do this in Maya as well it doesn't really matter you just need to basically fill the Gap so that we have a solid geometry that we can project onto so you can see here I just merge the vertices and that's basically how we get a mesh that we can project onto but not on the object that we're removing so that's kind of what we got and it doesn't matter that the textures stretch because we're not actually using this texture that came with the lighter scan so One important principle is to use what's real and use multiple projections so if I just do a very simple projection setup here with one projection we can see something like this so I go to the last frame here and I'm projecting this image so just to quickly run through it I'm just doing a frame hold here and I am painting out on that frame now normally you would do an undistort and you would do your paint and then you would redistort at the end but since we're dealing with grass and really small details and everything is in the center of the frame and it's an iPhone which has low lens Distortion relative to other lenses I've basically just turned them off here because the sampling is kind of making it a little bit blurry which we don't have to do if there's no noticeable sliding so that's just a quick thing to point out but anyways we have a frame hold we have the paint and we're projecting onto the light lighter scan that we've taken and this is kind of what we have but the idea is with multiple projections is to use what's real from different angles and so from our first angle here we see that this rock is actually covered by the object that we're trying to remove but on the left side we see this rock that's uncovered so when we're painting it out we're actually kind of having to restore the rock that's here but if we actually go earlier into the video and we go backwards in time we can actually see that rock so we don't actually want to make up what's there because at the start of the video you know we're kind of losing detail if we do it that way so rather what you want to do is do at least two projections if not more to establish as much real information as we can so here we might project the left side to get to this detail and then more towards the beginning before the Rockets covered up we would project another projection here and basically you just have multiple patches of projections that you layer over Just A over B uh after that point another technique that we need to keep in mind when doing clean plates is to dissolve your patches so a lot of times when you're doing projections um one projection will only work for like a certain portion of the frame which means like your geometry might not be completely perfect or you might see from a slightly different angle of that that geometry so the projection doesn't work from every single angle and that means we have to be kind of dissolving to account for those perspective changes now this can be a bit difficult to describe just through a video tutorial because you really have to see the really small details and Parallax shifts that are happening in footage which takes a lot of like scrubbing back and forth and looking very close so the best way to actually learn this is just by practice and actually running into the problems but I'll explain the example quickly here kind of where we use this example so up on this part of this precomp if you look at this rock towards towards a certain frame it starts to look a little bit flat which is actually not what it looked like in the real plate so if I compare to the real plate you see the angle of this corner here it's a little bit different it's kind of being shifted and so this is where we need to create multiple patches to make sure that the angle is changing with the perspective and The Parallax that's actually in the real geometry so your geometry will never be a hundred percent perfect 100 lined up so that's where the 2D techniques and also also dissolving multiple projections comes in so if we look here if we go towards the end of shot what I did here is a patch so if I disable it and I go to this Frame and I re-enable it you see I'm kind of correcting uh where our perspective started to shift and kind of just putting a patch over this area which is bringing some sharpness back as well now this projection actually only works for a few frames if I go backwards in time we see that it starts to become a little bit flat again and so either there's multiple ways you can correct this you could use tools like an I transform so if I create a I transform node you can get this on Wikipedia if you just type in I transform you can use a grid warp and you can use other just 2D tricks to kind of force the perspective meaning you would take this corner and just shift it over just a tiny bit so that it kind of matches with the plate so if you look at the plate we could just shove it over a tiny bit and kind of correct the angle that's not correct with the geometry and projection so you would use these just before you project it so just before the project 3D you kind of correct the patch the other way to do it is to use another projection so what I did here was I go backwards in time where it starts to break usually with patches you want to see like what's the last good frame that's kind of working and then go backwards in time and where it starts to break is where you're gonna have to do some more work so you would create a patch here and you would dissolve forward in time to create that seamless transition so it's pretty difficult but it takes practice so we can look here at what I did so I go to the key mix and what I did here was I actually restored some of the original plate again after I've done some projections so I kind of bring it back and then I frame hold that and create a new patch and so this is going to work for just a few frames this is like only five frames that we're actually using this new patch so if I look at the merge and we see what that change if I disable it and enable it we see that we're just correcting the angle on just a few frames just to get that perspective to work another thing we want to do is to hide your tracks so that is to say we want to basically try to blend our patches in a way that the viewer is not going to perceive that we might be losing a slight amount of Parallax through our projections so for example if we're projecting on the grass grass is going to have like you know an infinite amount of Parallax planes because each piece of grass technically has Parallax in front of the other now unless we plan on replacing the whole ground with CG grass and that's that could happen on a film where you just use CG grass you know you might have to come up with other ways to kind of fake that Parallax and so the two main ways of doing that in this scenario would be either to have a card that you have lots of subdivisions so if you create a card and you would increase these uh rows and columns to a high number and use a displacement so you could use a displace geo with like a noise pattern like really small you could do it that way and you could kind of just mess up this card a lot to kind of give it some little spikes to kind of simulate that movement of the grass over each other the other way is to just blend the edges where that Parallax issue might be obvious and so in this case that's kind of what I did I just took a roto shape and I used a fractal blur I'll attach the in the description where you can get this node essentially what it does is it just if you have a blurred Alpha like this or a sharp Alpha I suppose and you look at the alpha it's just breaking up the edge through noise so it's kind of going to make it harder to see where we place that CG patch over the real footage and that's going to help us kind of blend across uh what we're trying to add so this is kind of what I add over and I'm putting this over the shot so we're kind of replacing some different things and cleaning it up and we're also flattening the ground a bit because originally there was a hill here so I kind of flattened it so there would be less of a hill but I'm also just using this technique to kind of break up the edge so that's something that you can keep in mind just to hide any areas that The Parallax is going to change slightly so the last concept we'll talk about is projecting from the maximum resolution point of view and all that means is to find an area that is the highest amount of pixels for what we're projecting so if we're here at the sequence and we go further if we're projecting this rock for example we might want to go closer to it where this is actually a bigger on the screen because it means that there are more pixels meaning it's higher quality especially if we were walking even further away than this this rock is going to get really small so if you project from there and then we walk really close to it you're going to see that it's going to become blurry as we get closer and closer so in general we want to be at the closest point of view where we still can see things and that's going to be where you project now if your perspective changes too much that's where we go back to the technique we talked about earlier where we do dissolves between the patches so that we don't perceive any color changes or perspective changes being stretched from a different angle that's about it for this tutorial so if you guys liked it make sure to hit like And subscribe and for anyone who wants the project files these will be added to Nuke 202 so if you've already taken that class you'll have this additional project added and you can play with it there if you're a complete beginner looking for a full course on the 3D system in Nuke I'd recommend checking out nuke 202 because we cover Concepts such as camera movements Parallax tracking triangulation camera projections and a lot more so you can go check that out on composingacademy.com and the link is in the description

Channel: Compositing Academy

Views: 15,733

Keywords: nuke, polycam, lidar, projections, camera projection, cleanplating, vfx

Id: mEeCZFjpO8s

Length: 12min 42sec (762 seconds)

Published: Sat Jun 17 2023