Welcome everyone to another blender 2.8 tutorial.
In this one I'm going to be showing you how to track deformable objects which is something
I'm very excited about because I've not seen a single tutorial about this on Youtube. So
as far as I know we are going into new waters. So here is the footage I recorded with my
phone. You see I have a piece of paper that I'm bending. And hence this is a deformable
object. We have a bunch of tracker markers here, and our goal is to bring this into blender
and somehow track this. And have kind of like a 3d representation of this that bends with
the paper. So how are we going to do this? Well before we even hop into blender, the
first thing we need to do is take care of this footage. Well what do I mean? When we
go into properties, and by the way everything including this video and the scene files is
going to be available in a download link in the description. But when we go into the properties
of this video you're going to notice that it is 29.91 frames per second which is very
very inconvenient and is a fairly strange framerate. And again this is because I recorded
it with my phone. So what we want to do is take this and turn it into an image sequence
because blender sometimes freaks out when you give it weird framerates and we don't
want anything to de-sync. So before we go into blender let's make a new project folder.
We can call it paper tracking. And then inside of here we need to make a folder for where
we want our sequence. So we'll call this sequence and now we are ready to get started. So let's
open up blender. There we go. And we can delete everything to begin with, so a, x, delete.
Keyboard shortcuts are going to be up here. So first thing we need to do is import our
footage and turn it into an image sequence. So to do this I'm going to go from 3d viewport
over to movie clip editor. I'm going to click open for import. And then I'm just going to
navigate to the footage. Ok so our footage is now open. One thing you may notice is that
it's 259 frames, however, blender default scenes have 250 frames. So to go from 259
to 250 you either type in 259 or you click this button right here. And what that does
is it automatically does it for us which is very convenient. Cool, if you now go to output
which is where we can set our framerate. You're going to see a whole bunch of options. In
my case 29.91 is not here so you want to go to custom and when you type in 29.91 you're
going to notice that it rounds up to 30. So it does not like a lot of these fractions.
So how do we get around this? Well this is where the base comes in. So instead of going
29.91 and 1, we're going to represent this as a fraction of integers. So here is what
I mean. So instead we're going to write 2991 over 100. So image you're taking this number
and dividing it by that number and then we kind of retrieve this framerate that we want.
So this is kind of a hack that let's us do what we want to get any fractional framerate.
So now we have 29.91 fps. Ok so we want to take this and export it as an image sequence,
so to do this we're going to go to compositing. And then we're going to enable our nodes.
We don't need this render layer so x to delete. And then shift a, and then we're just going
to input our movie clip and we can just find it right here. So we're going to take this
movie clip and connect it into our composite, so this is what it's going to output. And
we have our, in output we have our resolution correct, our framerate correct now too. And
then we can pick how we want to export this. So I'm going to do a jpeg sequence. We are
going to lose a bit of quality because jpeg has quite a bit of compression. But I don't
mind this, if you do, again you have the scene files you can of course change this into a
png sequence or something more lossless with less compression. So jpeg sequence. Click
where you want this to save to. I'm going to go desktop and then paper tracking, sequence,
and we're going to put it in here. And we're just going to call this sequence. Ok so that
is now good to go. And make sure you do set this framerate. You;re going to get some weird
issues otherwise that we do not want. So now that we are all set up all we have to do is
render our animation I guess. You don't want to render a single frame. So either go to
render, render animation or just do control f12. And it's just going to start scrubbing
through this. And it's going very quickly because again all it has to do is look at
the frame and then save it as an image. So let's see what we have going on here. Yes
it is now populating our folder. And it's going to have to this 259 times. So I'm just
going to skip or fast-forward until that is done. Ok so we just had it finish the last
frame. If we go to our folder you're going to see that we have all 259 frames. And again
this is useful because we no longer have to worry about framerate which is an issue when
you record with a phone. So we are done with this and actually we're just going to abandon
this blender project right here. So we're just going to go to file, new, and then general.
So now we have a new instance with everything else gone of blender. And now we can actually
get started on this effect. So very first thing you need to do obviously is delete everything,
we don't want this here. So what we're going to do is save this project, control s. And
then we are going to save it into our folder paper tracking. And we can call this project
paper tracking, not too inventive there. Ok perfect, so now let's get started. So first
thing we're going to do is we're going to go to output and now this framerate is representing
something different. So when we set this framerate, basically this is saying what is it going
to output at. Whereas before it meant that but it also symbolized how is it going to
read our footage. So now we can set this to 30 fps without any issue. So again this was
an issue when we were using movie files but now we are in image sequences. So we are going
to go to the movie clip editor. And this time we are going to import our image sequence.
So you're going to get all these images, just click a to select them all and then open it
clip. And it will automatically recognize this as an image sequence. Now first thing
that you might notice is when you scroll through this holding right click it's super slow.
Very very slow. So I'm just going to go to the first frame, shift left to do that automatically.
So what we want to do is speed this up so we're not going very slowly frame by frame.
So first thing I'm going to do set scene frames so it's now 259. Then we're going to click
prefetch and you're going to see it's going to scroll through here. And basically it's
putting all this information in memory, quick access memory. So that now it's scrolling
by a lot faster. Ok perfect, going to save again. And now we're going to do the first
step which is tracking. Now again we are tracking a deformable object which means at some points
it's going to curve in like that. Now there is now, that's a lot of d's up there. There
is no way to very easily take a plane and have it track while this plane deforms. So
what we're going to be using is a trick of perspective which works pretty much perfectly
when you're looking at it from the right angle. So here is what we're going to do. For each
of these markers we're just going to put a tracker and track it all the way through.
So I'm going to be using perspective because I think that's going to be the best for this.
You have some change in perspective as we bend this paper. I'm going to turn on normalize
for different lighting conditions. And then right here in correlation, let me just drag
this over. In correlation I'm going to bring this up to .9. What this value is is it's
a tolerance when we're tracking automatically it's saying if it's 90% certain it's correct
from frame to frame keep tracking. If it's not stop the track. So the lower this is the
more reckless it is. And the closer it is to 1 the more very very precise it has to
be to continue is a good way to say it. So I'm going to open up track over here so we're
going to have all this open. And let's begin by doing a single track. And soon we're going
to be doing them all at once but let's start with a single one. So control click to do
what I just did. So let me just delete that. Control click where you want it. And then
we're going to just scale with s. g, s, and r works here too. And then we can also move
it from this window right here, holding shift to do it very slowly. So once you have that
setup we can track either manually or have it do it automatically. To do it manually
hold alt and then right click like that and it's going to go frame by frame. Just like
that. Now if we do not want to do this 259 times we can have it do it automatically.
And again it's going to stop when it's not 90% confident that it's correct from frame
to frame. So I'm going to click l to center this or lock it I guess is what you call it.
And then I'm going to click this right here or you can hit control t. And it's going to
begin tracking. And you see already it ran into an issue. So we're going to go into the
frame before this, left click, or not left click left arrow rather. And you see at this
point it kind of lost it. And it makes sense because there is quite a bit of motion blur.
We're going to see if we can actually help it do this without manually moving it. So
I'm just going to resize this and see if it can handle it now. Yep, so sometimes that's
really all you need to do. And we can go back and forth, scrub the footage. And you see
that it is dead on there. And you of course can look at this window up here. So that is
perfect. So once we're happy with that all we need to do is lock it and we are done.
However we do not want to do this, how may, 20 times. Ya we don't want to do this 20 times.
So what we're going to do is do them all at the same time. So how are we going to do this?Well
we're just going to put a tracker here and one here. And I'm not scaling them up yet.
We're going to do that all the same time. So this is all about bulk processes, processes
I don't know which one it is. And again these are very very good tracker marks. They're
very high points of contrast. Of course we do have the issue of motion blur occasionally
and we also have the additional compression from the jpeg sequence. But this is generally
kind of what you want, this is the ideal tracker marker. Let's adjust this one. And what I
did for this is I just made this design. You can quickly find a picture of this tracker
on the internet and then I put them in a grid and then printed it out on a piece of paper.
Of course that might be a bit less convenient if your deformable object isn't a sheet of
paper. But in my case it is. Only a couple more to go. And if anything in this tutorial
takes a super long time like exporting the image sequence of course I will fast-forward
it. But I think this is actually not taking too long, so I don't mind. Ok one more. Perfect,
so now we have all our trackers but we want to size them up so they consume the entire
circle. So we're going to hit a to select all and then s for scale. Now notice that
it's scaling exactly how I want it to. Each one is being scaled individually. If that
is not happening for you what you want to do is click this button right here. And this
is basically saying where do you want it to scale from. So if I instead said 2d cursor,
or actually I guess it is working. Either way you want to make sure it's set to, in
this case it doesn't seem to matter that much, but individual origins is what you want. Maybe
it's because I didn't define my cursor. I don't know, either way that's what you want.
I'm just going to scale it perfectly like that. And now when we have all of them selected
and hit this it's going to track all of them at once. And it's going to be a bit slower
because it's doing 20 times the calculations as doing only the corner. And some of these
might drop off, like this one dropped off. But hopefully the majority of them make it
to the end. And we're just going to go back and retrieve this information over here. So
it looks like all of them made it except for this one. So what we're going to do is we
are going to select this one right here and then we're just going to scroll back to, let's
lock in with l, we're going to scroll back to where it's actually working. Should be
the same as last time. There we go. So again I'm just going to move this and hope that
it solves itself or else we're going to have to move it manually. So something like that,
let's correct that a little too. Ok, let's try tracking it. It seemed to like that. You
won't always get so lucky sometimes you have to manually keyframe it. Ok so now it seems,
of course we can inspect each of these so if I want to see this one I can just click
it and now we can see what this one is doing, it seems like all of them are dead on. You
do want to make sure of that but I'm pretty sure that's the case here. So I'm going to
select all of them and then right click and then lock tracks. And that locks all of them
at once, and now I'm going to save. So what do we have? We have 20 trackers that should
be dead on the sheet of paper. And if you don't like these trails you can always go
to display, disable path, or make it a shorter path. I'm just going to disable it. So what
we want to do now is I'm going to split the view like that. And make this a 3d viewport.
And what we want is to bring these dots into our 3d viewport. So to do this we need a camera
first of all. So shift a and then we are going to add a camera which I'm going to hit n to
get all of these properties here. I'm just going to center in terms of rotation, so zero,
zero, zero. So it's pointing straight down and then g for grab, z for up. Just put it
pretty much facing down like that. Ok perfect. So now what I'm going to do is I'm going to
select in this window, I'm going to take all these trackers. Or you know what I can just
close that up. I'm going to take all these trackers, and let's turn off locked view.
I'm going to take all these trackers, go to reconstruction and then link empty to track.
And that should do exactly what we want. You see our outliner now has a bunch of tracks.
So again select all of them, reconstruction, link empty to track. So now let's bring this
back to the 3d viewport, and what do you see? You see our camera and a bunch of lines going
down to what looks like a grid of points. And if we play this, indeed you can kind of
see the animation in there. First of all let's kind of declutter everything. So I'm going
to scale these down. Now this is where the individual origins matter. So lets see if
I'm crazy. So 3d cursor if I put it over here and now we scale. Ya that's not what we want
right. So I'm going to first of all bring this back to the world origin. So shift s
and then cursor to world origin. So I'm going to bring this back to individual origins and
then scale it down like that. So that's what I was talking about before. Ok perfect. So
now we have all these points. Now one thing you might notice is again we are kind of tracking
a 3 dimensional object. So you'd expect when it's bending like that you can see it's kind
of bending here, you'd expect it to be 3 dimensional but it's actually all in the plane. And this
is going to work whenever we're looking from the perspective of this camera, this illusion
works because we have the same perspective. It's basically like advanced corner pinning.
But I will kind of describe this a bit more in a second. So let's go to our camera view
and in that camera, select your camera, go to settings. I'm going to turn on our background.
And then make that a movie clip and of course open up our sequence. And you see we have
our background, let me just turn this off. Or you know what we can turn only the axis'
off. You see now our trackers are doing exactly what we expect, now again they're not 3 dimensional
this is a trick. We're kind of cheating here we're putting everything all on the same plane
because we're distorting it in the right way it's going to end up looking right. So there
we go, we have all our trackers there. Now what we want to do is make some geometry.
So some kind of rectangle that represents this piece of paper that actually deforms.
And the way we want it to deform is using this information we have here. So I'm going
to select all of these. I'm going to click m and that makes a new collection. Or we can
put them in a new collection in blender 2.8. New collection and we can call these empties
or trackers whichever you want. How do I spell this? Empties. So now we have this collection
here and it's not cluttering everything. So now let's actually make our geometry. So shift
a, mesh, and then the obvious thing to do is a plane. And then we just need to resize
this so s for scale, g to move it, r to rotate. And now let's scale it so it kind of fits
the dimensions of this. Let me turn on our axis' again. So scale with respect to x, then
move it. Scale with respect to y and then move it. Now let's do a bit of rotation, doesn't
need to be perfect we'll adjust this in just a second. There we go. So I'm going to say
that's pretty good. And now what we want to do is note that we have a grid of 20 trackers.
So when we go to edit mode notice that this is only one polygon. I'm going to hit control
r, this is going to add a loop cut. I'm going to scroll up. And then select that, right
click to center. And then same thing on this axis. And basically what we did is for each
of these trackers there is now a vertex. So for example there is a vertex here for this
tracker right here. So for each tracker there is now a vertex, and we're going to link these
things together. So now the geometry is going to move with it. So make sure you add the
appropriate number of vertices depending on what your video clip is. So how do we actually
parent everything? Well there is a very nice way to do this and it's using hooks. So let's
start off with this one. I'm going to select it, and we want to move the cursor so it's
here. The shortcut is shift s and then cursor to selected. Or a fast way to do it is you
just go like that. I guess I didn't do it very well. But either way that's how you do
it. Then we're going to open up our plane. And select the relevant vertex. Then we want
to shift s and then selection to cursor. So the cursor went here and then we moved this
to the cursor. And now we want to link this vertex to this empty so control click this.
So hold control and then click it and then control h, hook to selected object. Ok what
did that do? Let's exit out of edit mode. What did that do? You see it's actually deforming
with it and that's kind of the whole idea here. So how many times do we have to do this?
We have to do it 19 more times. But let me just show you a couple more times. You can
do it pretty quickly one you get good at it. So I'm going to click this, shift s, and then
cursor to selected. Select this, tab, select your relevant vertex, and then like that that's
the fast way to do it. Control click, control h, and then hook to selected and now both
of these are moving together. So now let's see how fast we can do this. So double click
boom, bam. It really isn't that fast but I do wonder, I don't really know that much about
scripting python, python scripting but I feel like this is the kind of thing you can make
a python script for to do it automatically. Or maybe there is some kind of feature I'm
not aware of. So I'm going to do this a couple more times and then I'll fast-forward. Just
want to make sure you understand this part. So select the object, shift s, cursor to selected.
And then go into edit mode on your plane, shift s, selection to cursor. Control click,
control h, hook. And then at speed it looks like this. Ok and now we have also the interior
vertices inside this plane are all parented in the correct way. So I'm just going to continue
doing this and fast-forward the footage. Ok so I just finished up the last one. Let's
make sure that we actually have all of them. Ok, that is looking very good, awesome. So
I'm just going to save this. And now when we go over to the camera view, what are you
going to see? Well let's actually bring this to the beginning and play it. That's exactly
what we want. Look at that. So now we have this plane kind of tracked. And again it's
deforming correctly even though it's not 3 dimensional. When you look at it from just
the right angle. And that's the whole idea here, when you look at it from just the right
angle everything works out. And we have this system of parenting where if we select our
camera and rotate it it's kind of attached. And if we play it it all works out. Ok so
make sure all your vertices are set and now one thing we want to deal with is, you see
since we don't have that many points, everything considered, let me turn this off. Since we
don't have that many points everything kind of looks sharp. You kind of see these edges,
the outline. Well how do we deal with this? Well there are many techniques but the idea
is we want to smooth this out but still have it animate correcting. And there is a couple
ways to do this I'll show you the best one. So select your plane which I'm just going
to rename, I'm going to rename it paper. And then I'm going to rename our camera projector
because it's projecting the location of these points or something like that. So we're going
to go to our paper, we're going to go to modifiers and you're going to see all these tracker
modifiers. And that's basically these hooks, 20 of them to be precise. So we're going to
scroll to the bottom, or actually to the top. We're going to add, now your first instinct
might be a subdivison surface and I'll show you what happens when you do that. So scroll
to the bottom, you're going to bump this up. And you're going see that you know it's smoothing
and it's even tracking on, however it kind of warped our corners. And if you don't want
that to happen you might go to simple but then it doesn't smooth out everything else.
So subsurf is not really the way to go unless you stack 2 of them. And same think with multi-resolution
modifier. So the best way to go I've found and this doesn't mess with the UV's either
which is going to be important, is we are going to add a bevel modifier. I know, I know,
very useful. So let's zoom in. We're going to take this width, bring it all the way up.
And you already see it's smoothing. And then we just add segments, something like 4. And
now it's a lot smoother. Let's pick the part where I bend the paper. Look at that. So that's
without and that's with. Much smoother. We can actually bring this up to 5 if we want.
Ok and we don't even need to apply this, we can just keep it in the stack for now. Ok
amazing. Well now let's actually put some kind of texture on here. And I kind of think
of this as the harry potter, the harry potter newspaper effect, you know how they have these
newspapers with video on them. We can put anything one this including video. So let's
go to shading. Once we're in shading we're going to select our paper. We're going to
add a new material, we don't need this junk. And then in this folder, let me close this
up for a second, in our folder I have this file right here. It's going to be available
in the project files, again download link, link to download. You can download it in the
description. So I'm going to take this and then copy it into here. So that's our UV grid,
what are we going to do with it? Well we're going to shift a, I just did shift s that's
my bad. We're going to shift a and then texture, image texture, we're going to hook this up
to surface and then of course our image texture is just going to be this uv grid. Ok and you
see that's applied, let's go to layout. And bring this to rendered mode. So you see this
is actually deforming, and it kind of looks 3 dimensional. Of course it will really look
correct when we go to the right angle. But when you look at it from the side you can
kind of see what it is we're doing. It's this kind of advanced corner pinning is probably
the best way to think about it. So instead of just pinning the four corners we're pinning
many spots of the plane. And now let's go into our camera view and that's distorting
correctly. Look at that, it's not even moving it's also bending. Well what's the drawback
here, there is a drawback. Because this is 2 dimensional, during this spot in the footage
where we're bending it kind of like that. That's a lot of d's I wonder if I can fix
that. Let me erase that, so many d's. Either way, when it's bending this much we expect
it to kind of cast a shadow on itself because it's kind of in a giant parabola. Parabolic
cylinder to be precise. But because it's 2 dimensional we can't actually cast that shadow.
That's a drawback but you know what we can live with it. Ok so you're probably screaming
at me this is not big enough you can see the boundaries. And the issue is if we take our
paper and try to scale it with s nothing happens. And that's because it's hooked by a bunch
of spots. So it's kind of constrained and can't move. Ok well how do we scale it then?
Well think about it, I mean we are in this camera view and it works from this camera
angle. What if we took this camera duplicated it shift d, right click to bring it back,
and then g z move it down. Well what would that look like. Well let's take this camera
call it our viewer because that's the camera we're going to view from. And notice right
now it's just going back to the same camera, to make this the camera we go to you go to
view, cameras. set active object as camera. And now you see it's much bigger so if we
hit n which let's us control this. You can see exactly what's happening, let me split
the view so we get a side by side like that. So in this one, let me just go into this view
and we'll see what happens when we go up and down. So the camera is moving up and down
and that's effectively scaling this paper because we're going close to the source. So
let's close that out. So we want to make this as close to the boundary but not go too big.
So something like that. Because if we make it too big it's going to begin cutting into
my hand. It's going to look really weird. Let's see if that's right, see if it pokes
out any point. Ya it's kind of poking out a little bit so let's do that. And no matter
what you do it's still going to keep the right perspective. Now you see right here that there
is a bit of exposure over here. So here you want to choose, do you want to zoom this in
even further, do you want to paint this out, or do you just want to live with it? In this
case we're going to live it but just know that you do have options. So this is pretty
good, and by the way is this already mapped correctly like why did it work automatically?
Well the reason is because we're using a plane object, I mean we are deforming it, but because
we're using a plane object if you go to UV editing and select all your faces. It's just
a perfect square. And the image we imported, this one, is a perfect square so that's why
it works out. So anything you want to bring in here, you want to make sure it's a square
or rectangular at least and it's going to work out. Let's close out of this, actually
no, what am I doing. Oh well, we don't need this anymore. Let's just abandon that. Ok
so now we have this shaded. So let's go back into here and it's working out. So clearly
what is the issue here now? Well it's overlapping my thumbs, it's not supposed to do that. The
thumb is supposed to be over the piece of paper. So in some sense we want three layers,
we want this background, then we want this cg paper, and then we want a layer here and
here for my thumbs. And the way to do this is to mask out my thumbs. And if you have
never done masking buckle up this is not the fun part. This really is not the fun part.
So let's draw some masks. So go into the movie clip editor. Again this is where you track
and draw masks. So you just go into mask and now we are in masking mode. These trackers
are still here if you want to hide them just click pattern. It's going to hide them. So
now what we need to do is to draw a mask that goes over this part of my thumb and then later
on in the footage this kind of meaty part of my thumb. I guess you can call it. Because
again the paper is a bit bigger than this paper so it's going to cut it even more. So
we need to mask that there for all the frames and here for all the frames. And you might
think the pulling a luma key would be ok because this is white. But you know my hand, I'm a
pretty white dude so it would work ok, it would work better if my skin color wasn't
as white. So I'm not going to pull a luma key on this one. So how do we do this? Well
what we want to do is draw some masks. So now we're in mask mode up here. To start drawing
a mask all you do is, again it depends on if you have your settings for left or right
click, I believe mine is on right click so you hold control and right click. And that's
going to add a point. You click it again here and drag, it's going to define a spline. So
you just kind of keep control clicking and draw around. At any point you can take this
and hit g to move it. That's moving the arm, or hold right click here and then click g,
that works too. And then you can just keep control clicking and it's just going to continue
your spline where you left off. Now I'm pretty sure we can bring this into black and white
mode. Ya if it's easier to see what you're doing you can go to black and white mode.
It's more, I wouldn't say it's more contrasty but it's easier for the human eye to see what's
going on. So I'm just going to draw kind of like, nothing special, kind of an easy mask.And
then we're just going to close this off like this. And then you don't go back and click
this here, to make this a closed shape you go to mask and then you hit cyclic. And that's
automatically going to seal it although it's going to make this closing and starting point
very sharp. So right click it to select this point and then click v I believe. Ya v and
then change the handle type to aligned like that. And then we can scale this up and down
rotate it, exactly what you'd expect. So let's adjust this. Rotate this one. Same controls
as pretty much everywhere else. Now this is our new mask right, we just defined a mask.
We can rename it into left thumb. So this is our left thumb mask. And again like I said
we also need the meaty part of the thumb but not for the whole footage. Like right now
we don't need the meaty part of the thumb, it's not overlapping. But around here we need
to start masking it. So in our mask we can have multiple parts so this is the first part
right here. We can call it the tip as in the tip of the thumb and then later on we can
click here to add another part. So how do we do this? Well we have this display right
here and we have this one here. Which we are going to change into a dope, dope sheet. One
of my favorite things. Bring that into mask and now you see our left thumb is in here.
So at any point to keyframe you just hit i and it's going to add a keyframe. So if we
go seven frames in all we have to do is select all of them, g is going to grab them. Maybe
a bit of rotate, oh that's not rotating the correct way and that's because we changed
this from individual origins to median point that would work. So we've been doing a lot
of this pivot point changing. So you do this and then you just right click to adjust. And
generally the most efficient way to do this is to move the shape all together and then
just make your fine adjustments. And we can also track like the fingernail, we can make
another track so not like the tracks we did here. But a track for the thumb and then connect
the mask onto that. Although I have not been getting very good results with that, so I'm
not going to do it. So once you're happy with this just click i and it's going to keyframe.
And now if we go back and forth it's not perfect because we need to put some keyframes in-between
but you see it's traveling with it at least. And then one other thing, probably because
of my weird 29.91 frames per second issue, the first frame and the second frame are identical.
Which is not a big issue but it is a teachable moment because what I can do is just zoom
in here by holding that. So I want this frame and this frame to be the same so I just take
my keyframes these ones and shift d to duplicate and then jsut move them down like that. That
way it sticks on and then begins moving. So I'm going to go to the frame in-between, inside
this gap. Then we're just going to adjust this. And you can be very accurate with this
it really depends on what you're going for. I'm not going to be crazy about this. And
then i, and then there you go. This part is pretty well animated. And we're just going
to do a bit more. As you can imagine this is going to take a good while so I'm not going
to do all of it. I already have the masks pre-prepared which I'm going to import. Again
that's going to be in the scene files so you don't need to mask it either if you don't
want. But just for example we'll just do a bit more. Ok so we can go a couple more frames
down, move this, hit i. Make sure you hit i or just do automatic keyframes if you don't
want to hit i every time. And maybe adjust it in the gaps in-between. I mean the thumb
isn't that bad you can have a much worse shape. And then once you do this if it's not a perfect
mask then you can pull a luma key and that's going to get rid of the border at least in
this white area. And then let's say we want to track the meaty part of the thumb. Of course
we do not want to do this at this part of the footage but let's say we did. What we
do is click add and this is a second area in our mask. And we're going to call this
base as in you know the base of the thumb. So we're just going to hide that. And then
just control right click to define this shape. Zoom in here. And the first shape ended about
here so we're just going to have a bit of overlap just to be safe. And then we can just
kind of round it off here, turn this to cyclic. Right click or sealing point, v, aligned.
There we go. And now these, or this rather hasn't been keyframed yet. We've only keyframed
the tip not the base. So when we open this up you see the base, doesn't have any keyframes,
so I'm just going to add a keyframe. And then go down here, oh remember the first or the
second frame is a duplicate. So we just select this, shift d, put that there. Perfect and
now we animate it with it. Like that. And we can actually have both of them enabled
at the same time to see what's going on. And you see it's moving together. If you want
to make sure that your mask is looking good you can always go to mask display overlay.
That gives you this, or change this to combine. And then you're going to see what you masked
off. Just like that. Ok let's turn that off. And then of course you're going to need to
do this with the second thumb.You can do this all in one mask but I prefer to have two masks.
Not that important for this tutorial but in other cases it's pretty important. So you
go to this mask area over here and then click add a new mask so now we have our mask and
left thumb. So we rename this right thumb. And then we do the same procedure here. And
you need to do this, for how long? For 259 frames, screw that let's import the masks
that I made before. So I'm just going to take these masks and delete them. I guess if you
want to truly delete them, pro-tip go into blender file. This is where all your things
are including your masks. So you just select whatever you want, right click, or maybe do
you just click delete, do you click x. Why can't I delete them? Well it seems like for
some reason I cannot get rid of my mask. Save this data-block even if it has no users, unlink
data block. You know what, quite honestly I don't know what's happening right now but
that's fine. So whatever, maybe I'm missing something stupid not a big deal. So we're
just going to call this 1 just to not get confused. And 2. And now we are going to import
our masks. So let's do this. We are going to file, append, and then we have, where did
I save it. Is it in the desktop? So I have this file called tracking masks and then when
you click that you have all this stuff including the masks. And we're just going to bring this
and this in. Which is why I renamed it to 1 and 2 so we wouldn't be confused. There
we go. So now when we go to masks we have all of this. So we have this mask right here,
let me just show it off. So by the way this block right here that's the base of the thumb.
I didn't animate it in this portion because it's not important. It's disabled to if you
go to overlay you see it doesn't actually count. I did that by animating the opacity
of the base. So once you get into like here this area you see the opacity has gone back
up to one and it's tracking on nicely. Just like that. And then we have the same kind
of deal over here. Let me enable this so we see everything. Ok so now we have our masks.
So now we are in the home stretch which is putting everything together. So how do we
put everything together? Let's go back to 3d viewport. So again we want background,
piece of paper, and then thumbs, so how are we going to do this? Well first of all let's
render one frame of this. So we can do this in eevee doesn't really matter. If we are
in eevee go to film, alpha, and go to transparency. This is going to render it so that, I'm pretty
sure that should make the background null, but we'll see about that. Let's see what do
we have in our compositor, ya pretty much nothing. So let's hit f12 and it's just going
to render this one frame. Which is exactly what we want. So when we go into compositor,
when we go into compositor. Everything is here although you don't see it and that's
because you need to add a viewer node. Now you can do this two ways. Shift a, output,
viewer, that should do it. Or the fancy way, and for this you need node wrangler. So just
go to edit, preferences, add-ons, node wrangler, and enable that. You can just control shift
click whatever you want to see. So control shift click. And now we have our viewer automatically
made. I'm going to hit v, ya v to zoom out. And then alt hold middle mouse to move this.
Ok so we have our middle layer, we need the background so shift a, input, movie clip.
And this is going to be our background. And then we just choose the sequence. So how do
put this layer over the background? Well there is a bunch of ways. The best way, at least
I think one of the best ways for this, shift a, color, alpha over. So I'm going to put
this in the background. This in the foreground. And then we need an alpha which comes from
here to blend this. And then control shift click to view it. Perfect. Now if we get rid
of this factor, oh I guess it does work. Ok either way I'm just going to keep that there.
So now we have this and this but we want the thumbs to go over it. So we're going to need
our masks. So shift a, and then input, and then mask. So we are going to have our thumb
left. Shift d to duplicate our thumb right. Thumb left, thumb right. And then we want
to make this into one big mask that we apply to this and put it over. Well what do I mean
by this? Well we're just going to duplicate this, put it here. So this is going to be
the background and then in the foreground we have our thumbs. So put this here. Now
of course we don't want all of this, just the thumbs which is where these masks come
into play. So if we control shift this we see this thumb, and then this thumb. To combine
them shift a, and then we're going to do a math operation. So vector, no, converter and
then put math right here. And we're just going to add these two. Which does exactly what
you think it would. It adds both of them. So now if we put this here as our factor you
see the thumbs are now over it. Just trying to do something a little fancy over there.
Our thumbs are over it and that's going to be true for every frame. Although we didn't
render every frame of the middle layer so we're not going to deal with that yet. So
what does this look so weird? It's because, v to zoom in, no alt v to zoom in. It's because
we don't have any shadows and this is pretty sharp. So how do we take care of that? Well
first of all let's handle the shadows. So to make shadows, to make shadows I'm going
to put this under our thumbs right. So you have background, picture, shadows, thumbs.
So we need to put the shadow in-between here and here. So I'm going to add shift a a mix
should do it. So for one of the layers we're going to have this and then for the other
layer we're going to have black which is going to represent shadow. But we don't want the
shadow everywhere we just want it under the thumbs. Now you might think the way to do
that is to bring this mask into here. But then our shadow is directly under the thumbs
so you're not going to see it. Well how do we move it? Well you can add a transform.
Put that in here. And then bring this down or I guess up is what I'm going currently.
And then you see that our shadow is under here. So let's move it so it's below the thumb
like that. Ok perfect. So couple issues, one looks horrible, but two there is shadow where
there isn't even paper. So let's take care of that. So how do we take care of that? So
basically we only want shadow where it's, from under the thumb and where the plane is.
So that's an intersection or in other words it's a multiplication operation. So what we
need to do is shift d duplicate. Change this to multiply. I'm going to wedge this in right
here. So we're going to multiply this transform by, well how do we know where there is plane?
Well that's exactly the alpha, the alpha of the plane. There we go and now it's only in
this area. So you see this is before and then we cut this off after. Perfect, now let's
make it look less ugly. So before we multiply it let's blur it out a little so shift a,
blur, wedge that in right there. And then blur it until it kind of looks like a shadow
so maybe 12 in both directions. Something like that, I keep zooming In. And then we
also want to soften these shadows which we can do by, before we bring this in here shift
d add another multiply and in this case we took it and multiplied it by .5 so it's half
as strong. So 1 unaffected, 0 invisible, .5 half way in-between. So let's do .65, so now
we have a softer shadow. Ok I like the way that looks, let's save. Now another issue
to deal with is that the thumbs themselves not the shadows, but the thumbs look a bit
choppy. So how do deal with this? Well when we put the thumbs in we just used this addition
here, this addition. What we want to do is soften that addition. Well how do we do that?
Well that's just a simple, that's just a simple blur. So shift a, sorry shift a, blur. Add
that in right there. And then you want to be very very careful with this. You do not
want to overdo this. So we're going to zoom in. So let's try starting off with something
like 3 in both directions. And that's already looking a bit too blurred out so let's go
back to 2. And you're also going to notice that maybe in some of these frames a tiny
bit of white around here. So if you want to get rid of that before we blur it we can kind
of close this mask in a little. And that's called a dilation or a contraction. So shift
a, add a dilation or a dilate, and let's see what this does. Well when we move this up
it expands it, and when we move it down it subtracts it. So look at this and then this.
And then we are going to blur it. And that's going to go in here. And you can make this
a bit more extreme you can go to minus 2 and you see it kind of chops in the thumb a little.
But I'm just going to stay at minus 1. And these kinds of small touches really help,
really help sell the effect. Ok next this render layer is looking way too colorful at
least in this case. If we put something different that wasn't this image that might not be the
case. Oh by the way I didn't even mention this, well I did mention it but we didn't
even talk about it. You don't need this picture, you can make it anything. You can put anything
on this plane. For example you can put, let's see how do we change this quickly. What if
we go into shading, go into this, and change it from uv grid into, into well let's move
this into image sequence and then which image sequence? I guess we need to import it. This
sequence right here. Well then it's going to have the video of me holding the video.
Of course we need to f12 to update that. And now it's me holding me holding the thing and
we can have this be a nested kind of shebang. But let's go back, wonder if undo ya there
we go. Compositing, f12 to refresh. Ya so again you can put anything in here it can
be a video really doesn't matter. So we're going to take this, e're going to desaturate
it, so shift a. Actually I don't know what it's called so shift a, color, what's it called,
hue saturation value. I'm going to wedge that in here. And then we can dip the saturation
and you can see what that does. Make it a bit darker, anything that you feel like will
blend it in more. Change the color of it, think I did that for the preview of this tutorial.
There we go. And that is looking like a composite. So let's try a different frame, let's try
like one of the frames where it's really bent. I'm going to move this to non-linear. So let's
pick a frame like that right there, lots of distortion. Well we go into compositing, f12
to render 128. And now we have that frame. And you see these nodes kind of work for any
frame because now, now our masks look like this because we are on this frame. Dilation,
blur, everything works out. The shadows are, these shadows are underneath. And then we
get that. And you want to make sure your mask works for every frame. I already did that
for you so again use the project files, use these masks. So now I believe, let me think,
I'm trying to think if we forgot anything. We can add a little something extra. I think
that compared to everything else this is looking a bit sharp. Looking like I just kind of downloaded
a picture from somewhere. Which I did, so what I'm going to do, this is really a mess.
But the logic is there and that's what's important. So we changed the color saturation value,
now let's add a bit of blur. So shift a, and then what am I looking for, I'm looking for
a blur. Nice little blur, a very soft one, maybe 1 by 1. And to see what that did let's
actually zoom in. I keep zooming out I don't know what's wrong with me. Let's try 2, no
that's a bit too much let's go back to 1. Ok so I think that I am actually happy with
this. So to render you know more than just a single image I'm going to go to layout.
In output let's do png output, no compression. And then let's save these directly into our,
directly into, let's make actually a tiny video. We don't need to make a sequence. So
30 frames per second, let's output it as a ffmpeg, ffmpeg video. Ok, encoding we want
this to be h264 mp4 type thing. So mpeg-4, h264, medium quality, doesn't really matter.
Let's only do like 1 second of it so 30 frames, but it should work for all of these. So we're
restricting it to 30 frames. Ok no audio, all this is fine. Output it, output it in
our folder. Tutorial output. Amazing. And before we output this one final remark. Again
this perspective idea which again I keep stressing it's really powerful. As long as your camera
is from the right angle, and even if you move it down along the same angle, this really
does work. It really does. So even if your camera is moving in the shot like for example
in this shot, in this shot it's very not noticeable. But if you look at the edge of the frame the
camera does move so technically this is a moving shot. So these ideas, this corner pinning
works with a moving camera, that's why it's so powerful. Enough of that, output, 30 frames,
and then ok I think we're good. We'll see about that though, so control f12 and that's
going to render the animation. Oh wait, big issue, big issue! You saw in the composite
it didn't have any of the background even though we have all this. But that's because
we've been feeding all of this, all these nodes into the viewer. We need to also, also,
feed them into the composite. And that's basically what it's going to output. So now when we
hit control f12 I have hight hopes. There we go! So it renders updates, renders updates.
It's applying our color change, our bringing the thumbs back. So while this renders for
only 30 frames some critiques. The shadows could be softer and put in a better place.
Definitely some color correction that needs to happen here. I would recommend, I made
the piece of paper a bit too big but if we didn't we would need to paint out the first
one which is a pain. So I guess that's fine. But if you can help it scaling down this paper
is always going to help sell the effect. Ok so that's our 30 frames, let's save our project.
Where did I save this, this folder. Now hopefully this play nicely, let's try opening it. Ya
there we go, so it's only a second long. But it really is correct. You'll notice that there
is a few issues like right here the mask seems to be a bit off. That can either be a dilation
issue or maybe the mask was a bit off you know. But that can always be fixed, that's
my challenge to you to fix it. And there we go. Well what did we talk about. We talked
about turning things into image sequences. How to do, how to do tracking. We learned
a lot about tracking and how to do this corner pin effect especially how to use this kind
of corner pinning hooks onto empties idea. And then using a bevel to smooth it out. We
talked about shading this, about masking the thumbs over. And then finally this mess which
really isn't that complicated, hopefully it was easy to follow this node network which
kind of composites it all together along every single frame. Of course it kind of breaks
because we didn't update this, if we f12 that should update. Ya now everything looks correct.
But hopefully this tutorial was very useful again I have not seen anything like this on
the internet at least when it comes to tracking deformable objects. Oh no, I don't know if
you saw what I just saw, that means we needed to make the plane a bit bigger. You see it's
kind of poking out. Either way in my preview I made sure it was the right size. But that
is how you make this deformation tracking effect and it works with more than just paper.
Only downside is you kind of miss out on shadows and stuff like that so hopefully you really
enjoyed this tutorial. I really had fun making this one. Thank you guys for watching. If
you want to support me either keep watching the tutorials, I also have a patreon. But
make sure you download the resources for free of course, the project files below. Thank
you guys for watching and I'll see you guys on the next one.
