The following content is
provided under a Creative Commons license.
Your support will help MIT OpenCourseWare continue to offer
high quality educational resources for free.
To make a donation or to view additional materials from
hundreds of MIT courses, visit MIT OpenCourseWare at
ocw.mit.edu. so -- OK, so remember last
time, on Tuesday we learned about the
chain rule, and so for example we saw that
if we have a function that depends,
sorry, on three variables, x,y,z,
that x,y,z themselves depend on some variable,
t, then you can find a formula for
df/dt by writing down wx/dx dt wy dy/dt wz dz/dt.
And, the meaning of that formula is that while the change
in w is caused by changes in x, y, and z, x,
y, and z change at rates dx/dt, dy/dt, dz/dt.
And, this causes a function to change accordingly using,
well, the partial derivatives tell you how sensitive w is to
changes in each variable. OK, so, we are going to just
rewrite this in a new notation. So, I'm going to rewrite this
in a more concise form as gradient of w dot product with
velocity vector dr/dt. So, the gradient of w is a
vector formed by putting together all of the partial
derivatives. OK, so it's the vector whose
components are the partials. And, of course,
it's a vector that depends on x, y, and z, right?
These guys depend on x, y, z. So, it's actually one vector
for each point, x, y, z.
You can talk about the gradient of w at some point,
x, y, z. So, at each point,
it gives you a vector. That actually is what we will
call later a vector field. We'll get back to that later.
And, dr/dt is just the velocity vector dx/dt,
dy/dt, dz/dt. OK, so the new definition for
today is the definition of the gradient vector.
And, our goal will be to understand a bit better,
what does this vector mean? What does it measure?
And, what can we do with it? But, you see that in terms of
information content, it's really the same
information that's already in the partial derivatives,
or in the differential. So, yes, and I should say,
of course you can also use the gradient and other things like
approximation formulas and so on.
And so far, it's just notation. It's a way to rewrite things.
But, so here's the first cool property of the gradient.
So, I claim that the gradient vector is perpendicular to the
level surface corresponding to setting the function,
w, equal to a constant. OK, so if I draw a contour plot
of my function, so, actually forget about z
because I want to draw a two variable contour plot.
So, say I have a function of two variables,
x and y, then maybe it has some contour plot.
And, I'm saying if I take the gradient of a function at this
point, (x,y). So, I will have a vector.
Well, if I draw that vector on top of a contour plot,
it's going to end up being perpendicular to the level
curve. Same thing if I have a function
of three variables. Then, I can try to draw its
contour plot. Of course, I can't really do it
because the contour plot would be living in space with x,
y, and z. But, it would be a bunch of
level faces, and the gradient vector would be a vector in
space. That vector is perpendicular to
the level faces. So, let's try to see that on a
couple of examples. So, let's do a first example.
What's the easiest case? Let's take a linear function of
x, y, and z. So, I will take w equals a1
times x plus a2 times y plus a3 times z.
Well, so, what's the gradient of this function?
Well, the first component will be a1.
That's partial w partial x. Then, a2, that's partial w
partial y, and a3, partial w partial z.
Now, what is the levels of this? Well, if I set w equal to some
constant, c, that means I look at the points where a1x a2y a3z
equals c. What kind of service is that?
It's a plane. And, we know how to find a
normal vector to this plane just by looking at the coefficients.
So, it's a plane with a normal vector exactly this gradient.
And, in fact, in a way, this is the only case
you need to check because of linear approximations.
If you replace a function by its linear approximation,
that means you will replace the level surfaces by their tension
planes. And then, you'll actually end
up in this situation. But maybe that's not very
convincing. So, let's do another example.
So, let's do a second example. Let's say we look at the
function x^2 y^2. OK, so now it's a function of
just two variables because that way we'll be able to actually
draw a picture for you. OK, so what are the level sets
of this function? Well, they're going to be
circles, right? w equals c is a circle,
x^2 y^2 = c. So, I should say,
maybe, sorry, the level curve is a circle.
So, the contour plot looks something like that.
Now, what's the gradient vector? Well, the gradient of this
function, so, partial w partial x is 2x.
And partial w partial y is 2y. So, let's say I take a point,
x comma y, and I try to draw my gradient vector.
So, here at x, y, so, I have to draw the
vector, <2x, 2y>.
What does it look like? Well, it's going in that
direction. It's parallel to the position
vector for this point. It's actually twice the
position vector. So, I guess it goes more or
less like this. What's interesting,
too, is it is perpendicular to this circle.
OK, so it's a general feature. Actually, let me show you more
examples, oops, not the one I want.
So, I don't know if you can see it so well.
Well, hopefully you can. So, here I have a contour plot
of a function, and I have a blue vector.
That's the gradient vector at the pink point on the plot.
So, you can see, I can move the pink point,
and the gradient vector, of course, changes because the
gradient depends on x and y. But, what doesn't change is
that it's always perpendicular to the level curves.
Anywhere I am, my gradient stays perpendicular
to the level curve. OK, is that convincing?
Is that visible for people who can't see blue?
OK, so, OK, so we have a lot of evidence, but let's try to prove
the theorem because it will be interesting.
So, first of all, sorry, any questions about the
statement, the example, anything, yes?
Ah, very good question. Does the gradient vector,
why is the gradient vector perpendicular in one direction
rather than the other? So, we'll see the answer to
that in a few minutes. But let me just tell you
immediately, to the side, which side it's pointing to,
it's always pointing towards higher values of a function.
OK, and we'll see in that maybe about half an hour.
So, well, let me say actually points towards higher values of
w. OK, any other questions?
I don't see any questions. OK, so let's try to prove this
theorem, at least this part of the theorem.
We're not going to prove that just yet.
That will come in a while. So, well, maybe we want to
understand first what happens if we move inside the level curve,
OK? So, let's imagine that we are
taking a moving point that stays on the level curve or on the
level surface. And then, we know,
well, what happens is that the function stays constant.
But, we can also know how quickly the function changes
using the chain rule up there. So, maybe the chain rule will
actually be the key to understanding how the gradient
vector and the motion on the level service relate.
So, let's take a curve, r equals r of t,
that stays inside, well, maybe I should say on the
level surface, w equals c.
So, let's think about what that means.
So, just to get you used to this idea, I'm going to draw a
level surface of a function of three variables.
OK, so it's a surface given by the equation w of x,
y, z equals some constant, c.
And, so now I'm going to have a point on that,
and it's going to move on that surface.
So, I will have some parametric curve that lives on this
surface. So, the question is,
what's going to happen at any given time?
Well, the first observation is that the velocity vector,
what can I say about the velocity vector of this motion?
It's going to be tangent to the level surface,
right? If I move on a surface,
then at any point, my velocity is tangent to the
curve. But, if it's tangent to the
curve, then it's also tangent to the surface because the curve is
inside the surface. So, OK, it's getting a bit
cluttered. Maybe I should draw a bigger
picture. Let me do that right away here.
So, I have my level surface, w equals c.
I have a curve on that, and at some point,
I'm going to have a certain velocity.
So, the claim is that the velocity, v,
equals dr/dt is tangent -- -- to the level,
w equals c because it's tangent to the curve,
and the curve is inside the level,
OK? Now, what else can we say?
Well, we have, the chain rule will tell us how
the value of w changes. So, by the chain rule,
we have dw/dt. So, the rate of change of the
value of w as I move along this curve is given by the dot
product between the gradient and the velocity vector.
And, so, well, maybe I can rewrite it as w dot
v, and that should be, well, what should it be?
What happens to the value of w as t changes?
Well, it stays constant because we are moving on a curve.
That curve might be complicated, but it stays always
on the level, w equals c.
So, it's zero because w of t equals c, which is a constant.
OK, is that convincing? OK, so now if we have a dot
product that's zero, that tells us that these two
guys are perpendicular. So -- So if the gradient vector
is perpendicular to v, OK, that's a good start.
We know that the gradient is perpendicular to this vector
tangent that's tangent to the level surface.
What about other vectors tangent to the level surface?
Well, in fact, I could use any curve drawn on
the level of w equals c. So, I could move,
really, any way I wanted on that surface.
In particular, I claim that I could have
chosen my velocity vector to be any vector tangent to the
surface. OK, so let's write this.
So this is true for any curve, or, I'll say for any motion on
the level surface, w equals c.
So that means v can be any vector tangent to the surface
tangent to the level. See, for example,
OK, let me draw one more picture.
OK, so I have my level surface. So, I'm drawing more and more
levels, and they never quite look the same.
But I have a point. And, at this point,
I have the tangent plane to the level surface.
OK, so this is tangent plane to the level.
Then, if I choose any vector in that tangent plane.
Let's say I choose the one that goes in that direction.
Then, I can actually find a curve that goes in that
direction, and stays on the level.
So, here, that would be a curve that somehow goes from the right
to the left, and of course it has to end up going up or
something like that. OK, so given any vector tangent
-- -- let's call that vector v tangent to the level,
we get that the gradient is perpendicular to v.
So, if the gradient is perpendicular to this vector
tangent to this curve, but also to any vector,
I can draw that tangent to my surface.
So, what does that mean? Well, that means the gradient
is actually perpendicular to the tangent plane or to the surface
at this point. So, the gradient is
perpendicular. And, well, here,
I've illustrated things with a three-dimensional example,
but really it works the same if you have only two variables.
Then you have a level curve that has a tangent line,
and the gradient is perpendicular to that line.
OK, any questions? No?
OK, so, let's see. That's actually pretty neat
because there is a nice application of this,
which is to try to figure out, now we know,
actually, how to find the tangent plane to anything,
pretty much. OK, so let's see.
So, let's say that, for example,
I want to find -- -- the tangent plane -- -- to the
surface with equation, let's say, x^2 y^2-z^2 = 4 at
the point (2,1, 1).
Let me write that. So, how do we do that?
Well, one way that we already know,
if we solve this for z, so we can write z equals a
function of x and y, then we know tangent plane
approximation for the graph of a function,
z equals some function of x and y.
But, that doesn't look like it's the best way to do it.
OK, the best way to it, now that we have the gradient
vector, is actually to directly say, oh, we know the normal
vector to this plane. The normal vector will just be
the gradient. Oh, I think I have a cool
picture to show. OK, so that's what it looks
like. OK, so here you have the
surface x2 y2-z2 equals four. That's called a hyperboloid
because it looks like when you get when you spin a hyperbola
around an axis. And, here's a tangent plane at
the given point. So, it doesn't look very
tangent because it crosses the surface.
But, it's really, if you think about it,
you will see it's really the plane that's approximating the
surface in the best way that you can at this given point.
It is really the tangent plane. So, how do we find this plane?
Well, you can plot it on a computer.
That's not exactly how you would look for it in the first
place. So, the way to do it is that we
compute the gradient. So, a gradient of what?
Well, a gradient of this function.
OK, so I should say, this is the level set,
w equals four, where w equals x^2 y^2 - z^2.
And so, we know that the gradient of this,
well, what is it? 2x, then 2y,
and then negative 2z. So, at this given point,
I guess we are at x equals two. So, that's four.
And then, y and z are one. So, two, negative two.
OK, and that's going to be the normal vector to the surface or
to the tangent plane. That's one way to define the
tangent plane. All right, it has the same
normal vector as the surface. That's one way to define the
normal vector to the surface, if you prefer.
Being perpendicular to the surface means that you are
perpendicular to its tangent plane.
OK, so the equation is, well, 4x 2y-2z equals
something, where something is, well, we should just plug in
that point. We'll get eight plus two minus
two looks like we'll get eight. And, of course,
we could simplify dividing everything by two,
but it's not very important here.
OK, so now if you have a surface given by an evil
equation, and a point on the surface,
well, you know how to find the tangent plane to the surface at
that point. OK, any questions?
No. OK, let me give just another
reason why, another way that we could have seen this.
So, I claim, in fact, we could have done
this without the gradient, or using the gradient in a
somehow disguised way. So, here's another way.
So, the other way to do it would be to start with a
differential, OK?
dw, while it's pretty much the same content,
but let me write it as a differential,
dw is 2xdx 2ydy-2zdz. So, at a given point,
at (2,1, 1), this is 4dx 2dy-2dz.
Now, if we want to change this into an approximation formula,
we can. We know that the change in w is
approximately equal to 4 delta x 2 delta y - 2 delta z.
OK, so when do we stay on the level surface?
Well, we stay on the level surface when w doesn't change,
so, when this becomes zero, OK?
Now, what does this approximation sign mean?
Well, it means for small changes in x,
y, z, this guy will be close to that guy.
It also means something else. Remember, these approximation
formulas, they are linear approximations.
They mean that we replace the function, actually,
by some closest linear formula that will be nearby.
And so, in particular, if we set this equal to zero
instead of approximately zero, it means we'll actually be
moving on the tangent plane to the level set.
If you want strict equalities in approximations means that we
replace the function by its tangent approximation. So -- [APPLAUSE] OK,
so the level corresponds to delta w equals zero,
and its tangent plane corresponds to four delta x plus
two delta y minus two delta z equals zero.
That's what I'm trying to say, basically.
And, what's delta x? Well, that means it's the
change in x. So, what's the change in x here?
That means, well, we started with x equals two,
and we moved to some other value, x.
So, that's actually x- 2, right? That's how much x has changed
compared to 2. And, two times (y - 1) minus
two times z - 1 = 0. That's the equation of a
tangent plane. It's the same equation as the
one over there. These are just two different
methods to get it. OK, so this one explains to you
what's going on in terms of approximation formulas.
This one goes right away, by using the gradient factor.
So, in a way, with this one,
you don't have to think nearly as much.
But, you can use either one. OK, questions?
No? OK, so let's move on to new
topic, which is another application of a gradient
vector, and that is directional derivatives. OK, so let's say that we have a
function of two variables, x and y.
Well, we know how to compute partial w over partial x or
partial w over partial y, which measure how w changes if
I move in the direction of the x axis or in the direction of the
y axis. So, what about moving in other
directions? Well, of course,
we've seen other approximation formulas and so on.
But, we can still ask, is there a derivative in every
direction? And that's basically,
yes, that's the directional derivative.
OK, so these are derivatives in the direction of I hat or j hat,
the vectors that go along the x or the y axis.
So, what if we move in another direction, let's say,
the direction of some unit vector, let's call it u .
OK, so if I give you a unit vector, you can ask yourself,
if I move in the direction, how quickly will my function
change? So -- So, let's look at the
straight trajectory. What this should mean is I
start at some value, x, y, and there I have my
vector u. And, I'm going to move in a
straight line in the direction of u.
And, I have the graph of my function -- -- and I'm asking
myself how quickly does the value change when I move on the
graph in that direction? OK, so let's look at a straight
line trajectory So, we have a position vector,
r, that will depend on some parameter which I will call s.
You'll see why very soon, in such a way that the
derivative is this given unit vector u hat.
So, why do I use s for my parameter rather than t.
Well, it's a convention. I'm moving at unit speed along
this line. So that means that actually,
I'm parameterizing things by the distance that I've traveled
along a curve, sorry, along this line.
So, here it's called s in the sense of arc length.
Actually, it's not really an arc because it's a straight
line, so it's the distance along the line.
OK, so because we are parameterizing by distance,
we are just using s as a convention just to distinguish
it from other situations. And, so, now,
the question will be, what is dw/ds?
What's the rate of change of w when I move like that?
Well, of course we know the answer because that's a special
case of the chain rule. So, that's how we will actually
compute it. But, in terms of what it means,
it really means we are asking ourselves,
we start at a point and we change the variables in a
certain direction, which is not necessarily the x
or the y direction, but really any direction.
And then, what's the derivative in that direction?
OK, does that make sense as a concept?
Kind of? I see some faces that are not
completely convinced. So, maybe you should show more
pictures. Well, let me first write down a
bit more and show you something. So I just want to give you the
actual definition. Sorry, first of all in case you
wonder what this is all about, so let's say the components of
our unit vector are two numbers, a and b.
Then, it means we'll move along the line x of s equals some
initial value, the point where we are actually
at the directional derivative plus s times a,
or I meant to say plus a times s.
And, y of s equals y0 bs. And then, we plug that into w.
And then we take the derivative. So, we have a notation for that
which is going to be dw/ds with a subscript in the direction of
u to indicate in which direction we are actually going to move.
And, that's called the directional derivative -- -- in
the direction of u. OK, so, let's see what it means
geometrically. So, remember,
we've seen things about partial derivatives,
and we see that the partial derivatives are the slopes of
slices of the graph by vertical planes that are parallel to the
x or the y directions. OK, so, if I have a point,
at any point, I can slice the graph of my
function by two planes, one that's going along the x,
one along the y direction. And then, I can look at the
slices of the graph. Let me see if I can use that
thing. So, we can look at the slices
of the graph that are drawn here.
In fact, we look at the tangent lines to the slices,
and we look at the slope and that gives us the partial
derivatives in case you are on that side and want to see also
the pointer that was here. So, now, similarly,
the directional derivative means, actually,
we'll be slicing our graph by the vertical plane.
It's not really colorful, something more colorful.
We'll be slicing things by a plane that is now in the
direction of this vector, u, and we'll be looking at the
slope of the slice of the graph. So, what that looks like here,
so that's the same applet the way that you've used on your
problem set in case you are wondering.
So, now, I'm picking a point on the contour plot.
And, at that point, I slice the graph.
So, here I'm starting by slicing in the direction of the
x axis. So, in fact,
what I'm measuring here by the slope of the slice is the
partial in the x direction. It's really partial f partial
x, which is also the directional derivative in the direction of
i. And now, if I rotate the slice,
then I have all of these planes.
So, you see at the bottom left, I have the direction in which
I'm going. There's this,
like, rotating line that tells you in which direction I'm going
to be moving. And for each direction,
I have a plane. And, when I slice by that
plane, I will get, so I have this direction here
going maybe to the southwest. So, that gives me a slice of my
graph by a vertical plane, and the slice has a certain
slope. And, the slope is going to be
the directional derivative in that direction.
OK, I think that's as graphic as I can get.
OK, any questions about that? No?
OK, so let's see how we compute that guy.
So, let me just write again just in case you want to,
in case you didn't hear me it's the slope of the slice of the
graph by a vertical plane -- -- that contains the given
direction, that's parallel to the
direction, u. So, how do we compute it?
Well, we can use the chain rule. The chain rule implies that
dw/ds is actually the gradient of w dot product with the
velocity vector dr/ds. But, remember we say that we
are going to be moving at unit speed in the direction of u.
So, in fact, that's just gradient w dot
product with the unit vector u. OK, so the formula that we
remember is really dw/ds in the direction of u is gradient w dot
product of u. And, maybe I should also say in
words, this is the component of the gradient in the direction of
u. And, maybe that makes more
sense. So, for example,
the directional derivative in the direction of I hat is the
component along the x axes. That's the same as,
indeed, the partial derivatives in the x direction.
Things make sense. dw/ds in the direction of I hat
is, sorry, gradient w dot I hat, which is wx,maybe I should
write, partial w of partial x. OK, now, so that's basically
what we need to know to compute these guys.
So now, let's go back to the gradient and see what this tells
us about the gradient. [APPLAUSE]
I see you guys are having fun. OK, OK, let's do a little bit
of geometry here. That should calm you down.
So, we said dw/ds in the direction of u is gradient w dot
u. That's the same as the length
of gradient w times the length of u.
Well, that happens to be one because we are taking the unit
vector times the cosine of the angle between the gradient and
the given unit vector, u, so, have this angle, theta.
OK, that's another way of saying we are taking the
component of a gradient in the direction of u.
But now, what does that tell us? Well,
let's try to figure out in which directions w changes the
fastest, in which direction it increases
the most or decreases the most, or doesn't actually change.
So, when is this going to be the largest?
If I fix a point, if I set a point,
then the gradient vector at that point is given to me.
But, the question is, in which direction does it
change the most quickly? Well, what I can change is the
direction, and this will be the largest when the cosine is one.
So, this is largest when the cosine of the angle is one.
That means the angle is zero. That means u is actually in the
direction of the gradient. OK, so that's a new way to
think about the direction of a gradient.
The gradient is the direction in which the function increases
the most quickly at that point. So, the direction of gradient w
is the direction of fastest increase of w at the given
point. And, what is the magnitude of w?
Well, it's actually the directional derivative in that
direction. OK, so if I go in that
direction, which gives me the fastest increase,
then the corresponding slope will be the length of the
gradient. And, with the direction of the
fastest decrease? It's going in the opposite
direction, right? I mean, if you are on a
mountain, and you know that you are facing the mountain,
that's the direction of fastest increase.
The direction of fastest decrease is behind you straight
down. OK, so, the minimal value of
dw/ds is achieved when cosine of theta is minus one.
That means theta equals 180�. That means u is in the
direction of minus the gradient. It points opposite to the
gradient. And, finally,
when do we have dw/ds equals zero?
So, in which direction does the function not change?
Well, we have two answers to that.
One is to just use the formula. So, that's one cosine theta
equals zero. That means theta equals 90 degrees.
That means that u is perpendicular to the gradient.
The other way to think about it, the direction in which the
value doesn't change is a direction that's tangent to the
level surface. If we are not changing a,
it means we are moving along the level.
And, that's the same thing -- -- as being tangent to the
level. So, let me just show that on
the picture here. So, if actually show you the
gradient, you can't really see it here.
I need to move it a bit. So, the gradient here is
pointing straight up at the point that I have chosen.
Now, if I choose a slice that's perpendicular,
and a direction that's perpendicular to the gradient,
so that's actually tangent to the level curve,
then you see that my slice is flat.
I don't actually have any slop. The directional derivative in a
direction that's perpendicular to the gradient is basically
zero. Now, if I rotate,
then the slope sort of increases, increases,
increases, and it becomes the largest when I'm going in the
direction of a gradient. So, here, I have,
actually, a pretty big slope. And now, if I keep rotating,
then the slope will decrease again.
Then it becomes zero when I perpendicular,
and then it becomes negative. It's the most negative when I
pointing away from the gradient and then becomes zero again when
I'm back perpendicular. OK, so for example,
if I give you a contour plot, and I ask you to draw the
direction of the gradient vector,
well, at this point, for example,
you would look at the picture. The gradient vector would be
going perpendicular to the level.
And, it would be going towards higher values of a function.
I don't know if you can see the labels, but the thing in the
middle is a minimum. So, it will actually be
pointing in this kind of direction.
OK, so that's it for today.
