-- one and -- the lecture on
symmetric matrixes. So that's the most
important class of matrixes, symmetric matrixes. A equals A transpose. So the first points, the
main points of the lecture I'll tell you right away. What's special about
the eigenvalues? What's special about
the eigenvectors? This is -- the way we
now look at a matrix. We want to know about its
eigenvalues and eigenvectors and if we have a
special type of matrix, that should tell us
something about eigenvalues and eigenvectors. Like Markov matrixes, they
have an eigenvalue equal one. Now symmetric matrixes, can I
just tell you right off what the main facts -- the
two main facts are? The eigenvalues of a
symmetric matrix, real -- this is a real
symmetric matrix, we -- talking mostly
about real matrixes. The eigenvalues are also real. So our examples of
rotation matrixes, where -- where we got E- eigenvalues
that were complex, that won't happen now. For symmetric matrixes,
the eigenvalues are real and the eigenvectors
are also very special. The eigenvectors are
perpendicular, orthogonal, so which do you prefer? I'll say perpendicular. Perp- well, they're
both long words. Okay, right. So -- I have a --
you should say "why?" and I'll at least answer why
for case one, maybe case two, the checking the Eigen --
that the eigenvectors are perpendicular, I'll leave
to, the -- to the book. But let's just realize what -- well, first I have to say, it -- it could happen, like for
the identity matrix -- there's a symmetric matrix. Its eigenvalues are
certainly all real, they're all one for
the identity matrix. What about the eigenvectors? Well, for the identity, every
vector is an eigenvector. So how can I say
they're perpendicular? What I really mean
is the -- they -- this word are should really
be written can be chosen perpendicular. That is, if we have --
it's the usual case. If the eigenvalues
are all different, then each eigenvalue has
one line of eigenvectors and those lines are
perpendicular here. But if an eigenvalue's
repeated, then there's a whole plane of eigenvectors
and all I'm saying is that in that plain, we can
choose perpendicular ones. So that's why it's a can
be chosen part, is -- this is in the case of a
repeated eigenvalue where there's some real,
substantial freedom. But the typical case is
different eigenvalues, all real, one dimensional
eigenvector space, Eigen spaces, and
all perpendicular. So, just -- let's just
see the conclusion. If we accept those as
correct, what happens -- and I also mean that
there's a full set of them. so forgive me for doing such
a thing, but, I'll look at the I -- so that's part of this
picture here, that there -- there's a complete
set of eigenvectors, perpendicular ones. So, having a complete set
of eigenvectors means -- so normal -- so the usual -- maybe I put
the -- usually -- usual -- usual case is that the matrix
A we can write in terms of its eigenvalue matrix and its
eigenvector matrix this way, right? We can do that in
the usual case, but now what's special when
the matrix is symmetric? So this is the
usual case, and now let me go to the symmetric case. So in the symmetric
case, A, this -- this should become
somehow a little special. Well, the lambdas on the
diagonal are still on the diagonal. They're -- they're real,
but that's where they are. What about the
eigenvector matrix? So what can I do now special
about the eigenvector matrix when -- when the A
itself is symmetric, that says something good
about the eigenvector matrix, so what is this --
what does this lead to? This -- these perpendicular
eigenvectors, I can not only -- I can not only guarantee
they're perpendicular, I could also make them
unit vectors, no problem, just s- scale their length to one. So what do I have? I have orthonormal eigenvectors. And what does that tell me
about the eigenvector matrix? What -- what letter should
I now use in place of S -- I've got -- those two equations
are identical,1 remember S has the eigenvectors in its columns,
but now those columns are orthonormal, so the
right letter to use is Q. So that's where -- so we've got
the letter all set up, book. so this should be
Q lambda Q inverse. Q standing in our minds always
for this matrix -- in this case it's square, it's -- so these are the Okay.
columns of Q, of course. And one more thing. What's Q inverse? For a matrix that has
these orthonormal columns, So I took the dot product
-- ye, somehow, it didn't -- I we know that the inverse
is the same as the transpose. So here is the beautiful -- there is the -- the great
haven't learned anything. description, the factorization
of a symmetric matrix. And this is, like, one
of the famous theorems of linear algebra, that if
I have a symmetric matrix, it can be factored in this form. An orthogonal matrix
times diagonal times the transpose of that
orthogonal matrix. And, of course, everybody
immediately says yes, and if this is
possible, then that's clearly symmetric, right? That -- take -- we've looked
at products of three guys like that and taken their transpose
and we got it back again. So do you -- do you see
the beauty of this -- of this factorization, then? It -- it completely displays
the eigenvalues and eigenvectors the symmetry of the -- of
the whole thing, because -- that product, Q times
lambda times Q transpose, if I transpose it, it -- this
comes in this position and we get that matrix back again. So that's -- in mathematics,
that's called the spectral Spectrum is the set of
eigenvalues of a matrix. theorem. Spec- it somehow comes from the
idea of the spectrum of light as a combination
of pure things -- where our matrix is broken
down into pure eigenvalues and eigenvectors -- in mechanics it's often called
the principle axis theorem. It's very useful. It means that if you have -- we'll see it geometrically. It means that if I
have some material -- if I look at the right axis, it
becomes diagonal, it becomes -- the -- the I- I've done something dumb,
because I've got the -- I should've taken the dot
product of this guy here with -- that's directions
don't couple together. Okay. So that's -- that -- that's
what to remember from -- from this lecture. Now, I would like to say why
are the eigenvalues real? Can I do that? So -- so -- because that --
something useful comes out. So I'll just come back --
come to that question why real eigenvalues? Okay. So I have to start
from the only thing we know, Ax equal lambda x. Okay. But as far as I
know at this moment, lambda could be complex. I'm going to prove it's not
-- and x could be complex. In fact, for the moment,
even A could be -- we could even think, well,
what happens if A is complex? Well, one thing we can
always do -- this is -- this is like always -- always okay -- I can -- if I have an equation,
I can take the complex conjugate of everything. That's -- no -- no -- so A
conjugate x conjugate equal lambda conjugate x conjugate, it
just means that everywhere over here that there was a -- an
equals x bar transpose lambda bar x bar. i, then here
I changed it to a-i. That's -- that -- you
know that that step -- that conjugate
business, that a+ib, if I conjugate it it's a-ib. That's the meaning of conjugate
-- and products behave right, I can conjugate every factor. So I haven't done anything yet
except to say what would be true if, x -- in any case, even
if x and lambda were complex. Of course, our -- we're
speaking about real matrixes A, so I can take that out. Actually, this already tells me
something about real matrixes. I haven't used any
assumption of A -- A transpose yet. Symmetry is waiting in
the wings to be used. This tells me that if a real
matrix has an eigenvalue lambda what I was going to do. and an
eigenvector x, it also has -- another of its
eigenvalues is lambda bar with eigenvector x bar. Real matrixes, the eigenvalues
come in lambda, lambda bar -- the complex eigenvalues come
in lambda and lambda bar pairs. But, of course,
I'm aiming to show that they're not complex at all,
here, by getting symmetry in. So how I going to use symmetry? I'm going to transpose
this equation to x bar transpose A transpose equals
x bar transpose lambda bar. That's just a number, so I don't
mind wear I put that number. This is -- this is -- this is a -- then again okay. Ax equals lambda x bar
transpose x, right? But now I'm ready
to use symmetry. I'm ready -- so this
was all just mechanics. Now -- now comes the
moment to say, okay, if the matrix is this
from the right with x bar, I get x bar transpose
Ax bar symmetric, then this A transpose
is the same as A. You see, at that moment
I used the assumption. Now let me finish
the discussion. Here -- here's the way I finish. I look at this original equation
and I take the inner product. I multiply both sides by -- oh, maybe I'll do
it with this one. I take -- I multiply both sides
by x bar transpose. x bar transpose Ax
bar equals lambda bar x bar transpose x bar. Okay, fine. All right, now
what's the other one? Oh, for the other one I'll
probably use this guy. A- I happy about this? No. For some reason I'm not. I'm -- I want to -- if I take the inner
product of Okay. So that -- that
was -- that's fine. That comes directly from that,
multiplying both sides by x bar transpose, but now
I'd like to get -- why do I have x bars over there? Oh, yes. Forget this. Okay. On this one -- right. On this one, I took it like
that, I multiply on the right by x. That's the idea. Okay. Now why I happier with
this situation now? A proof is coming here. Because I compare this
guy with this one. And they have the
same left hand side. So they have the
same right hand side. So comparing those two, can -- I'll raise the board to
do this comparison -- this thing, lambda
x bar transpose x is equal to lambda
bar x bar transpose x. Okay. And the conclusion
I'm going to reach -- I -- I on the right track here? The conclusion
I'm going to reach is lambda equal lambda bar. I would have to track down the
other possibility that this -- this thing is
zero, but let me -- oh -- oh, yes, that's important. It's not zero. So once I know that this
isn't zero, I just cancel it and I learn that lambda
equals lambda bar. And so what can you -- do you -- have you got the
reasoning altogether? What does this tell us? Lambda's an eigenvalue
of this symmetric matrix. We've just proved that
it equaled lambda bar, so we have just proved
that lambda is real, right? If, if a number is equal to
its own complex conjugate, then there's no
imaginary part at all. The number is real. So lambda is real. Good. Good. Now, what -- but it depended
on this little expression, on knowing that
that wasn't zero, so that I could cancel it out
-- so can we just take a second on that one? Because it's an
important quantity. x bar transpose x. Okay, now remember, as far
as we know, x is complex. So this is -- here -- x is complex, x
has these components, x1, x2 down to xn. And x bar transpose, well, it's
transposed and it's conjugated, so that's x1 conjugated x2
conjugated up to xn conjugated. I'm -- I'm -- I'm really reminding
you of crucial facts about complex numbers
that are going to come into the next
lecture as well as this one. So w- what can you tell
me about that product -- I -- I guess what
I'm trying to say is, if I had a complex vector, this
would be the quantity I would -- I would like. This is the quantity I like. I would take the vector times
its transpose -- now what -- what happens usually if I take a
vector -- a -- a -- x transpose x? I mean, that's a quantity we
see all the time, x transpose x. That's the length
of x squared, right? That's this positive length
squared, it's Pythagoras, it's x1 squared plus
x2 squared and so on. Now our vector's complex,
and you see the effect? I'm conjugating
one of these guys. So now when I do
this multiplication, I have x1 bar times x1 and
x2 bar times x2 and so on. So this is an --
this is sum a+ib. And this is sum a-ib. I mean, what's the point here? What's the point -- when
I multiply a number by its conjugate, a complex number by
its conjugate, what do I get? I get a n- the -- the
imaginary part is gone. When I multiply a+ib by
its conjugate, what's -- what's the result of that -- of
each of those separate little multiplications? There's an a squared and -- and
what -- how many -- what's -- b squared comes in
with a plus or a minus? A plus. i times minus i is
a plus b squared. And what about the
imaginary part? Gone, right? An iab and a minus iab. So this -- this is
the right thing to do. If you want a decent answer,
then multiply numbers by their conjugates. Multiply vectors by the
conjugates of x transpose. So this quantity is positive,
this quantity is positive -- the whole thing is positive
except for the zero vector and that allows me to know
that this is a positive number, which I safely cancel out
and I reach the conclusion. So actually, in this discussion
here, I've done two things. If I reached the
conclusion that lambda's real, which I wanted to do. But at the same time,
we sort of saw what to do if things were complex. If a vector is complex,
then it's x bar transpose x, this is its length squared. And as I said, the next
lecture Monday, we'll -- we'll repeat that this is
the right thing and then do the right thing for
matrixes and all other -- all other, complex
possibilities. Okay. But the main point, then,
is that the eigenvalues of a symmetric matrix, it
just -- do you -- do -- where did we use
symmetry, by the way? We used it here, right? Let -- can I just -- let -- suppose A was a complex. Suppose A had been
a complex number. Could -- could I have
made all this work? If A was a complex
number -- complex matrix, then here I should
have written A bar. I erased the bar because
I assumed A was real. But now let's suppose
for a moment it's not. Then when I took this
step, what should I have? What did I do on that step? I transposed. So I should have
A bar transpose. In the symmetric
case, that was A, and that's what made
everything work, right? This -- this led
immediately to that. This one led immediately to
this when the matrix was real, so that didn't matter,
and it was symmetric, so that didn't matter. Then I got A. But -- so now I
just get to ask you. Suppose the matrix
had been complex. What's the right equivalent
of sym- symmetry? So the good matrix --
so here, let me say -- good matrixes -- by good I mean
real lambdas and perpendicular x-s. And tell me now, which
matrixes are good? If they're -- If they're real
matrixes, the good ones are symmetric, because then
everything went through. The -- so the good -- I'm saying now what's good. This is -- this is -- these
are the good matrixes. They have real eigenvalues,
perpendicular eigenvectors -- good means A equal
A transpose if real. Then -- then that was
what -- our proof worked. But if A is complex, all -- our
proof will still work provided A bar transpose is A. Do you see what I'm saying? I'm saying if we have complex
matrixes and we want to say are they -- are they as good
as symmetric matrixes, then we should not only
transpose the thing, but conjugate it. Those are good matrixes. And of course,
the most important s- the most important
case is when they're real, this part doesn't
matter and I just have A equal A transpose symmetric. Do you -- I -- I'll just repeat that. The good matrixes, if
complex, are these. If real, that doesn't
make any difference so I'm just saying symmetric. And of course, 99% of
examples and applications to the matrixes are real
and we don't have that and then symmetric
is the key property. Okay. So that -- that's, these main
facts and now let me just -- let me just -- so that's
this x bar transpose x, okay. So I'll just, write it
once more in this form. So perpendicular orthonormal
eigenvectors, real eigenvalues, transposes of
orthonormal eigenvectors. That's the symmetric
case, A equal A transpose. Okay. Good. Actually, I'll even
take one more step here. Suppose -- I -- I can break this down
to show you really what that says about
a symmetric matrix. I can break that down. Let me here -- here
go these eigenvectors. I -- here go these eigenvalues,
lambda one, lambda two and so on. Here go these
eigenvectors transposed. And what happens if I actually
do out that multiplication? Do you see what will happen? There's lambda one
times q1 transpose. So the first row here is
just lambda one q1 transpose. If I multiply
column times row -- you remember I could do that? When I multiply matrixes, I can
multiply columns times rows? So when I do that, I
get lambda one and then the column and then
the row and then lambda two and then
the column and the row. Every symmetric matrix
breaks up into these pieces. So these pieces have real
lambdas and they have these Eigen -- these
orthonormal eigenvectors. And, maybe you even could
tell me what kind of a matrix have I got there? Suppose I take a unit
vector times its transpose? So column times row,
I'm getting a matrix. That's a matrix
with a special name. What's it's -- what
kind of a matrix is it? We've seen those matrixes,
now, in chapter four. It's -- is A A transpose
with a unit vector, so I don't have to
divide by A transpose A. That matrix is a
projection matrix. That's a projection matrix. It's symmetric and if I square
it there'll be another -- there'll be a q1 transpose
q1, which is one. So I'll get that
matrix back again. Every -- so every
symmetric matrix -- every symmetric matrix
is a combination of -- of mutually perpendicular --
so perpendicular projection matrixes. Projection matrixes. Okay. That's another way
that people like to think of the
spectral theorem, that every symmetric matrix
can be broken up that way. That -- I guess
at this moment -- first I haven't done an example. I could create a symmetric
matrix, check that it's -- find its eigenvalues,
they would come out real, find its eigenvectors, they
would come out perpendicular and you would see it in numbers,
but maybe I'll leave it here for the moment in letters. Oh, I -- maybe I will do it
with numbers, for this reason. Because there's one
more remarkable fact. Can I just put this
further great fact about symmetric
matrixes on the board? When I have
symmetric matrixes, I know their eigenvalues are
So then I can get interested in the question are they
positive real. or negative? And you remember why
that's important. For differential equations,
that decides between instability and stability. So I'm -- after I
know they're real, then the next question
is are they positive, are they negative? And I hate to have to compute
those eigenvalues to answer that question, right? Because computing the
eigenvalues of a symmetric matrix of order let's say 50 -- compute its 50 eigenvalues -- is a job. I mean, by pencil and paper
it's a lifetime's job. I mean, which -- and in fact,
a few years ago -- well, say, 20 years ago, or 30, nobody
really knew how to do it. I mean, so, like, science
was stuck on this problem. If you have a matrix
of order 50 or 100, how do you find its eigenvalues? Numerically, now,
I'm just saying, because pencil and paper is --
we're going to run out of time or paper or something
before we get it. Well -- and you
might think, okay, get Matlab to compute the
determinant of lambda minus A, A minus lambda I, this
polynomial of 50th degree, and then find the roots. Matlab will do it,
but it will complain, because it's a very bad way
to find the eigenvalues. I'm sorry to be saying
this, because it's the way I taught you to do it, right? I taught you to
find the eigenvalues by doing that
determinant and taking the roots of that polynomial. But now I'm saying, okay,
I really meant that for two by twos and three
by threes but I didn't mean you to
do it on a 50 by 50 and you're not too
unhappy, probably, because you didn't
want to do it. But -- good, because it would
be a very unstable way -- the 50 answers that would come
out would be highly unreliable. So, new ways are -- are
much better to find those 50 eigenvalues. That's a -- that's a part
of numerical linear algebra. But here's the
remarkable fact -- that Matlab would quite happily
find the 50 pivots, right? Now the pivots are not the
same as the eigenvalues. But here's the great thing. If I had a real matrix, I
could find those 50 pivots and I could see maybe
28 of them are positive and 22 are negative pivots. And I can compute those
safely and quickly. And the great fact is that 28
of the eigenvalues would be positive and 22
would be negative -- that the signs of the pivots
-- so this is, like -- I hope you think this --
this is kind of a nice thing, that the signs of the pivots -- for symmetric, I'm always
talking about symmetric matrixes -- so I'm really, like,
trying to convince you that symmetric matrixes
are better than the rest. So the signs of the pivots
are same as the signs of the eigenvalues. The same number. The number of pivots
greater than zero, the number of positive
pivots is equal to the number of positive eigenvalues. So that, actually, is a very
useful -- that gives you a g- a good start on a decent
way to compute eigenvalues, because you can
narrow them down, you can find out how
many are positive, how many are negative. Then you could shift the matrix
by seven times the identity. That would shift all the
eigenvalues by seven. Then you could take the
pivots of that matrix and you would know how many
eigenvalues of the original were above seven and below seven. So this -- this neat
little theorem, that, symmetric matrixes have this
connection between the -- nobody's mixing up and thinking
the pivots are the eigenvalues -- I mean, the only
thing I can think of is the product of
the pivots equals the product of the
eigenvalues, why is that? So if I asked you for
the reason on that, why is the product of the
pivots for a symmetric matrix the same as the product
of the eigenvalues? Because they both
equal the determinant. Right. The product of the pivots
gives the determinant if no row exchanges, the
product of the eigenvalues always gives the determinant. So -- so the products -- but
that doesn't tell you anything about the 50 individual
ones, which this does. Okay. So that's -- those are essential
facts about symmetric matrixes. Okay. Now I -- I said in the -- in
the lecture description that I would take the last minutes
to start on positive definite matrixes, because
we're right there, we're ready to say what's
a positive definite matrix? It's symmetric, first of all. On -- always I will
mean symmetric. So this is the -- this is
the next section of the book. It's about this -- if symmetric matrixes are
good, which was, like, the point of my lecture
so far, then positive, definite matrixes are -- a subclass that are
excellent, okay. Just the greatest. so what are they? They're matrixes --
they're symmetric matrixes, so all their
eigenvalues are real. You can guess what they are. These are symmetric
matrixes with all -- the eigenvalues are -- okay, tell me what to write. What -- well, it --
it's hinted, of course, by the name for these things. All the eigenvalues
are positive. Okay. Tell me about the pivots. We can check the eigenvalues
or we can check the pivots. All the pivots are what? And then I'll -- then I'll finally
give an example. I feel awful that I have got
to this point in the lecture and I haven't given you a single example. So let me give you one. Five three two two. That's symmetric, fine. It's eigenvalues
are real, for sure. But more than that, I know the
signs of those eigenvalues. And also I know the
signs of those pivots, so what's the deal
with the pivots? The Ei- if the eigenvalues are
all positive and if this little fact is true that the pivots
and eigenvalues have the same signs, then this must be true
-- all the pivots are positive. And that's the good way to test. This is the good
test, because I can -- what are the pivots
for that matrix? The pivots for that
matrix are five. So pivots are five and
what's the second pivot? Have we, like, noticed the
formula for the second pivot in a matrix? It doesn't necessarily
-- you know, it may come out a
fraction for sure, but what is that fraction? Can you tell me? Well, here, the product of
the pivots is the determinant. What's the determinant
of this matrix? Eleven? So the second pivot must
be eleven over five, so that the product is eleven. They're both positive. Then I know that the
eigenvalues of that matrix are both positive. What are the eigenvalues? Well, I've got to take
the roots of -- you know, do I put in a minus lambda? You mentally do this -- lambda
squared minus how many lambdas? Eight? Right. Five and three, the
trace comes in there, plus what number comes here? The determinant, the
eleven, so I set that to zero. So the eigenvalues are -- let's see, half of that is four,
look at that positive number, plus or minus the square
root of sixteen minus eleven, I think five. The eigenvalues -- well, two
by two they're not so terrible, but they're not so perfect. Pivots are really simple. And this is a -- this is the
family of matrixes that you really want in
differential equations, because you know the
signs of the eigenvalues, so you know the
stability or not. Okay. There's one other related
fact I can pop in here in -- in the time available for
positive definite matrixes. The related fact is to ask
you about determinants. So what's the determinant? What can you tell
me if I -- remember, positive definite means all
eigenvalues are positive, all pivots are positive, so
what can you tell me about the determinant? It's positive, too. But somehow that --
that's not quite enough. Here -- here's a matrix
minus one minus three, what's the determinant
of that guy? It's positive, right? Is this a positive,
definite matrix? Are the pivots --
what are the pivots? Well, negative. What are the eigenvalues? Well, they're also the same. So somehow I don't just want
the determinant of the whole matrix. Here is eleven, that's great. Here the determinant
of the whole matrix is three, that's positive. I also -- I've got to check,
like, little sub-determinants, say maybe coming
down from the left. So the one by one and the two
by two have to be positive. So there -- that's
where I get the all. All -- can I call them
sub-determinants -- are -- see, I have to -- I need to make the thing plural. I need to test n things, not
just the big determinant. All sub-determinants
are positive. Then I'm okay. Then I'm okay. This passes the test. Five is positive and
eleven is positive. This fails the test because that
minus one there is negative. And then the big determinant
is positive three. So t- this -- these -- this fact -- you see
that actually the course, like, coming together. And that's really my point now. In the next -- in this lecture
and particularly next Wednesday and Friday, the
course comes together. These pivots that we
met in the first week, these determinants that we met
in the middle of the course, these eigenvalues that
we met most recently -- all matrixes are square here,
so coming together for square matrixes means these three
pieces come together and they come together in that
beautiful fact, that if -- that all the -- that
if I have one of these, I have the others. That if I -- but for symmetric matrixes. So that -- this will be the
positive definite section and then the real climax of the
course is to make everything come together for
n by n matrixes, not necessarily symmetric -- bring everything
together there and that will be the final thing. Okay. So have a great
weekend and don't forget symmetric matrixes. Thanks.
