OK, here we go with, quiz
review for the third quiz that's coming on Friday. So, one key point
is that the quiz covers through chapter six. Chapter seven on
linear transformations will appear on the final
exam, but not on the quiz. So I won't review linear
transformations today, but they'll come into the
full course review on the very last lecture. So today, I'm
reviewing chapter six, and I'm going to
take some old exams, and I'm always ready
to answer questions. And I thought, kind of help
our memories if I write down the main topics in chapter six. So, already, on
the previous quiz, we knew how to find
eigenvalues and eigenvectors. Well, we knew how to find them
by that determinant of A minus lambda I equals zero. But, of course, there
could be shortcuts. There could be, like,
useful information about the eigenvalues that
we can speed things up with. OK. Then, the new stuff starts out
with a differential equation, so I'll do a problem. I'll do a differential
equation problem first. What's special about
symmetric matrices? Can we just say that in words? I'd better write
it down, though. What's special about
symmetric matrices? Their eigenvalues are real. The eigenvalues of a symmetric
matrix always come out real, and there always are
enough eigenvectors. Even if there are
repeated eigenvalues, there are enough
eigenvectors, and we can choose those eigenvectors
to be orthogonal. So if A equals A
transposed, the big fact will be that we
can diagonalize it, and those eigenvector
matrix, with the eigenvectors in the column, can be
an orthogonal matrix. So we get a Q
lambda Q transpose. That, in three symbols,
expresses a wonderful fact, a fundamental fact for
symmetric matrices. OK. Then, we went beyond
that fact to ask about positive
definite matrices, when the eigenvalues were positive. I'll do an example of that. Now we've left symmetry. Similar matrices are
any square matrices, but two matrices are similar
if they're related that way. And what's the key point
about similar matrices? Somehow, those matrices
are representing the same thing in
different basis, in chapter seven language. In chapter six language, what's
up with these similar matrices? What's the key fact,
the key positive fact about similar matrices? They have the same eigenvalues. Same eigenvalues. So if one of them grows,
the other one grows. If one of them decays to zero,
the other one decays to zero. Powers of A will look
like powers of B, because powers of
A and powers of B only differ by an M
inverse and an M way on the outside. So if these are similar,
then B to the k-th power is M inverse A to
the k-th power M. And that's why I
say, eh, this M, it does change the
eigenvectors, but it doesn't change the eigenvalues. So same lambdas. And then, finally, I've
got to review the point about the SVD, the Singular
Value Decomposition. OK. So that's what this
quiz has got to cover, and now I'll just take
problems from earlier exams, starting with a
differential equation. OK. And always ready for questions. So here is an exam
from about the year zero, and it has
a three by three. So that was -- but it's a pretty
special-looking matrix, it's got zeroes on the diagonal,
it's got minus ones above, and it's got plus
ones like that. So that's the matrix A. OK. Step one is, well,
I want to solve that equation. I want to find the
general solution. I haven't given you
a u(0) here, so I'm looking for the
general solution, so now what's the form
of the general solution? With three arbitrary
constants going to be inside it, because
those will be used to match the initial condition. So the general
form is u at time t is some multiple of the
first special solution. The first special solution will
be growing like the eigenvalue, and it's the eigenvector. So that's a pure exponential
solution, just staying with that eigenvector. Of course, I haven't
found, yet, the eigenvalues and eigenvectors. That's, normally, the first job. Now, there will be second
one, growing like e to the lambda two, and a
third one growing like e to the lambda three. So we're all done -- well, we haven't
done anything yet, actually. I've got to find the
eigenvalues and eigenvectors, and then I would match u(0)
by choosing the right three constants. OK. So now I ask -- ask you
about the eigenvalues and eigenvectors, and you look
at this matrix and what do you see in that matrix? Um, well, I guess we might
ask ourselves right away, is it singular? Is it singular? Because, if so, then we
really have a head start, we know one of the
eigenvalues is zero. Is that matrix singular? Eh, I don't know, do you
take the determinant to find out? Or maybe you look at the
first row and third row and say, hey, the
first row and third row are just opposite signs,
they're linear-dependent? The first column and third
column are dependent -- it's singular. So one eigenvalue is zero. Let's make that lambda one. Lambda one, then, will be zero. OK. Now we've got a couple of
other eigenvalues to find, and, I suppose the
simplest way is to look at A minus
lambda I So let me just put minus lambda
in here, minus ones above, ones below. But, actually, before
I do it, that matrix is not symmetric,
for sure, right? In fact, it's the very
opposite of symmetric. That matrix A transpose, how
is A transpose connected to A? It's negative A. It's an anti-symmetric
matrix, skew-symmetric matrix. And we've met, maybe,
a two-by-two example of skew-symmetric
matrices, and let me just say, what's the
deal with their eigenvalues? They're pure imaginary. They'll be on the
imaginary axis, there be some
multiple of I if it's an anti-symmetric,
skew-symmetric matrix. So I'm looking for
multiples of I, and of course,
that's zero times I, that's on the imaginary axis,
but maybe I just do it out, here. Lambda cubed. well, maybe that's
minus lambda cubed, and then a zero and a zero. Zero, and then maybe I
have a plus a lambda, and another plus lambda, but
those go with a minus sign. Am I getting minus two
lambda equals zero? So. So I'm solving lambda cube
plus two lambda equals zero. So one root factors out
lambda, and the the rest is lambda squared plus two. OK. This is going the
way we expect, right? Because this gives the root
lambda equals zero, and gives the other two roots, which
are lambda equal what? The solutions of when
is lambda squared plus two equals zero then the
eigenvalues those guys, what are they? They're a multiple of i,
they're just square root of two i. When I set this
equals to zero, I have lambda squared equal
to minus two, right? To make that zero? And the roots are
square root of two i and minus the square
root of two i. So now I know what those are. I'll put those in, now. Either the zero t is just a one. That's just a one. This is square root of
two I and this is minus square root of two I. So, is the solution
decaying to zero? Is this a completely
stable problem where the solution
is going to zero? No. In fact, all these things
are staying the same size. This thing is getting
multiplied by this number. e to the I something t, that's
a number that has magnitude one, and sort of wanders
around the unit circle. Same for this. So that the solution doesn't
blow up, and it doesn't go to zero. OK. And to find out
what it actually is, we would have to plug
in initial conditions. But actually, the
next question I ask is, when does the solution
return to its initial value? I won't even say what's
the initial value. This is a case in which
I think this solution is periodic after. At t equals zero, it
starts with c1, c2, and c3, and then at some value of
t, it comes back to that. So that's a very
special question, Well, let's just
take three seconds, because that special question
isn't likely to be on the quiz. But it comes back
to the start, when? Well, whenever we have e to
the two pi i, that's one, and we've come back again. So it comes back to the start. It's periodic, when this
square root of two i -- shall I call it capital
T, for the period? For that particular T, if
that equals two pi i, then e to this thing is one, and
we've come around again. So the period is T is
determined here, cancel the i-s, and T is pi times the
square root of two. So that's pretty neat. We get all the information
about all solutions, we haven't fixed on only
one particular solution, but it comes around again. So this was probably
my first chance to say something
about the whole family of anti-symmetric,
skew-symmetric matrices. OK. And then, finally, I asked,
take two eigenvectors (again, I haven't computed
the eigenvectors) and it turns out
they're orthogonal. They're orthogonal. The eigenvectors of
a symmetric matrix, or a skew-symmetric matrix,
are always orthogonal. I guess may conscience
makes me tell you, what are all the matrices that
have orthogonal eigenvectors? And symmetric is the
most important class, so that's the one
we've spoken about. But let me just put that
little fact down, here. Orthogonal x-s. eigenvectors. A matrix has orthogonal
eigenvectors, the exact condition -- it's
quite beautiful that I can tell you exactly when that happens. It happens when A times A
transpose equals A transpose times A. Any time
that's the condition for orthogonal eigenvectors. And because we're interested
in special families of vectors, tell me some special
families that fit. This is the whole requirement. That's a pretty special
requirement most matrices have. So the average
three-by-three matrix has three eigenvectors,
but not orthogonal. But if it happens to
commute with its transpose, then, wonderfully, the
eigenvectors are orthogonal. Now, do you see how symmetric
matrices pass this test? Of course. If A transpose equals A, then
both sides are A squared, we've got it. How do anti-symmetric
matrices pass this test? If A transpose equals
minus A, then we've got it again, because we've got
minus A squared on both sides. So that's another group. And finally, let me ask you
about our other favorite family, orthogonal matrices. Do orthogonal matrices pass
this test, if A is a Q, do they pass the test for
orthogonal eigenvectors. Well, if A is Q, an orthogonal
matrix, what is Q transpose Q? It's I. And what is Q Q transpose? It's I, we're talking
square matrices here. So yes, it passes the test. So the special cases are
symmetric, anti-symmetric (I'll say skew-symmetric,)
and orthogonal. Those are the three
important special classes that are in this family. OK. That's like a comment that,
could have been made back in, section six point four. OK, I can pursue the
differential equations, also this question, didn't
ask you to tell me, how would I find this matrix
exponential, e to the At? So can I erase this? I'll just stay with this same... how would I find e to the At? Because, how does that come in? That's the key matrix for
a differential equation, because the solution is -- the solution is
u(t) is e^(At) u(0). So this is like the
fundamental matrix that multiplies the given
function and gives the answer. And how would we compute
it if we wanted that? We don't always have to
find e to the At, because I can go directly to the answer
without any e to the At-s, but hiding here is an e to the
At, and how would I compute it? Well, if A is diagonalizable. So I'm now going to put in my
usual if A can be diagonalized (and everybody remember
that there is an if there, because it might not
have enough eigenvectors) this example does have enough,
random matrices have enough. So if we can diagonalize, then
we get a nice formula for this, because an S comes way
out at the beginning, and S inverse comes
way out at the end, and we only have to take
the exponential of lambda. And that's just a
diagonal matrix, so that's just e
the lambda one t, these guys are showing up,
now, in e to the lambda nt. OK? That's a really quick
review of that formula. It's something we can
compute it quickly if we have done the
S and lambda part. If we know S and
lambda, then it's not hard to take that step. OK, that's some comments
on differential equations. I would like to go on to a next
question that I started here. And it's, got several parts,
and I can just read it out. What we're given is a
three-by-three matrix, and we're told its eigenvalues,
except one of these is, like, we don't know, and
we're told the eigenvectors. And I want to ask
you about the matrix. OK. So, first question. Is the matrix diagonalizable? And I really mean for
which c, because I don't know c, so my
questions will all be, for which is there a condition
on c, does one c work. But your answer should tell
me all the c-s that work. I'm not asking for you to
tell me, well, c equal four, yes, that checks out. I want to know all the c-s
that make it diagonalizable. OK? What's the real
on diagonalizable? We need enough
eigenvectors, right? We don't care what
those eigenvalues are, it's eigenvectors that
count for diagonalizable, and we need three
independent ones, and are those three
guys independent? Yes. Actually, let's look
at them for a moment. What do you see about those
three vectors right away? They're more than independent. Can you see why those
three got chosen? Because it will come up in the
next part, they're orthogonal. Those eigenvectors
are orthogonal. They're certainly independent. So the answer to diagonalizable
is, yes, all c, all c. Doesn't matter. c could
be a repeated guy, but we've got
enough eigenvectors, so that's what we care about. OK, second question. For which values of
c is it symmetric? OK, what's the
answer to that one? If we know the same setup if
we know that much about it, we know those
eigenvectors, and we've noticed they're orthogonal,
then which c-s will work? So the eigenvalues of that
symmetric matrix have to be real. So all real c. If c was i, the matrix
wouldn't have been symmetric. But if c is a real number, then
we've got real eigenvalues, we've got orthogonal
eigenvectors, that matrix is symmetric. OK, positive definite. OK, now this is a
sub-case of symmetric, so we need c to be real, so
we've got a symmetric matrix, but we also want the thing
to be positive definite. Now, we're looking
at eigenvalues, we've got a lot of tests
for positive definite, but eigenvalues,
if we know them, is certainly a good,
quick, clean test. Could this matrix be
positive definite? No. No, because it's got
an eigenvalue zero. It could be positive
semi-definite, you know, like
consolation prize, if c was greater
or equal to zero, it would be positive
semi-definite. But it's not, no. Semi-definite, if I put that
comment in, semi-definite, that the condition would be
c greater or equal to zero. That would be all right. OK. Next part. Is it a Markov matrix? Hm. Could this matrix be, if I
choose the number c correctly, a Markov matrix? Well, what do we know
about Markov matrices? Mainly, we know something
about their eigenvalues. One eigenvalue is always one,
and the other eigenvalues are smaller. Not larger. So an eigenvalue
two can't happen. So the answer is, no, not a ma-
that's never a Markov matrix. OK? And finally, could one half
of A be a projection matrix? So could it- could this --
eh-eh could this be twice a projection matrix? So let me write it this way. Could A over two be
a projection matrix? OK, what are
projection matrices? They're real. I mean, th- they're symmetric,
so their eigenvalues are real. But more than that, we know what
those eigenvalues have to be. What do the eigenvalues of a
projection matrix have to be? See, that any nice
matrix we've got an idea about its eigenvalues. So the eigenvalues of
projection matrices are zero and one. Zero and one, only. Because P squared equals P,
let me call this matrix P, so P squared equals P, so
lambda squared equals lambda, because eigenvalues of P
squared are lambda squared, and we must have that, so
lambda equals zero or one. OK. Now what value of
c will work there? So, then, there are some
value that will work, and what will work? c equals zero will work,
or what else will work? c equal to two. Because if c is two, then
when we divide by two, this Eigenvalue of
two will drop to one, and so will the other one,
so, or c equal to two. OK, those are the
guys that will work, and it was the fact that those
eigenvectors were orthogonal, the fact that those
eigenvectors were orthogonal carried us a lot
of the way, here. If they weren't orthogonal, then
symmetric would have been dead, positive definite
would have been dead, projection would have been dead. But those eigenvectors
were orthogonal, so it came down to
the eigenvalues. OK, that was like a chance to
review a lot of this chapter. Shall I jump to the singular
value decomposition, then, as the third, topic
for, for the review? OK, so I'm going
to. jump to this. OK. So this is the singular
value decomposition, known to everybody as the SVD. And that's a factorization
of A into orthogonal times diagonal times orthogonal. And we always call those U
and sigma and V transpose. OK. And the key to that -- this is for every
matrix, every A, every A. Rectangular, doesn't
matter, whatever, has this decomposition. So it's really important. And the key to it is to look
at things like A transpose A. Can we remember what
happens with A transpose A? If I just transpose that
I get V sigma transpose U transpose, that's
multiplying A, which is U, sigma V transpose, and the
result is V on the outside, s- U transpose U
is the identity, because it's an
orthogonal matrix. So I'm just left with
sigma transpose sigma in the middle, that's
a diagonal, possibly rectangular diagonal by its
transpose, so the result, this is orthogonal,
diagonal, orthogonal. So, I guess, actually, this
is the SVD for A transpose A. Here I see orthogonal,
diagonal, and orthogonal. Great. But a little more is happening. For A transpose
A, the difference is, the orthogonal
guys are the same. It's V and V transpose. What I seeing here? I'm seeing the factorization
for a symmetric matrix. This thing is symmetric. So in a symmetric case,
U is the same as V. U is the same as V for
this symmetric matrix, and, of course, we
see it happening. OK. So that tells us,
right away, what V is. V is the eigenvector
matrix for A transpose A. OK. Now, if you were here when I
lectured about this topic, when I gave the topic on singular
value decompositions, you'll remember that
I got into trouble. I'm sorry to remember that
myself, but it happened. OK. How did it happen? I was in great shape for
a while, cruising along. So I found the eigenvectors
for A transpose A. Good. I found the singular
values, what were they? What were the singular values? The singular value
number i, or -- these are the guys in sigma -- this is diagonal with
the number sigma in it. This diagonal is
sigma one, sigma two, up to the rank, sigma r,
those are the non-zero ones. So I found those,
and what are they? Remind me about that? Well, here, I'm seeing them
squared, so their squares are the eigenvalues
of A transpose A. Good. So I just take the square root,
if I want the eigenvalues of A transpose -- If I want the sigmas
and I know these, I take the square root,
the positive square root. OK. Where did I run into trouble? Well, then, my final
step was to find U. And I didn't read the book. So, I did something that was
practically right, but -- well, I guess practically
right is not quite the same. OK, so I thought, OK, I'll
look at A A transpose. What happened when I
looked at A A transpose? Let me just put it here,
and then I can feel it. OK, so here's A A transpose. So that's U sigma V
transpose, that's A, and then the transpose
is V sigma transpose, U sigma transpose. Fine. And then, in the middle
is the identity again, so it looks great. U sigma sigma
transpose, U transpose. Fine. All good, and now
these columns of U are the eigenvectors,
that's U is the eigenvector matrix for this guy. That was correct,
so I did that fine. Where did something go wrong? A sign went wrong. A sign went wrong because --
and now -- now I see, actually, somebody told me
right after class, we can't tell from this
description which sign to give the eigenvectors. If these are the
eigenvectors of this matrix, well, if you give
me an eigenvector and I change all
its signs, we've still got another eigenvector. So what I wasn't
able to determine (and I had a fifty-fifty
change and life let me down,) the signs I just
happened to pick for the eigenvectors,
one of them I should have reversed the sign. So, from this, I can't tell
whether the eigenvector or its negative is the
right one to use in there. So the right way to
do it is to, having settled on the
signs, the Vs also, I don't know which sign to
choose, but I choose one. I choose one. And then, instead,
I should have used the one that tells me what
sign to choose, the rule that A times a V is
sigma times the U. So, having decided on
the V, I multiply by A, I'll notice the factor
sigma coming out, and there will be a
unit vector there, and I now know
exactly what it is, and not only up to
a change of sign. So that's the good
and, of course, this is the main
point about the SVD. That's the point that
we've diagonalized, that's A times the
matrix of Vs equals U times the diagonal
matrix of sigmas. That's the same as that. OK. So that's, like,
correcting the wrong sign from that earlier lecture. And that would complete that,
so that's how you would compute the SVD. Now, on the quiz, I going to
ask -- well, maybe on the final. So we've got quiz
and final ahead. Sometimes, you might be asked
to find the SVD if I give you the matrix -- let me come
back, now, to the main board -- or, I might give you the pieces. And I might ask you
something about the matrix. For example, suppose I
ask you, oh, let's say, if I tell you what sigma is -- OK. Let's take one example. Suppose sigma is -- so all that's how we
would compute them. But now, suppose
I give you these. Suppose I give you sigma
is, say, three two. And I tell you that U
has a couple of columns, and V has a couple of columns. OK. Those are orthogonal
columns, of course, because U and V are orthogonal. I'm just sort of,
like, getting you to think about the SVD,
because we only had that one lecture about it,
and one homework, and, what kind of a
matrix have I got here? What do I know
about this matrix? All I really know right now
is that its singular values, those sigmas are three and
two, and the only thing interesting that I can see in
that is that they're not zero. I know that this matrix
is non-singular, right? That's invertible, I don't
have any zero eigenvalues, and zero singular values,
that's invertible, there's a typical SVD for a
nice two-by-two non-singular invertible good matrix. If I actually gave you
a matrix, then you'd have to find the Us and
the Vs as we just spoke. But, there. Now, what if the two
wasn't a two but it was -- well, let me make an
extreme case, here -- suppose it was minus five. That's wrong, right away. That's not a singular
value decomposition, right? The singular values
are not negative. So that's not a singular value
decomposition, and forget it. OK. So let me ask you
about that one. What can you tell me
about that matrix? It's singular, right? It's got a singular matrix
there in the middle, and, let's see, so,
OK, it's singular, maybe you can tell me, its rank? What's the rank of A? It's clearly --
somebody just say it -- one, thanks. The rank is one,
so the null space, what's the dimension
of the null space? One. Right? We've got a two-by-two
matrix of rank one, so of all that stuff from
the beginning of the course is still with us. The dimensions of those
fundamental spaces is still central,
and a basis for them. Now, can you tell me a vector
that's in the null space? And then that will be my last
point to make about the SVD. Can you tell me a vector
that's in the null space? So what would I multiply
by and get zero, here? I think the answer
is probably v2. I think probably v2
is in the null space, because I think that must
be the eigenvector going with this zero eigenvalue. Yes. Have a look at that. And I could ask you the
null space of A transpose. And I could ask you
the column space. All that stuff. Everything is sitting
there in the SVD. The SVD takes a little
more time to compute, but it displays all the
good stuff about a matrix. OK. Any question about the SVD? Let me keep going
with further topics. Now, let's see. Similar matrices
we've talked about, let me see if I've
got another, -- OK. Here's a true false, so
we can do that, easily. So. Question, A given. A is symmetric and orthogonal. OK. So beautiful matrices like that
don't come along every day. But what can we say first
about its eigenvalues? Actually, of course. Here are our two most
important classes of matrices, and we're looking
at the intersection. So those really
are neat matrices, and what can you tell
me about what could the possible eigenvalues be? Eigenvalues can be what? What do I know about
the eigenvalues of a symmetric matrix? Lambda is real. What do I know about
the eigenvalues of an orthogonal matrix? Ha. Maybe nothing. But, no, that can't be. What do I know about the
eigenvalues of an orthogonal matrix? Well, what feels right? Basing mathematics on just
a little gut instinct here, the eigenvalues of
an orthogonal matrix ought to have magnitude one. Orthogonal matrices
are like rotations, they're not changing the length,
so orthogonal, the eigenvalues are one. Let me just show you why. So the matrix, can I
call it Q for orthogonal Why? for the moment? If I look at Q x
equal lambda x, how do I see that this
thing has magnitude one? I take the length of both sides. This is taking lengths,
taking lengths, this is whatever the magnitude
is times the length of x. And what's the length of Q x
if Q is an orthogonal matrix? This is something
you should know. It's the same as
the length of x. Orthogonal matrices
don't change lengths. So lambda has to be one. Right. OK. That's worth
committing to memory, that could show up again. OK. So what's the answer
now to this question, what can the eigenvalues be? There's only two
possibilities, and they are one and the other
one, the other possibility is negative one, right, because
these have the right magnitude, and they're real. OK. TK. true -- OK. True or false? A is sure to be
positive definite. Well, this is a great
matrix, but is it sure to be positive definite? No. If it could have an
eigenvalue minus one, it wouldn't be
positive definite. True or false, it has
no repeated eigenvalues. That's false, too. In fact, it's going to
have repeated eigenvalues if it's as big as
three by three, one of these c- one
of these, at least, will have to get repeated. Sure. So it's got repeated
eigenvalues, but, is it diagonalizable? It's got these many, many,
repeated eigenvalues. If it's fifty by
fifty, it's certainly got a lot of repetitions. Is it diagonalizable? Yes. All symmetric matrices,
all orthogonal matrices can be diagonalized. And, in fact, the eigenvectors
can even be chosen orthogonal. So it could be, sort
of, like, diagonalized the best way with a Q,
and not just any old S. OK. Is it non-singular? Is a symmetric orthogonal
matrix non-singular? Orthogonal matrices are
always non-singular. Sure. And, obviously, we don't
have any zero Eigenvalues. Is it sure to be diagonalizable? Yes. Now, here's a final step -- show
that one-half of A plus I is A -- that is, prove one-half of A
plus I is a projection matrix. OK? Let's see. What do I do? I could see two ways to do this. I could check the properties
of a projection matrix, which are what? A projection matrix
is symmetric. Well, that's certainly
symmetric, because A is. And what's the other property? I should square
it, and hopefully get the same thing back. So can I do that, square and see
if I get the same thing back? So if I square it, I'll get
one-quarter of A squared plus two A plus I, right? And the question is, does that
agree with p- the thing itself? One-half A plus I. Hm. I guess I'd like to know
something about A squared. What is A squared? That's our problem. What is A squared? If A is symmetric
and orthogonal, A is symmetric and orthogonal. This is what we're given, right? It's symmetric, and
it's orthogonal. So what's A squared? I. A squared is I,
because A times A -- if A equals its own inverse,
so A times A is the same as A times A inverse, which is I. So this A squared here is I. And now we've got it. We've got two identities
over four, that's good, and we've got two As
over four, that's good. OK. So it turned out to be a
projection matrix safely. And we could also
have said, well, what are the eigenvalues
of this thing? What are the eigenvalues
of a half A plus I? If the eigenvalues of A
are one and minus one, what are the
eigenvalues of A plus I? Just stay with it these
last thirty seconds here. What if I know these
eigenvalues of A, and I add the identity,
the eigenvalues of A plus I are zero and two. And then when I divide by two,
the eigenvalues are zero and one. So it's symmetric, it's
got the right eigenvalues, it's a projection matrix. OK, you're seeing a lot of
stuff about eigenvalues, and special matrices, and
that's what the quiz is about. OK, so good luck on the quiz.
