Okay. This is lecture six
in linear algebra, and we're at the start of this
new chapter, chapter three in the text, which
is really getting to the center of linear algebra. And I had time to make
a first start on it at the end of lecture five. But now is lecture
six is officially the lecture on vector
spaces and subspaces. And then especially --
there are two subspaces that we're specially interested in. One is the column
space of a matrix, the other is the null
space of the matrix. So, I got to tell
you what those are. Okay. So, first to remember
from lecture five, what is a vector space? It's a bunch of vectors
that -- where I'm allowed -- where I can add -- I can add any two
vectors in the space and the answer
stays in the space. Or I can multiply any vector
in the space by any constant and the result
stays in the space. So that's -- in fact if I
combine those two into one, you can see that -- if I can add
and I can multiply by numbers, that really means that I can
take linear combinations. So the quick way to say it is
that all linear combinations, C -- any multiple of V
plus any multiple of W stay in the space. So, can I give you examples
that are vector spaces and also some examples that are not,
to make that point clear? So, suppose I'm in
three dimensions. Then one way to get us one space
is the whole three dimensional space. So the whole space R^3,
three dimensional space, would be a vector space, because
if I have a couple of vectors I can add them and I'm certainly
okay and they follow all the rules. So R^3 is easy. Now I'm interested
also in subspaces. So there's this key
word, subspaces. That's a space -- that's some
vectors inside the given space, inside R three that still make
up a vector space of their own. It's a vector space
inside a vector space. And the simplest
example was a plane. So, like, can I just sketch
it -- there is a plane. It's got to go
through the origin, and of course it
goes infinitely far. That's of that's a subspace now. Do you see that if I have
two vectors on the plane and I add them, the
result stays in the plane. If I take a vector in the plane
and I multiply by minus two, I'm still in the plane. So that plane is a subspace. So let me just make that point. Plane through zero, through that
zero zero zero is a subspace. Okay. And also, another
subspace would be a line. A line through zero
zero zero -- yeah, the line has to go
through the origin. All subspaces have got
to contain the origin, contain zero -- the zero vector. So this line is a subspace. Really, if I want to
say it really correctly, I should say a subspace of R^3. That of R^3 was, like,
understood there. Now -- so let me call this plane
P. And let me call this line L. And let me ask you about
other sets of vectors. Suppose I took -- yeah -- so here's
a first question. Suppose I take two
subspaces, like P and L. And I just put them
together, take their union, take all the vectors -- so now you've got P
and L in mind, here. So I have two subspaces. I have two subspaces
and, for example, P -- a plane and L a line. Okay. Now I want to ask you
about the union of those. So P union L. This is all vectors
in P or L or both. Is that a subspace? Is this a subspace? This is or is not a subspace? Because we're -- I just want
to be sure that I've got the central idea. Suppose I take the
vectors in the plane and also the vectors
on that line, put them together, so I've
got a bunch of vectors, is it a subspace? Can you give me,
like, so the camera can hear it or maybe the tape. Can you say yes or no? Do I have a subspace if I put
-- if I take all the vectors on the plane plus all -- and all
the ones on the line and just join them together -- but I'm
not taking this guy that's -- actually, I'm not
taking most of them, because most vectors are not
on the line or the plane, they're off somewhere else. Do I have a subspace? STUDENTS: No. STRANG: Right. Thank you. No. Because -- why not? Because I can't add. Because if I that
requirement isn't satisfied. If I take one vector like
this guy and another vector that happens to come from L and
add, I'm off somewhere else. You see that I've gone outside
the union if I just add something from P and
something from L, then normally what'll happen
is I'm outside the union -- and I don't have a subspace. So the correct answer is -- is not. Okay. Now let me ask you about --
the other thing we do is take the intersection. So what does intersection mean? Intersection means all vectors
that are in both P and L. Is this a subspace. Yeah, so I guess I want to go
back up to the same question. This is or is not a subspace? And you can answer me -- answer
the question first for this particular example,
this picture I drew. What is P intersect
L for this case? STUDENT: It's only zero. STRANG: It's only zero. At least, sort of this was the
artist's idea as he drew it that, that that line L was
not in the plane and, went off somewhere else -- and then the
only point that was in common was the zero vector. Is the zero vector
by itself a subspace? STUDENT: Yes. STRANG: Yes, absolutely. And what about, if I don't
have this plane and this line but any subspace and
any other subspace? So now -- can I ask that
question for any two subspaces? So maybe I'll write it up here. If I'm strong enough. Okay. So this is the general question. I have subspaces, say S and T. And I want to ask you about
their intersection S intersect T and I want -- it is a subspace. Do you see why? Do you see why if I take the
vectors that are in both one- th- that are in both
of the subspaces -- so that's like a smaller
set of vectors, probably, because it's -- we've
added requirements. It has to be in S and in T. How do I know that's a subspace? Can we just think through
that abstract stuff and then I get to the examples. Okay. So why? Suppose I take a
couple of vectors that are in the intersection. Why is the sum also
in the intersection? Okay, so let me give a name
to these vectors, say V and W. They're in the intersection. So that means they're both in S. Also means they're both in T. So what can I say
about V plus W? Is it in S? Yes. Right? If I take two vectors, V
and W that are both in S, then the sum is in S,
because S was a subspace. And if they're both
in T and I add them, then the result is also in
T, because T was a subspace. So the result V plus W
is in the intersection. It's in both and requirement
one is satisfied. Requirement two's the same. If I take a vector that's in
both, I multiply by seven. Seven times that vector is in
S, because the vector was in S. Seven times that vector's in T
because the original one was in T. So seven times that vector
is in the intersection. In other words, when you
take the intersection of two subspaces, you get probably
a smaller subspace, but it is a subspace. Okay. So that's like sort of
just emphasizing what these two requirements mean. Again -- Let me circle those,
because those are so important. The sum and the scale of
multiplication which combines into linear combinations. That's what you have to
do inside the subspace. Okay. On to the column space. Okay. So my lecture last time started
that and I want to continue it. Okay. Column space of a matrix. Of A. Okay. Can I take an example? Say one two three four. One one one one. Two three four five. Okay. That's my matrix A. So, it's got columns,
three columns. Those columns are vectors, so
the column space of this A, of this A -- let's stay with this
example for a while. The column space of this
matrix is a subspace of R -- R what? So what space are
we in if I'm looking at the columns of this matrix? R^4 , right? These are vectors in R^4,
they're four dimensional vectors. So it's this column space of
A is a subspace of R^4 here, because A was four by -- A is a four by three matrix. This tells me how
many rows there are, how many components in a
column, and so we're in R^4. Okay, now what's
in that subspace? So the column space of A,
it's a subspace of R^4. I call it the column
space of A, like that. So that's my little symbol
for some subspace of R^4. What's in that subspace? Well, that column certainly is. One two three four. This column is in. This column is
in, and what else? So it's got the
columns of A in it, but that's not
enough, certainly. Right? I don't have a subspace if
I just put in three vectors. So how do I fill that
out to be a subspace? I take their linear
combinations. So the column space of A is
all linear combinations -- combinations of the columns. And that does give
me a subspace. It does give me a vector
space, because if I have one linear combination
and I multiply by eleven, I've got another
linear combination. If I have a linear
combination, I add to another
linear combination I get a third combination. So that -- this is like
the smallest space -- like, it's got to have
those three columns in it, and it has to have
their combinations and that's where we stop. Okay. Now I'm going to be
interested in that space. So I, like -- get some idea
of what's in that space. How big is that space? Is that space the whole
four dimensional space? Or is it a subspace inside? Can you -- let me just see if
we can get a yes or no answer sometimes without being
ready for the complete proof. What do you think? Is the subspace that
I'm talking about here, the combinations of
those three guys, does that fill the full
four dimensional space? Maybe yes or no on that one. No. No. Somehow our feeling
is, and it happens to be right, that if we
start with three vectors and take their combinations,
we can't get the whole four dimensional space. Now -- so somehow we
get a smaller space. But how much smaller? That's going to come up here. That's not so immediate. Let me first make this
critical connection with -- with, linear equations, because
behind our abstract definition, we have a purpose. And that is to understand Ax=b. So suppose I make
the connection -- w- w- does A x=b always
have a solution for every b? Have a solution for
every right-hand side? I guess that's going to
be a yes or no question. And then I'm going to ask which
right-hand sides are okay? That's really the
question I'm after, is which right-hand
sides (b) do make up -- you can see from the way I'm
speaking what the question -- As it is. The answer is no. A x=b does not have a
solution for every b. Why do I say no? Because A x=b is -- like,
this is four equations, and only three unknowns. Right? X is -- let me right
out that whole -- what the whole thing looks like. Yeah. Let me write out A x=b. A x is -- these columns are
one two three four. One one one one and
two three four five. Then x, of course, has three
components, x1, x2, x3. And I'm trying to get the -- hit the right-hand
side, b1,b2,b3 and b4. So my first point is,
I can't always do it. In a way, that just says again
what you told me five minutes ago -- that the combinations
of these columns don't fill the whole
four dimensional space. There's going to be some
vectors b, a lot of vectors b, that are not combinations
of these three columns, because the combinations
of those columns are, like, going to be just a little
plane or something inside -- inside R^4. Now, so and you see that I do
have four equations and only three unknowns. So, like anybody is going
to say, no you dope, you can't usually
solve four equations with only three unknowns. But now I want to say
sometimes you can. For some right-hand
sides, I can solve this. So that's the bunch
of right-hand sides that I'm interested
in right now. Which right-hand sides
allow me to solve this? This is the question for today. It's going to have, like,
a nice clear answer. So my question is -- is
which bs, which vectors b, allow this system to be solved? And I want to ask you -- so that's, like, gets two
question marks to indicate that's -- this is the
important question. Okay, first, before we
give a total answer, give me just a partial answer. Tell me one right-hand
side that I know I can solve this thing for. So -- all zeroes. Okay. That's the, like, guaranteed. If these were all zero,
then I know I can solve it, let the x-s all be
zero, no problem. So that's always a -- okay. Okay. A x=0 I can always solve. Now tell me another
right-hand side, just a specific set of numbers
for which I can solve these three -- these four equations
with only three unknowns, but if you give me a good
right-hand side, I can do it. So tell me one? STUDENT: 1 2 3 4. STRANG: 1 2 3 4? If I -- can I solve -- is
that a good right-hand side? Can you solve -- can you
find a solution that -- X one plus X two plus
two X three is one, two X one plus X two plus three
X three is two and two more equations -- so I'm asking you to
solve in your head in -- within five seconds, four
equations and three unknowns, but you can do it, because
the right-hand side is, like, showing up here is --
it's one of the columns. So tell me what's the
X that does solve it? One zero zero. One zero zero solves
it, because -- well, so you can multiply
this out by rows, but oh God, it's much nicer to say -- okay,
this is one of this column, zero of this, zero of this,
so it's one of that column, which is exactly what we wanted. Okay. So there is a b that's okay. Now tell me another
B that's okay, another right-hand side
that would be all right? Well -- all ones? Actually -- and then what's
the solution in that case? 0 1 0, thanks. And, in fact, it's
much e- like, one way to do it is think of a
solution first, right, and then just see what
b turns out to be. What b turns out to be, right. Okay. So I think of a solution
-- so I think of an x, I think of any -- x1, x2, x3, I do
this multiplication and what have I got? Now I'm ready to answer
the big question. I can solve A x=b exactly
when the right-hand side B is a vector in the column space. Good. I can solve A x=b when b is
a combination of the columns, when it's in the column space -- so let me write
that answer down. I can solve Ax=b exactly when
B is in the column space. Let me just say
again why that is. Because it -- the column space
by its definition contains all the combinations. It contains all the Ax-s. The column space really consists
of all vectors A times any X. So those are the bs
that I can deal with. If b is a combination
of the columns, then that combination
tells me what X should be. If b is not a combination of
the columns, then there is no x. There's no way to
solve A x equal b. Okay. So the column space -- that's really why we're
interested in this column space, because it's
the central guy. It says when we can
solve, and that -- we got to understand
this column space better. Let's see. Do I want to think -- yeah, somehow -- oh,
well, let's just -- as long as we've got it
here, what do I get for this particular example? If I take combinations of
this and this and this, I'll tell you the question
that's in my mind. It's not even proper
to use this word yet, but you'll know what it means. Are those three
columns independent? If I take the combinations
of the three columns -- does each column contribute
something new or now? So that if I take the
combinations of those three columns, do I, like, get some
three dimensional subspace -- do I have three vectors
that are, like, you know, independent,
whatever that means? Or do I -- is one of
those columns, like, contributing nothing new -- So that actually only
two of the columns would have given the
same column space? Yeah -- that's a good
way to ask the question. Finally I think of it. Can I throw away any columns -- and have the same column space? STUDENT: Yes. STRANG: Yes. And which one do you
suggest I throw away? STUDENT: Column three -- three. STRANG: Well, three is the
natural, like, guy to target. So if I -- and why? Because -- what's so
bad about three here? Column three? It's the sum of these, right? So it's not -- if I'm taking --
if I have combinations of these two and I put in
this one, actually, I don't get anything more. So later on I will call
these pivot columns. And the third guy will not
be a pivot column in this -- with those numbers. Now actually -- honesty makes
me ask you this question. Could I have thrown
away column one? Yes, I could. I could. So when I say pivot
columns, my convention is, okay, I'll keep the first
ones as long as they're not dependent. So I keep this guy,
he's fine, he's a line. I keep the second guy. It's in a second direction. But the third one, which is in
the same plane as the first two gives me nothing new. It's dependent in the
language that we will use and I don't need it. Okay. So I would describe the column
space of this matrix as a two dimensional subspace of R^4. A two dimensional
subspace of R^4. Okay. So you're seeing how these
vector spaces work and you -- you're seeing that we --
some idea of dependence or independence
is in our future. Okay. Now I want to speak
about another vector space, the null space. So again I'm getting
a little ahead because it's in section three
point two, but that's okay. All right. Now I'm ready for
the null space. Let me keep the same matrix. And this is going
to be a different -- totally different subspace. Totally different. Okay. Now -- so let me
make space for it. Now -- here comes a
completely different subspace, the null space of A. What's in it? It contains not
right-hand sides b. It contains x-s. It contains all
x-s that solve -- this word null is going to -- I mean, that's the key
word here, meaning zero. So this contains --
this is all solutions x, and of course x is our
vectors, x1, x2 and x3, to the equation A x=0. Well, four equations,
because we've got -- so, do you see what I'm doing? I'm now saying, okay,
columns were great, the column space we understood. Now I'm interested in x-s. I'm not -- the only b I'm
interested in now is the b of all zeroes. The right-hand
side is now zeroes. And I'm interested in solutions. x-s. So t- where is this null
space for this example? These x-s are -- have
three components. So the null space
is a subspace -- we still have to show it
is a subspace -- of R^3. So this is -- and
we will show -- these vectors x, this is in
R^3, where the column space was in R^4 in our example. For an m by n matrix,
this is m and this is n, because the number
of columns, n, tells me how many
unknowns, how many x-s multiply those
columns, so it tells me the big space, in this
case R three that I'm in. Now tell me -- why don't we
figure out what the null space is for this example,
just by looking at it. I mean, that's the
beauty of small examples, that my official way to find
null spaces and column spaces and get all the facts
straight would be elimination, and we'll do that. But with a small example,
we can see that -- see what's going on without
going through the mechanics of elimination. So this null space -- so I'm talking about --
again, the null space, and let me copy
again the matrix. One two three four, one one one
one and two three four five. What's in the null space? So I'm taking A times x,
so let me right it again, and I want you to solve
those four equations. In fact, I want you
to find all solutions to those four equations. Well, actually, just
first of all find one. Why should I ask
you for all of them? Tell me one -- well, tell
me one solution that y- you don't even have to look at
the matrix to know one solution to this set of equations. It is zero vector. Whatever that matrix is, its
null space contains zero -- because A times the zero vector
sure gives the zero right-hand side. So the null space
certainly contains zero. A- so it's got a chance
to be a vector space now, and it will turn out it is. Okay. Tell me another solution. So this particular null space --
and of course I'm going to call it N(A) for null space -- this contains-- well we've
already located the zero vector, and now you're going
to tell me another vector that's in the null space,
another solution, another x, another -- you see what I'm
asking you for is a combination of those columns. That's what I'm always looking
at combinations of columns, but now I'm looking at the
weights, the coefficients in the combination. So tell me a good set of
numbers to put in there. One one -- STUDENTS: Minus one. STRANG: One one minus one. Thanks. One one minus one. So there's a vector
that's in it. Okay. But have I got a
subspace at this point? Certainly not, right? I've got just a
couple of vectors. No way they make a subspace. Tell me -- actually, why don't
I jump the whole way now? Tell me -- well, tell
me one more solution, one more X that would work. Student: 2 2 -2. STRANG: 2 2 -2? Oh, well, tell me all of them,
that would have been easier. Tell me the whole lot, now. What is the null
space for this matrix? It's all vectors of the
form -- what could this be? It could be one one minus one,
it could be it could be any number C, any number -- the
same number again and -- STUDENTS: Minus. STRANG: Minus C. In other words -- actually, any
multiple of this guy. Oh, that's the
perfect description, because now the zero vector's
automatically included because C could be zero. The vector I had is included,
because C could be one. But now any vector. And that's actually it. And do I have a subspace? And what does it look like? It's in -- how would you
describe this, the null space, this -- all these vectors of
this form C C minus C, like, seven seven minus seven. Minus eleven minus
eleven plus eleven. What have I got here? If -- describe that whole
null space of -- what -- if I drew it, what do I draw? A line, right? The null space is a line. It's the line through -- in R^3
and the vector one one negative one maybe goes down here, I
don't know where it goes, say, down here. There's the vector one one
negative one that you gave me. And where is the
vector C C negative C? It's on this line. Of course, there's zero
zero zero that we had. And what we've got is that whole
-- oops -- that whole line, going both ways,
through the origin. The null space is a line in R^3. Okay. For that example, we could
find all the combinations of the columns that
gave zero at sight. Now, can I just
take one more time, to go back to the definition
of subspace, vector space, and ask you -- how do I know that the null
space is a vector space? How I entitled to
use this word space? I'll never use that word space
without meaning that the two requirements are satisfied. Can we just check that they are? So I'm going to check that -- can I just continue here? Check that -- that the
solutions to A x=0 always give a subspace. And, of course, the key
word is that= "Space." So what do I have to check? I have to show that if I
have one solution, call it x, and another
solution, call it x*, that their sum is also
a solution, right? That's a requirement. To use that word
space, I have to say -- I have to convince myself that
if A x is zero and also -- and A x* is zero, or maybe I
should have said if A v is zero and A w is zero, then
what about v plus w? Shall I -- let me
use those letters. If A v is zero and A w is zero,
then what -- if that and that, then what's my point here? That A times (v+w) must be zero. That says that if v is in the
null space and w's in the null space, then their sum
v+w is in the null space. And of course, now that
I've written it down, it's totally absurd,
ridiculously simple -- because matrix multiplication
allows me to separate that out into A v plus A w. I shouldn't say absurdly simple. That was a dumb thing to say. Could -- we've used, here,
a basic law of matrix multiplication. Actually, we've used it without
proving it, but that's okay. We only live so long,
we just skip that proof. I think it's called the
distributive law that I can split these -- split
this into two pieces. But now you see the point, that
A v is zero and A w is zero so I have zero plus
zero and I do get zero. It checks. And, similarly, I have to
show that if A v is zero, then A times any multiple,
say 12v is also zero. And how do I know that? Because I'm allowed to s-
bring that twelve outside. A number, a scaler can move
outside, so I have twelve A vs, twelve zeroes -- I have zero. Okay. Just to -- it's really
critical to understand the -- oh yeah. Here -- I was going to say,
understand what's the point of a vector space? Let me make that point by
changing the right-hand side. Oops. Okay. Let me change the right-hand
side to one two three four. Oh, okay. Why don't we do all of linear
algebra in one lecture, then we -- okay. I would like to know the
solutions to this equation. For those four equations. So I have four equations. I have only three
unknowns, so if I don't have a pretty
special right-hand side there won't be any
solution at all. But that is a very
special right-hand side. And we know that there is
a solution, one zero zero. Were there any more solutions? And did they form
a vector space? Okay. So I'm asking two
questions there. One is, do -- so my right-hand
side now is not zero anymore. I'm not looking
at the null space because I changed from zeroes. So my first question is, do
the solutions, if there are any and there are, do
they form a subspace? Let's answer that
question first. Yes or no. Do I get a subspace if I
look at the solutions to -- let me go back to x1 x2 x3. I'm looking at all the x-s, at
all those vectors in R^3 that solve A x -b. The only thing I've changed
is b isn't zero anymore. Do the x-s, the solutions,
form a vector space? The solutions to this
do not form a subspace. The solutions don't, because -- how shall I see that? The zero vector is not a
solution, so I never even got started. The zero vector doesn't
solve this system. I can't -- solutions
can't be a vector space. Now what are they like? Well, we'll see this, but
let's do it for this example. So one zero zero was a solution. You saw that right away. Are there any other solutions? Can you tell me
a second solution to this system of equations? STUDENTS: 0 -1 1 STRANG: 0 -1 1. Boy, that's -- 0 -1 1. Yes. Because that says I take minus
this column plus this one and sure enough. That's right. So there are -- there's a
bunch of solutions here. But they're not a subspace. I'll tell you what it's like. It's like a plane that
doesn't go through the origin, or a line that doesn't
go through the origin. Maybe in this case
it's a line that doesn't go through
the origin, if I graft the solutions to A x equal B. So you -- I think
you've got the idea. Subspaces have to go
through the origin. If I'm looking at x-s, then
they'd better solve Ax=0. In a way I've got -- my two subspaces that I --
talking about today are kind of the two ways I can tell
you what a -- about subspace. If I want to tell you
about the column space, I tell you a few columns and
I say take their combinations. Like I build up this subspace. I put in a few vectors, their
combinations make a subspace. Now, when I went to -- let me
come back to the one that is a subspace here. Here, when I talked
about the null space, I didn't tell you what's in it. We had to figure
out what was in it. What I told you was the
equations that I'm -- that has to be satisfied. You see those -- like, those are the two
natural ways to tell you what's in a subspace. I can either give you a few
vectors and say fill it out, take combinations -- or I can give you a system of
equations, the requirements that the x-s have to satisfy. And both of those
ways produce subspaces and they're the important
ways to construct subspaces. Okay, so today's
lecture actually got, the essentials of
three point two, the idea of the null space. Now we have to
tackle, Wednesday, the job of how do
we actually get hold of that subspace in an
example that's bigger and we can't see it just by eye. Okay. See you Wednesday. Thanks.
