COS 302: Vector Spaces

so this video is about vector spaces the goal is to formalize some of the intuitions that hopefully we built in an earlier video sort of reasoning about vectors generally our starting point though for vector spaces is to talk about groups now i don't know about you but group theory sounds like a very intimidating subject that's very mathy but it's useful to think of groups as just an abstraction for reasoning about certain kinds of symmetries and these are symmetries that come up in lots of different settings from things like rotations permutations and graphs and these have applications in various kinds of like coding theory and in cryptography and lots of other areas a group really just has two ingredients you need a set g so this doesn't have to be a finite set it can be an arbitrary set and you need an operation i'm going to write like this to sort of emphasize that i'm being generic about it that takes two members of g and produces another member of g i'm using a very similar notation to notation that i introduced earlier in the course that's very common where we have some when we want to talk about sort of the type signature of a function we might write the function colon and then the set that represents the domain and then the set that represents the range in this case i'm using g times g to indicate that it's the cartesian product or that is it's the it's the set of pairs of members of g this is very similar to what we did earlier when we talked about like r2 and we or rn where we had an exponent this is exactly the same concept it's just i'm writing it as a i'm just writing it with a multiplication sign i could have just as easily done this to be consistent with what we did earlier in the course the key thing here though is that it takes two objects of g and produces another object that is always in g and that gets us to the requirements that we need for the group which is the pair of the set g and the operator represented here so requirements for the set g and its operator together the first thing is what we just mentioned which is closure closure is the idea that for all pairs x and y in the set g x operator y is also a member of g we need the operation to be associative which means that for all x y and z it needs to be the case that the operator applied to x y x operator y operator z needs to be equal to y operator x z it also needs to be the case that there is a neutral element we'll call this lowercase e so there exists an e and there's a member of g such that for all x and g e operator x equals x and also that this works the other way around and then the last property we need is the property that there are inverses this is a statement that for all x and g there exists a y in g such that x operator y equals e and also y operator x equals e equals the neutral element a twist on groups is the idea of an abelian group an abelian group satisfies all of these requirements to be a group but is also commutative that is to say that for all x and y in g then x operator y equals y operator x that does not have to be true in general this is just an important special case so let's talk through a couple of examples a first example might be integers and addition so we might write that as z for the integers and plus for the addition operator you can see this as a group because if i add two integers i get another integer and you can see that it has a neutral element that is zero if i add zero to an integer i get that integer back and it has an inverse element for every member that is the opposite of whatever integer you have in hand so if i have 5 then its opposite is negative 5 and that's its inverse add those and i get the neutral element 0. so that is a group now another thing we might think of is say the natural numbers and addition so let's uh take this to include the zero element i'll write the little zero there to indicate that the natural numbers so this is zero one two three four five so on and addition now i want you to think for a second about whether or not this is a group so if i add 2 of them i get another member of the natural numbers and it has the neutral element 0 that if i add that to any natural number i get that natural number back but it's not a group because it does not have inverses that is that since it doesn't have negative numbers then i can't add any two non-zero natural numbers and get the zero element so this is not a group what about the integers and multiplication so z again but with multiplication as our binary operator rather than addition okay so if i multiply two integers i get another integer that's good it has a neutral element which is one that's good if i multiply five by one i get uh you know i get 5 back but it doesn't look like it has inverses right because what is the thing that i need to multiply 5 by in order to get 1 well one-fifth so that's a fraction and of course that's not in the integers so this this candidate does not actually have it doesn't have inverse elements in general and so it is not a group okay well let's try to fix that by adding fractions and in fact let's be more general and let's uh say real numbers and multiplication so are the real numbers and multiplication group does that fix the problem we just uh that we just saw okay well if i multiply two real numbers i get another real number that's good it has a neutral element one also good so if i multiply some real number by one then i get it back and it looks like it has inverses right so if i have five and i ask what it's inverses then it's one fifth and that's a real number and that looks good the problem is this includes zero and zero does not have an inverse there's no number that i can multiply by zero that's in the real numbers that will give me one so not a group interestingly though if i take the real numbers and i remove 0 then i do get a group so that is a group so other things that of course we're going to care about here are vectors in rn so and addition so that is a group and then while we're at it matrices of a particular shape say n minus n times n and addition are also a group notably these are also abelian groups and a final important example is square invertible matrices under multiplication so let's just think through this particular example if i multiply two square invertible matrices i get another square invertible matrix so it's closed under the operation it has a neutral element which is the identity matrix that if i multiply one of these by the identity matrix i get that element back and by definition these matrices are invertible and so they have inverses and those inverses are also in this set and if i multiply those two together then i get the identity element back note though that because matrix multiplication is not in general commutative this is not an abelian group it's just a regular group now that we know what a group is we can formally define what a vector space is from our point of view we can just think of a vector space as being an abelian group that also has scaling so we have a couple of ingredients we have a set v we need addition which takes a member of v or takes two members of v rather and then maps them to another member of v and we need scaling i'll write that operator as just a dot and it takes a member of v and a real number and turns that into another member of v we have a set of requirements that we need to satisfy and these are requirements for the triplet of v plus operator and dot operator so our set our addition operator and our scaling operator first requirement is that v and plus together must form an abelian group also the scaling operation must distribute over addition we can think of that in sort of two different ways the first is that for all lambda in the reals so all scales we might perform for all vectors x and y in v it needs to be the case that lambda dot x plus y needs to equal lambda x plus lambda y we also need to be able to distribute over sums of the scales what i mean by that is for example if we had all say a and b in the real numbers and for all x's in the vector space it needs to be the case that a plus b scaling x needs to be the same as a scaling x plus b scaling x the scaling operation must also be associative that just means that if i construct a scale by multiplying two scales then it must be the same as scaling a vector and then scaling it again that is to say that for all again say let's say a and b in the reals and for all x vectors it needs to be the case that a multiplied by b x so b scaling x and then a scaling that result needs to be the same as a multiplied by b scaling x additionally our scaling needs to have a neutral element which is straightforward because we're treating it as real numbers so that neutral element is one so for all x vectors it needs to be the case that scaling x with 1 equals x one thing i should say here is i've not necessarily put the dot in when i think it's clear what the meaning is uh that's accidental but you can see for example here lambda times x that's lambda dot x according to what we defined up here uh and so on so i hope it's still clear i've put pluses everywhere there needs to be pluses but sometimes i've implicitly uh represented the the multiplication operation the scaling operation when we have this set up the members of v are called vectors in this abelian group the neutral element is of course the all zeros vector note that in this setup we have not defined vector multiplication that is we do not have a product between two vectors here when we talk about inner products and norms that's when we'll introduce that now let's talk about vector subspaces which are a key concept for linear algebra so if we have a vector space v with of course our operations addition and scaling we call a non-empty subset u of v a subspace if it satisfies all of the requirements of a vector space so u that is a subset of v is a subspace if the combination of u plus and scaling is a vector space u has all the same properties it's closed it's an appealing group it has scaling everything we listed just a second ago except that u is a subset of another vector space v that also satisfies all of those properties closure is the main thing you want to think about so a vector space requires that addition be a group that means when i take two elements and i add them together i need to stay within that set so that means if i'm in a subspace u then when i do scaling and when i do addition with other members of u i can never escape you so once i'm in you i can't get back into things that are in v but not in you so closure really has to be preserved so that's the main characteristic of the vector space is that you're kind of stuck in it with those operations one key thing is that you needs to be non-empty and so it's worth thinking about what you would need to have in the smallest possible subspace the answer turns out to be that the small subspace you can have is one just with the zero vector that is just with the neutral element of the vector abelian group so if i add zero to zero i stay in i stay zero if i scale zero i stay zero so i can perform all of the operations i needed the vector space with that one member that is with a set of size one just the zero vector you can also immediately convince yourself that all subspaces must contain that zero vector not surprising because they all need to be groups and they all need to have a neutral element under addition and so they have to have that one special point even if they don't have any other ones thinking back to our idea of vectors as directed line segments is helpful for thinking about subspaces if i think of a vector space in two dimensions i could have many different kinds of vectors and the idea being that i can add these in different ways and i will always stay in the set and so this might contain sort of all of these two-dimensional directed lines but i could just consider a set of directed line segments that are all co-linear say pointed in just a single direction then no matter how i add these guys up and no matter how i scale them i will always be stuck on that line segment so i can't access these other regions of the space because i'm on this subspace of the original vector space

Channel: Intelligent Systems Lab

Views: 195

Rating: 5 out of 5

Keywords:

Id: C43Q7jIU23A

Length: 16min 10sec (970 seconds)

Published: Sun Feb 07 2021