Eigenvalues & Eigenvectors : Data Science Basics

[Music] hello everybody in this video we're gonna be talking about eigenvectors and eigenvalues which are often a point of confusion for a lot of students who are just learning linear algebra and that makes sense these words are kind of scary and the concept itself does not seem super important or intuitive or useful in fact it is and we'll talk about the end of this video why eigenvalues and eigenvectors are so important from a data science perspective so that hopefully that helps to help you remember these concepts better but first things first we're going to talk about what is the definition of an eigenvector and an eigenvalue it's pretty simple if you have any square matrix a so a can be 2x2 3x3 any square matrix and eigenvector is any vector X which is not equal to 0 such that multiplying matrix a by vector X gives back some multiple of vector X where that multiplier is lambda which can be any real number so for example lambda could be 2 or negative 1 in which case you would have a times X gives back negative 1 times X or 2 times X and that's all an eigen vector and an eigen it's corresponding eigenvalue is so that's one important point to note is that an eigenvector has a corresponding eigenvalue so in this case the eigen vectors X the eigen value is lambda ok let's jump right into an example a real example of how to derive eigenvectors and eigenvalues by hand of course you'll use a computer program in real life but it's important to have the theory down okay here's our square matrix a 0 1 minus 2 minus 3 and our challenge is going to be let's figure out what are the eigenvalues and eigenvectors of this matrix okay let's start with the definition that because that's all we really have right now so we know that if we find an eigenvector x it's going to need to satisfy that equation from the definition now let's rewrite the right hand side a little bit and put the identity matrix right here so remember that the identity matrix in two dimensions is just given by 1 0 0 1 and it an analogue to multiplying any part of an equation by one it doesn't change anything it just gonna help us to go to our next step a little bit easier so this equation still holds now what we're gonna do is we're gonna subtract lambda identity X from the right-hand side on the left hand side so we get a X minus lambda identity matrix X is equal to zero we like in math when we have things equal to zero because they helped us a lot we see we have an X here and an X here let's factor the X out so we get a minus lambda identity matrix times X is equal to zero now let's pause for a second to think about what this equation means remember one key fact that X is not equal to the zero vector okay we assume that right here so that means that this new matrix we've devised and let's say where m is equal to a minus lambda I has some vector X which is not equal to zero in its null space or its kernel another way of saying that is that there is some vector X that I can do M times X and get zero back which means that this may this matrix M is not invertible we'll make a video on invertibility of matrices but the main point you need to recall right now is that if there's some nonzero vector X in the null space of matrix M that means matrix m is not invertible what else do we know if we know the matrix is not invertible well we know something important about its determinant right let me create some space right here so let me erase the definition here if we know matrix m is not invertible we know it's determinant has to be equal to 0 because the converse is true we know that if a matrix is invertible its determinant is not equal to 0 okay so we know for a fact that they determinant which we write as Det determinant of a minus lambda I has to be equal to 0 because of the non-invertibility of that matrix as we saw down here all right cool so what do we do with this let's go ahead and write this in its full form so this is zero this is zero one minus two minus three we subtract lambda times I which is just it looks like this and we get negative lambda we get one we get minus two and we get minus 3 minus lambda as that entry and we want to know what's the determinant of that matrix okay 2 by 2 determinants are something we can handle remember it's just this element times this element so let's do that first if we do that we get 3 lambda plus lambda squared minus this element times this element so minus negative 2 gives us positive 2 so that is the determinant this polynomial in terms of lambda is the determinant of a minus lambda I now we want to make sure the determinant is equal to 0 this is just a nice polynomial equation a quadratic equation so we don't even have to use the quadratic formula in this case why because we know we can just factor it so we can do lambda plus 2 and lambda plus 1 does that check out lambda square 2 lambda that goes it's 3 left yeah that checks out and we know that the roots of this polynomial equation are lambda is equal to minus 2 and lambda is equal to minus 1 awesome so we solved for the two eigenvalues so remember lambdas are the eigenvalues they are negative 2 and negative 1 now the only question is how do I find the corresponding eigenvectors let's do it for one of them let's pick lambda equals negative 1 let's get rid of everything on here except the matrix that we care about so we know now that negative 1 is an eigen value of this matrix how does that help us find the eigenvector well we write again the definition so we know that a which is 0 1 negative 2 negative 3 times some vector x1 x2 we don't know that yet is equal to lambda times that same vector in this case we're using lambda as negative 1 so if we do negative 1 we get minus x1 and minus x2 now this is pretty trivial it's just solving a system of equations and since there's a 0 in here it'll even easier so our system of equations becomes the first equation is just X 2 is equal to minus X 1 so the minus X 1 comes from here and the second equation is negative 2 X 1 minus 3 X 2 is equal to minus X 2 let's plug the first equation into the second one so we have X 2 is equal to negative X 1 let's well you know what let's first move this minus 3x 2 onto that side so that we get negative 2 X 1 is equal to 2 X 2 and then we divide by a negative 2 on both sides and we get positive here we get X 1 is equal to negative X 2 in fact we didn't even need to plug in the first equation the second one because they say the exact same thing is that you can use any vector in here any eigen vector such that X 1 is equal to negative X 2 for example 1 negative 1 will work or negative 2 2 will work so there's a whole family of eigenvectors that go along with this one heigen value and we can do the same thing for the other eigen value which was negative 2 instead we would put a negative 2 right here and then we would solve the system equations and we would get the family of eigenvectors that solve that ok so that's kind of how we do it we set the characteristic polynomial equal to 0 for the a minus lambda i equation determinant we find the eigenvalues and for each eigenvalue we plug it back in and we figure out what is the family of eigenvectors that is corresponding to that one so that's how we find eigenvalues and eigenvectors now as promised and i think the most important part of this video will be asking the big question of who cares we have this cool math concept called the eigenvalue and the eigenvector but why is it important why do we care about it in any real-world situation so let's use a graph for a second because I think visuals help myself and a lot of people learn so let's look at two dimensions because it's easier and let's set up a real situation let's say we are a biologist studying fish draw my best that's fish okay so we are biologists studying fish and we only care about two attributes of the fish we care about the fish's length and we care about the fish's weight okay so length and weight we measure them on this axis right here now that means since we only have two dimensions we can represent this fish as a vector for example here is a fish with length little L with weight little W so here's one possible fish here's another possible fish for example so we have all these different possible fish represented by vectors now let's say we have a matrix a remember at its core a matrix is just a linear transformation so if we take this matrix a and apply it to any vector in this length weight plane it's going to map it to a new vector in the length width plane we're gonna assume is a square matrix right so that means that if we take this fish right here and we hit matrix a against that fish we're gonna get some new fish we're gonna get for example maybe that vector and let's just say this linear transformation a represents given a fish a map set to the length and weight of the best friend of that fish or whatever okay so this fish has its best friend as this fish and so on so why is it important why where do I gain values and eigenvectors come into play well if we have an eigenvector in this context we know that a times X is equal to some scalar multiple of X that means if we have that eigenvector X and we apply this bestfriend transformation right here we're gonna get back some scalar multiple lambda X it could be shorter it could be longer but the point is it's a scalar multiple so it goes in the same direction as the original fish which is a big deal because the same direction means that it has the same ratio of length to wait for example that can't be said for any other vector in this plane if I have this vector the ratio of length and weight are completely different if I have this vector there again completely different so the our of eigenvalues and eigenvectors in a data science context is that if you know something as an eigenvector for a given matrix for a given linear transformation you know that that linear transformation will map that eigenvector onto a different vector which maintains the same ratios which maintains the same basically ratios of length to weight or any other dimension you're using which is a big deal in data science because we care a lot about how one quantity relates to another so hopefully that helps a little bit to motivate why eigenvalues and eigenvectors are important it's going to become even more clear when we talk about principal component analysis and we'll be seeing eigenvectors and eigenvalues come up again then so until then I hope this helped understand I gain values and eigenvectors a little bit better than you did before and until next time

Channel: ritvikmath

Views: 61,656

Rating: 4.9216933 out of 5

Keywords:

Id: glaiP222JWA

Length: 11min 57sec (717 seconds)

Published: Wed Sep 18 2019