Is BᵀB Always Positive Definite? (Also, Messi makes a comeback!)

okay so we'll talk about the product B transpose B for any matrix a you will recall that it is always compatible regardless of the shape of B so I'm not even supposing that B is a square matrix here and the result will always be a symmetric matrix a which there are many ways of proving now it's not the right time for it what it is the right time for is asking whether the results in matrix a is always positive definite and here is my proof that it is and you'll tell me if there is anything wrong with my proof and the reason why I'm doing it this way is because my goal is to teach you as much linear algebra reasoning as I possibly can so that's how I'm going to do it so how do you approach proving that something is positive definite well you consider the product X transpose ax and then you ask yourself the question is it always positive well let's see what happens is this positive for all nonzero X well let's recall that a is B transpose B a nice beautiful expression the deeper you get into applied math the more you'll be encountering expressions like this as relatively recent students of linear algebra you're not used to products of four things or more well actually you are and LU decompositions and QR decomposition but this is a very nice product do you see how things so the way we did things were originally grouped is that these two were together but of course the matrix product is associative so we can group them whatever way we want so which way we're actually going to regroup this product it's right we'll put B and X together let's put B and X together and call it Y I'll call that that's a vector right matrix times a vector so that's a vector Y it's getting a little messy then what would you call X transpose B transpose very important to notice that it's Y transpose that the transpose works like the inverse when applied to a product it's the product of the individual transposes in the reverse order so your eyes should be perfectly trained to recognize that if BX is Y then Y transpose is X transpose B transpose so this is right here is actually Y transpose whatever the vector Y is this is y transpose it's nice how it's almost like this problem was meant to be it's all working together so we're left with Y transpose Y now Y transpose Y is the fascinating expression as much as I don't like the notion of the standard product the standard inner product here it's kind of unavoidable because if you think about it what it is what happens when you multiply a vector by its transpose you're basically adding there's this the squares of the entries it's the sum of the squares of the entries does everybody see that this is y1 y2 y3 this is y1 y2 y3 so when you do this kind of product you get y1 squared Y 2 squared plus y3 squared the sum of squares for a non zero of Y this sum of squares is always positive why did I mention the standard inner product it's the centered in the product of Y with itself and when you stalk by the standard inner product it's the sum of the squares of Y and the sum of the squares is always positive am i right yes yes and therefore a x transpose ax is always positive I want to give everybody a chance to find the flaw with this argument so I will give you an example of B please go ahead and evaluate B transpose B that'll give you your a and then for a two by two a you know the criterion for positive definiteness the top left entry needs to be positive and the determinant needs to be positive see if it works if this argument is correct it will work the result will be positive definite but if it's not then there is something wrong with this argument and you'll have to tell me what it is you have to pinpoint the statement that was wrong that's B transpose B by the way as quickly as I just did it that's how quickly everybody should be able to do it because I know that their pairwise dot products of the columns of the matrix so I dotted this with itself 45 9 plus 36 I dotted with this with itself one plus four five and then I dotted these together three plus twelve fifteen and those are my off diagonal ones it's that easy is this a positive definite matrix no five zero do you see that so in other words this is my a so fine for my vector X chose three negative one I would get zero or negative yeah three negative 1 or negative three one which we've encountered before this vector so what's wrong with this argument what you just have to pinpoint it you say from here to here correct yeah I just plugged in what a is from here to here correct yes and here I made the statement that we're just looking at a sum of squares so clearly it's positive and that's where you said wait wait a minute yeah I said the sum of squares must be positive well that's not true because if all of the entries are zero then the sum of squares is also zero and you would say yes what the condition the positive definiteness is only four nonzero vectors so that doesn't count no no no but that's four nonzero X so X cannot be a non zero of that cannot be a zero vector but if B is singular like Michael said and we take X to be from the null space of B then even though X is nonzero Y can still be the zero vector and that's exactly what happens here so that's where the argument breaks down this is the best I can claim because why can still be zero if X is nonzero if X is in the null space of B so of course for the matrix B I chose a singular matrix and it's not really the singularity that matters because it might be as it doesn't have to be a square matrix it just has to have a linearly dependent columns if it has linearly dependent columns then there will be an X from the null space of the matrix for which you get zero and not a positive number and so the quadratic form a which is B transpose B is only positive semi definite and if the columns are linearly independent then the resulting quadratic form is positive definite so in summary B transpose B is always symmetric and it's guaranteed to be positive semi definite not surprising for something that looks like a square but it will be in fact positive definite if it has linearly independent columns and it will be positive semi definite if it has linearly dependent columns okay that's a good step towards many of the other criteria for positive definiteness for symmetric matrices

Channel: MathTheBeautiful

Views: 13,489

Rating: 4.961165 out of 5

Keywords: iMovie

Id: bp38BKP-xh4

Length: 8min 46sec (526 seconds)

Published: Mon Feb 20 2017