How to differentiate with respect to a vector - part 1

hi there in this video I want to define what it means to differentiate with respect to a vector so the idea is that we're quite capable of doing differentiating with respect to some sort of scalar but we haven't defined what we mean by differentiating with respect to a vector and the reason I'm introducing this particular concept now is because when we write down our econometrics quai ssin in matrix form then it turns out that in order to derive our least squared estimate of beta which in this case is a vector we have to differentiate some sort of sum of squares with respect to a vector so if we're differentiating this vector a vector we better know what we mean by that okay so let's think about a particular example whereby I have a vector X which has sort of P entries so I have sort of x1 x2 all the way through to X P and similarly there is vector a which we define as similarly having P entries so a1 a2 all the way through to AP so in both of these cases I hope you can see that this is a P by 1 vector and so is this and they both got the same sort of size the same dimensions so let's say we form something which is made by taking the transpose of X and then multiplying it by the vector a and we're calling the X Y so what what dimensions does Y have well we can do this by just sort of thinking about the dimensions of each of these things in our right hand side so what dimensions does X transpose have well it's just a 1 Y P because I've sort of transposed my column of entries and now it's just a row of P entries okay so that's 1 by P and our a vector is still P by 1 so I've got a 1 by P multiplied by P by 1 when in multiple in my trees multiplication when we have 2 and things next to each other and they have the same inner indices so they have the same sort of indices touching then those indices cancel and we just left with that of the outer indices so in fact we've just got a one by one sort of thing coming out of this so why is actually a scalar and the reason I've introduced this concept or this variable Y is because we actually know how to differentiate a scalar so we could actually write down what Y is explicitly and then we know how to differentiate that and that's going to be informative for other cases when we don't actually have a scalar we're differentiating let's say a better or differentiating matrix okay so let's think about what it actually means to differentiate Y with respect to let's say X well this is defined as having exactly the same dimensions as our X vector what are the particular components well the first component is just dy over DX 1 the second component is the y over DX 2 and then we sort of continue through to DeWine over D X P right so we have formed a vector a row a sort of column vector of all the differentials of Y with respect to each of the argument of X so the idea here again is that we have AP by 1 vector so that's what it means to differentiate Y the spektr X what does it actually equal in this case well the way in which we can go about finding that is actually writing out what Y is explicitly so Y explicitly if I take a transpose of x and multiply it by a then assuming that each of these X's in each of these A's are themselves scalar so X 1 for X be a scalar and a1 3 P a scalar then I can write this out as a 1 X 1 plus a 2 X 2 plus all the way through to AP XP and when I write it in this form it's quite easy to see that Y itself is a scalar if all they're sort of entries of a and X are themselves scalar because I'm just sort of multiplying them together rather than them up okay because we're written in this form it's actually quite easy to find out the particular entry in our sorbetto of derivatives so the first entry is regatta dy over DX one which when I sort of differentiate is back to h1 I get all these other terms other than the one with x1 cancelling or they go to 0 rather so I just get dy over DX 1 is equal to a 1 and similarly for some any other entry it would be some dy over DX I would be equal to a I so I sort of continue through to sort of a DX P and that's just equal to AP so actually in this circumstance it's quite easy to write out this vector of derivatives it's actually defined as in this case actually bandit in this case to be a 1 all the way through to AP but this looks similar to what we have up here in fact it's just the main is it's just a vector hey um we could have kind of seen that right because this is if we sort of didn't think about X and a is being vectors if you just thought about them as being scalar then when I differentiate this thing then with respect to X then I'm just going to kind of get a leftover but we have to be a little bit careful when we differentiate with respect to a vector because we have to make sure that we get out something of the correct dimensions so in this case it made sense to it for us to sort of write the product or the result of this differential as the vector a let's think about what it might means to differentiate with respect to not X but X transpose so what does this mean well it's quite easy to carry thee and our four processes through to what this might mean it just means that we actually just get a row of our sort of particular entries so I just get dy dx1 all the way through to dy over DX P so in this circumstance because we've got to have out something which has the same dimensions as the thing we're differentiating with respect to X prime door X transpose is a 1 by P vector so when we differentiate with respect to 1 by P vector we better get out at 1 by P vector so this is actually going to be a 1 all the way through to AP which isn't a it's a transpose so notice that this isn't perhaps as easy to see as the first one was because we were differentiating Y with respect to X Prime and it looks like if we just sort of did that we weren't thinking about these things as being vectors then we should just get a out but in fact that thought process would be misleading because we need to make sure that the thing which we're differentiating with respect to better be the same size as the sort of thing which we get out in the end so in this case the answer was an A but it was a transpose in the next video we're going to generalize this idea to think about what it actually means to differentiate sort of more complex things with respect to vectors I'll see you then

Channel: Ben Lambert

Views: 114,267

Rating: 4.8963284 out of 5

Keywords: econometrics, calculus, Derivative

Id: iWxY7VdcSH8

Length: 7min 37sec (457 seconds)

Published: Tue Jun 25 2013