Basics of Linear Algebra

hello and welcome to MATLAB programming for numerical computations we are now in week 4 of this course in this week we are going to cover linear equations so basically we are going to cover linear algebra in the first lecture lecture 4.1 followed by several methods to use linear algebra to solve the problems of the nature ax equal to B to find the values of the vector X before we do that let us recap some of the results from linear algebra for this the related videos are in the computational techniques course module 3 lecture 1 the link for which is given in front of you so let us review some of the results in linear algebra so what is a vector vector is nothing but an ordered set of scalars an example is shown over here so this vector has a dimension in this case the dimension of this vector is 2 as a convenience what we are going to do in this entire lecture series we are going to assume all the vectors to be column vectors which means that the vectors will have n rows but a single column as you can see over here we have two rows but a single column now this column vector has a dimension which is nothing but the number of elements if we want to look at the geometric interpretation this could be nothing but a point in this space in a two dimensional space which is the distance to on the x axis and a distance 3 on the y axis or more appropriately distance to on axis on the first axis and distance 3 on the second axis now this particular vector has a direction which is basically given by the value 2 comma 3 as well as it has a norm now the most common norm that we are used to so far is what is known as this 2 norm of this vector which is nothing but the square of the distance of that particular point from origin so square root of 2 square plus 3 squared is nothing but the 2 norm of this particular vector likewise we have the 1 norm of the vector N norm of the vector or infinity norm of the vector something we are not going to go into but essentially just the way we are associated we associate distance with a physical point in space a vector can be associated with a with a norm which is nothing but a an indication of the size or the distance of that particular vector ok now next case we have seen is what does it mean when we want to solve two equations in two unknowns so here I am showing three different cases of two equations and two unknowns the unknowns are X 1 and X 2 and the 2 equations can actually have a single solution can have 0 solution and can have infinite number of solution so when do we get a single solution 0 solution and infinite number of solution we are showing graphically on this slide ok so if we look at the first example we have the equation X 1 plus 2 X 2 equal to 1 X 1 minus X 2 equal to 4 these represents the blue represent the blue and the red line over here and these two lines intersect at a point 3 minus 1 so 3 minus 1 is a unique solution ok next is if these two equation instead of having X my X 1 minus X 2 as the second equation if we were to have 2 X 1 plus 4 X 2 which is nothing but the double of the left-hand side as the left-hand side of the next equation which is 2 X 1 plus 4 X 2 that is the left-hand side of the next equation and the right-hand side if we take that value as 4 we can see that these now represents represent two parallel lines these two parallel lines are not going to intersect and we are going to have 0 solutions in this case and now in this case if the second equation was nothing but double of the first equation so 2 X 1 plus 4 times X 2 equal - - which is nothing but double of this first equation we are going to have these two lines lying on top of each other and we are going to have infinite number of solutions so the question is when do we get single solution when do we get infinite number of solution and when do we get zero solution okay and answer to that is from finding out the rank of the matrix so if we look at the rank of matrix a and if the rank of matrix in this particular case was equal to two we are going to have a unique solution if rank of a square matrix of an N by n dimensional square matrix is equal to M then we will have a unique solution for the problem ax equal to B okay but what if the rank is not equal to two that is the case that we see in the middle column as well as to the right column in that case we need to look at the rank of the matrix a comma B if rank of matrix a comma B is equal to the rank of a then we are going to have infinite number of solutions but if rank of a comma B is not going to be equal to rank of a then we are going to have zero solutions okay let us go on over to MATLAB and look at these three examples that we have seen so let us look at the matrix a matrix a is 1 - 1 - 1 B is the right-hand side that is 1 4 ok so these are the two the values that we have when we want to solve ax equal to B we know that the solution X is going to be I and V inverse of a multiplied by B ok and X as you can see over here is 3 comma minus 1 something that we had seen in our PowerPoint slide a minute earlier okay now MATLAB has a more efficient way of solving this equations and that is using slash in this case backslash because we are pre multiplying inverse of a with B so we write this as a backslash B and when we press ENTER we are going to get the solution three comma minus one a backslash B for an A which has a rank equal to n is exactly same as this particular command ok let us do help slash and see what we get okay if you recall what we did in the first module we had looked at element by element division and element by element operations and this is what the latter part of this help slash showed us but what we are more interested in is to look at what the left division and right division actually mean so a backslash B is left division which is same as I and V a multiplied by B if a is an N by n matrix with rank equal to n okay it's something different if the rank of a is not equal to n we won't go into that right now and then we have an equivalent called a right division right division is be multiplied by inverse of a okay for the problem a X equal to B we needed to have a left inverse so X was inverse of a multiplied by B naught B multiplied by inverse of a and therefore we had a left inverse which is indicated by a backslash a backslash small B over here not capital B and that is what the solution that we got okay so that was the first thing that we saw how to solve these equations let us look at how to find out using MATLAB whether or not your equations are going to have a unique solution or infinite number of solutions or zero solutions and we can do that by using the command rank a and rank of matrix a is - - so a we say is a full rank matrix that means a the rank of a is equal to the number of rows or the number of columns of matrix a okay we can also find the size of matrix a by saying M comma n equal to size a M is going to be number of rows n is going to be number of columns we have already seen that in module 1 let us now look at the second example and the second example our B remains the same we change over a 2 1 2 to 4 let us make jet change a equal to 1 to 2 4 let me echo this by skipping this semicolon okay so our a is 1 2 2 4 let us get rank of a and rank of a equal to 1 okay and therefore a will not have a unique solution it sorry ax equal to B will not have a unique solution it can either have infinite number of solutions or 0 solutions and to do that we need to find the rank of matrix a comma B and the rank of matrix a comma B equal to 2 which means that the rank of a comma B is not the same as rank of a so we are going to have 0 solutions that is this particular case now let us change B to 1 to be equal to 1 semicolon to note that we have used a semicolon because we want a column vector so that is our B rank of a is not going to change because we have not changed rank sorry we have not changed matrix a let us now look at rank of a comma B and rank of a comma B in this case equals rank of a in which case you are going to have infinite number of solutions ok let us now look at the next concept in linear algebra and that concept is what is known as condition number in these two examples we had equation X 1 plus 2 X 2 equal to 1 and X 1 plus 4 X 2 equal to 2 instead of the second equation being X 1 plus 4 X 2 if you were to change this to 3 point 9 9 9 X 2 okay we are going to get this particular equation okay and our matrix a is going to be 1 2 2 3 point 9 9 9 if we solve this equation we are going to get the solution as 3 comma minus 1 now if we make a very small change in our B that means instead of 1 & 2 point 0 0 1 if we change it to 1 & 2 point 0 we are going to have a very large change in the solution and the reason for this is what is known as condition number or what is known as a poorly conditioned matrix condition number is nothing but the ratio of eigen values of the matrix so in this particular case a being an N by n matrix has two eigen values and the ratio of two eigen values in this case is approximately 25 thousand which is a fairly large number and therefore we see that the results change quite significantly for a reasonably small change in B matrix okay so let us look at the condition number of a when a was defined as 1 2 to 3 point 9 9 9 that was our a matrix we will use the command Co and D in order to find the condition number of a and Co and D a is 2.5 hours approximately 2.5 into 10 to the power 4 which is approximately 25,000 now we can find eigen values of a using the command e IG and e IG will give you two eigen values and we can also verify that the condition number is the ratio of the two eigen values okay and the absolute value of Mac yeah and this is exactly what our condition is that we obtained using the command Co nd so the MATLAB command Co and D is finding condition number of any matrix the MATLAB command rank is to find out the rank of that particular matrix and MATLAB command e IG is the command for finding eigenvalues of that matrix so now let us go back and look at what else our eigenvalues command can do so I will type help TIG okay and so we can see that I've in values so if we give a command e IG with just one output argument we are going to get a column vector containing eigenvalues however if we give two arguments V and D V is going to be a a matrix containing the eigen vectors and D is going to be the matrix whose diagonal elements are the eigen values so let us look at that in MATLAB v comma D equal to e IG okay so as remember a just a few minutes back we had said that we will define vectors as column vectors so this is the eigenvector 1 which is minus 0.8 9 0.45 that is the first eigen vector corresponding to this eigenvalue the second eigenvector is this guy which corresponds to this eigenvalue okay let us look at whether this satisfies the equation rather a multiplied by V equal to lambda multiplied by V whether this is satisfied or not okay let us try to look at that okay so let us say L was nothing but D 1 okay and our V was nothing but the first column the first column basically means all the rows and column number 1 this L is the first eigenvalue this is the first eigenvector let us say a multiplied by V minus lambda multiplied by V what we get is we get the value as 10 to the power minus 15 look let us look at the value of a multiplied by V and a multiplied by V is of the order of 10 to the power minus 3 so for all practical purposes this particular guy a multiplied by lambda is almost equal to lambda multiplied by V okay again keep in mind what we had said earlier in the previous two modules is computer is a finite precision machine what that means is that computer has a least count so when we are going to do problems using floating-point numbers using real numbers we are not always going to get when we do a minus B we are not always going to get the value equal to 0 if a and B are equal we are going to get some x value of a minus B as a very small number compared to value of both a and B and that is what essentially we are looking for is whether or not a a multiplied by V is close enough to a multiple aMDA multiplied by V so a multiplied by V is this value and lambda multiplied by V is also the same value let us look at the other example where we take the lambda as this second eigen value so L is d 2 comma 2 and V is nothing but V all the rows and second columns okay let us look at a multiplied by V this is the value of a multiplied by V and let us look at lambda multiplied by V and that is the same value that we get as lambda multiplied by B okay so to recap what we did we did use the command e IG to find eigenvalues and eigenvectors when you run the command with two arguments the first argument will give you a matrix containing the eigenvectors the first column of that matrix is the first eigenvector second problem of that matrix a second eigenvector and so on the the second argument the from AIG is a matrix whose diagonal elements are the various eigenvalues the first a 1 1 is the first eigenvalue a 2 2 is the second eigenvalue and so on a and n is the nth eigenvalue so that is the review of the linear algebra that is going to be useful for rest of this course and let us now recap all the useful matlab functions that we have done so far in in the previous lectures in module 1 primarily we have looked at matrix exponent matrix logarithm matrix square root matrix power and today we also looked at matrix division left division as well as right division so these were the commands that we have covered before in like in modules before today we covered the command I and V to find inverse of a matrix rank to find rank of a matrix Co and D to find condition number of matrix we will also in a minute use the command norm to find norm of a vector or a matrix we will just pick ourselves two vectors for now but the norm exists for a matrix as well okay we also used a I G to find eigenvalues and eigenvectors lu cha Leske QR factorization SVD sure decomposition these are something that a lot of Engineers use quite a bit and I have put them in this particular table just for the sake of completeness lu decomposition is something that we are going to cover in the fourth lecture of this this module okay however you do not have to worry about the remaining things singular value decomposition or sure decomposition or chalice key factorization and so on for the purpose of this course okay okay so let us finally now let us go and find out the norm so let us say let us look at our vector X now norm of this vector is nothing but three squared plus one squared square root of that we square root of 10 so norm of X is going to be square root of 10 which is nothing but three point one six two three okay so norm is an indicator of the size of any vector so with that we come to end of lecture 1 of module four so what we have done in lecture one of this particular module is to cover the basics of linear algebra so primarily we have looked at trying to solve equations of the form ax equal to B look at our values and eigen vectors of a what does the condition number mean what does rank mean and how we can use rank in order to find out whether we will get unique solutions infinite number of solutions or zero solutions to an equation of the form ax equal to B so that ends this lecture for recapping linear algebra and see you in the next lecture

Channel: MATLAB Programming for Numerical Computation

Views: 37,834

Rating: 4.9250002 out of 5

Keywords: Lec, Basics, of, Linear, Algebra

Id: celUu5aY6_Q

Length: 21min 2sec (1262 seconds)

Published: Mon Feb 01 2016