Least Square Method (Curve Fitting)

hi let's suppose that we perform an experiment which involves a resistance connected to a battery and then you change the voltage of the battery let's suppose first you take 1 volt then 2 volt then 3 volt and on and on and with each voltage you measure the current so here in this resistor circuit there are meter measures the current of voltmeter measures the voltage so if you gather a set of data points by changing voltage and then finding out current and then you use these data points to plot them in some kind of a graph so let's suppose the voltage is x axis Y current is in the y-axis and your data points are scattered around in this particular manner now you know from Ohm's law that voltage and current have a linear relationship that means voltage as it increases the current should also increase with the same proportion so therefore you are surprised you're surprised because if you look at your data points then your data points connect to each other in some kind of a zigzag path to one you expected a straight line but you are getting a zigzag path and why is that this is because your data points or your observation points have errors in them their errors could be because of instrumental that is maybe the measuring devices do not have sufficient resolving power the error zone can also be because of human errors maybe you made slight mistakes along the performing the experiments so the question is if your data points contain these kind of errors which makes these points deviate from the straight line then this time for circuit is following so how do you obtain the curve that best represents your set of data points you cannot just drop and take your pen and just draw a line randomly like that because this is just guesswork you need some - you need some method which will give you a mathematical way of determining the line which best represents these set of data points which minimizes the errors involved so to do that we are going to discuss what is known as the least square method so Li square method is a very popular method through using which we can draw some kind of a curve which best fit a set of data points to minimize the errors associated with your set of data points so let's get straight into it so here suppose that I have a general set of observation points all right or data points where the independent variable is X and the dependent variable is y so these are known all right you have these data points so these are known what you do not know what you want to find out is this line and what is the equation associated with this line okay this is the this is what we are interested in the line which best approximates the data point and what is the equation associated with your line so let's suppose the equation associated is just gonna give a general name effects and if FX is the function associated with your line then for each point X 1 X 2 X 3 X 4 X 5 the corresponding values will be f X 1 f of X 2 f of X 3 f of X 4 and f of X 5 now what we want to find here is that we want to find the error associated with this FX and the actual observation point so let's suppose for X 1 the actual observation point is here but if X 1 is basically here so the error is this much so this is an error let's suppose this is e1 ok so we can say that V 1 which represents the error between the actual observation point and the point that should fit into your straight line best fit line is known as the error alright so here I say that error at X is equal to X 1 is nothing but e 1 which is equal to Y 1 minus f of X all right similarly for every single point you will have errors associated with those points so for example at X is equal to X 2 you will have some error e 2 y 2 minus FX 2 and for all other points so let's suppose at any general point X is equal to X I you will have some error E is equal to y i minus f of X I know now that these e 1 e 2 e 3 basically represent the errors associated with each individual point so these are the errors associated with each individual point what do we do or what matter do we follow to find out the line that minimizes these errors now the errors can be positive it can be negative it can be positive it can be negative so I cannot just add up these errors because then positive errors and negative errors will cancel each other out and that would not give us any meaningful information what I want to do is I want to find the square of the errors okay I want to find the square of errors because the square of errors are always going to be positive so I'm going to define this quantity okay let's suppose this quantity is capital e and then this capital e is the square of the errors and then you sum it over the entire observation data of x1 to xn so for I is going from 1 to N if you have n number of data points you sum up the errors corresponding to each point you sum up the square of the error is corresponding to each point and add this is the quantity I define as capital e you know this is the quantity that is going to be positive and the further this line is from this data point so if I plot a line here then the greater is going to be this particular quantity for this line so I want to create a line or I want to create a curve which minimizes this particular quantity ok so I want to minimize capital e to find out the best fit line so this least square method basically gives us an idea of how I can minimize this particular quantity which is the summation of the square of the errors so my objective or the so the objective to follow in the v-square method is to find out this particular quantity capital e which is basically the summation of the square of the errors and to minimize it ok the important thing is that I want to minimize this so how do I minimize this to minimize it first let it let's write it down properly so summation is from I is equal to 1 to 1 what is e ie I is nothing but why I minus f of X I all right so I want to minimize this but before I minimize it I have some expectation from the nature of this line okay I either say that the line is a straight line or I say that n minus some kind of a curved line which has a parabolic relationship or something else ok so I need to decide what nature of a line it is so I'm going to do the derivation for a straight line all right so let's assume that our best fit line is a straight line so if I want to fit a curve of a straight line in this set of data points then I can assume that FX can have some kind of a relationship with X as ax plus B where a is nothing but the slope all right and B is nothing but the intercept all right and X of course is the set of data points tell you again so FX plus B represents a straight line if you want to a best fit some other kind of a curve maybe a parabolic curve in that case you will have to use some other equation FX is equal to let's suppose ax square plus BX plus C ok but I am NOT going to do for parabolic or some other polynomial expression let us do or devote this video for a straight line case if I do it for a straight line then what does capital e become capital e simply becomes summation I is equal to 1 to n y I minus a X I plus B whole square so I already told you that this represents the summation of the total amount of errors associated with your observation data and I want to minimize this how do I minimize this now Y I and X I represents your observational data what is the unknown here the unknown is aiona I don't know what is I don't know what is the slope of the straight line I don't know what B is I don't know the intercept so a and B are the unknown so I'm going to minimize it with respect to a and B so I'm going to say that Dell of capital e by Del of a is equal to 0 and I'm going to say that Dell of capital e by del of B is equal to 0 this is what I'm going to do if you use some other equation in this f of X so if you said f X is equal to ax square plus BX plus C in that case you would have to minimize with respect to a B and C but since in this case I have taken a straight line so if I assume that FX is equal to ax plus B so I'm minimizing only with respect to a and B separately so let's minimize it with respect to a and B and see what we get so first time I try to I say that del of del a summation of I is equal to 1 to M Y I minus a X I plus B whole square is equal to 0 to minimize I say this is equation is true and then if I find the derivative with respect to a for this equation then I can simply write the summation I take outside then this is square so I can write to here why I minus a of X I plus B and because I'm differentiate with respect to a there is a term associated with multiplicative factor of minus X so minus X I alright so now to coaster idents I become 0 minus 1 goes to RN side becomes 0 I am left with summation of I is equal to 1 to M so if I multiply X I Y I get X I Y I minus a of X I square plus no sorry minus B of X I and then this is equal to 0 yes so if I write it properly if I take these two terms to the right hand side and then I flip the equation I simply end up getting a summation of X I square plus B summation of X I is equal to summation of X I Y I and this summation goes from 1 to and ok so this is the first equation that I have obtained this is the first equation that I've obtained by trying to minimize the summation of the square of the errors with respect to a now let's try it so this is the first equation I have to let's try it for the second case so if I try to do derivative with respect to B then in that case I get why I minus a of X I plus B square is equal to 0 however here the derivative with respect to B ok so if I do this derivative then I can take the summation outside then I can again write to here just because a power is 2 here so why I minus a of X I plus B ok and then B has a multiplicative factor of minus 1 so I write mine one here okay now so again two equals the right hand side becomes 0-1 whose right hand side becomes zero I am left with if I take this as I am left with basically summation of I is equal to 1 to N and then I get Y I minus a X I minus B is equal to 0 if I take these two terms so right hand side and I flip the equation I simply get summation of so a X I I is equal to 1 to N then again I get plus B okay summation of I is equal to 1 to N 1 all right so if you add 1 n number of things what do you get you simply get n here so I can just write N and this becomes summation from I is equal to 1 to n Y I so this is the second equation that you get so basically you get these two equations let's suppose equation number one and equation number two which will minimize the sum of the square of the errors with respect to a and B separately so let's write down these two equations so these are a system of system of linear equations so how do you solve a system of linear equations so what we can do is we can just convert the system of linear equations into matrix form all right how do you do that you can simply convert this system of linear equations into matrix form so basically here X I square is known all right X I is the set of data points corresponding to the independent variable and X I square represents the square of these X values and the summation of that so X I square X I X I and X I Y I Y these are all known the only unknowns are a and B ok so I am going to do is I am going to write it in terms of matrix multiplication so in the first matrix I can say that this consists of X I square X I and then X I summation and then N and then I can write a B is equal to summation of X I Y I and then summation of Y I so this can this basically represent so let's suppose this here represents some kind of a matrix this represents another hand matrix and this represents another kind of matrix alright so let's suppose that this represents the matrix a alright this is known this contains the observation data points which is known and this again can this is also B which is also known and this is let's suppose I named it as X because X is unknown so if I do that then simply this matrix equation simply becomes an equation that looks like a X is equal to B how do you solve this solution of this simply is X is equal to a inverse of B that's all so if you take inverse of a and multiply with B then you get X so where X 1 is nothing but a which is a slope and X 2 is nothing but B which is the intercept so basically our challenge is to construct a matrix of this particular fashion and this particular fashion and then use this equation to obtain the solutions of a and B here all right so that we can easily do you can easily do it either on your own hand using pen or paper or you can program some kind of you can make a program where you find the inverse of a and then you multiply it with B to obtain the values of the slope and the intercept all right so I have actually created a program here using Scilab you can you can do it with any other program you can do in MATLAB you can do in C++ you can do in Python where I have given a set of data points so these are my data points corresponding to X these are my data points corresponding to Y then this here represents the sum of X basically the summation of X only and this represents the summation of Y this represents the summation of XY this represents summation of X a square alright and this is the matrix that I've defined as a this is a matrix of the finest P and this is a matrix that I just now mentioned to be X here everybody's SP this is X ok if I run this then I end up getting these ok this graph so this red he represents my data points but this dotted black line represents the best fifth line corresponding to these red data points I have also obtained the slope the slope for my data points is not going to end the constant or intercept is 1 point 1 so it will of course be different corresponding to different sort of data points so for the different data points you can get a different slope and a different intercept so you can do this either manually using pen or paper or you can do this using some kind of a program so this is how you square fitting method works you have a set of data points we do not necessarily fall in a straight line but then you use the nice square method to obtain a straight line whose slope you can find whose intercept you can find and then you can plot this particular straight line ok so let us do a little bit of a revision of what we have just now seen so here we had a set of data points all right that we wanted to fit into some kind of straight line we just found out the errors associated with each data point and then we define this quantity capital e which is a summation of the square of the errors and which is nothing but this expression here where FX I have assumed to be ax plus B now I need to mention that this derivation is going to be different if you take some other expression like a X square plus BX plus C which represents a parabolic curve or some other polynomial expression if you do that you will have to minimize it with respect to the other variable also so you will get more number of equations but in this case we have only done for a straight line so if we do its first straight line you try to minimize this with respect to a first and then with respect to B and then you obtain these two equations these two equations are the system of linear equations which you write it in matrix form and then if you solve it you obtain the slope and then you obtain the intercept and once you obtain the slope and the intercept you can plot the straight line corresponding to your set of data points that is all for today's video thank you very much

Channel: For the Love of Physics

Views: 35,964

Rating: 4.936975 out of 5

Keywords: least square method curve fitting, curve fitting least square method, method of least squares, least squares, curve fitting, curve fitting straight line, least square method, least square method regression, method of least square regression, method of least squares fit a straight line, least square method straight line, method of least squares straight line, bsc physics, for the love of physics, physics, Linear regression

Id: GPEE8JviDGU

Length: 16min 50sec (1010 seconds)

Published: Mon Sep 09 2019