Fourier Series Is Nothing but a Least Squares Problem!

so let's talk about Fourier series and I think that and I hope that some of you are just a little bit shocked by what they see on the board because just a moment ago we talked about vectors and in a product and linear spaces and all of those matrices all of those other wonderful things having to do with linear algebra and now all of a sudden you see something on the board that's distinctly calculus analysis functions infinite series and so forth so this shock that you might be experiencing and that I hope you're experiencing is a very very important emotion to go through when you're studying mathematics that's the moment when you realize that two fields that you didn't think we're related are in fact related and something that you thought maybe you didn't know you actually know already so let me a state what the classical problem of Fourier series is and then I will have to really try to make it longer than a five minute discussion because you already know everything there is to know pretty much about what's going on here so we're considering an arbitrary function f of theta f of theta define on a segment from minus PI to PI and we want to represent it as a sum written on the board of sines and cosines and yes all of these functions are periodic so we can only focus on one period of this function and a function happens to continue in some other way beyond that segment well that we will not be able to capture at all so we're talking about what's called periodic continuation of of this function so you imagine this thing repeated over and over again periodically that's the only thing that we can hope to expand as a series like that expressed as a series like that so that's the goal you've seen something like this in calculus before where we where every function could be represented as a polynomial the only compromise was that you need infinitely many terms that's the breakthrough in calculus which I discussed briefly in the video when I talked about Fourier series in the PDE class but I was really looking forward to talking and about it in in this linear algebra class okay so the same compromise is made here we're going to we're going to add up an infinite number of sines and cosines and then maybe there is hope of representing every function in some calculus sense exactly as a sum of a convergent series but as you will see in a moment since we've discussed least squares we can actually eliminate calculus from this discussion well not I am limited calculus eliminate infinite series from this discussion completely so what we'll do now if you understand the problem from the point of view of functions representing a function as a sum of other functions let's interpret this problem from the point of view of linear algebra and to recognize that what we're looking at here if you just bear with these infinities for a moment is the decomposition problem we're being asked to decompose a given function could be theta squared doesn't have to have anything to do with trig as a linear combination of the constant function a 0 times 1 perhaps of the cosines and of the sines and we have to determine these coefficients so yes it is very much a decomposition problem can we solve it exactly let's say from the linear algebra point of view probably not because the space of functions I don't want to say infinite dimensional but it's certainly not finite dimensional there is no way that I could come up with n functions no matter how big n is that will span the whole space of functions in if you say sufficiently smooth functions so this is probably the a yet that ax equals B case where a is tall and not very wide where the solution is not possible but when it comes to this regime at least we know how to do as best we can and so we're going to do as best we can so what we need what's absent from this discussion is and then a product because we could start doing this decomposition problem if we had an inner product we would just use this approach and what's nice is that this approach works when you have a complete basis or when you have an incomplete basis so right now let's actually switch to this precise mode let's eliminate these infinities and just assume that we're taking n terms capital n in each in each instance okay and leave the question of convergence series completely outside of this discussion so what we have now is classical least squares situation we have to solve a decomposition problem that's obviously not feasible unless this function happens to be a finite sum of sines and cosines and so we have to use in a way this formula not this formula but this formula interpreted via in the products we can certainly not use this formula directly because we don't have matrices here we just have vectors so we have to use this interpretation where will we will think of these sines cosines and the constant as our basis wanna be incomplete basis for the whole space of functions and then we'll form the inner product matrix we'll dot our target function with each of the elements of the basis and then we'll solve this decomposition problem so what we need what we need is an inner product do you guys agree and once we have the inner product then becomes a hundred percent the linear algebra problem but we just solved so let's choose this standard inner product I don't know if it's called this end a standard inner product I'll just call it exactly the sort of thing you would expect where F dotted with G is defined as the integral from minus PI to PI of f of X or what are we using theta f of theta G of theta D theta that's my inner product this one these cosine in these signs I have two n plus one functions as my count correct two n plus one function is my basis for the subspace let's find the coefficients such that this is not exact this is not exact but is the best possible one it can be with respect to this inner product so that if I take this function and its best approximation and I find their difference their length according to this inner product would be as small as it possibly can be okay first thing that I would like to assess is whether or not this basis is orthogonal or not wouldn't it be nice if this was if the basis was orthogonal if the basis is orthogonal then this matrix would be diagonal and we're right back in that mode where alpha I this linear system reduces to this solution when the basis is orthogonal and so we end up with simply a diagonal matrix here we're all in agreement on that right we can either see this as a direct conclusion or is a special case of this linear system with the diagonal matrix is the basis orthogonal let's ask this question well let's first see is this guy which is 1 maybe I should write it in here explicitly 1 the constant function orthogonal to the cosine any cosine 2 theta 3 theta and theta let's test is this zero is this integral zero yes of course it is just imagine the cosine it's perfectly periodic as much below as it is above so this is 0 so 1 + cosine n theta are orthogonal let me write it this way what about to the signs well it by the same token will be orthogonal to all the signs right so far so good what about cosines and sines are they orthogonal and when M is not equal to n there's a trig identity that tells you that the product of two cosines I believe it's this cosine of the sum plus the cosine of the difference or minus the cosine of the difference doesn't matter but this will end up being something which clearly integrates to zero so this will to be zero and so all of the cosines are orthogonal to each other and by the same token all of the signs are orthogonal to each other by a similar trig identity and then all of the cosines are orthogonal to all of the signs whether n is the same or not so this is a perfectly orthogonal basis that I should write down it's an orthogonal basis now all that's left is to write down the answer all that's left now that we've established that our basis orthogonal and that and when so when we're applying the least-squares formula which we are because we have a finite number in here oh it reduces to this formula right here all we have to do now is apply so let's see what it becomes all right mmm so we have well have three different formulas one for a zero one for a sub n + 1 is 4 B sub n the formula for a zero you guys agree can't be anything else pure linear algebra and this equals what is one dotted with one you guys tell me according to this definition 2pi so it's 1 over 2pi perfect so the interpretation of a zero as you guys informed me in the PDE lecture is it's the average value of the values of f right it's the integral of f divided by the length of the segments so that's what a zero is it captures the average value of the function a sub n equals this will be interesting the integral of cosine and Theta dotted with itself yes this basis is orthogonal but is it orthonormal well obviously not because a is not unit length okay so this will probably also not be one so let's figure out what it is now if you remember the half angle formula this is 1 plus cosine of 2 n theta divided by 2 so it is basically 1/2 plus something that integrates to 0 because it's just a nice cosine and because it's 1/2 plus something that integrates to 0 the whole thing integrates to PI it's a calculus detail we don't have to discuss it now but it is PI so it becomes 1 over pi okay and the wine remaining one is B sub M and sine n theta dotted with sine n theta by the exact same logic is also pi so it's 1 over pi so familiar formulas from the other class but much more insightful and structured derivation and we see that in this problem so when we did it in the PD class I'll bet you felt like it's 90% calculus and 10% Jonas a while but now it feels like it's 90% linear algebra and 10% calculus

Channel: MathTheBeautiful

Views: 13,734

Rating: 4.9541545 out of 5

Keywords: iMovie

Id: u8ccubUfhKY

Length: 14min 9sec (849 seconds)

Published: Thu Mar 30 2017