Inner Products in Hilbert Space

[Music] welcome back so before I start talking about the Fourier transform and the Fourier series I want to remind you a little bit about inner products of functions okay so we're really familiar with vector spaces two-dimensional three-dimensional n-dimensional vectors of coordinates or of data but now we're going to be talking about inner products of functions and what I want to do here this is pretty confusing the first time you see it for some people and so what I want to do is convince you that the definition of an inner product of functions is kind of very consistent with our definition of inner products of vectors okay so I'm going to do this with an example where we're going to take some functions of X so this is my X direction and we're going to have some function f and some function G and what we're gonna do is we're going to define the inner product between these two functions okay so I'm gonna define my inner product of f of X with G of X as the integral and let's say that this is defined on some domain from A to B then this is going to be defined as the integral from A to B of f of X times G of X DX okay now if this was a complex valued function which we're going to use later on like if this was some e to the I Omega X or something like that then this inner product would have a complex conjugate over this G so this would be f of X times complex conjugate of G of X but for real valued data for real valued functions f and G the complex conjugate of G is just G so you can kind of forget that so don't worry about that right now okay so this is how we define the inner product of two functions this essentially tells me how similar these two functions are just like the inner product of vectors tells me you know if my two vectors are orthogonal then my inner product is zero if they're very well aligned I have a large inner product it's exactly the same thing with these functions so the two that I drew here are actually very close to each other so they should have a large inner product when I multiply them and integrate from A to B okay but what I want to show you now is that if we discretize these functions at a discrete set of X locations and we collect data vectors of the function evaluated at those positions then this inner product essentially comes from that that inner product of sampling so I'm going to do that right now so what we're gonna do is we're going to sample this function f at a bunch of discrete locations F 1 F 2 F 3 and so on and so forth all the way up to F n so I'm going to have n samples of this function I'm gonna do the same thing with G so G 1 G 2 G 3 and so on and so forth all the way up to G N and I'm going to be sampling these at regular intervals X 1 X 2 X 3 dot dot dot all the way up to X n okay and so I'm going to for now just assume that there's a constant Delta X that I'm using to sample between between these points okay so I think I would have something like Delta X is equal to B minus a divided by n minus 1 something like that okay so as I increase n or as I decrease Delta X I increase the number of sample points and that I have okay and so what we're gonna do is we're gonna think about the inner product of data vectors containing these sampled points F and the sampled points G and I'm gonna show you that as you take the limit as Delta X goes to 0 as this becomes infinitely finally resolved as you recover these functions then you recover this this function definition of an inner product ok so let's do that so my F function is I'm just gonna put an underbar tota note that this is kind of a vector of data like we're used to seeing an N dimensional vector containing F 1 F 2 dot all the way down to F n and similarly I'm going to have G underbar is G 1 G 2 dot dot dot all the way down to G and good now it's relatively straightforward to take the inner product of these vectors we know exactly how to do that we have lots of intuition for how to how to compute the inner product of these two vectors and what I'm going to show you is that when you do that and then you take n goes to infinity or Delta X goes to 0 you recover this exactly ok so let's do that right now so what we're gonna do is we're going to take the inner product of vector F with vector G kind of my data vectors here and we know that this is going to be I'm gonna just define this as G transpose F so what that essentially means is I take G I knock it over on its side so let me a draw a picture it looks like G as a row vector times F as a column vector ok and what that means is that I take the first element of f1 times the first element of g1 plus the second element of f2 plus the second element g2 plus f3 G 3 plus f4 G 4 and so on and so forth ok so this is a sum from k equals 1 to n of f k g:k and again if this was complex valued data this would be g bar and this would be complex conjugate transpose I'm just gonna neglect that for now but I want you to remember that if this was complex data this would be complex conjugate transpose and this would be a complex conjugate on all of these little G's ok now there's a little bit of an issue here as as far as I see it which is if I double the number of data points if I have two n data points instead of just n this gets twice as large because I'm adding up twice as many things so that's a little strange so what we're going to do instead is we're going to normalize this by Delta X which is going to make it so that if I double the resolution the doesn't get twice as big just because I have twice as many data points okay so if I look at my f G on my data vectors times Delta X now what I have is this equals my sum from k equals 1 to N of F of K G K times Delta X now what I'm going to say here is the F K we know that F K is just F evaluated at XK so I'm going to say that this is F at XK G at X K times Delta X and so this is where kind of it all comes together if I this is just the Riemann approximation of my continuous integral up here remember if this was complex valued data these would have bars bars this would be a complex conjugate transpose and so if I take the limit as Delta X goes to 0 so I have infinitely fine resolution this vector becomes infinitely tall and it contains the whole function from A to B then this riemann approximation becomes my continuous integral formulation ok so i just wanted to show you that this definition of the inner product for functions again it tells you exactly the same information that the inner product of vectors tells you it tells you how close these two functions f and g are to each other how aligned are they in function space but there's nothing kind of magical happening here this is just what you would get if you took the inner product of vectors of data as the resolution of that data became infinitely finally resolved ok so I think that's really nice to kind of convince you that nothing fancy is happening here we're going to use these function inner products a lot in Fourier series to represent arbitrary functions with sines and cosines all right thank you

Channel: Steve Brunton

Views: 45,738

Rating: 4.9568343 out of 5

Keywords: Hilbert Space, Inner Product, Fourier transform, Fourier Series, FFT, Fourier analysis, Wavelets, Machine Learning, Data science, Linear algebra, Applied mathematics, Compression, Python, Matlab

Id: g-eNeXlZKAQ

Length: 8min 40sec (520 seconds)

Published: Thu Mar 12 2020