POD for Partial Differential Equations

[Music] so today we're going to be talking about reduced order modeling this is from chapter 11 of the book on data driven science engineering and you can find the website here data book u-dub comm and I really wanted to spend some time thinking about what we can do with all of this data analytic methodology in particularly looking at future spaces which was some of the things we've been talking about throughout the book you know there was ways to represent eight represent the data or representations that are advantageous and so we're going to talk about is reduced order models and reduced order models are all about exploiting low dimensional patterns or features in your data to build models so ROMs is they're often called are often used in partial differential equations to provide we're called proxy models that are much faster to simulate than the full governing equations of a system so that's the goal is to find a way to take a what's normally a very high dimensional system find a way to simulate it to look for a space in which you can trade out your full dimensional model to a proxy model which is much faster to simulate so that's what the ron's architecture is gonna do for us so here's where we start partial differential equations and what I'm wanting to think about here with partial differential equations is to think about some generic structure U of T so this is a variable to represents space and time and so the evolution dynamics themselves are soo some non linear prescribed through some non linear operator let's call it n which depends upon u its derivatives maybe time space itself and beta which represents some kind of parametrizations of the PDE itself so it could be fact X oftentimes like a bifurcation parameter but in general it's a set of parameters that characterizes the evolution equations of the of this of this PDE typically this is specified on some domain here are negative lol so the idea here is that oftentimes if this were a linear PDE we oftentimes would have potential access to analytic solution techniques that could allow us to maybe make some progress in writing down solutions however when you generically have a non linear PDE such as this then it's difficult and we do not have general solution techniques and we have recourse then to computational methods and that's normally what we do with nonlinear PDEs that are difficult to solve we just say okay well let's discretize or find a computational algorithm that will in fact solve this PDE that I cannot write down solutions for so that's the modern way of doing these things and so let's talk about how that might work so first way to maybe do this is through standard numerical discretization is to take the U field and discretize it into a discrete number of spatial points so here u is traded out which is normally a continuous variable of X and T is now depending upon U of X of K of T where K goes from 1 to n so the idea is if I have a domain which is negative L to L I take the U field and I chop it up into a discreet number of points n of them and those are the representational points for my continuous function U of X T so I'm doing this disk organization at this point just in the spatial domain now importantly once I've done this discretization I have a way to represent derivatives in this representation using what are called the finite difference formulas so this is fairly standard from computational sciences which is a first derivative is exactly what we learned in calculus which is a slope formula rise over run and so a first derivative in this discretized system would be a very simple representation of our first derivative is you have K plus 1 so the point in front u minus U of K minus 1 the point behind divided by 2 Delta X so this is a rise over run formula for the approximation of the derivative in this discretized space second derivative you can follow this up with these finite difference ideas which is the derivative of the derivative and you get a formula something like this the second derivative is the point in front of you or to one side the point to the other side minus twice the current point divided by Delta x squared and so what both of these are is it gives you a way to compute derivatives as a function of neighboring points remember all the derivative is is a slope so you just have to use your neighboring points to compute that slope so that's that's the idea behind using a numerical discretization scheme is if I chop this up down to a finite number of points each point I can now start going back to that original PDE using these discretization schemes to come up with a approximation for the PDE evolution so let's see what this would look like so here's kind of what it would look like this is in fact the evolution dynamics here now in the discretized coordinates now notice what I've done I've traded out this continuous variable U of X of T now for in discrete points describing the U field and the evolution of the K point is given by the following do u K DT is equal to n now and notice what I've done here I've put in here the discrete values of the U field so in other words evaluated each of those K points and notice how now it depends upon neighboring points at k plus 1 K minus 1 and it depends on those points because of the derivatives the derivatives make it a point a localized point talked to its neighbors they're related to those derivative relationships which I showed you here which tells you if I have a first derivative it uses two neighbors if I have a second derivative uses two neighbors plus the point itself and then you can compute these four third derivatives fourth derivatives and you can even do higher accuracy schemes which not just rely on one neighbor but potentially two neighbors in fact if you wanted to get better approximations for derivatives you would use some of these higher order approximation schemes to do that okay now what we've effectively done is we've taken this spatial domain U of XT where we have some solution continuous solution now broken it down to a finite number of points and so now I have this representation of that system as a discrete number of points and I can use something like a rung a cut a scheme to march this forward in time because now what I've done is I've traded out this partial differential equation for a high dimensional set of differential equations which I can now solve with something like you know runge-kutta fourth order runge-kutta so this is the problem with actually a lot of computational schemes is there's no problem with it except for for a very large system a two dimensional system a three dimensional system the the dimension of this system is here no there's number of points you've discretized it and can get very large so in modern computing if you're thinking about large fluid flows you can have billions and trillions of differential equations now to model a spatial temporal system so this is very high dimensional and typically extremely slow to run and this is really getting at the heart of why reduced order models are important because what we'd like to do is find a way to approximate this further find a way to run simulations of this system much more quickly than running this massive system of differential equations so that's the goal of reduced order modeling how do we take this find a proxy model and just like the title says it's reduced order which means lower dimensional and that's what we want to get active now before we go this direction let's talk about a different way to discretize or numerically approximate that PDE and this is actually going to get to how we're gonna do model reduction here so what we're going to think about instead of doing the discretization and finite differences we're going to think about well oftentimes one of the standard solution techniques we use for linear different equations is to do a separation of variables argument and then the separation of variables argument is to say take the U field U of X T and separate it into time and space so a of T contains all the time dynamics size of X has all the spatial dynamics now one of the problems with doing such a thing is that of course this works for a linear system and linear PDE theory this is one of the standard out ways to solve analytically for PDE s but now we have a nonlinear system so there is a question about how good could such a method be for a nonlinear PDE but in either case what we really want to do is take that idea of this separation of variables and build a numerical algorithm out of it okay and how we're going to do that is through doing model expansions and the way we're gonna do that is think about U of XT I'm still gonna use the structure of a time space separation a of T sy of T but notice what I've done here is I've now said I'm gonna represent this in terms of South case and a of case so what the side kazar are gonna be modal basis functions and I'm explain what those are and so what they are is a set of orthonormal functions then I'm going to use to represent the solution U of XT okay and by doing this I can basically take any of this u field and take a sum of n of these basis functions to represent my solution so that's going to be the goal and one of the questions of course will come immediately is what basis functions should you use to make a reasonable approximate to the PDE okay so oftentimes this goes under what are called spectral methods for solving PDE which spectral methods you use some basis functions often it's the fast Fourier transform or Fourier modes to reprobates functions or chebyshev polynomials to represent the basis functions and by the way chebyshev and Fourier are used heavily because you can compute the a of K in n log n time so they're very fast so both those are very fast methods remember part of what we're going after here is not just can I come up with an algorithm to solve that PDE it's can I come up with an algorithm and saw that PDE in a very fast and efficient way because the system often tends to be extremely large so what would we do if we have this basis expansion this modal expansion well we put it back into the governing equations and so if I do that here's what you would get so now what I've just done is simply taken that time space separation solution put it back into the governing PDE here it is and then to get out the evolution equations again it's it's gonna be just like finite differences what I'm gonna get now is some evolution equations but now it's for the a of K terms that in other words the coefficient of each of those basis functions and I'm gonna rely on one of the most important properties they have here which is orthogonal Atia of the basis functions so here is your thought analogy properties so the modes are orthonormal so there are thought to each other and they have an inner product of one with themselves now if you apply that orthogonal any condition here in other just take this thing take the inner product with respect to Phi of M and what you end up getting is some evolution equation here so this is now the evolution for each of the modal coefficients so the a of K is how the fee of K mode is evolving in time now what's nice about this kind of representation is the finite differences thinks about the discretization as I have a point it's talks to its neighbor so it's very local in nature so the air is all cumulated around that local sort of representation here when you do these modal expansions you're using modal bases which are modes that expand the entire domain negative L to L and so each one of these modes you're doing here or evolving is as you evolve it you're looking at its modal expansion the the error now is each mode expands the entire domain so it's global in nature so like it has a sort of a global structure in terms of its air versus the local which is the finite differences and oftentimes these spectral methods have exceptional properties in terms of air bounds but that's the representation you would have now part of the reason I'm going through these numerical procedures whether find a difference or something like the spectral representation is because this kind of spectral representation is gonna be exactly what we're gonna go after for building restore two models we're gonna try to find a set of modes to project our data onto and this here is usually called a galacon projection which is you find a basis set you project and then you so you get the evolution dynamics of the coefficients of each mode okay so it's a that's the idea is to take the full PDE find a basis project the dynamics into that basis and that's the evolution equations if you do that okay so what that's what we're gonna build up to and part of what reduced order models modeling efforts are about is to try to find out what are the best bases that you should be using to get this right okay so let's talk about one of the ones we use all the time Fourier mode expansion and so one basis set that you could just pick generically is Fourier modes and here's what they look like these are your this is what the FFT is built out of in fact one of the really nice things about this FFT is that you can get the projection to the amplitudes base a of caves in n log n time so it's very fast to move from your original state space to this new representation space a of K so this is sort of your your your canonical go to it's the basis of a lot of fast spectral methods and but one of the things I want to highlight here is I've just generically picked this basis and often what you do in for emos you're gonna go back here and you're gonna represent this in terms of n of these Fourier modes now oftentimes when you discretize and if I have I diss critize them two endpoints I use in Fourier modes okay so I still have to find a of K and so that my state space is still very high the goal of reduced order modeling at least from the point of view of the basis set is to say I want to be able to expand my solution in terms of a basis set where instead of using N and n is very high-dimensional I want to be able to go to our modes where R is much much smaller than n that is going to be the key idea of reduced order model this here this formula but instead of n R where R is much less than n so the idea would be then when I solve this system of OD E's there's our of them not n of them where R is much less than n ok so that's going to be the idea behind reduced order modeling now if I do just generic basis like Fourier modes I don't get to necessarily find that reduction space I would just use this thing and by the way a lot of times the the value in using a Fourier transform is that it's very fast to do it that's exactly why you would use it because you have the FFT which was developed in the 60s that helped us compute this in n log in time ok ok so let me just show you if I were to take some generic function here a Gaussian and project it into that Fourier mode space in other words if I want to use a basis expansions which are for modes and here I want to think about representing this Gaussian what it would look like in the for e mode in particular how many foreign modes what I need to do and give an accurate representation of that Gaussian so one of things we've been I can show you here it depends a little bit on what that Sigma is but here's a chart and I want to show you three different cases here so what I'm showing you here is the function which is the Gaussian and so there's yellow kind of a white and a cyan so I have a Gaussian depending upon how I pick Sigma it's either so if if I pick Sigma to be large that means it's a very tight Gaussian and if it's small then it's a very spread out galaxy so those are those two extremes and because it's a Fourier transform pair if I do the Fourier transform something very localized in X will be very spread out in the Fourier domain so the yellow which is very tight here is very broad in the spectral domains in other words the number of Fourier modes I would need to represent something so localized is very high whereas if I look at this very broad Gaussian the cyan so an X it's very broad in the Fourier domain it's highly localized okay so this is one of the properties of Fourier transform pairs if you squeeze this way in space it gets very broad in frequency if you squeeze in frequency so it's very localized in frequencies very broad in space in fact the limits are that it will have a delta function in X it's infinitely broad in spectrum if I have a delta function in in the mode space in other words in a single frequency Omega it's a constant solution in X okay so this is this Fourier transform pairing and I just simply say well ok let's let's take let's try to represent these by a fixed number of modes by the way you can clearly see here if you look at the cyan which is well it's a sort of a very fat Gaussian here in the X domain it's very localized in the Fourier domain and I'm gonna plot here on a log scale the error versus the number of modes I need to use to represent that so notice what happens here the cyan which I don't need very many modes for by the time I've got 15 modes notice that the error is already down at 10 minus 5 10 minus 6 in contrast if I take this highly localized yellow here which is a small patch of space very compressed it's very broad in the frequency and notice it's air decay properties even at 40 modes I haven't done that well in terms of bringing the error count so one of the things you need to do with numerical schemes is really work on trying to reduced your error so any localized structures in your PDE domain it's gonna cost you a great deal in representation okay so this is one of the big challenges and it's partly also why simulations of many systems especially multi scale physics systems where you have fine scale features cost you a great and a mountain computational cost because you have to resolve those features and it's gonna cost you a great deal number of 40 modes to resolve them for instance and you're gonna have then to take a large set of modes to get accurate representations and by the way this middle panel here just shows you when I look at the intermediate case which is the white one here and I look to see how well do you reconstruct it as a function of the number of modes so you know I'm using from three modes all the way to 39 modes and this is supposed to look like a Gaussian and you can see with three modes it's just a big broad thing as I use four or five six all the way up to 39 modes it nicely shaped up into this beautiful calcium so this is a nice representation of what of what we're altima trying to do here with reduced outer models is I'm gonna we're gonna construct a basis set to try to represent the solution and so which is fine except for the we're gonna have to have deal with these issues around how many modes do I need to use because ultimately we're like you what you really like to use is as few modes as possible to get some prescribed accuracy every mode you use here is another differential equation you have to solve right so you'd like to use as few as possible because if you look at the difference between these the simulation of this blue one cyan one I could probably use 15 modes in my simulation and do a good job whereas the yellow one I'm actually going to have to probably use a hundred or two hundred modes to get the same performance in the same error that I get here from the blue so that's that's the trade-off so the yellow would take significantly longer maybe an order of magnitude longer to simulate even though it's the same function ok just that I need more resolution to get it dialed in so this brings us then to the heart of it which is ultimate let's go back to our system and I've written written it a little bit differently here and I've written it in the form that often is used in reduced our modeling which is I take my PDE and I have some linear part of that PDE and some nonlinear part so I've broken it apart now I've now for this point for this purposes right now I've just said let's put an epsilon from the end in other words let's assume that the linear part dominates for a minute and I'd say what would we do with this if we were doing some analytics well if we were doing analytic solution techniques we would say well what I have here is a linear operator and oftentimes these linear operators have very nice properties right we can take linear operators and find eigenfunction expansions that we're Augen values and eigen functions that are representations of that operator l so it's equivalent to taking a matrix a and finding its eigenvectors and eigenvalues and once you have those eigenvectors and eigenvalues you can represent your solution in terms of those eigen coordinates and we're going to do the same thing here because remember what we're really after how do i construct the basis that I actually want to use for building the traduced or two models and so when I look at this this is already starting to indicate or foreshadow what we really want to do with this which is if I look at that linear part and if I know that the linear operator has such an eigen value eigen function expansion then that's exactly what I'm gonna do in fact I'm gonna take that and instead of expanding in Fourier modes or whatever generic basis set that I would have I expand in the eigenfunctions of that linear operator and when doing so this is my evolution dynamics okay this is just a generic numerical algorithms spectral expansion methodology so so now I get the evolution equations dadk so I'm gonna do an expansion remember for this you filled in terms of a sum of a of k's sigh of case and so i know what the sigh of kaizar they're right here so the only thing i kind of determine is what's going on with the a of case and the a of case I get from putting that expansion back into the PD taking the inner product using your thought analogy properties of these eigenfunctions and here is your evolution dynamics okay so that's the that's in some sense in a nutshell what we've learned is spectral theory or spectral representation for solving PDEs okay spectral methods that's this is this is what they are find a basis expand in that basis projective to that basis and get your evolution dynamics and we're gonna use this same concept and reduce sort of modeling the real big deal we're gonna do that is making reduce order modeling effective is we're gonna be very smart about how we choose these basis functions that is the entire key for reduce order modelling pick those pick the best five case you can now huh what do I mean by pick the best I want these dadk be T's I want K goes from 1 to N typically that's what you do I want them to go from one are I want the smallest number of OD E's for me to represent that full high dimensional PDE so I want an our dimensional system to accurate represents the in dimensional system and that's we're going to talk about in the next lecture by the way all this can be found here databook qw comm more codes more notes the notes are can be downloaded here and also some code and the future lectures are all dilated at this site

Channel: Nathan Kutz

Views: 3,101

Rating: 4.9459457 out of 5

Keywords: POD, reduced order modeling, ROM, proper orthogonal decomposition, scientific computing, model reduction, SVD, data science, kutz, brunton

Id: I8n8a7q8wLk

Length: 27min 16sec (1636 seconds)

Published: Thu Jun 11 2020