Machine Learning for Reduced-Order Modeling (Prof. Bernd R. Noack)

so I off yes had a good lunch break and this morning we have heard two really fantastic talks by Steve one two three no hello everybody no okay it works can you turn on the volume ah hello everybody okay that's the right thank you so this morning we have heard these fantastic talks of Steve and you can ask what I can possibly enter a add to these nice talks about machine learning and so attempt number 10 so the question is what what kind of kind of possibly add to these beautiful talks and number eleven can you hear me okay well I will focus I will try to present a spectrum from reduced order modeling starting from first principle navier-stokes paste to purely data-driven and there are a number of more things we can learn and as mentioned already in my first talk I would not be here without the phenomenal support of smart students and equally smart colleagues the package which I'm going to present for the reduced order modeling has been developed together with Daniel Feeney and which I am Zeeman he is the mastermind of machine learning control and and young its paeonia in deep mean-field modeling auto alert it's locally linear embedding and this loop past or frank suarez are on now Etna array Marc Mezvinsky Steve Brunton and avocado say my senior colleagues who have opened me the eyes to many different directions and we did very nice joint work first I will motivate why we need reduced order modeling we have seen already a couple of examples and then I will acquaint you with a method proximity map who knows proximity maps of you okay few so i this top it will be many more and i think every before doing anything else with your daughter you should do proximity map because they really tell you a lot about your your data if you have outliers on or not your you can color and use them in many different directions and then i will plunge more deeply in Kaioken modeling navier-stokes based Kaioken model modeling and present our programming package then I will give you a couple of examples of PUD models more examples and some critical and ableist then I will continue what Steve has discussed this morning money Ford models two ways of arriving at these money Ford models and then I will end with a with cluster based model which is a purely data-driven model so you do not need to know anything about the navier-stokes equations this is something which you can fully automate so this gives you as a spectrum of reduced order models for different and purposes so why could we be interested in reduced order models to some extent this question has been answered by the previous speakers over at one perspective so you could say that the reduced order model is kind of the glue between your theory where you have equations which are very limited and validity to your data and so the in in the theory you have understood a lot but you have a very low accuracy and if you look at your flow data you're incurably very accurate but you understand very little and so these reduced order models kinds of lifts your understanding by focusing on the gist you can also take a put a computational view so these maybes are coherent sizes of the coherent structures which you want to resolve so these are the finest structures Maeby's are called Makarov vortexes these are the large-scale structures if you want to resolve all of them navier-stokes based then you have to do a direct numerical simulation and we know that this is extremely costly and even when I'm walking here up and down I create a fun common vortex shedding it will be next to impossible to tort or to solve this numerically so one simplification is you remove the small scale structures and replace it with a model of this macalinski a model and you assume that this transition is universal enough so that you can deal with one closure model and you can go a step further and you can say okay I will take the unsteady runs equations your dynamic resolutions is now limited to something like an oscillatory flow so it's very very limited and the reduced order models are somewhere in between they resolve less than large Eddy simulations but they're resolved significantly more than unsteady runs of call your cost you pay for this accuracy a price and the prices that these are more configuration specific and this is a range where you also want to do uncontrolled design and optimization at the moment you do run simulation essentially you issue you believe the Quian structures are not interesting to you you only want to know the mean flow here so this is a computational perspective then there's another perspective understanding if you reach parentals paper the fluid dynamicists at that time we're very inspired by the by the orr-sommerfeld equation and there was a hope that by understanding the instabilities of the flow you can also learn something about the coherent structures and you can also learn something about turbulence and one of these passes have been mentioned by Steve this morning one process of following of course you know that the flow the dynamics of the flow changes with a Reynolds number if the Reynolds number is small enough you have a steady flow so you have a fixed point at some point at some critical Reynaud number it become oscillatory we have seen the cylinder wake already then this instability is called a Hopf bifurcation and one scenario which has been hypothesized by Landau and Hoff 41 and 48 is that you you find more and more of these Hopf bifurcation in fact infinitely many as you increase the Reynolds number and turbulence can be understood as a sequence of infinitely many hops bifurcations creating more and more frequency so this looks very appealing but this picture is wrong because at some point you have something like a cascade so you have an activation of many frequencies at the moment for instance you have a vortex and you have a vortex filament deformation you create turbulence and this is not a bifurcation anymore so nice in our you're good to have see smart people on board but this was not not a good theory then there were some theories in the 80s from from chaos theory so the question is how can a limit cycle become unstable apart from exhibiting another Hopf bifurcation you can have a period CC flying bomb scenario a period doubling scenario these have been observed in a very narrow data range and you end up with some Kotik flow which might be described in similar spirit like the Lawrence equation from which we have learnt this morning you can also have an intermittency scenario where essentially you you create larger and larger spikes and this leads to the strange attractor and you have also passes forms or tutorials and the three Charles that means from two oscillations and three oscillations the problem was this I would claims a strange attractor has never be found in fluid mechanics if you look at the correlation dimension so people have always measured the length of their data and fluid mechanics as a following you of no orchestra was almost infinitely many played players and when there are some leading violinists and everybody Tunes in and it is impossible to get them somewhat synchronized so everybody esto hasta Colinas the flow and if you look at the data essentially you see it be it's much closer to a deterministic stochastic system than to a strange attractor so these are dreams with a termination date so it was the end of the eighties but they have still inspired some approaches which turned out to be right now we go a bit engineering and we want to do turbulence control so with turbulence control we need some actuators and some sensors the question is of course where do we place these actuators and sends us what type of actuators should we use which where do we place them which amplitude range should they have this frequency range and so on and you can ask the same thing for the sensing for the same thing is easy but if you if you if the equation is wrong you have lost and now we do not have time for myriads of high fertility simulations so one approach is typically you make a few well-chosen large Eddy simulation or you use a plant you truly believe them and from there on you try to distill some reduced order model and the reduced order model allows you to explore a new better minima and of course at some point you have to test them so essentially is a reduced order models has less accuracy but a broader range of operating conditions or controllers actually this approach occasionally works now the next question is how should we characterize Korean structure Xavier did a wonderful talk all right and on the first day I will remind you how the vortex shedding behind the cylinder of Wake has been viewed so here you look at a smoke visualization around a cylinder and if you would not know anything about a guy Jochen model you would say oh there are vortices as a vortex here which goes like that another vortex here and so on so the so the vortex is move down strength by the way it's called fun common vortex street because when Carmen has built the vortex model describing this shedding so here you adopt a Lagrangian view you say these coherent structures localized important you look at the vorticity and you characterize the vortices which may be a ranking vortex and and some circulation and the dynamics is described by biot-savart so this is our vortex is a vortex model so this vortex model are the other models of the first hour and and hundreds of them have been constructed they are very robust we even used them in industry because of their robustness but there are some disadvantages the disadvantages typically they are limited to two dimension for three dimensions they essentially become CFD another approach is you look for instance at the V component here on this x-axis so these two vortices essentially they create a downwash this one creates up force this one creates it downwards so it does not create much fantasy to see that oh there's a traveling wave it goes like that that that that if you have a traveling wave say cosine x minus t you know you can write it as cosine x times cosine t minus sine x times sine T so you have your space dependent modes and your time dependent mode amplitudes and of course you can generalize this concept you are saying okay you have a modal expansions we have seen this or already so this would be the guy Jochen approximation of the flow and then you have to find a way how to derive a dynamical system for this mode amplitudes but cinavia Stokes equation can help you in this respect and of course there are many other ways of characterizing coherent structures we have seen the Lagrangian Korean structures they don't lead to a dynamical model there are also other dynamical model but I do not want to give a complete review about of this anybody knows another reduced order model than the yoke in a vortex motor well we have thought already one parabola stability equations so here are some milestones of low dimensional modeling or reduced order modeling you can argue that it might may have started with Leonardo da Vinci he has pictured the vortices very nicely and was most of his work so 50% of the pictures were right and 50% of the pictures were wrong Helmholtz has laid the foundation of vortex method of methods with his famous helm of vortex laws gallo keen has pioneered galerkin methods not in fluid mechanics but for membranes and then lawrence essentially made us aware of that there could exist it chaotic motion is something which we consider as generic so that means if you have two states which are closed we should expect them to exponentially diverge as you can see the photos have become increasingly better over the last 500 years now we go to a tool which I which we are using essentially in every application proximity map and the main idea is very simple so you have say you have your snapshot daughter maybe of 10000 snapshots 1 billion grid points each and you would like to see how much sense they make and how close they are so what you are doing is you consider here your original snapshots in a high dimensional state space and they um have a certain distance from each other so it could be done measured with a Hilbert space norm or whatever so they have a distance from each other and you would like to project it in let's say a two dimensional plane such that the distances are preserved if you have only three snapshots it can be done it's very easy if you have more of course you can only do this job approximately these coordinates are called a feature point and they typically give a very good guidance on the proximity of your data but they can of course also fool you so here's the mathematics behind it so the optimization is a following you say here as you say okay to try to find a mapping from the high dimensional space to the low dimensional space such that the error which you make average error on the distances is minimized so these are the snapshot in the high dimensional space these are the feature vectors in the low dimensional space this is the distance between the EM snapshot and the M structure this is the same for the features vectors what you want you want these two differences to be small and in fact you want say cumulative error to be minimized so find the T that e is minimal and this problem can be solved it turns out if you if you don't add further constraints then gamma 1 and gamma 2 are the first two PD modes but typically you would add more constraints and and and then you would link your feature space also to your performance and other things so what did I do where the executive summary for professors is look in Wikipedia and for students apply a CMD scale in an in MATLAB there are several degrees of freedom so one degree of freedom is you have to you can move the point in the feature space what do you do you Center them so that the average is 0 you can also rotational degree of freedom you can rotate the feature space as you want so essentially you make sure like in pud that the largest fluctuation happens in gamma 1 and then gamma 2 and so on and you have to live with mirror symmetry so you can reflect as a triangle or the configuration around some some X's so gamma 1 could be positive or negative there's no way how to fix that again like with computing now I'll show you one example and wake behind an awkward body use this is where you see UX you see a lot of fine scale structure what you don't see there also large scale structures and here you have a feature coordinates each dot corresponds to one point of the simulation now we want to interpret this a bit so what I did or what we did in addition week we clustered the data and we looked for every cluster we looked at the drag so this is a drag this is a lift and this is a side force and when you look a bit closer you see here on the left as a lift is negative the side force is negative and here's a side force is positive and here it's nearly neutral essentially what we see here in gamma one is kind of the by stability of the flow and the I 4 2 is related to some base flow and variation of the flow so we did gives you one example of the proximity map and we see we'll see others in the future you could in this talk you can also apply to functions to too many other things we apply two controllers as we have seen in my first talk so now we go to the first real reduced order model the PD Kaioken method and it started with Barska Jochen he was born 1871 in in some small town he was born his family was poor and so he initially struggled by being the writer in the court and then at some point but he was his interest was always engineering and mathematics at some point he made his way through the Technical University and was also an engineer as the Kharkov locomotive industry so in 2006 he was part in an anti self demonstration where a policeman got killed and Cesare was much nicer than Stalin so he pressed put him in a prison a way could work so he says this was a most productive one-and-a-half years he had where he got all the ideas for his future so this is could be considered the Russians a particle and then yet problems getting a job at the end he entered as a Polytechnic Institute eight years later he has published his book about his work about the guy Jochen method he became head of the faculty he became corresponding member of the academic society he became part of the academic society so one of the 48 academicians on top ranked in the US and yes at some point he has died a natural death enjoying a lot of Fame afterwards so these methods have been used everywhere so there are the basis of any CFD message and Lawrence was the first who suggested proper orthogonal decomposition in some report Lumley has coins that worked it took them something like twenty years to get the first model the first dynamic model of the fluid flow what was interesting about this this model was much more complex than any of the following model in the next couple of years and when I taught reduced order modeling I too read this paper every time again 50 pages condense with a lot of valuable information in you see there were few people who are really ahead of the time and have anticipated a lot of problems and have partially softs them in this model and then there were a couple of activities using this for flow control and here an example of a guy Akiane method so we want to solve this equation you know what it is convection equation flow goes from left to right and you have a boundary condition so let's say it's periodic from 0 to 2pi we have an initial condition let's say it's cosine X and of course you do not know how this equation evolves right and now we make a nun sets we say ok let's say maybe we can represent it with the first two fukui modes and some mode amplitudes but now we have a problem so we have parameter our solution in terms of two parameter what we have a PDE so the best we can hope for is that we can minimize an error but not that we can actually solve a PDE so what are we doing so one method to do this is using the inner inner product so projecting the residual of this equation on the two modes so we need two equations for two unknowns which are the mode amplitudes and we take the underlying equation so we project it here on u1 and you can see what happens so first we see the residuals and we put in the solution and the residual and at the end we get a1 dot equals minus a2 and which is the same with you two we get a 2 dot equals a 1 so this is our guy Orkin system our initial condition with cosine X this a 1 equals 1 something like that and this has a solution the solution is a1 cosine T a to a cosine T if he puts these things together with cosine X minus T so we are lucky we get the exact solution of this convection equation so this is so this is a spirit of the guqin model and we can embed it in a space poetry here we try to envision everything in terms of a1 and a2 so a few D mode amplitudes when we are on this point we have the cosine when we on this point we have the sine minus cosine a minus sign so I'm moving around so so as a wave travels from left to right we are moving on a circle and at some point we go to the initial part of course when we go to the navier-stokes equation we have a bit more work to do we have to find an approximation here and we've learnt already one approach pud and we have to find a system a guy european system for the navier-stokes equation and this is what i'm going to derive now it's very simple what you have to keep in mind now our inner product is essentially the two velocity fields with not Lydian inner inner product integrated over the domain so standard enough inner product of a two so first we will first we compute the PD modes and we take the average of the we take take a basic mode as a as a mean flow does anybody of you have an idea why we do this well it will become obvious in the next step so now a Scott has asks what is a request for the data what we are doing we are doing a second order statistics statistical order analysis of our flow data so at minimum the first moments must be right and the second moment must be right and just as a reminder if you have the cross correlation between you primes a fluctuation at x and y this can be expanded like this so disk this is a simple representation in the few demos so the next step what we are doing we are building the correlation matrix so here you see we take the same snapshot we subtract the mean and the end snapshot and we subtract the mean and really divide by by M this is our correlation matrix just just a volume integral not very difficult and I want to emphasize we are subtracting here you zero and I see a lot of applications where this has not been done surprised which you pay if you do not look at the fluctuation but if you take the whole velocity field is large so number one your first pewdie mode is likely to be something like a mean flow number two the fluctuation now has to be orthogonal to the mean flow well this is not what you typically have in a second and the convergence of the PD mode is not guaranteed the lambda I cannot be interpreted as variances anymore as a PD approximation does not fulfill the boundary condition for any a MA so and now you have to make an extra constraint on the mode amplitudes the PUD model is not physical for instance if you if you do the galerkin projection then you suddenly have a varying oncoming velocity in your dynamical system so this is a real screw-up and misuse of the approach so all the beauty of the PUD modeling is lost if you do not subtract you zero in in your expansion and in the fluctuation so the next step what you are doing is very easy you solve the fredholm equation so essentially you look at you do a spectral analysis of your correlation matrix you get a couple of vectors this C is positive semi T semi definite so the eigen values have to be all larger or equal to zero the modes have to be orthogonal and so you have a real non negative spectrum your the question yeah my question is about the means of traction if you have a stationary flow of course but if your flow is not session re and so your mean is not converged what is your point of view about that good good good point the question there's another reason why you should remove it and as some switch I make but that we have stationary boundary conditions for instance you have a cylinder Waqf is a no slip condition on the cylinder and you have uniform flow at infinity or far away so the mean flow now takes away the uniform flow condition and enter remainder you prime satisfies the homogeneous deviously condition and if u prime satisfies the homogeneous deviously condition that means or the fluctuation must fill it fulfill it that means it's a PD mode must fulfill it and now you can in the expansion you can choose the Pyramid amplitudes arbitrary and that's what you want you want you you want a guy you're consistent without constraints so this is this is what you have to do to arrive at this point good so now you have the egg mode and essentially what the pewdie models the field emote is nothing but a linear combination of your snapshots done in a smart way and of course a more and more snapshots you have you have to rescale it somehow and you'll remove some other fluctuation intensity in the direction so at that at the end you have also normal emotes and you can check this also by looking at the it's it's race so that some of the trace of the correlation matrix would be the sum of the eigenvalues should be the average fluctuation energy and the flow so these are the PD mods and now you have the AIS and and and and and these are the amplitudes again are somewhat proportional to the to to to the modes so if we have again a couple of prop and properties so the PD modes can be obtained by projecting the fluctuation on UI the average should be zero and the second moment should be centered so you have not only also normal normality and space you have also also unity in in the frequency if sorry yeah in in the beauty motor domain good now we call a one step further so what can we do having this fluctuation we can get a lot of inspiration when we look at moon in a glum and so all the equations which you see the value of the flow decomposed in a mean flow and a fluctuation can be written much more accurately in terms of a PUD mode and basis so we start with now with a Reynolds decomposition I will remind you of an equation which you should know from the from fluid mechanics so navier-stokes equation is written like this the Reynolds equation is essentially the residual with the Reynolds decomposition equals to zero a weak formulation means you take the residual magic and you can multiply it with any test function and you should get zero why is this effort well you can include for instance no shocks in your solutions and if you multiply the residual residual prime then you get an interesting equation you get a equation of fluctuation energy and here you see the navier-stokes equation your it can be expanded with a Reynolds decomposition now it looks something like that so it always has some constant and linear quadratic terms here a constant and linear terms here if you average them these are the survivors here so this is a Reynolds equation if you now subtract the Reynolds equation from the navier-stokes equation you get an equation for the fluctuation energy if you multiply this now with u Prime you get these volume integrals so this essentially is your production term so this leads to the production term this leads to the convection term this leads to the transfer term this leads to the dissipation and this leads to the pressure power so just a reminder of your navier-stokes a fluid mechanics lecture now we can do the same thing with with the modes here's a picture of the fluctuation there's some one term called production so this essentially determines how much energy the fluctuation gets from the mean flow there's one term participation this is how much energies are heat reservoir or the molecular structures get forms of coherent structure and there are two terms which are determined by the boundary one is a pressure power and one is a convection so in an infinite domain or periodic pipe and so on these two terms would vanish and now you can do the same thing also with you can do the same thing a motorized but what you have to do now you have to expand you Prime in terms of the guy Jochen expansion instead of projecting this residual on you prime for the global energy balance you're projected on AI UI and what you see immediately from the structure if I'm adding up all these contributions here then this becomes u Prime and essentially this gives the basis why we enter interpret this as a modal energy flow balance there's another equation which is vital if you project it and on UI we again get the guy Jochen system and seen in my last lecture the guy Jochen system has some some linear term it has some quadratic term and it also has some impression related to a Chung just you can make many many more variations or wizards for instance you can imprint it for projects a residual on e^x the unit vector in X direction and this gives you the equation for the drag so now you are capable of explaining the drag in terms of your Kaioken expansion or you can multiply it with E Y then you can interpret lift you can look at the Reynolds stress term and so answers a lot of equations which you can derive just with these simple answers here I want to focus just on the modal energy analysis we do again the analysis here and here the guy Oakland system looks like that and if he multiplied with AI UI we get these terms using all the PD and properties so there's one term left over this one here which leads to the production there's another term from the guy your consistent times the energy leading to the convection term there's a irreducible charm which essentially this is the essence of turbulence which leads to the transfer term and dissipative term and some other terms you see that the motor production convection and dissipation scale a proportional to the fluctuation energy in each of the modes and the guy oaken system essentially determined something likes again and these nonlinear terms are the one which has a troublemaker the picture which now emerges is the following so you have the different modes here there are some terms essentially they get some energy from the mean flow lose it to the dissipation they lose it at the boundary the most important thing is a transfer term if you remove the transfer terms and non-linearity maybe the first mode will explode the modes will either explode other way decay you have no sociology so you need the non-linearity in order to have this sociology and here's just a remainder that the average global energy flows are the sum of the modal flows and they all should add up to em to zero and so this is a recipe now so if you have something like m snapshots you have to write routines essentially for these three things for an inner product for a tree linear form from the convective term Abby linear form which comes from the dissipative term and maybe you want to consider the pressure power as well then you need a surface integral compute the mean flow compute the correlation matrix determine the spectral analysis computes the PD modes computer Fourier coefficients you see these operations are very simple and if I show you my Fortran code it's only something like like like three pages long so as an as an augmentation you can look at the dynamics so this is your dynamical system after the guy Jochen projection you can perform a motor analysis for each of the motor and the motor analysis will tell you for instance what is anything is missing in your in in your approach for instance you can see from the from fi if you should include the pressure term you can see from the difference to zero if you should add some stabilizing terms and so on any questions so far you have to choose again hands up I will try to start from you questions good so I've shown you now a procedure which I think most of you can program easily yourself say in a weekend there does it typically work or not well there are a couple of things which you should keep in mind there are some things which make your which which you may have to add to stabilize your guy organ system so first thing is maybe you are missing some modes in the PUD which have are stabilizing so in my first lecture I've shown the shift mode if you don't add the shift mode to the PUD guy European model it will be inherently a fragile and and potentially I mean robust so the shift mode is one possibility essentially your this is a difference from the steady solution to the mean flow now it has another aspect which Steve has mentioned you can derive from the navier-stokes equation from from most boundary condition that the quadratic term should be energy conserving and that set its energy conserving if the contribution at every instant to the whole fluctuation energy is zero and you can show that this coincides with another condition so you can decompose the Q I J case with the three tensor into in one quantity which is average and another quantity which is the remainder so you you need to remove this part because you will always have some numerical error to be navier-stokes consistent this is the remaining part and this often stabilizes an otherwise unstable and galerkin's system because the further you go

Channel: von Karman Institute for Fluid Dynamics

Views: 1,833

Keywords: Big data, machine learing, data-driven, von Karman Institute, Université libre de Bruxelles, VKI, ULB, model order reduction, system identification, flow control, machine vision, pattern recognition, artificial intelligence, mathematics, mathematical tools, discrete LTI systems, Bernd Noack, reduced-order modeling, galerkin method, POD

Id: ETDnjN33D2I

Length: 41min 56sec (2516 seconds)

Published: Fri May 29 2020