Steve Brunton: "Discovering interpretable and generalizable dynamical systems from data"

all right thank you and thank every think you everyone for being here so it's really exciting to be here and tell you about the work indeed dynamical systems modeling and control and so there is gonna be a pretty heavy element of fluid mechanics in this talk but I want to talk a little bit more broadly about sensing control and dynamical systems modeling with machine learning so before I go on I want to acknowledge my many colleagues co-authors and collaborators on this work so a lot of the data driven discovery work is what Nathan cuts and a lot of the fluid mechanics work is with John Christophe laws oh and it's actually a treat JC is here in the audience and he's giving the second talk so this is our first chance to give back to back talks about modeling fluids with machine learning and then a lot of the the work I'm going to show has been done by by various postdocs and grad students over the years okay so just quick show of hands how many of you have actually seen this movie before I play this a fair amount okay so I love this this video I find it to be an incredible inspiration for the work that we do on learning and complex systems ok so we are all here because of machine learning and the ability to discover governing laws or dynamical systems or controllers purely from data but I think that a lot of the inspiration from this does come from biology which are expert learning systems expert control systems and have an incredible grasp of both modeling predicting and interacting with the physical and natural world so a few features I want to point out about this eagle you can see this eagle interacting with an extremely complex unsteady turbulent gust field petros talked a lot about the challenges of modeling fluids yesterday and I would argue that this eagle although it looks like it's effortlessly interacting with this environment this is far far beyond our current engineering ability to predict and model at all scales so maybe we could take a box of fluid and discretize it and simulate this with a million or a billion ordinary differential equations for a few seconds but that leaves out a lot of the complexity of the feathers and the wings the compliance surface the musculature let alone the kind of nerve mechanical control system that's going on inside of this Eagles brain okay and so this is truly a vast and multi scale system and and I and I think there's a number of things I want to point out about about the inspiration we take from this Eagle so for example I don't think the Eagle needs to simulate all of those millions or billions of degrees of freedom because there are dominant patterns that exist for the Eagles daily life there are patterns of vorticity and fluid mechanics over its wings that are required for it to maintain lift and drag and turn effectively and efficiently and the existence of those few patterns facilitates low-dimensional reduced order modeling and sensing and control so it doesn't need to be running supercomputers in the background necessarily okay and it's worth pointing out that this eagle does not have full three-dimensional velocity field measurements does not have a PhD in fluid mechanics it probably doesn't even know about the Navy or Stokes equations it has its experience of itself in its ancestors it has a pretty incredible sensing system both in terms of its feathers its visual system and various other sensing modalities but again its goals or not to do full DNS predictions its goals are to effortlessly and efficiently control in its fluid environment okay so if you're the the themes of what my group works on and what I'm going to talk about in this talk is that often in complex systems so I'm going to talk a lot about fluid mechanics but this applies much more generally to lots of lots of complex systems that we're all interested in equations are often either unknown in modern systems or too complex to model so if you think about neuroscience or climate science disease modeling we don't have fundamental governing equations for these systems or we can't agree on them even in navy or Stokes fluid mechanics the equations are kind of one-size-fits-all they're too complex to be readily used for optimization prediction and control in real time it's a one size fits all model and so we want tailored reduced order models for example for specifically the relevant flows over a wing so we can do fast effective control like the Eagle another key challenge that motivates us is nonlinear and high dimensional dynamics so most of the systems we're interested in have non-linearity and they're high dimensional either in space or in time or both and those really really do challenge our ability to to interact with these systems to design to optimize and to control and so a lot of what we work on is discovering coordinate transformations that linearize these dynamical systems or make them simpler in some way and so I want to point out most of the great breakthroughs in mathematical physics over the last centuries have focused on coordinate transformations and coordinates discovery okay so if you think about the Fourier transform Fourier didn't cook up this Fourier transform for fun he did it to linearize the heat equation to diagonalize the heat equation into eigenvector coordinates into a coordinate system where things became simple in that case being sparse and diagonal and increasingly with machine learning were able to learn these coordinate transformations these effective coordinate systems where things become simpler okay and so the perspective that we take is essentially to learn physics from data putting a premium on models that are both interpretable and generalizable so I don't want kind of big black box models that have a million degrees of freedom if I can get away with a model that has three or four or five degrees of freedom because then I can communicate it analyze it and interpret it and even more I want those models because I'm thinking about control is my ultimate goal I want to make sure that my models are as general as far away from my training data as possible okay and so if you think about Newton's F equals MA that's essentially the ultimate generalizable model right it works you learn it watching an apple fall from a tree it still works when you launch a rocket to put people on the moon okay so you want generalizability if you're gonna do control and modify your system okay good so this is kind of the the framing for for the entire talk I like to put the whole world in one of these two boxes okay so everything I care about is a dynamical system pretty much everything and what we want to do is measure and affect that system with sensors and actuators and we wanted to develop some control logic to effectively manipulate the behavior of the system to some engineering outcome so a lot of what I'm going to talk about in this talk is essentially using machine learning to learn the dynamical systems directly to model the system we can also effectively learn the controller without without resorting to to learning a model okay and in fact I think this is a really important point for fluid flow control it's something that Petros also brought up is that often if you spend all of your effort up here modeling your fluid system the first thing you do when you turn on your controller if you did a good job is you drive your system away from the region you trained on and so your model might not actually be that good and so there's a lot of work developing effective control laws using using machine learning optimization so I'm going to talk a little bit about that right now and in fact I'm going to talk about kind of three flavors of control modeling using machine learning the first one is going to be work with with parent Noack on essentially turbulence control okay and I want to point out at the outset that the way I see machine learning is very simple I see machine learning as a collection of advanced optimization and regression techniques to build models from data okay and if you think about it almost every task in engineering and fluid dynamics is an optimization problem at the end of the day reduced order modeling is trying to find the best model that's optimization given some dimensionality and given some data control is trying to find the best intervention strategy to manipulate the system under some cost function sense they're an actuator placement or optimization problems okay so flow control is also an optimization it's a really nonlinear non convex high dimensional optimization problem and that makes it challenging and interesting okay so one of the things that berrineau acted a few years ago with collaborators is build this beautiful wind tunnel mixing layer where he can essentially mix up a fluid so you have a fast fluid dynamic fluid flow on top and a slow fluid flow on the bottom separated by this metal splitter plate and you get this this nice kelvin-helmholtz roll up and eventually it breaks down until turbulence and so one thing you might want to do is you might want to measure your flow at some downstream location for example using these hot wire rake and you might want to affect upstream so that this pressure this plate this metal plate that separates the flow has 96 independently actuate able pressure ports you can blow air out of these things up to 500 Hertz independently and so you might want to take these measurements and then do some kind of some kind of blowing on the flow to change the profile downstream to either increase the mixing or decrease the next thing and if you could do this here you could translate this to for example airplane wings or chemical mixing applications and so one of the techniques that Behrendt and colleagues approach this with is using genetic programming to learn effective control laws from trial and error so it turns out that simulating this flow is actually quite challenging and it's actually only gets harder when you simulate full aircraft and automobiles but it's relatively easy to test a control law in an experiment in about 10 or 20 seconds of an experimental trial you get a pretty good idea of if this control loss of the control input U as a function of sensors and constants you can tell if it's good or not in about 10 or 20 seconds okay you can tell if the mixing went up or went down and so what you do is you essentially run a hundred or a thousand randomly selected controllers you try them all and you rank them you take your best performers and then you run through these genetic operations mutations crossovers things like that to breed successively better generations of control laws that are more effective at minimizing your your cost function okay and this is it pretty effective in a number of turbulent turbulent fluid flows so you can get about 20% better mixing enhancement you can get drag reduction in other experiments so I thought that was pretty cool just using genetic programming so another application we look a lot at is renewable energy in an offshore or or river generated hydropower so this is this is what we call a cross flow turbine cross flow just means it's a vertical axis turbine if you knocked it over and put it in a river okay so it wouldn't make sense to call it vertical axis if it's on its side in a river and we have a lab scale version of this that's about yea big in a water channel University of Washington and so Ben Strom who is an excellent PhD student with Brian Paolo G and me wanted to maximize the power of these cross flow turbines for renewable energy in remote sites now what you would do in a helicopter that had you know rotating blades is you would dynamically pitch these blades at different phases of advance and retreat as it goes around and it's it's circular loop but because this isn't a remote village in Alaska you don't want to have any moving parts because you can never fix this thing you don't want any maintenance cost you don't want this thing to break down every you know hundred hours like a helicopter does and so what Ben realized is that instead of dynamically pitching the blades because this thing is moving around in a circle you can optimize the acceleration and deceleration profile you can essentially accelerate into certain parts of the turn and decelerate into other parts of the turn and it gives the effect of the bait blade pitching up and pushing down in its reference frame very cool and so this becomes again a high dimensional optimization problem we're essentially trying to optimize this blade acceleration profile as a function of its phase angle to give the maximum power output possible and so so Ben essentially tried various optimization strategies and was able to find a very very effective one that gives about 60 percent power increase over industry standard control in our lab Armand and the thing I think is really cool about this is that the procedure we didn't put any physics in at all we essentially just wrapped an optimization loop around the experiment and it learned that for certain profiles you can create this large leading-edge suction vortex here at the right at the right phase so that it coincides with the power stroke of the turbine so you get the maximum torque at the maximum velocity and you get this maximum increase in power and so here's kind of a movie of this thing going around you can actually see it forming this large leading-edge vortex at the right time and then it will shed it at a time where it won't affect the downstream blades this is a two bladed turbine pretty cool okay you just have to frame it as an optimization problem with a well defined cost function and essentially this is a plot showing kind of the velocity and torque and the overall resulting increase in power because you can align them with the control but the ultimate goal here is not just to have one turbine it's to have an array of turbines okay so you want to essentially put these things in a dense array pack together and this is actually exactly like what Petrus was showing you want to use the same principle that's fish use when they school to kind of get this favorable interaction of these these turbines together so we need to understand not just how to optimize the power of one turbine but where the weight structures are going and how to place them for the downstream turbine and so this is actually a teaser for work that Isabelle Sherrill at my lab is doing right now is applying these machine learning optimizations to place a second turbine in the wake of this first turbine these are finite timely oppan of exponents of the first turbine and so if you can picture what Petrus is showing with the fish yesterday you essentially want to place the second turbine at a favorable location so that these blades can kind of piggyback on these very very strong coherent structures generated by the first and again this is all experimental data from from a lab scale test that we're doing in in Washington okay so the third flavor of machine oh yeah question yeah so it does depend on flow speed and Reynolds number it depends on turbine geometry it depends on a lot of things absolutely so it's easier to optimize these in a river than in a tidal basin so the tidal basin is a little tricky because you have a whole range of velocities and so you get to decide where to optimize so there's two things we're trying to do one is we're trying to understand the fundamental physics and how it scales so that we get you know a good idea of what would work most of the time but the other thing we'd like to do is run this optimization loop online so that as the flow speed varies or if there's a disturbance or if a rock moves and starts creating you know vortex shedding we can kind of adaptively correct for that on the order of minutes which we think is pretty feasible based on what we've seen in the lab okay great question and please do ask any questions in the middle okay so the third flavor of control is essentially using deep model predictive control for for fluid dynamic control so here we see this is a recent paper by Katharina beaker and Sebastian pipes and what we see here is a single cylinder the vortex shedding behind a single cylinder and this is the new fluidic Pinball which is a really cool name for three cylinders in a triangle in case of the three cylinders in a triangle you can independently spin these around and you can get very different kind of Richard dynamics so you can you can have almost turbulent wakes behind this which are a little bit more complicated than the regular cylinder flow and so what we're gonna do here is we're essentially going to use deep learning to build a predictive model for the evolution of this flow under different control strategies and then wrap that into a model predictive control loop okay and so this is essentially what you would what you would normally do if you could afford it you'd have the full model the full DNS simulation in real time you'd be predicting the control output but this is far far too expensive so instead what we do is we use this surrogate model and we're gonna learn this essentially using a deep neural network and then wrap this control around the the simulated system and so when you do this here these are the commanded lifts and dashed for the three cylinders so we we make one of them have high lift one of them have a low lift and one of them have zero lift and you can actually see that our actual system is doing a pretty good job there's a little bit of ripple from the vortex shedding but more or less we're able to to control this system to our reference lifts pretty effectively one thing I thought was really cool though is that our architecture is actually online trainable so what you can do is as you apply this control law you can retune your neural network prediction to characterize that new attractor and if you if you're retrained in time as you go along your control cost goes down and your mean error goes significantly down and you can see that we're actually getting a much much better tracking of our reference lifts if we do adaptive learning of our neural network model as we go okay so that's kind of neat okay good so I've talked a lot about fluid dynamic control that was a simulation yeah we don't have that that experiment yet okay so I talked a lot about control but the first step in control generally is picking sensors and actuator actually picking the hardware that you're going to use to control your system so we also spent a lot of time thinking about how we can use these kind of high dimensional nonlinear non convex optimization techniques to learn effective sensor and actuator placement so that we can do good control in the future okay here's just a cool schematic I like of you know if you had some library of behaviors how would you sense that system effectively to tell what what behavior you're looking at and I want to go back to the bio-inspired perspective so this is work that I with Bing she has a Miri to study insect flight dynamics and control using sparse sensing which i think is really really interesting and I want to just take a little bit of an aside to tell you about how we think about insect flight and how it kind of informs some of the work we do so I showed the video of the eagle on the first slide it's a kind of amazing demonstration of real-time flow control but insects also interact with very complex turbulent fluid environments and to some extent I find this even more impressive because the eagle actually does have pretty sophisticated computational hardware it has lots and lots of sensors it's a very sophisticated animal the moth less so but the moth is also able to navigate these extremely complex turbulent gust fields and plumes so a couple of interesting facts about the moth I want to tell you about things I didn't know until working with biologists every single flying insect all of them have on the order of tens to hundreds of strain sensitive neurons on their wings all of them there are these little strain sense of neurons can Panna form sencilla that are distributed in a pretty regular pattern on the wings of all insects and they use these to inform their flight dynamic control system to measure forces and accelerations and things that are happening disturbances and they use these for very fast effective control so another thing that's really interesting is that if you hit a moth with a gust you can do these really cool experiments where a moth is flying and you whack it with a gust the moth can correct its flight pattern its wing beat frequency faster than that information can go to its brain and back to its muscles that's remarkable and so what's happening we believe is that the insect is assimilating this sensor information into a local computation in its shoulder muscle and it's using that information directly to infect it's it's its control system it's muscle control for its flight and so this is what Bing is proof by existence that you can get away with sparse sensing and very low dimensional computations for an extremely sophisticated control system pretty amazing so we've been studying this as long as our daughter has been alive so a long time trying to understand how this insect would why it would place its sensors here how it would process the information and how it would use it for effective real-time control and so we back engineered a lot of this technology and I just want to point out if you're interested in sensors if you're interested in compressed sensing or sparsity there's a lot of cool applications in engineering systems that you can use so once you kind of back engineer how these sensors could be used to inform either classification tasks or estimation tasks or control tasks again it's an optimization problem to figure out where you would put these sensors you can use that same technology to design sensors to help Boeing either in its manufacturing or in its flight dynamic systems to place our sensors in the ocean to estimate flow field temperature or for for input output control systems and so these are all applications Kritika manohar worked on when she was a PhD student and then this is worked by by Tomas and Bing in actually understanding kind of the insect okay so I've hopefully convinced you that machine learning can fit in lots and lots of different aspects of this control loop for the remainder of the time I'm going to spend my time up here this is kind of my passion is using using data driven modeling to get reduced order models of complex dynamical systems okay and I want to point out that there is a ton of great work on this this field I mean this is one percent of the work I'm actually highlighting this specifically because this work in the last ten years on model discovery has really taken a change in perspective from classical system identification of the 80s and the 90s to really put a premium on parsimony and sparsity in the model okay don't want the best fit model I want the best fit model with the least complexity okay and that's obvious I think when you say it but that's a relatively recent trend in data-driven modeling I would say major work in the last 10 years okay and so that's what I'm going to talk about for the rest of my time okay so I'm gonna walk you through the sparse identification of nonlinear dynamics the Sindhi algorithm here for obtaining nonlinear dynamical systems models from data and then I'm going to show you how we're using this to model interesting complex systems and fluid dynamics and and other fields okay okay good so I'm gonna illustrate this on the lorentz system which i think is a pretty simple simple toy system you have three states XY and z and you have simple quadratic nonlinear dynamics now I really want to point out and emphasize that the right-hand side here is very simple in fact the right-hand side of most of your models your textbook models in physics are simple they out of all the possible right-hand sides of all the complex models you could have most of the things we observe actually have only a few terms various reasons for this you could argue that it's a Taylor series expansion of the real physics you could argue that it's asymptotics but it's an observation we made that most models are very simple and when I say simple I think of sparse there's only a few terms here out of the possible terms so the way that we work this algorithm assume you have data XY and Z of your state in time so each column is a time series of data and assume that you have the derivatives X dot y dot and Z dot in time now that's a lot of assumption generally you don't have full state measurements and full derivatives we're gonna relax that in a little bit but it's easier to illustrate assuming you have all of the data how many of you heard of dynamic mode decomposition okay most of you have heard of DMD so what DMD would give is a little three by three linear model right here okay we give the best fit linear regression model that fits X dots given X's it would be a three by three linear model but we know that that's insufficient to handle truly nonlinear dynamics on a chaotic attractor so instead what we do is we build an Augmented library of possible candidate terms that could be in our right hand side dynamics okay in this case polynomials up to fifth order and each of these gray columns importantly can be computed from the data I can take the data X and I can square it or cubit or multiply it by Y and I can build this whole library and now if you write the dynamics in this library each of these X dot y dot and Z terms is essentially a sparse combination given by these coefficients C of these these library elements and theta okay and so the inverse problem is essentially to try to find which of these columns which small subsets of these columns adds up to explain X dot y dot and z dot okay now if I had posed that problem 20 years ago it would be unsolvable it would have been a combinatorial e hard brute-force search through all possible models with two terms and three terms and four terms now it's a relatively straightforward sparse regression problem there's many many techniques there's a wide range of techniques to solve this sparse regression Terk a problem to find with high probability the fewest columns of theta that describe X dots that describe y dot and that describes Z dot and when you apply these techniques what you do is you simultaneously find the structure of which nonlinear terms are active and the coefficients for those model terms okay so you can actually discover nonlinear dynamical systems purely for measurement data using this method now there's a lot of assumptions here right I assumed for example that I had the right measurements XY and Z and their derivatives that's the big first assumption I measured the right thing the second big assumption is that I can build the right library where the dynamics become sparse polynomials tend to work for lots of systems especially fluids but you could have built-in sines and cosines and Bessel's functions whatever you want and then the third assumption is that I can actually solve the sparse regression problem to find the fewest terms that are active my dynamics turns out that third one is the easy easy one sparser Christian is the easy part getting the right measurements and getting the right library is typically much harder okay so I'm gonna start relaxing these various assumptions I assumed I measured XY and Z and the derivatives now let's start systematically relaxing those and applying this to harder problems okay so the first thing you can do is you can get rid of the assumption of measuring the derivatives and we can instead just measure noisy collect noisy measurements of the state X Y and Z's so these are my actual measurements here this kind of fuzzy a tractor and what you can use is total variation regularized differentiation to approximate the derivative of XY and Z and when you plug these approximate of derivatives into the Sindhi algorithm you find you actually get the right structure of your model and almost all of the coefficients are pretty accurately preserved except this one's off by about 20% interesting we're minimizing total variation and I'm really glad Stan is in the audience because we're actually using this this original method that there's your name so we're actually using the code by by Rick char Trond here based on the total variation minimization but I think this is pretty incredible so you can take really noisy measurements and this is a testament to again total variation regularize derivatives but you can take very noisy measurements and actually still discover the right structure of the model you can also extend this to partial differential equation modelling so instead of ordinary differential equations you can collect the same you know high dimensional measurements and instead of building X dot equals you can essentially build partial derivatives in some state vector for example vorticity and solve the same sparse regression problem to find the fewest terms that give you the time derivative of Omega and in this case what you find is a very simple model that matches the Navy or Stokes equations for example if you give it fluid data and here you even get the Reynolds number to within 1% which I thought was really cool okay this is clean simulation data you can also do this because there's so much there's so much data here if you take points in space and in time you can massively subsample your data I don't know if you can see these little green squares but you can massively subsample your data in space and time and you can discover the same the same partial differential equation yes I think you could also do this with use it just as a it's a simpler representation here and it's easier to write down and visualize yeah I think this is actually mostly because I like plotting it in vorticity and so we're current so so we've essentially used this for rediscovery of partial differential equations we already know but we're currently working with collaborators to now discover coarse-grained models that we've never seen before for example in plasma physics and fluid mechanics so there are hierarchies of models of different descriptive ability ok so modeling is always a balance between how accurate you want your model to be and how effective or if I start efficient you want your model to be and so navier's stokes is very very accurate pretty slow and what we're trying to do now is find kind of reduced order models coarse-grained models that are less accurate but less computationally intensive so sparse regression essentially means some principle relaxation of an l0 minimization that that's what I mean it as is I'm trying to find I'm trying to solve this regression problem with a sparse vector of coefficients rather than a least squares vector of position so there's various ways you could relax this so for example the lasso is kind of the most the most obvious approach would be just relaxes to an l1 that doesn't have the best performance so we have our own homebrew process that is also a formal relaxation of the l0 and in fact that's actually I'm going to talk about about next ok so I want to take a step back and just think about what we mean by parsimonious modeling in general ok so this is the problem we're trying to solve we have X dot we have a library of candidate terms and we're trying to find the fewest linear combination the smallest linear combination of those terms that describe the dynamics and we're doing it by essentially penalizing the zero norm of my coefficient matrix I want to have as few terms as possible here so we get to choose how aggressively we penalize sparsity or number of terms in this model with this lambda parameter okay and essentially what we're gonna have is this trade-off between error and complexity so I'm measuring complexity and how many terms my model has an error and measuring using using this this you know two norm or fro norm or whatever you like okay so as I sweep through lambda I can get very very simple models or very very complex models let's take the limit of lambda goes to infinity if lambda is infinity then I'm that's the ultimate penalisation on sparsity and i basically can in that basically imposes that I have no terms active in my dynamics it's so expensive to have a term that I have zero dynamics okay so if lambdas infinity then I get the model X dot equals zero clearly under fit very high error very low complexity okay in the other extreme if I make lambda zero I'm not penalizing sparsity at all that's just doing a least-squares regression and if I do a least-squares regression here I'm gonna get a coefficient vector that has a little bit of all of the terms active in this case there's 81 terms so I'd have a model with 81 terms than without damp dynamics we know that's over fit right we know that our physics doesn't probably have 81 terms but it is gonna give me lower error on my my training data okay now what you hope is that there's a whole family of models between these for different values of lambda and in fact you hope it looks like this where there's a Pareto optimal curve it's also really really important to test your models on actual test data that's not used to train because then you can actually see that your least squares solution is incredibly overfit okay those 81 terms might even be unstable if if you apply it to a new trajectory and so what you're really trying to do is sweep through lambda to find this sweet spot where you have just the right amount of terms that is necessary to describe the dynamics accurately and no more so that's where these kind of parsimonious models live down here in this cross validated regime and there's a lot of techniques to do this and a lot of people think about parsimony and there's a lot of techniques to select these models so I mentioned that we're not using lasso because the lasso algorithm tends to get stuck in in the wrong the wrong solution early in the path so you often either kill the right terms early and you can't get them back things like that so what we do is something called sequentially thresholded least squares super simple algorithm it was born out of essentially desperation because we couldn't get compressed sensing to work on this what you do is you do a least square solution you get a little bit of all of these terms but a lot of them are small and so you hard threshold those then you do another least squares regression onto the remaining terms that are nonzero now some of those are small you kill all of those and you do this again and again and again until it converges that's the quench hole thresholded least squares and there's variants of it where you can kill one term at a time or multiple terms at a time but it's very simple you do at least squares you kill the small terms and you do it again until it converges now there are some chief benefits of doing it this way in addition to the fact that it doesn't get stuck as often and it's more robust to noise it's also based on a least squares regression which means that you can do lots and lots of other things while you're doing that least squares regression so we didn't realize this at the time but actually JC found out that you can impose physical constraints on your model by doing constrained li squares with sequential threshold is what I'm going to talk about in the next section on fluid mechanics okay so we have this tool cyndy that can discover toy systems that we already knew the answer to it's particularly useful if we can use it to solve or to model systems we've never seen a model for before like this fluidic pinball so so that's what I'm going to tell you about for the rest of the time so I'm going to look at this fluid flow past a cylinder characterized by a Karman vortex treat this is at Reynolds number 100 and so this fluid flow was a nice test case for Cindy because there's a really interesting about 30-year history of how reduced order models were discovered for this system and I'm going to tell you a very abridged and partially true partially complete history of this of this system to kind of motivate that it's interesting and challenging to get a model for this so in in the 70s rural and talk ins the same talking's of the talk ins embedding time delay embedding had a hypothesis that the chaotic and rich turbulent dynamics that were observed in fluid flows might be explained through a sequence of hop bifurcations as you increase the flow velocity or the flow Reynolds number as you increase it you go through one Hopf bifurcation and you get periodic motion like this if you go increase it more you go through another Hopf bifurcation and you get quasi periodic behavior and if you go through a final Hopf bifurcation you might get chaotic and turbulent behavior okay that was a hypothesis in the 70s about 15 years later numerically researchers discovered that in fact these Hopf bifurcation --zz do exist in in fluid systems but that actually begged more questions than it answered because for those of you who know dynamical systems the hop normal form is a cubic non-linearity but the navier's stokes equations are fundamentally governed by quadratic nonlinearities the convective term and then that navier's stokes equation is quadratic not cubic and so that begs the question how could you have a cubic hop for non-linearity in the quadratic system and so it took another 15 years before Baron Nowak and collaborators explained how you could have this this this Hopf bifurcation with a quadratic system and so what he did essentially right this fluid flow system in a reduced coordinate system given by 2p OD modes these are the the principal components or singular vectors of the data and a third mode called the shift mode which essentially there's an unstable equilibrium here that you would never see in experiments and the shift mode connects this unstable equilibrium to the average flow if you just average this time series okay and in this coordinate system he derived this reduced order model here where the z direction this z direction here is much much faster than the dynamics in the first two p OD modes and so what happens is very rapidly z equals x squared plus y squared if I start off of this make this slow manifold I very rapidly glom on where Z equals x squared plus y squared and if you plug that in to the dominant dynamics up here you get the effect of a cubic non-linearity and it looks exactly like the hop normal form so what he discovered was that through this slow fast separation of time scales you could get the effect of a cubic non-linearity in this quadratic system and so I kind of put a nice close on this this 30-year history of Hopf bifurcation in fluid mechanics so we saw this as a challenging test case for Sindhi we were going to take the exact same coordinate system again I'm assuming I'm measuring the right thing I'm still using Barents coordinate system we measure these three variables we plug that into the Sindhi algorithm and what we found was that the identified system actually had the right slow fast separation of time scales it had quadratic nonlinearities and it had this this parabolic slow manifold so I had all of the main features that we were looking for from this model which which was pretty neat and gave us some confidence that we could actually use this for discovering new models because this is you know a real research problem that took people a long time to discover yes no no so that's a really really important point so actually there's two different sets of simulations we tried the three you could just start on the limit cycle which is the movie I'm showing and that would not give you this model at all the simplest model that describes this limit cycle behavior is a linear DMD model by far the simplest model then what you could do is you could start near this unstable equilibrium and you could watch this thing slowly unwind on the slow manifold that also didn't work so in addition to that trajectory we also started with a trajectory here at the mean flow that goes down and then spirals up and so you need transients to be able to capture these extra nonlinear terms hugely important thank you for asking you absolutely need to have transients in your system kick the system in some way off of the attractor to really get the dynamics okay so this is where the story gets I think pretty interesting for me we put this paper on the archive and within a relatively short amount of time we got an email from JC and he pointed out a few key issues with our model so I don't remember exactly all of them but I think we had the wrong unstable growth rate mu we had I think a spurious fixed point down here for some reason I think there was a constant term it wasn't perfect the model okay and we were a little bit despondent but fortunately about two weeks later JC email back and had completely solved it and so this is one of my favorite things in this whole cindy modeling framework is JC's solution to the cylinder problem so essentially he came up with two incredibly important ideas for modeling fluid flows with Sydney that we're still using today the first one is again because our sparse regression is built on a sequentially thresholded least squares we essentially are just doing least squares and then killing terms it's relatively easy in our least squares regression to add linear constraint equations okay so constrained least squares is approximately as easy as regular squares and so there are lots of things we know about the fluid equations the navier stokes equations for example we know how energy is conserved in incompressible navier's stokes or at least JC did at the time it's because of skew symmetry in the quadratic terms of our model and you can actually write down what those skew symmetric terms look like for a model with three equations you know XY and z and you can enforce that the Sindhi model has to have those symmetries in the equations through an additional I think eight or nine constraint equations here so that's amazing you can build models that are stable by construction they're guaranteed to conserve energy they can't do anything otherwise because it's fundamentally baked into the into the regression so that I think is a very plausible way of incorporating partial physics knowledge either in some known symmetry or energy conservation mass conservation something like that the other innovation I think is really cool when we were developing this model I was going for a quadratic model because that was the history the 30-year history was how do you get cubic hop with a quadratic model but if you relax that and you allow yourself to have cubic quintic septic higher order terms in your dynamical system essentially what you can do is you can account for the energy that you threw away in your in your principal components so remember I'm building this model on the first two principal components well there's principal components three and four and five and six and seven and eight and those have energy two and are important for the dynamics those also are faster and faster time scales and substitute back in as higher-order nonlinearities in the asymptotic analysis and so what JC realized is that if you allow yourself to have higher order terms you're essentially getting a closure model for the effect of those higher frequency lower energy p OD modes that you're truncating so with fewer terms but higher model order you can account for all of the energy that you're discarding in that anecca lurkin system so I won't say that is you know I mean this is a laminar problem so it's not turbulence closure but I think it's highly suggestive that you can use some of these methods in more complex fluids so very very exciting okay I have a couple of slides here to show actually so this is how you get energy conservation for an incompressible fluid it's essentially this in it roll of you dotted into the convective term this is what it looks like in the reduced order coordinates a1 a2 a3 and these are the concerns that JC derived that have to be true this all has to equal zero so again you can do this for pencil and paper for for your symmetry and you arrive at these constraint equations that have to be true for the cindy model if this is true your model preserves energy by construction okay and again you can then write that as this matrix set of constraints and solve it using KKT constraint least squares so I want to compare this against classical irken projection because I think that's you know glurk in projection is the gold standard for reduce or remodeling of fluid systems what you see here in white is the a proxy for the drag coefficient the drag of this of this cylinder starting from that unstable equilibrium as it goes into steady state and this is what the galerkin's model does for three modes and seven modes okay so it gets the right qualitative behavior but there's a pretty big overshoot it doesn't have the right steady state and the growth rate is wrong and if you plot it in PID coordinates you can see that it's you know qualitatively the right dynamics but it's it's quantitatively off by quite a bit but if you include either these constrained or cubic Sindhi models you almost perfectly capture the growth rate the steady state behavior and there's there's no overshoot so this is really remarkably more accurate jaesi also applied this to the cavity flow and I believe since then to many many other flows but you can see that you get latane of lien better models with fewer fewer PID coefficients okay and they they capture more of the elements and something I thought was really cool is that when JC rewrote the equations that were discovered out you can essentially write it as this van der Pol oscillator so I think to date this is the simplest model for the cylinder that has been written down that captures all of the transient dynamics and it's just a it's just a van der Pol it's a nonlinear oscillator with a nonlinear damping coefficient that describes the the cylinder something else that's really cool is that you can apply this not just a p OD modes of the fluid but you can apply this to physical measurements like lift and drag so one of the big problems that we have in in fluid mechanics is the things we do in simulations and in the lab don't always translate over to Boeing and Airbus right in the lab or in a simulation I have this full flow field and I get to measure the whole thing I get to compute p OD modes you know I get these these global modes that describe the behavior of my system but in reality I don't actually know what my entire wake is because I don't have measurements what I had measurements of are either on the surface of my wing or maybe some acceleration or lift or drag something like that and so what we tried next was essentially on this flow instead of building cindy models on p OD of the whole wake it turns out it's possible to build these cindy models just on the lift coefficients and their derivatives so if all you had was the lift measurements in time for these three cylinders you can build a nonlinear model that remarkably is remarkably accurate in capturing those nonlinear dynamics for this system okay so this also fixes the problem that if you slightly changed this geometry or if you accelerated the flow to a slightly higher velocity all of these PRD modes would change but the lift and drag measurements are essentially intrinsic measurements those don't change if I change the geometry so in the past you couldn't build AGGA lurk and model for different geometry because your modes would change it would be apples and oranges but if you build your models only on lift measurements its apples and apples even if you change your your geometry okay so I thought that was pretty pretty remarkable okay it looks like I'm running out of time here so I think I'm just going to to kind of end here with a couple of key themes just a summary we spend a lot of time thinking about how to discover models for our systems when we don't have any even when we do have them we still want reduced order models that we can use for prediction estimation and control so think about those last models I showed those are far far simpler than the full navier's stokes dns okay i'm modeling to ordinary differential equations is better than a PDE for for our purposes i didn't get to talk too much about how we handle non-linearity and coordinate transformations if you're interested please you know come talk to me afterwards and if you have any interesting fluid systems you know definitely talk to 2jc and me okay thank you all [Applause]

Channel: Institute for Pure & Applied Mathematics (IPAM)

Views: 1,349

Rating: 5 out of 5

Keywords: math, mathematics, ipam, ucla, steve brunton, machine learning, dynamical systems, data science, nonlinear models, fluid dynamics

Id: MDdkXFZJ7t4

Length: 52min 33sec (3153 seconds)

Published: Tue Nov 12 2019