AI/ML+Physics Part 1: Choosing what to model [Physics Informed Machine Learning]

welcome back so we are just starting off this lecture series on physics informed machine learning essentially how to uh improve machine learning when we know something about physics or how to learn physics uh from data using machine learning and in the last lecture we talked about how there are these five major stages of machine learning and all of them can benefit uh from embedding physics and we can actually use these stages to learn physics too so today's Le is really just going to zoom in on this first stage of the machine learning process which is deciding on a problem what it is we're actually trying to model um or what's our goal of our machine learning model you know at first I actually thought this was going to be the easiest video to make um because you know how complicated could it be how much could I actually say uh about how you know important deciding on the problem is and and where to embed physics but it turns out I've actually been spending the last couple of hours thinking about this and every time I think I have uh figured it out there's something new I want to tell you because this is actually maybe the most important out of all of these stages um this is where uh you know the scientific method really does still apply or the Engineering Process the design process still applies uh in this phase one even if you're going to use machine learning to build your model okay so this is you know the most foundational step in any kind of model building process is deciding what it actually is you want to model what's the purpose of of this model how are you going to use it um and so I'm actually going to back up and talk about how this uh fits into these other stages before then again diving back in uh just this deciding on a problem stage one so you know sometimes we have a pretty clear idea of the problem we're trying to solve maybe I you know I'm doing turbulence modeling and I know that there is a closure problem there are things like Reynold stresses that I don't understand I don't have a good mathematical model of how they behave but I have a bunch of data and I know that I'm going to try to model those you know uh quantities um better from my data but sometimes I don't know exactly what the problem is I have a vague notion like I Want to Build a Better race car or I want to build a better you know a better wing and that's not really a specific enough problem and so you have to go down the line and think okay well what data might I use to improve uh that process or to you know accomplish that big goal so you start breaking it down maybe you have a vague idea of the kinds of architectures that are good at those kinds of problems for the data you have but once you actually start you know really looking at what data you have access to what data is cheap or expensive how much of it you have and the architectures you might actually have to go back and refine the problem statement so very often we find oursel going down uh this path even all the way down you youve trained your model and you realize it doesn't do exactly what you wanted it to or it's not exactly the right performance or it's not useful in the way you thought but you learned a lot in that process and you go back and you refine your problem statement now we know better the thing we're trying to model so again I really really want to emphasize in classic engineering and science uh we're used to using experiments and numerical simulations to you know either design a new system like an iPhone or a race car or you know an aircraft we're used to using simulations and experiments in that engineering design process uh and also in the scientific discovery process this really isn't that different the same way you would design an experiment to answer some question that you need to know uh for some bigger goal it's exactly the same way that you would design this kind of machine machine learning experiment okay so you're going to have some hypothesis I think I can model this phenomena maybe the lift uh as a function of different Wing shapes so I'm going to you know get the data I have actually build the model see if it worked I might have to tweak that experiment I might have to refine the problem statement so you know all of your intuition for engineering design uh and scientific discovery from the laboratory and from numerical experiments a lot of those same principles apply here okay good uh and there's a lot more I can say we're going to come back to this diagram at the end and we're going to revisit what are some of the lessons we learned what are some key themes uh on deciding on a good problem versus a bad problem okay but there is some trial and error it is a lot like you know building an experiment or building a simulation to start you know answering a question um but maybe we can do this better with data okay so we're going to zoom in on deciding on the problem uh and again you know in my first video I said said well if you are you know modeling some aspect of a physical system you're kind of inherently doing uh this you know intersection between physics and machine learning your machine learning models should be capturing some of that physics um but that's a little vague so I want to go a lot more in depth on the kinds of problems we might be looking at and some of their nuances uh in terms of you know is machine learning even the right tool for modeling those problems and we're going to get some kind of uh lessons along the way about good ways of setting up these problem statements okay so I think data science in general is in my view the process of asking and answering questions you know from a data Centric uh perspective so a lot of the history of science has been driven by the data the observations and this really isn't isn't that different okay um I I really like this quote and I think it's extremely apt for this section on you know what we're modeling what is the problem statement I actually got this slide from my friend uh Nathan Kutz who likes to give this in his um academic seminars and I think it really is uh exactly what we need to be thinking about so often people look at machine learning as uh the solution to a problem and you know a lot of times I see people just building machine learning models because they can and I think that is the wrong approach I think this Picasso quote really Nails it um computers are useless they can only give you answers and it's kind of the exact same story with machine learning okay these datadriven models are useless they can only you know model your data you have to be the one to decide what is the right model what is the right Fidelity what would be a useful model and how are you going to use it Downstream that's a human endeavor and once you figured out that really important piece then training the model you know is uh is not necessarily the challenging part so uh we're going to just go through some principles um before we we look at some examples I just want to be thinking about why do we need a machine learning model in the first place okay again it's going to be a expensive process time consuming you know that data wasn't cheap training those models is often expensive and timec consuming so you better understand why you need this machine learning model in the first place and how you're going to use it that's going to um dramatically kind of impact all of the downstream decisions and again we're not just talking about any old machine learning model we're really talking about machine learning models at the intersection of uh applying this for engineering and physics uh systems systems that are governed by physics um you know like designing materials or aircraft or modeling turbulence things like that so first um one of the areas where machine learning really is you know critically important and advancing our capabilities Beyond where they are without machine learning is learning new physics there are many many systems where we have a pretty good understanding of how the system works um you know we understand how fluid flows work we understand how pendula and mechanical systems work we have lots of governing equations for systems but there are tons of systems where we don't know the physics so if I think about um Neuroscience we don't actually have governing equations for the brain or an epidemiological system or even the climate system there are big missing chunks of physics uh that we don't know how to model as humans yet but we have tons of data and so maybe we can start using that data to improve and build on our first principles physics models to learn new physics that we've never been able to write down before that's a huge opportunity for machine learning and then you know maybe you can use those models just like you would have used a physics based model but you can apply this to a much more complicated system where humans haven't been able to you know discover that physics like f equals ma okay so actually a lot of um the work that that my lab does and our collaborators groups also is in this realm of of discovering new physics with machine learning but that's just one of many uh uses of a machine learning model another reason you might use a machine learning model or build a machine machine learning model is you know maybe there is some system or some Physics that you actually do understand quite well we we know how to simulate it or or how to build experiments it's just really expensive so things like uh simulating fluid flows or simulating you know materials at all of the scales in principle uh you know we have the physics for that maybe simulating a protein folding or uh some you know biochemical molecule we understand first principles physics we technically can simulate at those but they're super super expensive so can we use machine learning to augment uh those models to help us build surrogate models that are faster and still accurate enough to do things like design and optimization um and things like that so can we accelerate drug Discovery or materials design or fluid simulations even when we know the physics uh because that physics is so expensive okay and I I put a little third bullet point down here on the bottom I think this is going to be really important later on in this uh lecture series machine learning models are often not always but often kind of inherently automatically differentiable meaning uh you can kind of compute the sensitivity of outputs to inputs uh and vice versa using you know back propagation things that we use to train the neural networks and the models you can often use that automatic differentiability of the machine learning model in classical engineering design and optimization Loops okay so if I want to do shape optimization of a wing uh or you know uh an automobile or something like that we might be able to use this automatic differentiation that we're already using to train things like neural networks we might be able to use that to do better faster cheaper uh design optimization okay you're going to hear a lot more about this later it's really really important uh and interesting uh okay so learning new physics that makes sense there's things we don't understand maybe we can learn models encapsulating or capturing existing expensive physics um totally makes sense that a machine learning model might help us do that um better and there's this kind of in between uh what I'm going to call capturing multiphysics interactions so a lot of times um we are good at modeling you know single scale physics with single type of physics we're we're good at modeling a single fluid or you know a planetary system system in orbit or you know a simple material but when you start adding complexity if you have multiphase flows or if you have you know you're you're simulating um the motion of the planets but they have tidal forces and oceans and you know multiphysics interactions that becomes really complicated really quickly um one of my favorite examples of this actually I learned um from tapio Schneider's group at Caltech um in climate science realm is cloud physics so the Way That clouds form the way that ice crystals and condensation and precipitation and all of these physical processes that go into a simple Cloud what we think of as just one thing it's an incredible multiscale multiphysics process that we aren't that close to knowing how to model using first principles and so we have some you know uh existing models and there's some parts of the physics we don't know capturing these multiphysics interactions this complexity that is beyond our um you know at least our our past or historical ability to capture with analytic methods or even simulations that's something machine learning is actually uh helping us do okay so um simulating or building experiments for these really complex multiphysics systems has typically been uh out of our reach machine learning offers some ways of helping so some ways you might do this are you know if there's a multiphysics interactions where you understand part of it you might uh use that physics model where you understand part of it and you might model the discrepancy or the difference with machine learning again we'll talk a lot about uh all of these things later and then the last Point um is we often also have the opportunity with when we have machine learning models that we can update these models with new data okay so f equals ma doesn't change right like when I model uh a pendulum in my lab that equation doesn't really change but as that system that physical system ages over time maybe the bearings get a little uh less lubricated maybe you know there's a fan blowing one day and not the other maybe it's you know the height of Summer and the temperature is a little bit different if you're using machine learning there is an opportunity to update those models with new data with new information uh and that's a really exciting uh opportunity for lots of things like data simulation uh digital twin modeling of systems that might actually change in time so all of these are just things that we should be keeping in mind why do we need a machine learning model is it because our physics is unknown expensive really really complicated or you know changing uh those are all things we want to think about good so let's go into some examples of the kinds of things I'm talking about here uh and then look at some takeaways so um one of the tasks that comes up a lot in the physical sciences is this task of super resolution so super resolution is an idea from image Sciences where you have a low reses image uh and with kind of the statistical information you've collected over lots and lots of data you might be able to learn how to fill in that low reses image into a highres image this is super important for physics applications this can help us uh improve experimental measurement techniques if I have limited experimental measurement Fidelity I might be able to increase the resolution uh using these techniques it can accelerate uh numerical computations maybe I can simulate things on a course grid and use ideas from super resolution to speed up High Fidelity simulations and give you higher resolution people are using the these ideas in uh climate simulations today uh and fluid simulations so this is you know one of the kinds of ideas of a physical type problem you might write down I would and the way you know I would think about this is if my task is to take low reses data and fill in highres details I already know that my training data I'm going to need lots of pairs of highr low res data to train that model okay so I can't do that if I don't have the high res data for training good uh and other things you should think about are is you know I know that this is a physical system this is a jet uh you know a jet of of fluid are are there ways of doing the super resolution task that actually respect the physics that respect Mass conservation or momentum conservation that if I took a little patch and I did my boundary element you know kind of uh conservation calculation it would actually make sense with physics can I improve the super resolution by baking in that physics okay those are all questions that really honestly as a community we still trying to to figure out but it is pressing if I'm going to do super resolution and use this flow for you know some Downstream task I would like this flow to be actually solving the Navy your Stokes equations okay that would be important to me as an engineer um other tasks again we said discovering new physics that's um a huge opportunity so if I took measurements of the night sky you know the planets and and things like that could I actually discover either Kepler's laws or even better um Newton's Second Law f equals ma can I discover physics from observational data um we're going to have a whole section on how to do this and how to discover interpretable models like this so a ton of information I'll probably put links either in the description or like clickable links above and you know it's not just rediscovering things we know like f equals ma um what if there is something that we can't explain in our data can we start discovering new physics to explain discrepancies in our data uh and so this happened you know in general relativity right there was this observation of in the the the um Transit of mercury there was a small discrepancy that was not explained by f equals ma but it is explained by you know these Corrections uh given to us by relativity so there are lots of examples of that today where maybe I'm uh I'm dealing with a plasma in a spherical you know Fusion reactor and my model is not perfect but I have data can I discover the new physics that improves my model and allows me to do better uh control and design and things like that okay so design uh discovering new physics is a problem statement in a sense um materials Discovery is another big one can I do I want to accelerate uh the materials Discovery process do I want to characterize materials using machine learning do I want to use machine learning to uh find a better design space to parameterize this is a very very high-dimensional uh optimization space when you're designing a new material there's lots of things you can tweak so maybe machine learning can help us learn kind of low dimensional patterns that simplify our search as humans um lots of opportunities for machine learning and and materials Discovery um another one I think is just super important and fascinating is in uh computational biochemistry or computational chemistry at large so things like understanding protein folding uh drug design for better drug discovery understanding combustion processes and other you know chemical reactions computational chemistry and computational biochemistry in particular is an area that has been rapidly Advanced with machine learning techniques um so I don't know there's a lot to unpack here um but but again there's some tasks that you should be thinking about what is the actual use of your machine learning model often times what researchers are trying to do is they're trying to accelerate existing uh High Fidelity codes that they would use to simulate you know these big molecules over many many time scales really really fast time scales really really longtime scales and they're using machine learning to essentially build surrogates to speed up those computations that can give us you know faster drug Discovery faster um uh protein design and things like that and there have been some pretty big success stories and things like developing the covid vaccine this is not a picture of Co but um you know developing the co vacine that relied on these kind of accelerated uh computational Frameworks so lots and lots of stuff there and again you know we know that their system govern is governed by physics there are conservation laws and quantum mechanics and rules that govern the physics you want your machine learning models also to uh respect those physics um digital Twins and discrepancy modeling uh we're going to talk a lot about this in a lot of contexts but again if I have some physical asset I might want a digital representation of how it behaves for a lot of reasons maybe I want to control that object and I need some model that that allows me to do model predictive control that would be a great use of a machine learning model maybe I want to understand how that um how that asset ages over time uh and I want my model to be updated with data that's another great use of machine learning for this this kind of digital twin um discrepancy model is another one maybe I have a first principles physics model for this robot but it seems to disagree sometimes because this robot has you know nonlinear bearing chatter there's wind resistance there's all kinds of things that would cause small discrepancies between the actual device and your idealized physics model so we might be able to model that discrepancy and close the loop and make our digital uh our digital twin more accurate using a machine learning model um other things I think you know we're going to talk about a lot and are going to be really really important are things like shape optimization um shape optimization is a really important really hard engineering task in a ton of fields um you know building artificial hearts or surgical interventions for cardiovascular flows um designing you know a wing for an aircraft or um you know the body of a Formula 1 race car or you know maybe you just want to make your shipping uh vessel that's going to go across the Pacific more fuel efficient so that we're burning less you know less uh producing less carbon dioxide in many many cases we're trying to optimize the shape of an object that's traveling through a fluid for some kind of an optimization goal you know maybe we're trying to maximize lift and minimize drag or you know maximize downforce things like that um sometimes our optimization goals are relatively straightforward and sometimes they're very very complicated um so you know this is an example of a really complicated multi-objective optimization problem where again you might be thinking to yourself big picture I want to design a better Wing okay well what does better mean List what you mean by better more fuel efficient um you know more flexible whatever your criteria are you need to Define what you mean by better and then what is the thing that is challenging today about designing a wing is it because it's really expensive to simulate the fluid flow is there physics that's missing you know what is the thing that's holding us back today from doing this optimization that we want to do and that is a good place we can start focusing our machine learning models so for example if we can build better faster fluid simulators we can just take that module and plug it into this shape optimization design uh procedure there's a lot that we're going to unpack here we're going to talk a lot more about this um but in general you know shape optimization is a huge huge problem in many many Industries think about designing a wind turbine or um you know a bullet train all of these are our shape optimization problems at the end of the day but again it's not just shape optimization it's not just having high lift over drag there's all of these other factors you need to think about you need to think about what materials I actually have to build it what would be its weight how reliable is it is maintenance going to be you know uh easy or hard so the real world we're dealing with multi-objective optimization problems and again that's something in principle machine learning can help us with um I can build a model that spits out lots of of figures of Merit or quantities of interest and I can use that machine learning model as a surrogate to do this kind of complex multi-objective optimization over and if my objective function changes let's say 5 years from now the price of fuel is a lot different than it is today or we have a material that's you know a lot more performant in one dimension you can tweak your objective function and you can still optimize over that machine learning surrogate model so this is essentially uh the idea of a digital twin okay so we're going to talk a lot we're going to have like a whole section on digital twin modeling using these physics and form machine learning models um and the idea here is is that the digital twin uh comprises a hierarchy of models at different fidelities some will be you know really crude simulations from a traditional you know simulation code some will be uh High Fidelity models that are really really expensive and we might have these machine learning models kind of in the middle uh stitching together these different fidelities okay so we're going to have hybrid physics and machine learning models in the digital twin um and that really means that your your machine learning models need to satisfy physics they need to be able to like actually work with physics models so again we're going to talk a lot more about this but when we were talking about shape optimization and multi-objective design optimization the digital twin gives you a lot of um flexibility in doing that and again I want to point out this third bullet um it's a subtle point that people often miss but it's one of the most important parts um of bringing machine learning into this modeling process is if we have a machine learning model like a neural network to describe some piece of this digital twin the same kind of automatic differentiability of that neural network that we use to train the model to back propagate um you know loss function signals to to tweak the parameters of that model we can also use that in the actual optimization process if we're trying to optimize this aircraft or if I'm trying to optimize a wind turbine if my model of that is automatically differentiable like most machine learning models are I might be able to do this kind of adjoint free um optimization easier optimization uh cheaper okay so we're going to talk a lot about that uh too okay but again you need to think about what your ultimate goal is is it a design goal is it a modeling goal is it just to speed up your simulation for some Downstream purpose all of those matter um and we're going to talk about this in the next lecture on on um you know where the data actually comes from kind of curating the data but roughly speaking if I'm actually designing a complex system like a new you know aircraft Wing or an automobile or a wind turbine in reality I don't just have one data source I'm going to have lots of different fidelities that probably are more and more expensive as I go up this uh pyramid so I might have a lot of low Fidelity simulations less High Fidelity simulations even less you know expensive labor laboratory me uh measurements and eventually I'm actually going to build the thing and fly it or you know actually collect this uh in the field and those are going to be the most expensive so again the goal of this digital twin maybe maybe what I would want to do is build a surrogate model that takes in this data and can kind of predict how that system will behave in different use cases in different scenarios because then instead of spending a lot of money to actually tweak my system over here and do all of this testing if my surrogate model is good enough I might be able to do a lot of the optimization in this kind of virtual world okay so that's one of the the the things we might want to do with a machine learning model and kind of the picture of what the benefit of this would be is that you know you have all of these different sources of data at different you know some are more expensive and more accurate some are less expensive but more error and you would hope that your surrogate machine learning model can take all of that data and build a model that's a little bit of The Best of Both Worlds you know low cost lower cost and lower error so that's the kind of of thing that a good machine learning model uh can do for us okay uh and you can use this for things like shape optimization or materials design things like that okay good um so let's get a little more Technical and a little more ma mathematical right so we are almost entirely going to be dealing with with systems that vary in time that's not always the case um you know in materials Discovery maybe that's not a Time varying process although to make a composite you have to bake it um you know in an autoclave and that is a Time varying dynamical process but if I'm using machine learning for example to simulate the climate or to simulate a turbulent fluid flow or the flow over a wing or an automobile at the end of the day it's a dynamical system it's a differential equation there's some differential equation that probably governs that process this is just a cartoon of the Lorent system uh with a little cube of of initial conditions in red and as I propagate it you see that those initial conditions kind of spread out it's an illustration of chaos and so if I'm trying to model this system again with machine learning I have a lot of choices to make that we don't often talk about you know am I trying to actually model the right hand side of a differential equation maybe I can measure the state of the system you know the X Y and Z coordinate in time do I want my machine learning model to build a model representation of this function f ofx so I can you know integrate that machine learning model and predict the future um and do I want it to be in discreet time a continuous differential equation or do I want it to be a discrete time stepper so that if I have my point uh at time t or time K here I can build a machine learning model capital F that will step It Forward one discret delta T in the future and then another delta T and another delta T and another delta T so even that is a is a choice you have to make and there's a huge uh difference between these so the entire resnet architecture lives over here and the neural OD architecture lives over here they're solving related but different problems okay learning the the differential equation that governs your data is a machine learning problem so sometimes that's what we're trying to do um or you know maybe I know that this is a mechanical system this one isn't but maybe maybe the system I'm modeling is a mechanical system maybe I know that it's governed by you know uh fals ma Force equals mass time acceleration so I know that my acceleration is you know minus the gradient of some potential function maybe my ma machine learning task is to learn that potential function V so instead of learning the differential equation maybe I'm learning the potential that describes the system you know in the computational chemistry maybe I'm trying to learn a free energy potential or something like that okay maybe I'm trying to learn a hamiltonian I know my system's hamiltonian so I want to learn the hamiltonian or I want to learn the lonian all of those are problem statements that have unique advantages uh and disadvantages based on you know what we know about the physics okay and in this case I actually think this is a really good example modeling the system as it evolves forward in time might be useful but because the system is chaotic if I have a small uncertainty in my initial Condition it's going to massively amplify uh in the future so getting a deterministic prediction of the behavior of the system forward in time might be too much to ask for maybe what I want is a probability of being at some location in a future time given some initial condition so this again it's a lot like the weather versus climate if I want to met model the weather if I want to know what the weather is going to be like 5 days from now that is one trajectory and I have to get that one trajectory right but if I want to know what the climate's going to be like 50 years from now I'm looking at probabilities distributions uh that go through this dynamical system so again we're going to talk about all of this more but these are really getting to questions like uncertainty quantification uh and chaotic Dynamics need different descriptions and different models I might be trying to model probabilities instead of deterministic Dynamics in some systems um good so yeah I mean that's essentially what I just said chaos changes the kinds of problems you can and and can't answer so if you know your system's chaotic that should change the way that you set up your problem statement predicting the exact trajectory of your system a 100 years from now is a silly thing to try to model with any method with simulations with experiments or with machine learning instead maybe we should be trying to learn the probabilities or the distribu utions things like that okay so you got to think about you know knowledge of the physics of your system really does tell you a lot about how you can model it with a machine learning model machine learning is not magic it's not going to allow you to do things that are just mathematically impossible like predict the future of a chaotic system well into the future deterministically um and again you know climate modeling is a really good example a ton of effort is going into using machine learning to improve climate modeling um by speeding up the fluid simulations by improving our uncertainty estimates um by propagating probability densities for there's lots and lots of different aspects to The Climate modeling problem I pulled this from uh the clima X group at Microsoft where you can kind of see there are all of these different tasks there's downscaling there's now casting there's you know subseasonal predictions and all of these tasks have different data requirements different architectures and different things you can ask your machine learning model to do so knowing the physics of the system still matters a lot uh in getting you know a good uh a good machine learning model um which really points to the need for these Benchmark systems so I've already mentioned this and and again we're going to have a longer series on Benchmark systems we need to have uh Benchmark systems kind of like image net for you know image classification we need similar benchmark works for dynamical systems for engineering systems for Control Systems so that we can test our machine learning models and make sure that they're actually um solving the tasks we want them to be and to learn which tasks are good with Which models and vice versa what problems can we ask for a fluid or for a mechanical system things like that um and you know I think a lot about fluid mechanics it's my you know domain of of expertise um I wrote a a paper with Ricardo in USA from kth um he took the lead on this which was essentially looking at you know how machine learning can help us with computational fluid mechanics so just like we were talking about you know we can accelerate simulations that's the kind of task we might want we might want to improve turbulence modeling um things that we don't know how to model in turbulence uh we might want to discover new physics or reduced order models there are all of these different flavors of the ways that you can approach a problem like modeling a fluid with a with a computer where machine learning can help um so one of my absolute favorite examples of this is this uh classic now I mean in machine learning a paper that's you know eight years old is a classic classic paper by Julia Ling where uh in her collaborators where they are modeling a turbulence closure problem so there's a turbulence model called Reynolds average nav Stokes it's here and these Reynold dresses in yellow we don't have really really good models of those terms we have to approximate them that's the closure problem we have to approximate them from things we can measure and so you know Julia and her collaborators essentially realized this is a really good opportunity to use machine learning this is something that is hard to model but we have a lot of data and so we can start focusing our machine learning model to predict these quantities and kind of by construction it is going to be a physical model because it's modeling something that we know physically how it enters an equation and they went a step further and actually constrained their architecture to make sure that their predictions are you know satisfy certain symmetries that the physics is known to satisfy so this is one of my favorite papers and kind of the moral story here that I really like to relay to my students is turbulence closure modeling Reynolds average Navy or Stokes modeling is something where you know we had a huge amount of progress up in until you know the 1970s and 80s a lot of analytic first principles physics prog progress of of what these these terms had to mean what symmetries they adhered to and at some point researchers hit a wall because these are really nasty you know functions to approximate and so that was difficult for years um you know people just with pencil and paper couldn't go any further that's a great example of a problem where machine learning could really be helpful where a lot of smart people got us to a certain point where we know a lot of partial knowledge of the physics things like symmetries that need to be satisfied but the actual expression was so nasty that they couldn't you know write them down in closed form with pencil and paper that's the perfect example of what machine learning is good at because it's essentially gives you this ability to approximate arbitrarily nasty functions with enough data so these really nasty functions we can get a neural network representation that's very accurate and satisfies those symmetries so this is one of my favorite examples of the kind of thing you can do and we already mentioned the super resolution problem super resolution in principle can help you you know with faster turbulence modeling using things like large Eddy simulations which are a main stay in you know High Fidelity fluid simulations and climate simulations Okay um as much as fun as it is to talk about examples of where this is you know machine learning is is uh useful in solving these problems it is as important or more important to talk about when you should not use machine learning to solve a problem okay so I see this all the time um I'm not going to you know name names and there's also no um judgment or or shame like we all are guilty of this but often times we'll take a system where there is a perfectly good easy simple model like I can model this fluid flow with a simple uh two-dimensional linear system of equations using a really easy method like Dynamic mode decomposition which you could argue if that's machine learning or not but you know like simple machine learning can describe the system I've seen papers where researchers model the system using a massive deep neural network this thing you know might have a million free parameters it takes forever to train you need tons of data like yes it can model the evolution of this fluid flow but it's massive Overkill okay uh it just doesn't make any sense to to use that tool for this problem so it's really important also to to ask yourself do I need machine learning for this problem is my physics based model good enough is there a simpler method would linear regression work if linear regression works please use linear regression now I don't know if you get uh ads for like you know pelaton exercise bikes or um you know those new smart fitness centers that are powered by AI I don't want to you know uh be held to this but my guess is that a lot of that powered by AI is actually just linear regression um anyway when to use and not use machine learning is is as important as setting up the problem um and again there's a lot of parallels here um before we knew astronomy before we knew about Kepler's laws and Newton's laws and you know Einstein's relativity before we actually had physics that described astronomy there was a lot of bad astrology essentially people were asking the wrong questions they were trying to use their data to model things that couldn't be modeled to solve the wrong problem things like you know should I get married on this day or am I going to be unlucky should I stay at home like does the motion of the planets really affect those things I mean I don't really believe that now knowing about f equals ma but there absolutely are problems that the motion that the data we have does impact um it does tell us when to launch a rocket to get to ours it does tell us when is a good time to sew our crops so that we get maximum yield based on the seasons there are lots of useful problems that this data can model but there's a lot of silly problems it can model too so you don't want to be in the crystal Energy Group you want to be doing you know actual physics with your machine learning models good okay so that's uh essentially just the first layer of this this is the tip of the iceberg we haven't even gotten into the data the architecture the loss function or the optimization and it's already getting interesting but what you're going to find is that you know this deciding the problem might be the most important stage and even if you start with a problem and you start going down this pipeline you're probably going to have to go up and at least tweak the problem tweak the data this is not just a linear process there are couplings between all of these stages these two are you know surprisingly linked there's a lot of links between deciding on the problem and what data you have you might have to refine the problem depending on what data you have access to and vice versa um but again you know really really important and you should be thinking about not throwing out everything we know about physics and engineering design the same way we would try to solve a physical problem or to understand something we don't understand in physics the same way we would design you know an engineering system we need those same principles when we decide on a problem of what to model with machine learning okay thank you

Channel: Steve Brunton

Views: 69,664

Keywords:

Id: ARMk955pGbg

Length: 43min 26sec (2606 seconds)

Published: Fri Feb 23 2024