Machine Learning Control: Tuning a PID Controller with Genetic Algorithms

welcome back okay we're talking about machine learning control and in particular we're talking about how to use genetic algorithms to tune the parameters of a control log with a known or specified structure so here what we're gonna do is apply the genetic algorithm to tune a PID controller in MATLAB so here's just a quick recap of the diagram so I have some PID controller that I'm gonna wrap around my system to obtain some desired objective function I am representing the parameters of this PID controller the the proportional integral and derivative terms as this kind of genetic sequence of numbers and I'm going to try to find the parameters these parameters that optimize some objective function basically finding these hotspots where you get the best performance so we're gonna have to define an objective function we're going to have to construct a kind of control law with this the specified structure and then we're going to use the MATLAB GA command the genetic algorithm to tune these parameters using this this evolutionary strategy okay so that's what we're doing right now I want to caution you in real in real life you don't want to use a genetic algorithm to tune your PID controller there's lots of other easier better ways to tune this that would be less expensive and you know simpler but this is a demonstration of how you use the MATLAB GA command for some kind of a control optimization so this is an easy one where there's three parameters we want to optimize you could then take that exact code and apply that to your control system where you have parameters that you want to optimize that's not a PID controller okay so I don't want you to think that this is you know only useful or particularly useful for a PID controller you can do this for for anything where you know the basic structure and you're just trying to optimize the parameters so people have done this tons using genetic algorithms to optimize the parameters of a structured control law okay so we're actually going to do this in MATLAB we're going to cook this up in MATLAB and so what I'm going to do is a few things I'm going to define a system so I have to define some kind of transfer function I'm going to do this in the frequency domain so I'm gonna define a transfer function I also have to define some kind of an objective function this is kind of my J function it's a little hard to read but this is my my J function and then what I'm going to do is I'm going to wrap this GA command around this okay so that's what I'm gonna do so here's a script you can download this the code will be online this is in our book the book data driven science and engineering okay so I clear all close all CLC and to find a time step I defined some basic parameters of my genetic algorithm so I'm going to iterate for ten generations and each generation is going to have a population size of 25 individuals okay so I'm gonna have 25 individuals and I'm gonna enter 8 and advance that for 10 generations so I wanted to find my system that I'm going to be optimizing my PID around and I'm going to start with just a transfer function S as a transfer function s so this establishes my Laplace variable s in the frequency domain and my system is G equals 1 over s times s times s plus s plus 1 don't ask me why I chose this one it probably has some interesting properties so I have G equals 1 over s times s squared plus s plus 1 ok that's my transfer function that I'm going to use 1 over s times s squared plus s plus 1 that's G and now I have to define some objective function J so what I decided to do is kind of this interesting hybrid approach so I'm going to define my cost function J my objective function to be the lqr the linear quadratic regulator cost function and I'm going to then find PID control law that minimizes that lqr cost ok so this is kind of weird hybrid deal so here's the transfer function this is my dynamical system it's just some input/output system that has these dynamics and I have this function called PID test and what PID test does is it takes in the dynamics G the time step DT and a vector K of my control lock proportional integral derivative and it basically tests that control law on that dynamical system with this cost function that I'm going to define so let me show you what PID test n is so in PID test essentially again this function takes in the dynamic transfer function G a delta T and the parameters of my my PID controller and so basically it takes the first entry of the parameters and that's my proportional term the second entry of my parameters is my integral term so it's essentially this parameter is K proportional K integral K derivative okay D and so what I do is I take the first element and it's proportional so there's nothing multiplying it I take the second element and it's times 1 over s which is the transfer function for an integrator and the third term is you know KD times s that's the transfer function for a derivative now in MATLAB it won't let me build a transfer function and run this control loop with an actual derivative because that would be kind of unstable so what I do is I divide it by 1 plus something really really really small times s so that at very very very high frequencies this derivative term rolls off to zero and becomes unimportant okay so this is basically a derivative KD times s and then normalized by this really really really high frequency roll-off so this is basically just actually a high pass filter a high pass filter with a very high frequency okay so that's that's what I'm doing here in this PID test I take these parameters and I build a PID control law this last term here is just a little funky because derivatives are weird in MATLAB and then what I do is I build my my open-loop I take my my system G and my controller K and I build a series a series I have my system G I have my controller K and I build a transfer function from the left to the right and then what I do is I feed back on this loop like that okay so I build this feedback this is just nice two lines in MATLAB I build a series than a feedback and then what I do is I step the system my close loop system and I see what the step response is I want a nice fast rise time so when I step the system I want to go from zero to one as fast as possible with this little overshoot and ringing as possible okay and to quantify that what I'm going to do is I'm going to use an lqr cost function okay so this is my control transfer function I'm basically and also simulating I want to know what the control input was when I give this step I want to know what my control signal does because I'm going to penalize really big control and puts you and so I build this cost function J which basically adds up how how long it takes my state to get to the set point so basically the error of my tracking squared times Q plus how expensive it was u squared times R that's just the MATLAB way of doing this I take my vector of outputs and my vector of use and I dot square them and sum them up so that basically adds them up over time okay long story short what I'm doing is I have this system I want to wrap a feedback controller a PID controller around it and I want to tune the proportional integral and derivative terms to minimize this lqr cost function so I want really good state tracking I want to track this step as fast as possible with as little error as possible and I also want to do it with as little control input u as possible okay and then I have some plotting commands here again you can download all of this and play around with it yourself but basically I want to actually see what the step response looks like so we can plot on the screen for all of these these tests okay so almost everything up until this point has just been setting up the problem I set up the the transfer function the input output dynamics that I'm going to control I set up this function PID test that actually tests a PID controller and sees if it was a high J or a low J i want small j cost functions those are good controllers and then the GA command is super simple I run GA and I basically tell it what parameter I'm optimizing over so I take PID test and I wrap it so that it is only a function of K these three control parameters and I lock in G and delta T so now PA the genetic algorithm knows that all I'm doing is I'm running PID test and I'm trying to tweak these K parameters to get the best output of PID test possible and then there's some other commands here some other options that you can you can do the GA help and see what to do I think basically this is telling it that you know there's three variables that's optimizing over it gives it some initial condition and things like that okay so you can you can look into this yourself I also have an option for an output function called my fun my function which basically and every iteration and every every time you evaluate an individual and every time the generation advances I essentially save some internal variables so that I can plot them later so I'm in I'm plotting what is what are the three control parameters what's the system response what's the cost function J and things like that and eventually this gets dumped into a a file called history mat so I can load that and after the fact after I've run this genetic algorithm I can plot a bunch of things and diagnose how it learned the right controller okay so I kind of just want to recap there's a system I want to control there is a function that evaluates a controller and tells me was it good low J or was it bad hi J and all I do is I I plug that into the GA algorithm I tell it what to optimize this K parameter and it goes and I have to tell it you know how many individuals per generation and how many generations and I run it okay so I'm gonna hit go and what this is going to do is you're gonna see a bunch of step responses plotted to the screen and down here you're gonna see the cost function for each of those and we're just going to run through all of these ten generations twenty-five individuals each so we're gonna see two hundred and fifty control laws step responses and what we want when we do this step response in time what we want this is our set point we're trying to step up here I want this thing to go up as fast as possible and stay stay close to the set point what I don't want is huge oscillations that take a long time this would be a big state deviation that should give me a bad cost function and what will be even worse this is an unstable system so if I pick a bad PID controller what would be even worse is this this goes unstable okay so that would be a huge cost function integrated over ten time units okay so that's what we're gonna see is these kind of different canonical step responses for different PID control values and then genetic algorithms is going to optimize okay let's try it out I think we're ready to go let's hope there's no errors I'm gonna run this code okay so very quickly we should be seeing a bunch of step responses pop up so these are the step responses these are the cost functions down here you see a lot of them were unstable kind of blow up but some of them are okay and even after one generation you see many more of them are going to the set value of one and not blowing up you still have some unstable ones but now these cost functions are already getting a lot smaller now they're around one point one or two and only rarely do they get really big so this from generation to generation these control laws are getting better and better and better and finding better and better PID control gains and so we're just going to watch this evolve for a little bit now you see it's like kind of just jiggling around these really good solutions those are the the good and it's just you know mutating and crossing over trying to get slightly more optimal solutions and we found kind of this you know 1.4 seems like a pretty good minimum value it's not doing a lot lower than that okay it's done so super fast to evaluate if you're if it's if it's fast to evaluate a control law you can run through hundreds or thousands of these things and really find good optimizing parameters okay let's see what X and F Val what's X okay X is the optimizing so this will spit out the single best K the single best PID gains those are these and the best optimizing function value this is my cost function J so F Val the lowest it ever found was one point four seven six for these proportional integral and derivative terms so if I want I can just run PID test of G with delta T and this best PID control gains X these three values here and this is the best the best response apparently according to the lqr cost function according to minimizing the state deviation and the control cost this is the best response that genetic algorithm found okay now I don't know if this is a particularly good one or bad one you can definitely squash down this peak but it'll take longer to rise and apparently that costs more according to my cost function so this is only optimal with respect to the cost function that you select and maybe this was a bad cost function so that's very important is that these algorithms are no smarter than how you set the problem up so if you want different performance then you need to put in you know put that into your cost function so if I wanted to squash this peak down maybe I should put a penalty on a maximum peak one of the cool things about genetic algorithms you know the lqr cost function is is this quadratic smooth cost function and it was chosen that way because there are known optimal closed-form solutions but with genetic algorithms this can actually find solutions to non convex problems so I could put in constraints on you you can never be bigger than a value I could put in you know inequality constraints or I can basically penalize kind of the maximum value that this hits any number of things that would be very hard and non convex traditionally you can do in genetic algorithms okay so this is the best control law that was found pretty easy to run you know most of the effort and most of the time was actually just setting up the problem and the cost function right running the genetic algorithm was super fast and simple okay so in the next video we're going to load that history map file and actually start to see what this genetic algorithm did under the hood okay thank you

Channel: Steve Brunton

Views: 46,336

Keywords: Control theory, Machine learning, Data science, Dynamical systems, Machine learning control, Matlab, Optimization, Control, Model reduction, System identification, Reduced order models, Regression, Feedback control, Applied mathematics, Linear algebra, Balanced model reduction, balanced truncation, singular value decomposition, dimensionality reduction, Genetic algorithms, genetic programming, artificial intelligence

Id: S5C_z1nVaSg

Length: 16min 18sec (978 seconds)

Published: Sun Jun 10 2018