6.8210 Spring 2024 Lecture 1: Robot dynamics and model-based control

it's a beautiful machine that was going to go badly what's that uh so actually it does that um David who's is in our lab actually uh wrote a program that in fact at the major robotics conference you could it would take a picture of you turn it into a silhouette and you would draw your picture on a board and you could take it home with you yeah all right all right 235 welcome everybody this is underactuated robotics I hope you're in the right place um we're going to talk talk about things like walking robots in the class right so uh I want to give you an overview of the class and give you some of by the end of the class you should at least know what underactuated means that's sort of a you know a checko if uh if you don't know that then I've failed uh but I also want to dig in a little bit to robot Dynamics and uh you know show you some of the math we're going to use throughout the course and just even the philosophy of the course as well as getting some of those definitions across so uh just the administrative stuff quickly so I'm Russ um Jun is not here today Sava is here maybe stand up Sava uh and Pete is also here right here think we have an excellent teaching staff this time um most of the information is all on the website the the book is the website and from the book you can you'll click a link and you can see the the specific to this semester information for the website we have our email at underactuated staff if you haven't found it your way to the website already um you know please sign up for Patza that's the one action item the link is on the website it's also on the slide if you figured out how to get the slides.com um and watch this if you want to you know watch along uh you can use slides.com to watch along too okay yeah so then the textbook looks something like this some of you took manipulation with me last term um so it's meant to it's a I decided to write the notes in HTML because PDF just couldn't handle all the robot videos and the interactive visualizations and other things that I was trying to put into the content some features just to point out um almost every chapter has a a link to deepnote so you as you can follow along by sort of running the code that goes along with the examples that's just an online if you've used Google collab it's a lot like Google collab it supports the class a little better this here is the link that goes to the course website a course being taught at MIT so that's going to be this year our goal is to put all the videos onto YouTube they're going to be immediately on panopto so if you wanted to you know watch live from home or watch it immediately then you could watch it on panopto but it'll be posted on YouTube a day or two later after we we just have to do a med minor minor editing step Quincy knows that that well he was he he carried the weight last semester to do that uh okay and the other thing is there's um like random highlights you'll see okay someone highlighted and said hey I'm from Michigan too uh that's cool um but it's meant for um you know highlighting and asking technical questions uh also right uh so this is a sort of an online living Google doc kind of thing if you have a question something that's not clear about the notes feel free to just highlight ask the question uh and I will try to answer that's one of many places is you can try to get a hold of me and the staff to ask questions and you'll see lots of people have asked questions um over the years and that's fun and then um you can actually also if you want to use that if you find that useful you can make an account and make a private posting for yourself and just keep your notes there too I'm not sure if that's useful or not good and then um you know the course website has the administrative details for this I won't hand out paper and we stopped doing that a while ago but um we're or all of the office hours everything will be found there one thing I want to explain because I'm trying to get a little differently this time the grading distribution um we're going to do something I've heard has worked well in other venues okay so you will automatically whatever is best for your grade be graded in one of two ways one of them has an attendance and participation 5% okay so if you're here and you're participating and you're asking questions on Patza then um then you'll get that 5% if you think you just know the material you don't need it then you don't you can just put that 5% into the midterm right so if you feel like I could i ace the midterm I don't need to sit in class fine you don't have to sit in class and uh and and that's that's the trade-off I've heard that's pretty good I appreciate your feedback if you guys hate that or love that um but that's something I've heard has worked well in a few other classes and all of our other uh um you know policies about collaboration about generative AI all these all the standard things are on the website so please take a look okay so uh let's start to dig in the first thing I want to make sure I say again and again is that robots are awesome you are living in the best possible time to be thinking about robotics they have we are in the Golden Age of Robotics right the fact that uh you can just walk around campus and see these things walking around right um and it's not the exception anymore there's been a massive proliferation of beautiful robot Hardware companies that are building Hardware trying to build an entire ecosystem around this Hardware it's uh it's I've never seen such an exciting time the the entry of course of well first computer vision started working that made a lot of things possible large language models came on the scene that's awesome suddenly we can ask we can start you know asking how do I make a pizza give me step-by-step instructions tell me how my robot should make a pizza right that's amazing and you know the foundation model philosophy of GPT for instance is coming into robotics pretty fast okay so you're living in a beautiful time to be playing with robots uh right so what is this class about roughly this class is about making Atlas dance H it's a little bit more than that but there's that's kind of a theme is that we're going to try to understand the math behind this because it's a very general recipe um for for making robots do things like this okay uh it's like sort of awesome for me over the years to have been working on this and just you know watched these things come to life it's just incredible right this is you know Atlas doing parkour you've probably all seen this if you excited to thing the class but the the capabilities of this Hardware of the software are just like absolutely phenomenal and there's a lot that goes on behind them and there's a lot of things we can do to make it better still so by the end of the class you should roughly be understanding what's happening in these amazing videos okay but that's not the only so so the atlas control system is almost sort of famously now it doesn't use much learning okay it uses learning for perception but it's got a it uses equations of motion from from Newton to do a lot of the plan online planning you know contrast that with uh animal and a lot of the quadrupeds these days spot is a mix I would say these days um they're using a lot more learning right so this is sort of the reinforcement learning uh success story of the last few years and now it's spread we see lots and lots of quadrupeds using similarly successful reinforcement learning pipelines and in other areas um for instance when you get to some dextrous manipulation there's other types of learning that are working well right so imitation learning in this particular example is we're watching robots that can do I'd say surprisingly dexterous things with their hands like spreading peanut butter on toast and peeling potatoes and turning the pages of a book right these are things that just weren't possible a few years ago and uh you know we're going to try to understand a bit of about what makes those things tick this is actually the my favorite version of that right so this is the robot was trained to like a particular page of the book okay which it Knows by its picture and if you flip the pages of the book it'll flip the pages back that's all it does all day long if you flip the pages it'll flip it back but it has you know it has the understanding that if it sees a particular picture that was to the left and I should flip this way and if it was sees a different picture that was to the right and I should flip the other way right and it does the task pretty reliably and it's a non-trivial Dynamics and control problem okay so the goals for the course is that is spot too loud slightly easier for me to leave it on but if it's loud I can turn it off it's fine okay um so the goals for the course and if you've taken manipulation you you'll see this is a fairly different set of goals but we're going to talk a lot about Dynamics okay Dynamics and control uh we're going to talk about optimization we talk about some types of optimization in in the manipulation class but we get much more into some of the details of of convex optimization and how it applies to dynamical systems in this class if you've taken classes on control this will complement those but this is going to give you the nonlinear control and the optimization based view of control and then the machine learning view of control right so this is about dynamical systems control uh achieved through optimization and machine learning the way I try to go through that um I really try to capture sort of in each you know in each sort of when we introduce a new algorithm we try to um do rigorous thinking about a model system so for instance when we introduce walking robots you'll see that we can actually introduce extremely simple walking robot models starts off actually with a rimless wheel as the simplest model of walking and then I can out of a two-link walker and a three-link walker okay we be able to derive the equations of motion we'll be able to understand to some extent their nonlinear Dynamics they're going to have stable limit Cycles they're going to have hybrid impacts when the foot hits the ground okay we're going to be able to understand that and then we're going to think about how to design control systems that can increase the stability of the region of you know increase the region of Attraction of a limit cycle these are the kind of tools we'll be developing okay and you'll have a toolbox of of algorithms from optimization and and learning when you're done I get a lot of questions about what do you need to to have to take the class okay um and I it's it's a little bit tricky in the sense that I try to make the class accessible to course six course two course 16 those are the dominant we have some NIH folks um and that is explicitly the goal of the class is to be able to teach some Dynamics and control to maybe people that have thought more about software in their life and to bring in some optimization and and learning to people that have maybe thought a little bit more about Dynamics so because of that I try to bring in uh I try to not make a lot of prerequisites on either side but that does not mean it's an easy class uh we actually we use linear algebra heavily dyq heavily when people feel behind it's typically because they're not comfortable with the sort of they don't have maybe the maturity of the basic linear algebra DQ sort of tools okay we use Python heavily that's going to be important okay and I do use so I don't assume you've taken a machine learning class okay but I do I will introduce neural network jargon right uh as we go so if you have a comfort with using p torch and the like then that would be great if not you'll I'll try to keep links to to sort of uh drop in some of that language as I introduce it okay uh I would say that yeah of these things that I try to not assume you have I think at the end of the term the way the people that most wish they studied something it would probably optimization there's so much you can know about optimization and I can only introduce so much of it in the class so um that's one place if you're going to spend a little you know time studying on the side I would I would probably grab a book on optimization and there's some references on the website okay so let's let's dig in a little bit so I said it quickly but I like kind of putting up those three examples right so right now Atlas which is the Boston Dynamics humanoid is mostly using online planning okay with models from mechanics so it's sort of mechanics plus optimization animal okay and I'll think a lot of quadrip heads and this is true of a lot of of bipeds if you watch uh Tesla's biped walking around and you've watched enough Walking robots for that's not an RL policy that's a zmp policy walking that that thing around I don't know that I don't have any inside information I'm just saying that's what it looks like to me all right animal and most quadrupeds these days they started off and a lot of them have an underlying controller spot's main controller is going to be a lot like atlas's main controller okay but we've seen a lot of movement towards reinforcement learning as a successful strategy for handing handling quadrupeds especially when the terrain gets dicey and it's closer to the limits of performance okay and they're dealing with a lot of uncertainty in the world okay and then manipulation this is a gross oversimplification of a complicated Dynamic field but I would say that thing that's kind of winning right now in manipulation is imitation learning okay so it's interesting yeah go ahead right yeah I mean so so I in some sense I'm going to give you a semester long answer to that question okay so um there are definitely examples of reinforcement learning for bipeds okay but bipeds fall off fall over more more easily there's also another subtle thing which is um quadrupeds are easier to simulate okay Point contact uh on the ground is actually a pretty easy thing to to simulate simulate fast right so that that plays strongly to the RL workflows um and quadrupeds are sort of inherently stable uh if you have especially if you look at a quadruped that was built uh 10 years ago or more they were built very differently than the quadrupeds you see today from from BD and unry and the others okay uh what you see of these is they have very light distal Mass so very light legs and pretty big actuators at the hip so quadrupeds can basically not you know you can you can almost never fall down in a free in an open environment if you can move your leg arbitrarily fast so what happened is I think the actuators got fast enough that robots are got very stable and then you can start exploring the boundaries of your performance much more quickly it's harder to get a humanoid to have super fast like Dynamics okay not impossible some of the D you'll see some very low distal Mass bipeds too but kinematically they tend to be different yeah yeah yeah so we're going to talk so there's a lot of trajectory optimization okay that happens on this and that'll be one of the main topics in the course so I I I actually that the reason I put this up is not because I expect you to know the details of imitation learning or or reinforcement learning yet but just to say that there's subtle reasons that underly some of these things okay and by the end of the course you should really understand I think some of the subtle reasons why these are the the current state-of-the-art and it'll change maybe next time I teach the class okay but things like this that we were just talking about there's some subtle points about what works well for simulation you know what um that we want to get to the heart of okay and the other point I want to make here is that people often ask and I you know I've been on panels at conferences or whatever where they say you know which is better mechanics or reinforcement learning or reinforcement learning or imitation learning okay uh this is sort of you know physics based models versus pure learning for instance there's a lot of like this versus this and if you take away something from the the whole semester I want you to think that these things are actually a lot more similar then they are different really there's a mechanical system that's under test okay that's the the object of study is the is the physics that we have that underly our robot there's an optimization problem that we're trying to solve it can be described in terms of imitating a human or in terms of a cost function okay and we have many different ways to sort of optimize that function okay these are slightly different preferences on the way you do that optimization they give you a slightly different vocabulary for what you what's easy or hard in that optimization but the Crux of the class is understanding the optimization landscape and maybe why you would use one of these tools versus another okay but to do that really well you have to dig dig into the details I'm going to stop the robot it's a little noisy for me okay so let me say a similar thing a slightly different way okay um if you've seen a reinforcement learning talk or read a reinforcement learning book there's a diagram that everybody always makes we have an environment this is also the this is like the robot is included in the environment okay conceptually the outputs of that environment are the observations that we get to make okay so in the examples that we'll use in the class the observations might be joint um position measurements velocity measurements could also be cameras torque sensors contact load sensors okay on the other side you have your ctions coming in and this can be for instance motor torque commands or other actuator commands if you have different something other different on your robot okay and our goal is to build a controller I guess the RL folks would call it a policy I call it a policy too it's all good um synonym for that would be a controller okay that's doing sort of the the other half of that feedback loop right by the way um the controller you know is this is the maybe the controls name for it okay policy sometimes too the controls name for this is plant which is because control theory is old and it used to be we studied chemical plants okay but I will it is impossible for me to not say plant almost all the time okay so when I say plant that's the robot or the environment under test okay now you can do different um you know it might be that we we include the low-level controller that's running in the driver of spot inside this box or outside this box there's different modeling choices that could Define that boundary slightly differently but this is sort of the the big abstraction that everybody talks about in control and in reinforcement learning and in imitation learning now a lot of the learning based class if you're take if you're taking an RL class for instance we would try to make very few assumptions about the the equations that are underlying that this is the the in some sense the strength of reinforcement learning is that it's very general the environment could be the game of Go or it could be an Atari game or it could be Dota or something or it could be a robot right and the same algorithms should work in all of those settings roughly but this is not that class this is a different class okay we're going to try to open up the box we're going to say I care I don't I mean not that I don't care about go I didn't mean to say that but but I I care mostly about robots Okay so I want to look about the specific things that happen inside here when the the environment is given by the Dynamics of mechan of Newton right of L gr right I want to I want to understand specifically when it's a robot under there and the the equations are structured right energy is conserved you know all the all the types of things we know about mechanics and we'll continue to to learn about mechanics okay and that changes the optimization landscape partly I can often write much stronger algorithms than a general purpose if I make no assumptions about this then I'm limited in what algorithms I can I can use if I open it up and try to exploit the structure of the equations I can write stronger optimizations and I can write stronger guarantees and even when I go back and want to have a general algorithm that works for everything I can understand it maybe more rigorously by having examined it closely on an important class of problems okay so the class here compared to a typical RL or learning control kind of class really emphasizes a lot about the Dynamics um and the implications that has any questions about that didn't actually get as quiet as I hoped it would get think there's a harder off I can try motor power is cut I'll disconnect good boy okay so um different robots different um tasks that we might try to make our robot do are going to have different complexity in the optimization problem okay sometimes it's easy to write a control system for some robots some robots it's very hard to write a control system it's going to be similar when we're trying to use learning to do this okay there's sort of a complexity class if you will it's not as neat as in the of computation okay but there's sort of a sense for there's some things about your robot that can make it easier or harder to write a control system for okay so some examples for instance um if you have so maybe I'll I'll call this what makes control challenging or difficult okay Rich more positive okay if you're your observations are very noisy or if your they are um insufficient right U so if you have a lot of uncertainty or partial observability if you are trying to to you know solve a problem and you you're equipped with um sensors that only give you a partial view of what's happening okay that's one thing that can make make the control problem a lot more difficult or more Rich there's another thing which is there's a property of the Dynamics that can make things more Rich okay if your actions have long-term consequences what's an example of that if I were to you know Chuck this chalk right I took an action here here but the Chalk's going to keep going no matter what I do on the next time step right so I have long-term consequences of of the actions I took at the early time step so and if you've taken a class on sort of linear feedback control or something you know these are the questions about observability and these are the questions about controllability for instance and we'll have nonlinear generalizations of those we'll Define those carefully later okay um if your Dynamics or observations are nonlinear that can make things more Rich than uh than linear of course it's funny I used to always emphasize dimensionality like I used to say high dimensional here but that hurts some methods not all methods you know learning has really uh changed the perspective on what it means to be high dimensional right some things actually get easier again in very high Dimensions okay but for some of our algorithms dimensionality will be a a challenge if it gets to high dimensional so these two are sort of the topics that we're we're not going to talk as much about partial observability there's good classes on state of estimation and perception on campus okay but we're going to talk a lot about these uh Dynamics aspects yes yep good so the simplest one that that where the scaling works is it will typically have a state space representation of what's happening in here and the number of states in the model is what blows things up that often correlates with high actions High observations and stuff like this those complicate things in a different way but I think the fundamental one is the state Dimension I love questions by the way so feel free to question anything okay so that's a very abstract let's start using some of the sort of equations and Machinery that we're going to use in the class okay we're going to be working a a lot with differential equations in particular we're going to be working a lot with nonlinear differential equations unlike linear differential equations although there are some specific cases where you can solve them analytically our tools are mostly going to be numerical so if you think I haven't solved the nonlinear differential equation for a long time nobody has it's don't worry that that's not that prerequisite there's very few that you can solve okay but the language of differential equations is something that you'll get reminded of quickly uh and hopefully you have some background in okay so um I'll write a lot of equations that sort of take a form like this this would be my state Vector this would be my control input all of these are vectors okay so this is a a vector equation Vector valued function right you give me a vector X in a vector u in and I'll compute F and I get a vector out that vector VOR is the time derivative so when I write x dot right I think most people have seen it but that's like DDT of X right the time derivative of x this equation will often refer to as our Dynamics model we will spend a lot of time on the Dynamics part of this later we'll add an observation model that might look like this for instance would be the simple simplest form or Y is the um Vector of observations okay so um if you've taken your differential equations class then the a simple form of this you've probably seen is a linear differential equation which is just a specific example of that a particular choice of F I could say I could restrict F to be linear equations like this this would be a linear input differential equation right driven linear one you might have seen these in 1803 or if you've taken a signals and systems class or a feedback control class that's the the standard stuff of the of those classes our focus is um on these equations but we're going to typically not be able to get away with linear differential equations they're going to be nonlinear and they're going to be nonlinear in a particular form right we care a lot about mechanics first of all they're they're second order by Nature okay right Newton told us it was f equals ma which is an acceleration so that means I'll write a version of this we'll often write equations that look like this so This the except acceleration is a function is a vector function of my current positions and velocities okay so Q would be let's say the generalized positions this could be for instance the joint angles of my robot so if you give me a scalar number for each joint on spot that could be I stack them into a Vector that could be my Q The Joint velocities Q dot still my action input which could be a a motor torqus for instance that should determine my accelerations um this you should see that this is just a general uh or sorry a more specific version of this I could always write this in the more general form by just taking choosing to stack q and Q dot into a vector and then I would be able to say x dot equals I'll call it maybe F Bar of X just I have a different X U which would be Q Dot and then this F Q Q dot U right yes Q is the joint angles the generalized positions be my my list of joint angles this would be joint velocities joint accelerations let's see so when we're going to see a couple examples of like pendulums and the like that subscribe to this form and you'll see that Q is exactly the joint angles of a double pendulum for instance okay so these are the sort of um structure equations that we're going to work with a lot but again like I said we're going to dig in and try to leverage specific structure of the robot equations mechanics so I'll do that in in steps here the first thing is okay we said it's second order good it's actually even more it turns out the way that you enters into the equations is limited okay okay it turns out for basically every mechanics model If U is a torque okay then I could actually write it in a slightly more specific form okay so this is just that's even more specific than that it says I have to be able to write this in a way that you only enters in an apine way okay aine because if I were to set q and Q dot then the dependence of this on U is just a linear function of U if these two are given plus a constant linear plus a offset that's aine these are called control apine nonlinear systems and we're going to see it has even more structure than that but this structure is enough for me to give you an easy definition of underactuated okay so I had to work up to that at least okay let me give you the definition here I'll do it first u a system let me call this like equation one or something okay or system one that thing is fully actuated in the state q q dot if and only if F2 q q dot this is a matrix okay it Maps a vector to another Vector a matrix okay if this thing is full row rank that seems a little specific pretty fast Russ what did you just do there right but uh but that is actually a very useful definition okay and I'm gonna I'm going to try to argue that that is the thing that uh sort of makes control interesting when this F control gets a lot more interesting okay if this is full row rank then it has a right inverse that's the that's the key observation if F2 is invertible in the way we want it to be invertible then that's a much easier case for control than if it's not invertible because I'm going to be able to basically pick a u to do to make the robot do whatever I want yes oh good so I'm just tell I'm this is you have to take it on faith I haven't derived that I'm saying we're going to see a lot of robots and they're all going to I'm going to be able to write them in this form Y which is a more specific form than that it's a property of lran mechanics if torque is an input then it turns out that the the equations of motion are going to be linear in that torque impli yeah that's good um when we actually when Michael and I we used to work with to get with the same person before so um but uh a long time ago we debated and tried to look through if there was a really accepted general definition and uh it was a little messy we've got one i' got I put one in the not notes that we're that everybody's been happy with yes so I think it roughly goes with if you take the first derivative of the highest order basically take a linearization of this for a nonlinear system then you can Define underactuated in that way and I've never found a system that I was unhappy with that definition yeah but the you know all of our systems for the class are going to fit this form yes no that's a that's a great question is the question was is fully actuated the same as fully controllable you can be um underactuated but controllable in fact that's the interesting case controllability in the linear systems case means it might it might take time to achieve a goal you can't do it instantaneously but you can use torque over time to achieve the goal this is a question about can you instantaneously cause an acceleration let me just make sure this is um clear so if I have dimension of Q is we'll Define that to be M and dimension of U is n then F2 is a matrix right F given a Q and Q dot this thing U is a n ouble dot is m so this must be a matrix that size M by n so to say it's full row rank what I mean is the rank of this of that Matrix given a Q and A Q dot equals m okay that can only happen if n is at least as big as m it's a short fat Matrix it can't be full row rate but if you have lot if you have a big n you can you can have you know the the rank of your number of rows it might be less than the number of columns yeah that's a that we're going to handle that separately there's there's a separate case of If U has got limits on it for instance we're going to handle that separately good question let's just take this as arbitrary U for now okay so the rank of that Matrix cannot be more than M cannot be more than n either right um it's underactuated if the rank of F2 q q do is less than M so see I I I kind of got to depict this myself maybe but fully actuated is not really a word okay it's not clear if underactuated is a word but I just said come on I'm gonna name the book after it I'm gonna so there's no hype in an underactuated okay uh or or anything I I called that one word we're going to just run with that yes oh no sorry that's a one that was me like uh autoc completing from using gbt to write the word yeah that was your question okay good that's still one thank you yes in both cases yeah so in QQ dot thank you in both cases it's a condition that depends on your position and state but okay for many systems these properties hold for all Q or Q dot which we would call the system is underactuated yeah the fully actuated one is actually a little bit hard um you will call a fully actuated system even if there are some singular configurations that make it drop Brink but certainly for underactuated we'd say it's underactuated for all q and Q dot this is true yes yes sorry what uh this so this is saying I'm going to call the system the the the equation underactuated in a particular q and Q dot if for that particular one the rank is low and if for all Q I'm going to call the system not the state under actuated yeah um you can come up with quirky ones but I think the The Bu and large we're going to see it's it's going to hold for for almost all you can have systems that latch or clutch or other things that could somehow lose a degree of Freedom or gain a degree of Freedom those are the the ones you turn to okay let me try to convince you then that I mean I I think it's sort of clear but let me just kind of make the point if this Matrix is invertible and right inverse is what we need that's why we need the the row rank whole row rank okay then control is easy so let's make that super clear right so the name for this is feedback equivalence okay if you give me um a second order system that takes that form and if you tell me I'd like to achieve a certain acceleration okay then I can always apply a control input I write my controller to be the function F2 inverse right inverse times Q do D and I'll subtract this one fq Q dot this would be like you know this is what the the world gave me I'm going to type this into my controller I have to assume that I know F1 and F2 we'll we'll relax that assumption later in the class let's just assume we have a perfect knowledge of the Dynamics then here's something you could type into your controller as long as that inverse exists what does it do if I put this U up here F2 cancels this F2 I get a q double dot plus minus This this term cancels that term and the result is that I can follow exactly the acceleration that I wanted you give me any acceleration I can tell you the U that should that's that's what you'd expect just when I said that was invertible right you give me an acceleration I can give you a u yeah there's it so okay good so what this means is that we are now feedback equivalent to a double integrator so I can basically think of my system as just being this system and you still have to sort of take actions over time to get to the the origin in that system but we know everything about how to control that system okay it's called the double integrator that's why it's feedback equivalent so basically you give me any robot you put enough Motors on it and the Dynamics of the robot sort of don't matter if you're willing to run this controller it's as if you can just take the Dynamics erase them completely and write them as the Dynamics of a double integrator okay now you might want to not want to do that because it uses a lot of energy it's uh you know it could even be that inverse could get big or something right it could be very large torqus okay but it's so tempting that this is basically what robotics did for 50 years right and it's the reason that some you know that robots have have been limited honestly is that there's it's so tempting to be able to sort of write that form just to make the point here let me switch to code that a lot bigger so if I take the equations of motion which we'll derive in a second here of a single of a double pendulum okay and I simulate got to open my mesh cat instance here this is just the course uh notebook so you can just run this yourself okay I'm going to take the Dynamics of a double pendulum we'll I'll write that down in a second okay but that's just a standard two link robot let's assume I have Motors at both of the joints okay my claim is that if I have Motors at both of the joints then I can make that robot do anything I want and well it's only two links I can't make it act like a three link robot uh but I can make it do anything a two link robot is capable of so for instance I wrote a simple controller that basically took the twool link robot and I tried to make it act like a onelink robot so I took the Dynamics of the single pendulum I said act like that okay and that's just you can just drop in and make it act like whatever you want basically and then simulate okay make I can just convert the two- link robot into a one link robot effectively with feedback and even more maybe more is it more surprising I don't know I could have just chosen the gravity to be upside down so I can make it an upside down two link robot right and the math just all goes through right because I can just apply torques and I can can erase the Dynamics and put in whatever Dynamics I want and I've got a different robot okay so if I'm a walking robot and it's you know used to be kind of annoying that if you swing your leg forward there's like a lot of momentum that goes there and you have to worry about balance and stuff like that you could cancel the Dynamics of your swinging leg impose potentially with a lot of like potentially a lot of torque required a lot of energy required I could impose a different Dynamics and make it look like a a simpler robot yes uh it's it's I think it's partly limited by the motors but it's partly limited by the control philosophy right and that's one of the great things that's been happening now is that we're starting to see people break out of that so like Atlas doing flips is a big deal to me because we're not restricting ourselves in the same way we're using stronger optimization tools to break out of that old mold yeah that's true that's true but uh you could pick an arbitrary you could do like the you know Ministry of silly walks whatever if you've got a fully actuated robot then you can just program it in okay the question is are walking robots fully actuated okay the answer you know spoiler alert no except unless in the case you have a big flat foot and you keep that flat foot attached to the ground and you pretend there's no degree of Freedom between your foot and the ground then you start acting like a fully actuated robot that constrains the Motions you can take but then in that regime you can you can make yourself A Clockwork man right and that's how we that's how the field started okay so uh I mean you asked a question earlier about what happens if I have input limits right if I have torque limits then it's possible I can't execute this controller right and it is true that different things that can break um I can break the feedback equivalence idea with for instance input saturations so if I said that U had to be between -10 and 10 can't be arbitrarily big then depends if that controller ever goes near 11 right if it stays low for all the Motions I care about then those that input saturation doesn't affect me but if it starts asking me to produce torqus I can't produce that's going to run into problems you can have constraints like State constraints can interfere we'll talk we'll examine a few different types of the constraints but um a type of State constraint might be that my robot hand can't be inside the table right that would be a an inequality constraint that is limiting the number of the cues that I can take because the world won't let me enter the table or my foot can't go below the ground okay um model uncertainty will complicate this this derivation but we we'll see there's adaptive control approaches that can deal with it okay so input saturations actually by the generalized definition of underactuated would make a system underactuated okay um State constraints it depends how they enter it gets a little bit more subtle because they can be homic or non-holonomic um but but that can also complicate the derivations all these things basically make control interesting they mean that you have to reason about the long-term consequences of your actions right when you're fully actuated you can just erase things and act like clockwork when you're when you have any of these limitations you have to think about the long-term consequences of your actions and that's where control gets Rich okay side note just how many people have played with um like RL on the standard gym environments yeah so some of them you'll see things like acrobot and copper and stuff like this um you can make an underactuated system an acrobot in the gym environment has one less actuator so it's underactuated we'll see that we'll use this we'll study the acrobot if you red if you put very large torqus and have no limit on bandwidth then you can effectively turn in under actu if there's any coupling you can make it look fully actuated so there's some subtlety in these definitions okay this is this rigorous definition but you can make an underactuated system look almost fully actuated if you hack the parameters so some of those gym environments drive me crazy because they don't capture the essential dynamics of the robot that they were intended to model they were kind of tuned a little bit to make learning curves look good but they no longer resemble the physics that I that I love um some of them are great but some of them are are do not look physical to me cool okay so does that make sense is that the rough definition so now you know what underactuated means yeah so why why is a humanoid I I said the foot thing but maybe if I were to write the equations of motion for a humanoid it would be very natural to use Q as all of my joint angles but I also need some configuration that describes the location of my body in space right and even though I have many Motors or muscles like more than muscles than I have joints but I'm still so so in that case n is much bigger than M I have lots of parallel actuation in my muscles and my tendons and everything okay but still an a humanoid is underactuated because when I'm I can't immediately control the equations of motion of my center of mass as soon as I jump through the air assuming I don't have aerodynamic effects I'm going to take a ballistic trajectory nothing I can do with my M Motors makes me can control those degrees of freedom so that Matrix is low range again it gets more subtle if you have a big foot that you're willing to say is bolted to the ground that's the one case where a walking robot can look fully accurate but really the study of walking robots is the study of underactuated robots y so a classic example of a fully actuated robot would be like a robotic arm that's bolted to the table okay if I really I don't have a degree of Freedom between my table and my my arm and the table and I have joint joint joint motor motor motor then I can be fully actuated that's where a lot of our best you know adaptive controllers and other things work beautifully so it was natural for the community to to sort of consider the regime where you could think of yourself as a you know one of your feet bolted to the ground the there's criteria what we'll learn about which you can look at the center of pressure basically your instantaneous Center of pressure and if it's safely inside your foot then that assumption is sort of reasonable but it led us to build a lot of robots with big flat feet that kind of walk a little bit like this right because you want to you know instantaneously keep yourself as if you're like a robot arm there and then a different robot arm here and you can kind of use a lot of those control ideas okay so I will I was I had a Choose Your Own Adventure here I was could either derive this or not I will not um okay uh but it's I want you to know it's not scary you won't have to do it in class the software will always do it for you that's but you should know it's not scary how would I have derived it I could write a very simple equation that tells me where so this is a so what what is the picture first this is a double the double pendulum I just showed you L is the length of the link m is a point Mass that's the simplest way to derive the equations of motion is you just assume there's no Mass along the length only a point mass at each joint okay so I called the magnitude of that mass in kilograms M1 M2 The Joint angles are Theta 1 Theta 2 I can write the kinematics I can write the position of M1 is just you know L sin thet negative L cosine Theta basically very simple kinematics do the same thing for M2 I can write the kinetic energy it's just 12 mv^2 for M1 12 mv^2 for M2 that's simple I can write the potential energy that's just you know MGH MGH simple and then I can turn the crank of lran mechanics I put that into my lrange equation I take T minus u i take some partial derivatives and Boop what do I get out I get this out okay this is the sorry that's the L mechanics right and what do you see here okay so I I called tow the torqus at those joints what is the form I'm seeing here okay uh I see Q double dot on this side Q double dot actually always enters linearly every time you pop a robot in the top you turn that crank you're going to see a a dependence here on qou dot that's linear okay and you're going to see the Torx on the other side that's linear which is how I justify this form okay I get I could separate out my Q dot turns out that I mass is always positive so the things that are in front of the Q dot are always invertible so I could take it over to this side okay the exact form of the equations again the software will do that for you but you should know you could do it wanted to for two links Three Links it's a little messy five links I wouldn't recommend it okay but um but it's not scary okay and you should understand that it has this structure one of the things about so the software in the class is called Drake um but one of the things about the software it's it does simulation like we just saw but it actually also because we're trying to open up the box and uh and see inside it also has a symbolic engine so you can actually generate the equations the example code for the first chapter you put the robot in and it'll symbolically tell you the equations of motion you'll almost never I mean so some algorithms will require symbolic forms you'll never almost never want to print them they get big fast okay but there's a symbolic engine that exposes all the structure of the equations uh for our algorithm okay so if you do that again and again then what you'll see is a very important common structure which is called the manipulator equations so the thing I was just Heming and ha about with the mass on one side okay we're going to take yet one more level of specificity this was more specific than that this was more specific than that and now I'm going to write the manipulator equations which is the form that captures most of the structure we care care about okay so this is the accelerations positions velocities you know this is the mass Matrix this is the Coriolis terms this is the torque due to gravity the generalized torque gravity terms this is the torque input let's say and this is sort of the actuation selector Matrix every time you roll those equations through the through the recipe you'll see that they always fit in this form for almost every system you want if you start putting in like Laps on an air foil or something you'll break it but for for regid body mechanics this is the formul see over and over again there's one exception I I made a choice so um if you're working with querian then you would use a slightly different notation I would separate I would not write Q dot here I would write the derivative of quan slightly different ly we're just going to avoid cians and underactuated okay we talk about it a lot in manipulation you can't avoid it in manipulation but here we'll just assume no querian in the state representation it's easy to extend once you do it's not I'm not hiding anything it's just keeps the notation simpler okay so one great thing about mass is that it's positive or non- negative let's say okay and in this Mass Matrix form sometimes also called The inertial Matrix okay um now this be the generalization of that is that that's going to be a positive definite Matrix you in fact the kinetic energy the general form of the kinetic energy is going to be 12 Q do transpose mq Q do that's 12 mv^ s in The Matrix form okay to say that kinetic energy can never be negative is synonymous with saying that that Matrix has to be positive def that's the definition of positive semi-definite right not negative so what that means since this Matrix is positive definite I can always take the inverse I could always rewrite that equation as Q Dot equals m inverse Q CQ dot plus to g plus b u okay so that is the reason the further justification that it could fit in this form F2 in this case is M inverse time B full stop moreover since m in M is is positive definite m inverse is also invertible you get back to M okay the condition of saying F2 is full rank reduces to just whether B is full row rank Y how does q dot appear in F2 um yeah in this so in most cases it doesn't yeah in most cases it doesn't so that's more General than it needs to be yeah it would be pretty weird to have it actually appear I can't think of a simple case where that would appear cool yes yep you got a little Q happy that's a gravity term thank you yep good so um it's a question of that's a great question friction would typically appear here as an additional term here like a force of friction if you will or to friction so it probably wouldn't appear in front of it would be in F1 for sure but I don't know that it would enter in F2 if you had a um torque dependent friction you know depending on your actuator model or something I guess you could get there okay so B tells you if your systems interesting no sorry underactuated right you can just look at the Matrix B if B is full row rank for instance if you have an actuator for every one of your Motors a one toone correspondence that's the simple case b would just be the identity Matrix or it could have a your Ratio or something in it okay but if B is full row rank then you're fully actuated control can be easy when B is low rank you have to think harder about your Dynamics you can't just do feedback linearization your control actions have long-term consequences yeah so I'll end with some stories okay since I'm old now and I've seen the robots you know evolve it's kind of maybe fun sometimes to to tell you how that happened a little bit and how that relates to how that where why we're doing what we're doing here okay so um when I got to MIT in 2000 I went down to the building of ne43 which was the leg laboratory at the time Mark rabert had already left and started the company Boston Dynamics down the street it was a simulation company at the time but there was a guy named Pete Dilworth that was building dinosaur robots and that basically sealed my fate I always wanted to do walking robots but when I saw that I was like okay drop everything this is what I'm going to do with my life I just really like Trudy okay those are all little hobby servos but Pete is a absolute design you know he just has something special about the Aesthetics of these things okay but around that time this was like in 97 is when they announced this but I remember the uh when I was sitting in the leg Lab at MIT and we had a visit from Honda and they came in and they're like just you know we've been working on walking robots for 20 years behind the scenes look what we did okay and um and they they came up with they started showing the world this robot this was P2 went on to P3 went on to azimo if you've seen aimo this is the slightly flat foot Walker I talked about but this was mindblowing mind like absolutely a GameChanger when this happened we couldn't believe it there was a lot of you know uh we obsolete kind of thinking but it moved the field forward uh in just fantastic ways okay but at the same time um this is sort of like this is the fully actuated sort of view of the world if you want that's a little too simple for they're actually doing very Advanced Dynamics things but this is kind of what you get when you try to overpower your Dynamics and act a little bit more like a robot arm but around the same me earlier it happened but I got introduced to it around the same time I learned about these things called passive Dynamic Walkers okay so these are passive Dynamic walkers this is like a beautiful gate of a robot just started taking a stroll down the down the through the park or something right and what is so compelling about that maybe you can see it it has no Motors it has no controllers it's powered only by gravity and it's walking down a very small ramp that's the name passive Dynamic Walker okay so for me this was sort of the beginning was to see this this dichotomy of the way we're making our fully act you know our heavily actuated robots walk is so different than the way I perceive robots to should walk right and somehow this captured it and the lesson for me was don't try to erase your Dynamics stop doing that that's bad physics is good we should be pushing and pulling the natural dynamics of our robot with minimal control and that's the way we should be thinking about Dynamics and control right and that's algebraically very you know a bit different but sort of in the in the rollouts very different right yeah yeah that's right it the philosophy came from that but but they are doing more advanced things to regulate their zero moment point for instance they're they're doing a lot of work to make sure that that assumption is good I think that's that's fair so basically think of it as a hierarchy of controllers some of them are making that assumption true and then the rest are leveraging that assumption yeah it also requires by the way that your biggest Motors are at your ankle which seems seems a little backwards right spot for instance as Bigg as Motors are up at the hip that seems to make a lot more sense but a lot of those previous robots the big the big motors are at the ankles um before that time the leg lab was already a thing uh this was the beginning of the leg lab was actually about very Dynamic robots this is Mark rabbert who's the founder of Boston Dynamics started a lab at CMU originally and then it moved to MIT it was called the leg laboratory and he made these pogo stick robots we'll understand his controller the Dynamics of those systems in a future lecture okay but these was the sort of the history of the leg lab before Honda showed up right was these very Dynamic robots which look very different than Asimo they were throwing themselves through the air they're very simple in a weird way running turned out to be a little easier than walking because you could just throw yourself through the air and do sort of intermittent control just a beautiful series of robots Okay and we have azimo that came out which is a very capable I mean it can dodge people and uh it's still a beautiful one of the best you know engineered robots we've seen really really good it's super light and small beautiful machine and now we have Atlas so just think about how amazing that transition well that transition specifically but also the change from these very simple robots the sort of robots that can throw themselves through the air and a lot of that is really going in exploiting the Dynamics writing optimizations that can leverage these Dynamics and not trying to cancel yourself out okay um and it's like a different time now I couldn't believe you know um sorry I think it's wisdom from the sky but no the so you know now we're in a place where everybody thinks humanoid robots are a thing I used to spend all my energy like explaining why I was working on humanoid robots right like uh um I'm not saying it's a product I'm just saying it's a you know it's a scientific challenge humans have motor control systems that solve this problem that we don't know how to solve as Engineers that's a scientific challenge I used to say that all the time but now apparently it's going to be big so fully Justified um no exp necessary right and they're actually like I don't know this just came out this week did you guys see this one oh my God so good right it just you could almost see that in a factory right that's the first time I've seen one that's like wow that actually would be useful to have around it's incredibly good that's Pat Maran who took underactuated a few years ago okay this is the RL case you know oh my God that's so good right this is now robots finally getting sort of to the limits of their performance I think right and it's just it's such a good time I love it okay the cool thing is it's not just legs you might if you you might be like oh my gosh he's talked a lot about legs I didn't come here to learn about legs it's true in other fields too right so let me just tell you a quick version and if you care about um drones let's say right so what's sort of the moral equivalent to feedback linearization for an aircraft okay when you stay in a low angle of attack and your air flow stays attached to your wing then you have considerable Control Authority okay and that's sort of the safe Zone where we have a lot of Control Authority our flaps and you know aerons and everything can have significant authority over our pitch and the like okay birds don't do that um they don't restrict themselves to the small envelope and the example that I like the best maybe is when they land on a perch right so they go into these severe poststall Maneuvers there's I like this picture if you can see there's like smoke Trails there showing clear separation behind the wings the air flow did not stay attached to the wings they're in a regime that most people would call installed and like you've lost control Authority they do it all the time and they land on a purches and it mostly works out pretty well right beautiful stuff right so just to say that a little bit in pictures right if you're at a low angle of attack you have good control including over your control surfaces when you start separating you know naively and in practice you don't have the flow over your control surfaces and there you you've lost some Control Authority okay that might be a velocity dependent B Maybe it's a little bit complicated but uh all right and so I just got fascinated watching birds do these things right and so this is like an owl that was clearly baited with something at the camera but uh it's playing a little slower than I remember if you care about aerodynamics then watch the Leading Edge feathers on this this owl it's a little grainy okay you're going to watch right around there flip up he's going to be in sort of separated flow on his Leading Edge and does this sort of beautiful beautiful Landing separated flow right there all right you can find lots of these like on the nature channel right we started building airplanes that could do this this was early underactuated I tried to make the simplest airplanes possible that could do the same thing this one didn't have a prop it had one control surface okay we shot it out of a cannon but we made it land on a perch just by thinking about the nonlinear Dynamics and that was some of the early work sort of in this line of thinking and it was similarly in a very post stall aerodynamic regime right and we had airplanes that could sort of land on a perch whenever drove through it and now again we have like drone racing this is this earlier this year or I guess last year now um and drones are now you know at competition level Human Performance right they're champion level human performance for drones we have robot controllers that are using perception operating close to the limits of the vehicle and this is just awesome just awesome birds still have a number of things on us right they're way more efficient Albatross can fly across the ocean on you know basically without flapping its wings drones don't have that yet they can like dive bomb at 200 miles hour and catch a smaller bird I'm not sure we can do that yet so there's still lessons from nature but um this is one of my favorite and then I'll I'll end right so um this is a hummingbird likes to feed out of a feeder the cool thing is do you see it just looked like it was kind of hanging out there right it's actually doing a back stroke at like 8 meters per second you know it it's in a huge wind tunnel that's just blasting it with air and it's like yeah yeah I'm all good you know and it's it's it's amazing like they could change the air flow in all these different directions and it's keeping its head Rock Steady boom you know like no problem all right last one um this is a fish same same fish three well this is maybe the early version of the fish and the um later version is the big one okay it's a rainbow trout it's we're looking down at in a water tunnel this is how a rainbow trout swims in a water tunnel that's just a n natural gate of the rainbow trout we put a rock we didn't it was George Lauder at and Company at Harvard put a rock in front of this and The Rock sheds vortices and the rainbow trout changes its gate in order to adapt to those vortices it's called the Von Carmen gate pretty awesome but this is the one that I just love Okay so this is the that rock okay here we go this fish is dead there's a string making sure it doesn't go back caught in the greates but Watch What Happens that dead fish just swam Upstream right just because the mechanics of the body is designed to resonate with the vortices that it experiences in the world turn the energy of those Vortex streets into forward propulsion no brain gone uh but it can still swim upstream okay so that's Dynamics Dynamics is beautiful we should Master it not cancel it out see you next time

Channel: underactuated

Views: 7,097

Keywords:

Id: uyyBT-MHhLE

Length: 85min 7sec (5107 seconds)

Published: Fri Mar 01 2024