Lecture 8: Multi-Robot Assignment Methods

hi everyone welcome back to the mobile robot systems course this lecture is all about assignment methods which lie at the core of many multi-agent coordination problems we'll start with a seminal method which is the hungarian algorithm for which we use discrete countable representations of multi-robot systems and i will also talk about a method that relies on a continuous representation of uncountable robot swarms and besides looking into several centralized methods we'll also study one fully decentralized method without further ado let's get started so this lecture is all about assignment methods and i'll begin the narrative by motivating the topic on hand of examples from nature as well as some industry applications in particular we'll be studying four different ways of executing assignments starting with a classical algorithm the hungarian method and then looking into different ways and methods that allow this us to do this for distributed and potentially also decentralized multi-robot and even swarm systems so i'd like to start with the concept of division of labor which arguably is bio-inspired and quite prevalent in nature so the division of labor is simply the separation of tasks in any system so that participants may specialize or in other words it refers to the allocation of tasks to individuals or even organizations according to the skills and or equipment that those individuals or organizations possess so in nature however the division of labor is usually favored by high efficiency benefits or a reduction of conflict within the societies and it has been shown that natural selection actually favors extreme specialization where some individuals are completely dependent on the help of others and this example here this panel actually shows you an example of one of these societies where we we actually have extreme specialization as demonstrated by the composition of the society into physical cases so two different types of of ants in this case and so you can see that these different types of ants have specific behavioral repertoires um that belong and that don't overlap essentially right so we have these minor ants that have um ten times as many different behaviors as the major ants belonging to the same natural group for example here the minor ants tend to spend more time doing self-grooming for example whereas the major ants have specialized to guard the nest entrance we can also look at different types of specialization in this particular case we refer to this kind of specialization as temporal polyethylene where specializations adapt and change during the lifetime of the organisms for example here we're looking at worker bees where younger bees tend to spend more time cleaning cells and older bees for example tend to spend more time patrolling and defending the nest so this is another type of division of labor that you can observe in natural systems now bioinspiration also gives clues to the utility of certain mechanisms and so we've incorporate and so we've incorporated this idea of task allocation into many different application domains and it turns out that assignment mechanisms are what actually fundamentally allow us to coordinate multi-agent or multi-robot systems and the reason for this is that the and that they answer the fundamental question of which robot goes where and which robot does what right and so arguably this is really the foundation of coordination in systems where you have multiple actors and this slide here actually shows you numerous examples that illustrate potential applications where you'd have to solve an underlying task allocation problem in order to get these robots to do something together that is conducive to a higher order goal applications will include things such as monitoring and surveillance which for example consider the assignment of robots to locations in partition space we can also consider the assignment of robots to tasks that for example need to be attended through services that robots can provide given that they're specialized with certain skills or equipment and it can also include the assignment of robot cars for example for mobility on demand systems where robots are sent to pick up persons or goods and are potentially part of a much larger larger transport ecosystem so these are just a couple of examples of very classical examples of how you might want to exploit assignment algorithms to solve industrial application domains and what's really interesting is that assignment algorithms can be used also for slightly um different types of of problems such as trajectory generation in larger robot systems and the reason for this is because they answer the question of who goes where and how i might how am i going to do this in an efficient way right so if i'm a robot i need some assignment algorithm to help me understand what role i should be playing in this larger system of robots that are trying to maintain formations potentially under different constraints right and the underlying goal here in these multi-robot trajectory generation problems is the robots want to optimize speed transitions for example whilst obviously not colliding with each other nor with any of the obstacles in the environment so for example in this formation control type of problem you would want to consider an assignment objective that minimizes overall travel distance for all the agents in the robot team whilst considering any constraints that are given by the workspace so to recap the assignment problem answers or the solutions to the assignment problem answer the question of which robot goes where and which robot does what however to use such solutions we first have to understand how to model our systems so that we can map the solutions to the underlying problem domain and basically there are two key components to assign problems so we're considering agents which are the the one the the components that are actually providing services and we have tasks so what is a task a task can be discrete for example a robot may pick up a parcel x from a location y or a task could be continuous for example we could require a robot to monitor a building x for a duration of time or to search a given area y and the key is assumption that we're going to be making in this first instance is that different tasks are independent right because considering our modeling dependency of tasks actually leads to a different type of problem problem formulation which is known as or referred to as scheduling problems and this is something we're not going to be considering within this lecture so assignment methods are drawn from multiple fields so there are there's a lot of active research ongoing and lots of new assignment solutions are being proposed continuously um the most prominent field active in the area of assignment algorithm development is operations research closely related to economics and also within industrial manufacturing domains there's scheduling network flows and also combinatorial optimization areas more closely related to core mathematics actually however one of the most classical ways of formulating or representing the assignment problem is actually known as bipartite graph matching and we can do this for for the simplest systems where we for example don't necessarily include task dependencies and we'll be working with this representation quite a bit in this lecture so task assignment is all about optimizing a global objective where we have multiple agents and a bunch of tasks that we want to solve and the key to formulating this objective is a notion of what utility is so the utility for us will be that an individual robot or some centralized agent knows the value of executing a certain action and of course it can depend on the context so the utility could be a value or it could also be a cost or simply a fitness and knowing the exact and true utility is key to finding optimal assignments so clearly if we only have estimates of utilities then assignments may actually end up being sub-optimal but we'll work with what we have right so one example formulation might be so in this little example we're considering that q is a utility and c is a cost and so the utility the overall utility um is oh so well actually q would be quality uh here and u would be utility so the overall utility of assigning a robot r to a task t would be given by the quality of or how well that robot is able to satisfy that task minus the cost of that robot executing that task and if a q minus c is is not um or if q might is not greater than c then we would simply say that the utility of assigning r to t is zero so this would be one way of modeling a task assignment problem and giving it a quantitative substrate now assignment is all about optimizing a global objective function and the key question here is how are we going to write down that function what does it look like so let's start by assuming that we have a variable x sub ij that tells us whether a robot i is assigned to a task j okay so we can assume a value one for for an assignment and a value zero for when it is not assigned and we can assume that x i j is simply an element that stems from an assignment matrix capital x and what we want to do is we want to find a best possible assignment matrix x that maximizes some performance so the key is that in many multi-robot systems clearly we'll have a given set of constraints for example one robot can only do one given task at any given point in time and maybe one task can only um afford one robot at any given point in time now this kind of formulation can be represented graphically in a very elegant way as is shown here through this complete graph so the completeness of this graph is telling us nothing other that any robot is basically capable of performing any given task now the question here is how are we going to solve this assignment problem and you've probably thought of this a very naive solution or or the the most intuitive thing that comes to mind to solving this assignment problem clearly would be to check all the possible assignments to calculate the cost of each and every one and then to simply select the set of assignments that maximizes our utility function but the problem with this is that this may be very inefficient since if we have a system with an agents and end tasks we end up with n factorial different assignments and clearly this would lead to rather long evaluation or computation times in order to solve the problem optimally so what we're going to do next is we're going to look at a better way of solving this kind of assignment problem so there is a better way of of doing this and it's known by the name of the hungarian algorithm or the hungarian method so originally this algorithm was published by kuhn in 1955 based on the earlier works of two hungarian mathematicians so dennis koenig and you know a givery which is the origin of the name of the algorithm and the what this paper showed was that it was possible to solve this assignment problem in o of n cubed now there are a couple of steps to to solving the assignment problem following the hungarian method and without going to the theory or the proofs of why this algorithm converges to the optimal assignment i just want to give you an intuition of the mechanism itself so there are four key steps and we will walk through an example in the next slide and this is just to give you a summary of what these individual individual steps do so in the first step we are simply going to be subtracting row minima the second step does the same for columns the third step covers all zeros that are remaining in rows and columns with minimum number of lines and if the this number of lines corresponds to the number of assignments that you're trying to make then you're done otherwise you can continue with step four creating additional zeros and then you reiterate with step three all right so let's have a look at an example to see or to convince ourselves that this algorithm actually does does boil down to producing an optimal assignment so what we're looking at here at the very top um left hand of this slide is we have a matrix or a a grid essentially that is assigns costs uh to agents and tasks so at step zero this basically just consists of producing the cost matrix at step one what we're going to do is we're going to subtract a row minima from all rows so for example if we look at the first row 69 is the minimum we subtract that from all values and hence the the resulting values that we have there and we do the same for all the other rows in the matrix we repeat this procedure for the minima over columns example again and in the first column the minimum is zero so we subtract zero from all the values and this returns us the new column uh essentially in this case unchanged at step two for this given matrix at step three what we do is then we then cover all the zeros with a minimum number of lines in this case this gives us three lines that are covered now we know that three is not enough because we're trying to find an assignment between four agents and four tasks so we know that we now have to move on to step four to see if we can do some more manipulations in order to find a full satisficing assignment here so what we do in step four is we create additional zeros and the way we do this is by finding the smallest uncovered element we subtract it from unmarked values and we add it to the doubly marked values so in this case 6 is the value that we're going to subtract or add we subtract it from the yellow values and add it to the red values which produces the matrix shown to the right of that matrix at step four now what we do is we check again if we can if we have four lines that are now covered with the minimum number of lines or rows that we can use to cover them and indeed we find four lines that are now covered and this now tells us that an optimal assignment exists and that we have found it so if we then look at the final panel on the right hand side the green values show us the assignments that are going to be made essentially and of course these are the values that produce one-to-one assignments right so that use only one agent or one task for any given one assignment right so this is just a summary of how the hungarian method would work on a concrete example note that most numerical software libraries actually do have a function for the hungarian algorithm and you don't or rarely would need to implement implement it yourself although for those of you curious who and for those of you who want to develop a more deeper understanding of it i encourage you to do so and i also encourage you to look at the proofs and and for a deeper understanding of why this algorithm actually boils down to the optimal assignment and why it actually works in ofn cubed so next i want to show you or motivate the use of of the hungarian algorithm on a given example so in this particular case we're looking at vehicle to passenger assignments which are a core part of mobility on demand systems where we're looking at trying to pick up passengers in say given time windows where we know that there's a set of passengers that are waiting to be picked up and we know that there's a given set of vehicles that are currently free or available to pick up these this querying set of passengers now how do we solve this with an assignment algorithm well we have three main components we need to define so the agents are our robot cars the passengers are at their locations correspond to the tasks that need to be serviced or or attended to and the cost function here well well that's the question what is our cost function going to be well the most reasonable or intuitive cost function we can think of here is we want to minimize travel time right because this essentially minimizes the waiting time that the passengers go through whilst they're whilst they're essentially waiting to be picked up so so that is the cost that we want to minimize so in other words we're going to pose a minimization problem which simply says i'm trying to find an optimal assignment matrix a star such that the cost so the right hand side of this equation is minimized and this is simply telling us for all assignments of cars i to passengers j um give me the minimum sum of all uh resulting travel times or waiting times all right so in this case we run the hungarian algorithm and if we have an exact estimate of what these travel times are and if we know where our passengers are located and if we know where our cars are located then this is a trivial problem given that we have the algorithm that solves it now there are a couple of caveats to this so the first major caveat is that finding an exact travel time estimate is in many cases not trivial not only because you might not exactly know where passengers or cars are located but also because traveling along traffic heavy roads is an uncertain process and you don't know how long it really will take you at the end of the day so these algorithms are often working with estimates and the consequence of this is that the assignments end up being suboptimal because the estimates are only approximate another catch to the system is that as i mentioned before we need to know where the cars as well as the passengers are located and this can lead to privacy concerns so perhaps the passengers don't necessarily want their locations to be precisely known especially if they're heavy-duty users of such services then you might be able to infer some patterns over there over passenger behaviors and this can lead to privacy breaches or or or certain things about passengers habits becoming known to the centralized computational units that is undesirable for the for the clients of such systems and finally because of the noise that we might have in these systems overall we we don't ever really know if we unless we um are using more sophisticated methods whether or not the assignments are truly optimal or whether we're actually operating at a highly sub-optimal level um there are actually numerous methods that have been published in the recent decade or so or more recently in the last few years that deal with the various issues that i've addressed so privacy on the one hand side and uncertainty of the cost estimates on the other hand side and i really do welcome you to have a peek at those papers for those of you who are interested in what kind of methods we could develop to circumvent these two complications or challenges right so let's now summarize the assumptions and requirements of actually using the hungarian method to solve these kinds of um centralized assignment problems so we we know now that the costs and utilities need to be communicated or known to a centralized computational unit because that computational unit will collect them compose this assignment matrix and then use that to solve the assignment problem we also know that these costs and utilities are deterministic so there is no noise they don't vary over time they're constant and we assume that we know them precisely right so that's the underlying assumption to being able to assert that the assignments will be optimal and also we're saying that we're only making one-to-one assignments so there's one robot per task and there's only one task per robot now there are certain complications that arise in real world systems so often there is uncertainty around the true utility there are systems that operate in dynamic environments which in one on one hand sign changes utility or cost functions and on the other hand side might change something about the state of of agents or tasks and also we we didn't talk about how we're actually modeling robot or task dependencies especially in systems where we don't actually have homogeneity across all agents and we might not have homogeneity across all tasks so imagine if in the case of mobility on demand we're actually trying to assign certain robot cars to certain passengers where the fit might be different so maybe maybe a passenger at one location is not one passenger but a family of passengers and then you want to think about sending bigger cars to those kinds of tasks and smaller cars to singleton passengers right and these kinds of constraints or considerations really complicate the assignment process so consequences of all of the above as i've touched upon before are so in some cases this might lead to sub-optimality in other cases we're looking at algorithms that are no longer solvable through the linear sum assignment method because they become np-hard and require versions of combinatorial matching algorithms and and in some cases um centralized uh solutions are just simply not possible either because certain information is not available to centralized computational units or because they're just too computationally complex to be solved by one um centralized component so next i'd like to describe one particular example of a system that is difficult to solve through through centralized algorithms as a motivational example for some of the different solutions i'll be talking about later in this lecture so this example that i'm introducing here is an example where we're no longer interested in one-to-one assignments instead we are interested in tasks that require more than one robot to be solved right so this example here will really help us highlight the issue of assigning more than one robot to a task and at the same time many to one assignments actually make a lot of sense so you could think of robots or agents that are actually working together to solve a task more efficiently because potentially they're complementary or they're collaborating in a way that they couldn't do they would be trying to solve a task alone so essentially what we're doing here is we're creating agent or robot coalitions that together are capable of solving a task now this has certain repercussions one of them being that we now need a new cost model right because the cost of solving a task is no longer the cost of just simply assigning one robot to a task it's more how do we actually model the assignment value of a robot coalition to a task right and this is actually a really hard problem and i'll say a few more words about this in the following slide just to give you an intuition of how difficult this problem is if we have a system of two tasks and five robots we know that the the number of possible coalitions or partitions in the system is is 15 right you can you can probably even work this through by hand but as we scale the system the sterling number of the second kind actually tells us what this number is for an arbitrary set of of agents and tasks or agents and coalitions tasks and coalitions so for example we have 10 robots and five tasks the sterling number of the second kind tells us that there are 42 525 different possible partitions right so clearly this is a very large number of possible solutions you'd have to evaluate in order to find out which partition here or which assignment of coalitions to tasks is the optimal one so what kind of problem is this really and in order to introduce this more formally i'd like to go through a little bit of a formal introduction to this problem so so let's try to formalize this with a bit of notation and then we'll find out what kind of classical problem we're actually referring to so let's assume that we're trying to form robot coalitions and ultimately these coalitions belong to let's say a set of of robots which we we're going to be calling our ground set of components or elements or or units right and we're we're going to try to then assign these coalitions to tasks right so if e is the ground set of all of our robots and x is a family of subsets then we have two conditions one is saying that our robot subsets must be mutually disjoint right because we want distinct coalitions that we're going to be assigning to distinct tasks and we know that we're going we want to use all of the robots available to us so we know that the union of all of these subsets or partitions must be equivalent to the ground set that is available to us so these conditions and the problem together form something that is called the set partitioning problem which is defined as a given a finite set e and a family f of acceptable subsets of e and the utility function u that maps these family of subsets to a scalar find the maximum utility family x of elements in f such that this solution x is a partition of e okay so this is the formal definition of the set partitioning problem and in 1978 gary and johnson actually showed that solving this problem is strongly np hard so clearly even if we have centralized computational units as we try to scale these systems we're going to be running into a computational bottleneck and what i'm going to introduce in the next couple of slides is one potential solution or one potential approach about trying to solve or trying to think about how to solve these kinds of problems and it's all based on the assumption that we can actually relax the problem which is here very clearly defined in the discrete domain to the continuous domain and i'm going to convince you that continuous domain solutions can also be quite elegant in their own way so what's the key idea behind this relaxation from the discrete to the continuous domain now the key idea behind this is that instead of saying exactly how many robots need to go where we're going to be saying what proportion of robots need to go where so instead of count instead of modeling multi-robot systems as countable systems where every single robot counts we're going to be looking at these systems more like swarms where we're more interested in proportions and a distribution of math rather than an exact allocation of specific identifiable robots to specific identifiable tasks right so let's take a second to think about the consequences of modeling systems in this way and clearly you'll be thinking right now well there are a number of applications and problems where we cannot model systems as uncountable for example simply looking at warehouse and logistics problems where it is essential that every single item and product is picked up and we know exactly which robot is is is doing this but there are a number of problems for example if you look at environmental monitoring and surveillance problems where it's more interesting to have a certain coverage of certain areas with a rough number of robots or capabilities in those areas in order to have an estimate or feel confident that you're solving the task to specification so clearly there is a difference between modeling systems as countable versus uncountable and where in the case of uncountable systems what we're going to be doing is we're going to be looking more at a method called mean field approximation which is it's clearly an approximation but it's definitely we're aiming to show or what i'm able to show you is that it's definitely good enough for a number of a real world problems right and these pro approaches are very um attractive especially when you're trying to solve um allocation or distribution problems for very very large robot numbers where salt where looking at combinatorial approaches can be complete not only computationally demanding but um unnecessary so i'd like to illustrate the illustrate this problem of redistribution with a simple example that's that's going to be our running case study for the next couple of slides so what we're looking at here is a problem of continuous task assignment essentially where you can imagine that these different grey areas here are potentially geographical sites that we want a swarm of robots to to monitor for us so we want them to reach those sites and perhaps circle around them or just be there and be present at those sites to to um perhaps collect information or or just simply execute certain services in those areas okay so what's interesting about this geographical monitoring problem is that we don't really care about the exact number of robots that is every at present at every given task location but we kind of want to have an estimate or or knowledge about the average number of robots of any different kind for example in this heterogeneous system that is present at each different site so let me just say a few words about the mechanism behind this mean field approach so we will be assuming right that we care about distribution of robots over these tasks and the mechanism by which the individual robots will be functioning is essentially to say well the robots will be switching between different tasks according to certain transition rates right and here since we're looking at a heterogeneous system we'll assume that every robot type in the system has its own average switching frequency and the reason why we're actually considering switching frequencies and transitioning rates is because we're modeling these systems in the continuous domain as essentially dynamical systems given that we're using this mean field approach right so we're looking at the whole system this this whole swarm of robots as a dynamical system and we're aiming to have a steady state of this dynamical system we're aiming to have a stable distribution of the robots across these tasks such that this distribution corresponds to the desired distribution that we specified to the system at the outset okay so that's a bit of a low down on on the specification or the the the modeling of of how we're going to be attacking this problem so a few more words about this uh the way we're representing our problem so we're actually using a very important abstraction here which is that we have an underlying graph structure which is representing the topology of our our system of tasks essentially right and we're assuming that robots are allowed or unable to move along the edges of this graph right so this these could be potentially roads connecting different sites and the nodes in this graph are the sites or the tasks decides to be monitored or simply the tasks that are are to be solved right now note that this problem formulation could also easily be viewed much more broadly for example as a collection of tasks represented by the nodes of a connected graph where robots can switch from one task to another at a given rate and the edges then simply represent from which to which tasks are robots can transition so i did mention that we're considering a heterogeneous multi-robot system in this case it's not essential to to the mean field approach but it does give some richness to the problem formulation in the sense that we can actually now at the different so at the various tasks in the system we can actually say well i need a certain amount of capabilities to be present at these different task locations and this is actually an interesting way to think about you know heterogeneity because we can think about robots as for example as carriers of capabilities you know think of a robot carrying a certain amount of sensing capabilities or actually a quantifiable number of sensors or perhaps a quantum quantifiable number of actuators and perhaps we know how much of a given capability we need to have present at any given task in the system right so so let's assume we have this heterogeneous system and that we know roughly how much of any given capability we need to have present at any given site or task location in order for us to adequately be able to solve the overlying problem that we're looking at here right so now the question is how do we model this overall system so we're going to start by thinking of this whole system as a dynamical system and as i did mentioned before we're going to be taking a mean field approach that in this case is modeling the swarm of robots as a linear dynamical system so we start by saying we have a state of our system right where x is essentially the distribution of a of a given robot species over our tasks in the system right so here we have a collection of six tasks and the example that i'm giving you in this vector here is telling us that in the first task we have 0.3 or 30 of our robots present right and we have five tasks not six tasks right so out of the five tasks 30 of the robots are present at the first task in the system and um so x is x sub i is is just basically the proportion of robots at um task i now um we're doing this because we're relaxing the problem specific the problem formulation to the continuous domain and then we go about saying well we now need a dynamical equation and deriving this dynamical equation is relatively straightforward because we now know x is the state of our distribution then we all all that is remaining to be specified is x dot which is the change in this robot species distribution over the tasks in the system and how does this distribution change well we simply pre-multiply it with a transition rate matrix that is of size m by m where m is the number of tasks in our system right and there's so there are certain properties that we have on on k that may that allow our system or that guarantee that our system is stable for example the rows or columns might need to sum to one etcetera to make sure that we don't um so that the transitioning between tasks will eventually converge to a steady state um so and this matrix k well what does it describe for us well an entry i j in this matrix k simply tells us how a robot species s transitions from a task i to a task j so how can we now use this dynamical systems model to solve a given distribution problem so first what we need to do is to define a desired distribution of robots over tasks which we're going to call x star right so this is the desired distribution we want to have at steady state of the system and the problem then consists of solving an optimal control problem to find the best possible transition rates k star that for example are fastest to satisfy x-star so what does fastest mean in this case well obviously here you are free to define your own optimization objective but one very um i'd say legitimate optimization objective is to say well we have a given distribution of the robots over the tasks at the beginning and we want to find the transition rates um k-star that leads to the fastest possible redistribution of these robots from an initial state to the desired state right so this would be one legitimate optimization objective that would allow us to formulate this optimization problem and solve or back out k star okay so again we have us our our differential equation that tells us how our robot distribution changes over time and we know from literature and basic fundamental [Music] theory on on differential systems how to solve this kind of ode and so the solution for this ote is given to us by the second equation on this slide which basically tells us that we take we have to take the matrix exponential times t and multiply that by the initial distribution of robots and that will tell us how a a distribution or what the state is of the distribution of robots of a given species s's for a specific time t right so in other words given that we have a transition matrix k and an initial distribution x sub zero we know how to derive the state of the distribution of a given robot species at any given time t okay now finding k star is our optimal control problem and and how would we go about solving that so state-of-the-art has actually considered a number of approaches one of them is by saying well we can approximate k and we can then formulate a convex optimization problem and solve it with semi-definite programming methods or other approaches might consider stochastic optimization which are equally feasible and applicable to this kind of problem specification okay now assuming that we have solved this optimization problem and we now have an optimal transition matrix k star what do we do with that matrix k star and how do we use it to tell the robots what they have to do once they're on site so the idea here is that we're now going to perform controller synthesis from this transition rate matrix so what does that mean well we know that for a given robot species us we have this value kij that would tell the robot how it's supposed to be transitioning from a task site i to a task site k and this transitioning rate is no other than basically describing a transitioning probability and so all we have to do is take these values and actually use them as um or as use them as probabilities that a robot will at any given point in time actually execute this transition right so it's like using these values to estimate the probability of rolling a dice that tells you whether or not you're going to transition and so we use those values to calibrate the probabilistic behavior of robots as they reach sites and as they make decisions on whether or not to move to a next site within the graph topology so as i just described deriving probabilistic controllers is immediate from our transition rate matrix but how about if we want deterministic controllers well we can also back those out from these transition rate matrices because we know how to compute the steady state distribution of the robots over these sites and if we know the steady state distribution we can just tell the robots well this is where you're going to end up and we're just going to have you transition from this to that site until you reach your steady state site essentially so this is a way of actually backing out a deterministic controller and just a final note here we can use this kind of our mechanism in both open loop as well as a closed loop [Music] control paradigm so if we have feedback of how the robots are transitioning we can close the loop if we don't have that feedback we're just going to be using our transition rate matrix k um blindly so an open loop control so next i actually want to show you a video that illustrates um such a system because i appreciate everything has been very abstract until now and admittedly the video is still a little bit this movie is still quite quite a bit abstract but i just want to walk through it or talk you through what we're actually viewing here before i actually go about playing the movie so imagine that we have these six tasks here in this given system of tasks and what these white wire frames are showing me is the amount of capabilities that i want to have present at any one given task sites okay and i have um these four robot species that are carrying um each robot species carries um one of these capabilities so the capabilities are color coded and so are the robots species and we'll we'll see how the robots as a function of what capabilities they're carrying and what final distribution over k over the sites of these capabilities we're aiming to achieve the robots will be finding will be optimizing for these transitioning rates so that at the steady state of the system we will be satisfying the amount of capabilities that need to be present over these um sites right and so what you'll see happening at the beginning of this movie is that the robots first swarm into their initial distribution which is over the three sites on the right-hand side where we're essentially saying we don't need any robot capabilities right because the wireframes are all down to zero and then i'm applying my optimal transition rate matrix which was computed beforehand so that these robots then redistribute as quickly as possible into the three remaining sites that require their attention so now you see them swarming into their initial distribution and redistributing in order to satisfy the capabilities that are required by these white wireframes and as you'll see as the system approaches its steady state the average distribution of our capabilities will be such that these white wireframes are filled out with the right amount of capabilities and i can use this exact same type of system on a real robot system so no longer a simulation but an actual physical experiment so what you're seeing here is an admittedly a much smaller multi-robot system of of just five robots but again we have this problem that we want to redistribute the robots and we don't really care about potentially the exact number of robots um but rather we just want to use a probabilistic control law so that ultimately our steady state system here will con correspond to the desired distribution of robots over sites so we have two boat types here in the system we have species one which is a little boat and we have species two which is a big boat and they start off in this initial distribution what we want is we want the big boats to be in the upper left hand corner and we want um smaller boats to be in the upper right hand side corner and as i play this movie um you'll see how the the boats here are you are executing this probabilistic transitioning law in order to go from one side to another and you'll see that the steady state distribution corresponds exactly to the one that i specified at the outset so at this point we've seen two different methods for solving the assignment problem one that works well in countable systems and one that takes an approximative approach and is well suited to swarms of robots or uncountable systems now i want to talk briefly about a third method which is referred to as market-based coordination methods which is quite popular due to its flexibility in accommodating various underlying scheme assignment schemes through its overarching architecture so here we model robots or agents as self-interested agents that operate in a virtual economy and the tasks are essentially commodities of measurable worth that can be traded amongst the agents so an example scenario could be we have three robots exploring mars and the robots need to somehow gather data around these sites or craters and they they need to somehow distribute over these different sites here so we have three robots and seven sites and the problem consists of defining or finding out which of the robots is going to visit which site so market-based coordination functions on the idea that we're using auctions to decide which agents will will get to do which tasks and as i kind of already intuited the underlying mechanism are auctions so the auctioneer is basically or the idea of of using auctions is that we can offer items where the items are the tasks or commodities or resources in an announcement to the robots and the robots then make bids and negotiate which robots will get to do which uh tasks essentially there are different ways of winning bids or winning the auctions so there there are sealed bids versus open cry so robots can know about other robots bids or they can or this can be private um and as an example for for the actual opening mechanism so i just list two here it could be the first price so the best bid or it could be vicry uh which essentially uh where the sales price is the value of the second highest bids and other types of bidding mechanisms could also be considered here which is arguably one of the interesting things about using such market-based coordination schemes that you have a lot of flexibility in underlying in choosing the underlying components then we can be considering single item auctions where highest bidders win given tasks and if no bid beats the reserve price then the auctioneer can retain an item there can be a combinatorial auctions where we actually have multiple items and the robots bid on bundles of items or bundles of tasks and in this case a bid essentially expresses synergies between items so maybe a robot sees that two tasks are close to each other and hence it wants to bid for both in one in one bid and there are multi-item auctions where a robot can win at most one item a piece but there's a special case of combinatorial options for bundle sizes of bundles of size one so let's have a look at a specific example here so this illustration here shows an example of a multi-item auction where robots are making bids which correspond to their costs of servicing each task in the system so a robot like say robot one here would it would cost robot 150 to do task a and 100 to do task b um and it would cost robot 230 to do task a and 70 to do task b and for each of these tasks the reward is fixed so it's 120 for task a and 150 for task b and then an auctioneer might say well we we have a reserve price for task a and that would not be met if robot 2 executes task a so there's an uninvalid bid there and then based on the other costs and bids the auctioneer might might compose this matrix here and decide well it's going to assign robot 1 to task a and robot 2 to task b because that's how it's going to be maximizing it's it's profit here which is essentially essentially just a combination of bids and costs uh bids and rewards so a vanilla implementation of actually solving this assignment problem would be for example to consider a greedy algorithm so if multiple robots are bidding for tasks you can simply choose the best bidder and then rerun the auction for each item or you could also consider an optimal version which is essentially using a hungarian algorithm as the underlying algorithm and in this case what you need to do is you need to batch all the data and create a centralized synchronous scenario so that you can compute over the full cost to task or agent to task cost matrix all right so so this is one of the benefits of using the market-based coordination schemes that you have a lot of of choices or flexibility in how you want to actually implement the underlying mechanism and note that there are differences in in the choices that you make so obviously greedy is computationally more efficient than using optimal but has uh but is not proximative or or heuristic method in the sense that it would not lead to optimal assignments necessarily and also what i want to note here is that given this flexibility of such market-based coordination schemes you can implement them in decentralized as well as centralized manners i have listed a couple of examples here of specific implementations and for those of you who are interested in this area i do invite you to have a look at these various research papers um so so they're they're different i'm not going to go into the details here but there were different implementations or instantiations of market-based allocation mechanisms that showed in each of these respective papers that showed different properties and actually also implemented them on on either simulated or real robot systems showing them showing how they can be used these mechanisms can be used to solve the assignment problem so so far all assignment mechanisms we've studied have had a centralized component so just to summarize the hungarian method we saw how it we need a centralized computational unit that assembles all costs centrally and in a synchronous manner in our continuous relaxation in the swarms method we also saw how we rely on knowledge of the swarm's initial distribution so that was x sub zero so that was a global variable of interest that we needed to have specified okay so this is also centralized knowledge and in the market-based uh assignment method we saw how we require a auctioneer that collects all the bids and then makes the assignments so all of these three methods that we saw thus far in some way or another made use of a centralized component so now the question is is there also a way of solving the assignment problem without relying on a central component to compute any part of the system at all and the answer is there is a way of doing this and i want to walk through one very specific example of such a mechanism that would allow us to do this in a decentralized manner so threshold-based assignment methods are one potential way of implementing the assignment method or the assignment solutions to the assignment problem in a fully decentralized way and arguably it's not trivial to come up with fully decentralized mechanisms for the simple reason that robot systems as a whole need to gather information about what tasks have already been attended to and which haven't okay so this is global knowledge now the question is how can we provide the systems with a decentralized capabilities to get this knowledge so that the robots can act upon it again in a fully local and decentralized manner so how does the threshold-based assignment method work so in this mechanism each robot is assumed to have an activation threshold for which um each task of which is which essentially tells the robot whether or not it's going to be performing or needs to perform a task so if it's activation threshold is not satisfied it won't be executing any tasks the robots will also rely on this idea of a stimulus that reflects the urgency of a task and this is something that is continuously perceived by robots in the system so let's consider this example here which use threshold-based control for an aggregation problem so the aggregation problem in that is described in this paper that i reference here consists of a system of multiple robots that need to that are going to use their grippers to go and collect sticks and line the sticks up in a cluster so these robots somehow need to coordinate to go grab the sticks and put them together in a cluster the location of the cluster is not predefined but what needs to happen here is it needs to happen quickly and hence the robots are going to work together in order to solve this this problem in a coordinated manner so how can we implement a local decision making scheme that allows the robots to do this without relying on centralized coordination so the stimulus in this case here is represented by the time needed to find a stick to manipulate right so the longer the time the lower the stimulus associated with the task of finding sticks and placing them in clusters and what's important to note here is that the threshold for a robot to decide whether or not to engage with the stick clustering task is totally local so it's totally decentralized which means that this threshold has to be self-calibrated okay so how do we how do we how do we decide on how do we come up with a heuristic that calibrates the threshold so the first insight that we make or the first assumption that we're going to make is that the number of manipulation sites right so the number of sticks um or the the either end of the line of stakes decreases as the global task nears completion right so the robots are looking for sticks and as more and more sticks are lined up in a line the robots aren't going to be seeing very many sticks or sites where they can actually grab sticks right because as they're lined up in a line this whole line only corresponds to two grabable sticks right whereas within with the initial distribution of sticks many sticks are grabbable by the robot's manipulators okay so this is what what this um what the what the stimulus essentially is going to be relying on now the next point is that if the time to find the next stick goes beyond a certain threshold t the agent is simply going to switch to resting behavior okay so this is what we want to obtain through this heuristic that we're going to devise that self calibrates the threshold okay now the insight for getting this equation is that t sub k right and if we look at this equation which is the time to find the k stick will start going up disproportionately as the task nears completion and hence the stimulus will fall below the threshold and hence the robots will start to become idle right they will start resting okay so this is how the robot will be self-calibrating its threshold right by estimating or by realizing how long it's taking the robot to find um the next given stick so if we look at and by using the self-calibrated threshold equation by looking at these two panels here on the right hand side we can see how this system actually does converge to a steady state at the end where essentially the average cluster size reaches its full cluster size and the average number of active workers um is is approaching zero so the workers are essentially realizing that there's nothing left to do and they will start remaining idle okay so this is just to show you that we can use a fully self-calibrated local decision-making mechanism to achieve coordinated labor which are arguably requires the assignment of robots to sticks in a completely decentralized manner so i'd like to conclude by showing you an overview of the various assignment algorithms that we've seen in this lecture thus far so the first algorithm we looked at was the hungarian method which is a centralized mechanism because it relies on full knowledge of the costs of assigning agents to tasks it's optimal and the completeness of the algorithm is guaranteed we've also looked at mean field approaches that are generally centralized but one could of course imagine that if we have an estimate of the the initial distribution of robots that can be obtained in a decentralized manner we could we could imagine the system also being somehow developed in a decentralized manner it's approximative and completeness is only guaranteed in the approximative sense so when the system converges with very high probability we are approaching towards the distribution of proportion of robots over tasks that we desired at the outset with market-based approaches well cost estimates can be local but auctioneers are generally centralized um but estimates don't have to be batched or bids don't necessarily need to be batched so we can think of these doing these things in highly decentralized manners there are different ways of executing the auctions which can be which can produce either sub-optimal or also optimal assignment results and the completeness really depends on the task reserve prices so whether or not the reserve prices lie beyo or above below or above what robots are currently bidding or offering and finally our single fully decentralized approach was the threshold-based approach that i described previously it's clearly sub-optimal we don't have any guarantees on this and we also don't know whether or not these systems reach completeness but the great advantage of the system is that we require absolutely no centralized computational units and can be executed based only on robots or agents local sensing and actuation capabilities so with this i'd like to conclude with a few pointers to further reading so there are a couple of seminal papers um specifically on market-based and threshold-based task allocation algorithms i also highly recommend the first paper in this list of bullets uh bullet points here the formal analysis and taxonomy of task allocations algorithms for multi-robot systems it's a really good overview of all the various problems you might encounter and what what formal problem instances these map to and why certain algorithms are better suited than others so it's a really interesting paper for its overview that it provides and then i also listed a couple of more recent approaches that tackle a couple of the challenges that i highlighted at the various stages throughout this lecture so with this i'd like to conclude this lecture and as always look forward to speaking to you in the next one

Channel: Prorok Lab

Views: 694

Keywords:

Id: pWPZfmqcJA4

Length: 63min 8sec (3788 seconds)

Published: Mon Feb 15 2021