Bundle Adjustment - 5 Minutes with Cyrill

[Music] good morning today i want to explain in five minutes what bundle adjustment is about so bundle adjustment is a state estimation technique that is used to estimate the threat location of points in the environment and those points have been estimated from camera images and we do not only want to estimate the location of those points in the world we also want to estimate where the camera was when taking the images and where it was looking to so every point typically has three coordinates an x y and that coordinate in some coordinate frame and the camera and can be described by six parameters and what we want to do is we want to estimate the location of the cameras and the points jointly so that the error of where the points are actually projected to is minimized so what kind of error are we looking into we're looking into the so-called reprojection error that means we assume we know the location of the camera and the location of the points in the environment and then we are projecting the points into a camera image and this gives us a pixel coordinate of that point so where would that point be projected to if my estimates would be correct and then we compare this location to the actual location where we observe this point in our image and what we're trying to do we're trying to minimize this discrepancy and we minimize this over all combinations of observations of feature points treating them typically as independent of each other so this leads to a large least squares problem that we need to solve and this technique which has been developed in photogrammetry in the 1950s and then has been used to solve a large number of problems then has later been kind of redeveloped in the computer vision community and later on in robotics as it is very similar for example to the visual slam problem and the button adjustment approach is a statistically optimal solution making some assumptions such as gaussian noise and the dependencies how the mapping of the features into your camera images actually happen also assuming known data association there are several assumptions which are not necessarily justified in the real world such as known data cessation so knowing which point i'm actually observing at an image location knowing to which feature point this corresponds to in the real world that is something that is often not the case i mean to estimate this data association but how do we solve this approach button adjustment approach so we are using a least squares approach which typically leads to very very large systems so we need to solve a large system of linear equations and that is so large that typically we can't solve it unless we exploit the structure that underlies this bundle adjustment problem and the key thing to exploit in here is the sparse nature of the problem that means not from all camera locations we can observe all features so there's only a small number of features that we're observing in every camera image and so there's only a small number of dependencies and if we build up our big system of linear equations a lot of those entries in this design matrix will actually be zero and as a result of this we can exploit this sparsity and need only to take the non-zero values into account which allows us to solve the underlying least squares problem in a much more efficient way if you solve these problems in general so taking structure for motion systems that use bundle adjustment to perform the minimization then finding the data association is typically the computationally most complex problem so solving this least squares problem can be computationally demanding um but it's typically not the limiting factor in most applications so looking for data cessations in your images in your observation is still the part of the problem which takes most of the time and this is also something where you can easily make errors so if you screw up your data association then you will not converge to the correct solution so practice you again will need robust state estimation techniques or robust kernels integrate them into your least squares approach in order to be able to deal with a certain number of outliers in your data association so that was useful and gave you an idea how bundle adjustment works thank you very much for attention

Channel: Cyrill Stachniss

Views: 11,953

Keywords: robotics, photogrammetry

Id: lmj2Jk5tl60

Length: 4min 52sec (292 seconds)

Published: Wed Oct 07 2020