Photogrammetry I - 16a - DLT & Camera Calibration (2015)

okay so welcome everyone I would like to look today into the problem of estimating the parameters of a camera this can be the intrinsic parameters you can also be the extrinsic parameters and will basically propose or present you two methods the first one is a direct linear transform which is a technique given a few known control points in the environment so points in the 3d world about which I know they are 3d location how I can actually estimate where the cameras and what the parameters of these of this camera is so the intrinsic parameters likely the camera constant or the principle point and different aspects like that the second technique x method is a different technique that one doesn't use a natural scene about which we know some non control point but it uses a special pattern so basically a to do is called a checkerboard pattern so like a chessboard with black and white pixels and you can actually hold this pattern in front of your camera and by exploiting the knowledge about this checkerboard we can actually estimate the intrinsic parameters of the camera in this way calibrate our camera before we go in the field do measurements and then do not require special control points during operation so what's the idea consider we have images of a scene and we know something let's say the location of those red crosses over here there's something which you can call control points or points about which we know they are three location in the world and the question is if I have such an image can I actually tell something about the parameters of the camera so based on these images if I know the location of those images can actually tell from which location this image has been taken and can also say something about the camera constant or other intrinsic parameters of the of the of the camera which was used to take that picture and this exactly what we are addressing so given some known geometry about the scene can we actually estimate the parameters of the camera intrinsics as well as extrinsic parameters so this is in this case a single image it's just two buildings nearby which look similar but the scene looks and reality like that so what are things we can do if we have such a technique in hand in order to estimate the parameters of a camera so intrinsic as well as extrinsic s-- so several things are possible the first thing I may be interested in is just calibrating my camera so I have a camera I don't know anything about and I actually want to estimate the intrinsic parameters let's sum one of the topics that we are actually looking into today the fact is thing what I might get interested in I want to estimate the position in terms of X Y that position and the three angles for the heading from where an image has been taken so if I know something about the build a building and I take an image from the building can actually estimate where this image has been taken from or the other way around if I know where a camera is from the scene can actually use it to estimate the 3d location of objects in the scene so these are all relevant problems that we address in we are looking today into a camera calibration and estimating at the same time the extrinsic parameters given some information about the 3d scene and we do that again in two different ways one if we have control points that mean we have some point in the environment about which we know their X Y Z location and the second technique is something where I'm not necessarily very interested in knowing certain things in the environment I take a 2d pattern like this checkerboard with black and white areas and hold that in front of the camera take multiple images of this checkerboard and then can use this to estimate the intrinsic parameters in this case I'm typically not interested in estimating the extrinsic parameters also I may need that during the process but the result are the intrinsic parameters of the camera and not the location of the camera so we start with the first task which will lead us to the so-called DLT or direct linear transform what we want to compute is the extrinsic and intrinsic s-- of my camera so where the camera in the world and what are the five parameters for the intro of the intrinsics and what i assumed to have be given is Corden it's of objects so I know the 3d location of 3d coordinates of certain points or certain objects in the scene and these are my control points this is given I need to know this is background information in order to do this process then what I observe or what I measure are the XY location of the pixel in my image that correspond to the objects for which I know their 3d coordinate so basically take pictures of a known object and then I want to use this information of where this to which pixel the objects have been mapped to in order to estimate the parameters of my camera so the word direct linear transform is once used for this mapping process that we discussed last week but it's also used for estimating the parameters of this of this process so again we have our camera we have the kind of the the reference system of the camera so where is all the where the camera is with respect to my world reference system also I know where they're heading for we have our image plane and we have an object and another 3d location of the point and by measuring where this point is mapped to in my image I can actually estimate where the I can estimate the parameters of that mapping in order to make sure this point is actually mapped to this pixel location so this was the mapping we have the point in my image it depends on the calibration matrix which is an upper triangular matrix for the affine camera it has 5 degrees of freedom so on the main diagonal we have the focal lengths and the kind of difference in the scale for the x and y coordinate we have this year and we have our principal point then we have a rotation matrix which describes where the camera is actually looking to and we have the location of the pinhole in my pinhole model in the 3d world and then any object point X is actually mapped to this image point and if I combine this into one matrix to be described as a matrix ii-in to 3 by 4 matrix so this is a 3 by 3 matrix this is a 3 by 3 vector so this describes here a 3 by 4 matrix and this matrix realize the projection from the 3d vault to my to my camera image okay so that's a task that I want to do and this matrix P knowing calls all the end extrinsic and the intrinsic parameters so these are the encodes the intrinsics and the R and the x0 are the extrinsic parameters and I want to estimate that at the same time the intrinsic and the extrinsic from known correspondences so knowing the 3 location of the object with the scene and knowing where the point of some aptitude and this is all in homogeneous coordinates therefore the 3 by 1 and a 4 by 1 although this in the three-dimensional world and this is in the 2 dimensional world so it's the task or the task is to estimate the inorder orientation of the intrinsic and the extrinsic s-- that means X 0 and R and the calibration matrix K so they're kind of two different ways to do that we will look today in or start with something which is called the direct linear transform in this case I'll only observe where an object is mapped to and I know the coordinates of my control points I assume this to be given so this is given this is what I measure when our estimate is are the red elements so drip three degrees of freedom for the rotation matrix three degrees of freedom for the translation and five degrees of freedom for my calibration matrix so that's a task that we will address in the first hour today based unknown points the second part of the course today we will replace this with our checkerboard and then a few things will change in here we can simplify a few terms and then do the fill the checkerboard there's a second task that is pretty important it's called space resection or objective three-point algorithm are two names for exactly the same thing oh look that's net in German and what we are trying to do here is we only want to estimate the extrinsic s-- so where the cameras and where it's looking to given win okay that's something which will happen on next Tuesday so they obtained a week so this is just kind of an outlook if we know the intrinsic s-- our problem gets easier we only need to estimate the extrinsic parameter there's something we are not going to do today it's just an outlook of what will happen next week we address today this problem we want to estimate all the parameters at the same point obviously we need more observations if you try to solve this task so we need to have more ignore more control points and need to observe those points in our images and we need the last for the projected three point algorithm but that's something which will actually up next week okay so the first thing we may ask ourselves how many points how many control points do we need to know so each of those points gives us how many observations why so these points here are given so let's expand it so those points so this is just a normalization constant this is a three coordinates that's true but those are given so we have and those are the guys we are actually observing yeah so we only can measure the pixels in our image or the location on our sensor so we only get kind of two of them so again we're immobilizes you see this is one and these are three parameters these are two parameters and you can perfectly see these relation if you write down the pics the mapping from the 3d world to the pixel coordinates in our image in your treating coordinates so we have all those elements of the projection matrix the XY point of the 3d world and that's what we measure this is what our model tells us so we basically get two equations for every point that we have okay so if you have a calibrated camera it means we have six unknown parameters three for the translation and three for the rotation that means you need at least three points that we need to observe in order to estimate where the camera is so it at least three points the 3d location of three points in the environment we need in order to estimate the extrinsic of a calibrated camera giving me know the calibration and this is something which will happen next week we're looking today into the problem of the uncalibrated camera that means we do not know the calibration matrix the calibration matrix has five additional degrees of freedom that means we have eleven unknown parameters over here in order to estimate those eleven unknowns we need to have at least six points now in order to estimate those unknown parameters because we can have our mapping where we get two observation equations for each point and we have eleven unknowns we didn't have at least six points in order to do the estimation and that's something which we are going to solve with the so-called direct linear transform and this is exactly the task that I'm one address now okay again estimate with eleven elements of P so P is only eleven degrees of freedom because it's homogeneous so one is for the normalization or voters twelve elements or we have the 3d location X with index I of at least six points and we have the observations where this point is actually mapped to in our image okay so the first thing we do is say okay let's simply write down what this matrix actually means so we can actually write this down at the three by four matrix with our twelve elements of our matrix the first thing we do we just rearrange will just rewrite those equations by saying okay let's define a new vector a which is corresponds to this row over here vector B which corresponds to this row and the vector C which corresponds to this row if you do this we can simply write the same thing as a transpose B transpose c transpose but these are the individual vectors so this vector a is p11 p12 p13 p14 so we transpose it so it becomes a row and so the colors indicate the correspond of these this is just a definition how we defined it nothing has happened except of rewriting this in this form if you use this way of writing it ups then we can actually specify that this point in homogeneous coordinates so you IV I WI can be expressed by multiplying this vector with this vector over here it's corresponds to you the vector B transpose Phi X I gives me the second coordinate and C times X I gives me the third coordinate okay so no no black magic has happened we just kind of introduced some auxiliary variables a B and C in order to simplify our life okay that actually come like this so this is just copy-paste what we have written in the previous slide through this part let's copy-paste and we know that this corresponds to X I Y I normalized we want to normalize this this means we take you I divided by W I and VI divided by W I so I can say X I is you I divided by WI his X transposed X I this line divided by the third coordinate and I can do exactly the same thing for the second dimension well the only thing what we did so far we looked into what happens if you do the multiplication of the projection matrix and our part of the three worlds we introduce this exterior variables a B and C and then we can express the court the x coordinate and the y coordinate where we expect this point to end up on our image plane and all chip in this way okay okay so just put them here and then we can do a slide rearrangement can multiply with it by this expression so this one goes over here I can subtract the remaining expressions also goes over here and equal M equals to zero so we have x times C transpose X I minus a transpose X I equals zero and you can you exactly the same thing for our y coordinate pick our y coordinate V transpose X I divided by C transpose X I and rewrite it exactly in this way okay remember those ex were the point that we know these were are the coordinates of our control points what we're interested in is estimating this projection matrix P which consists of a B and C so we now have this here equations and those equations are actually linear in a B and C so I can slightly arrange that those things by basically performing the transposition and then it's minus X transpose a plus X I X transpose C and the same thing down here just with B so I have my unknown parameters a B and C and a system of equations which is linear in the parameters a B and C okay so that looks good that we something that we actually can plan to exploit so what I do now is estimate that only a B and or a B and C all at the same time so we can again slightly rewrite that by saying the introduce a vector P which is just a B and C stacked on top of each other which simply the vector of all elements in the projection matrix so it's p11 p12 p13 P 1 4 P 2 1 P 2 2 and so on and so forth all stack up at the vectors so this is a in this case 12 by 1 vector krispies has 12 elements so these are the rows and column vectors one below the other so it's a 12 by 1 vector over here and that means we can actually rewrite our system linear equations that we have here in a special way with two equations with some magic vector a transpose times P so P contains all the elements in a B and C and a second a vector a for the y-coordinate times the same major P equals zero and we get that if we define this make this vector a B and C in this way so this may sound a little bit counterintuitive or two why do I actually set up something like this I hope this will become clear because it will simplify the method we are going to do substantially but the first thing is I kind of provide you this two vectors a and I claim that this equation it is exactly the same equation over here so just climb that which actually verify that this is correct so let's simply give it a try and perform this computations and see if we actually end up with the same system of equations okay so let's verify that so this was my ax B I just the X Y that coordinate or the negative X Y that coordinates of the of the point in the 3d world and one and a couple of four zeroes and then the product of the point in the image coordinate system and the vector the coordinates of the back door of the point in the 3d world and I multiply those just all twelve elements of my of my projection matrix we just need to check if this if this equals zero this gives me the same constraints as a system of linear equations so again this corresponds to a this corresponds to B this corresponds to C so I can actually rewrite this vector as again ABC and those guys over here is this point X I transpose this is a zero vector for D and this is my point x times X transpose so the x coordinate of the observed point times the coordinate of the coordinates of the object point of 3d world what I then can do is if I multiply this vector with this vector I get minus X I transposed a second element is zero no dependency on B and then XK I forgot to remove the prime this is a mistake I try to unify unify permutation I forgot to remove this prime so this is exactly the same variable so this Prime should go away and so this equation is exactly the same equation that we had over here and this is we go back on our slide exactly the first line over here so this mod performing this multiplication leads us exactly to this line over here ok and we can do now I take the same thing in exactly the same way for y then we don't have the coordinates of the point here as we had it before for X it just moves over here and we have the Y coordinates over here that's how we define the vector a Y and the rest is called happens completely next in the same way and we get the same system of linear equations again the prime here should be eliminated so what we have shown so far is that the system of linear equation that we derived is exactly the same as those two constraints over here ok everyone happy up to that point if the nothing very fancy we just rewrote our equations by defining some new vectors and then show that the product of this vector a and the vector rised matrix is exactly the same condition as which resulted from all system of linear equation so we do now we have that done for one point X I so we could do it for all points so we said we need at least 6 points so we need to do that for is least 6 points so every point has the same matrix over here because they're all taken from the same image so they have the same matrix P we just have different coordinates over here where the point is in reality and we are we observed the point in the image so we simply do that for at least 6 points or 4i points at least 6 and then we can actually again stack them all up in form of a matrix over here and multiply this with this matrix P so we have here a 2i times 12 matrix and a 12 by 1 vector we say this product should actually be 0 okay if you take real observations from the real world the thing will not be 0 as you all know so we will get some contradictions take if you multiply em with P we will have some contradictions we won't be 0 so we are trying to do so it should be obvious to you we take W transpose times W we actually want to minimize that so we want to estimate the projection matrix which minimizes the sum of the squared errors so that's our goal find the projection matrix that minimizes the squared error that that results from those constraints over here and minimizing this which many mind exists just through replacing the definition in here and then we have yeah that's that's correct the only thing we need to take care about this matrix P is this matrix P is of no genius matrix what does it mean to have normal genius matrix in homogeneous coordinates a matrix M and a scaled version of the matrix M are exactly the same so have a scaling parameter in here and add up and this P all the elements of my matrix what I'm basically looking for so this actually shouldn't be written with M but of course is arbitrary but I what if you want to say P equals number P so because we talked about the projection matrix so my vector P here can be abitt orally scaled it's really nothing I want if I want to minimize an error so I add the additional constraint that can be fixed a normalization and this is by requiring that the norm equals to one there's one way for kind of fixing the scale so we do is we have to solve the least squares problem where we need to make sure that we had this problem that's actually basically what I was told before this squares problem let's look at a brag minimization so there's no direct geometric relation to what I see in terms of the errors and pixels that I have because we have this scaling in there we need to fix this scale parameter at some point okay so how do we solve the system of linear equations actually minimize the squared error but for that you can treat that as an eigen vector and eigen value problem and then use SVD or to solve that something you should have seen before nevertheless I would like to go through this individual steps so that you see the direct correspondence is how we can use SVD in order to to solve this problem because that's something that we will actually need in the remainder of the course so in the beginning of the photogrammetry to course we will use exactly the same technique in the same way for estimating different matrices when we look into stereo images and the orientation of stereo cameras okay so that's what what I have what we can do is we can compute the SVD of this matrix M what was the SVD so we have two matrices over here and here you transpose as VV transpose or identity matrices so you can see this works as a rotation and we have this matrix S which is a 12 by 12 matrix which is 0 everywhere except the elements on the main diagonal these others singular values this can be arranged in a way that those the singular values as one is the largest one and as one 2 is the smallest one those just computing the SVD of this matrix M this SVD decomposes this matrix into three main matrices and now we will actually exploit that SVD so this decomposition in order to show how we actually should select this matrix P so that minimizes the expression over here okay so this was our linear system that we wanted to solve but of course we will have contradictions in the others will be nonzero so what we actually doing we are minimizing the contradictions and we using this the SVD so this decomposition or to solve that problem if you apply the SVD we start with this expression over here and simply replace the W's by M times P and then we'll actually see that we can use the SVD directly to actually find the solution this happens as follows so the sum of the squared errors is w transpose W so I can say W equals M times P so this is P transposed M transpose MP just the definition of our contradictions then I use the SVD and decompose the matrix M as well as M transposed which gives me V as you this the SVD of this matrix times us V transposed this matrix P and now we had the effect that we said uu transposed was the identity matrix so this goes away so what I have is P I'm missing a transposed here so there should be a transpose it's back here on track so you goes goes away of F squared which is just the matrix because it's only elements on the main diagonal we just have the singular values on the diagonal squared so I can express this as my vector P the product of the 12 diagonal elements this SI squared and then this vector V okay so whatever but I'd rather just it so far is using the definition of the squared errors using the SVD applying the SVD to the matrix M do some trivial replacements I'm just saying you transposed use the identity matrix and then squaring the elements on the main diagonal and then I actually end up with this equation okay so this was just copy/paste from the previous slide now I know that those SVD s at the L sorry that the vectors in this V are all orthogonal so VI VI transpose is 0 if they are identical is 1 if they're not identical so if we now choose so they are as we are going over the main diagonal this gives this is 1 all the time so the remaining part the part which remains is if we set our vector P to the 12 element then this one then all will be 0 except the last one and so the squared error turns into the corresponding singular value squared so just by choosing P as a vector which corresponds to a small singular value this sum simplifies in a way that only the smallest singular value remains so it's as small as one of all of them so the squared error reduces to the square of the singular value so by choosing P as the again as the eigenvector corresponding to the smallest singular value as IV we minimize the squared error so what we basically in the end need to do in order to perform this minimization we just compute the SVD on this matrix M and choose from this matrix V the vector corresponding to the smallest singular value and this is in the solution to our matrix P or to our vector P which then turns into the matrix P this way for using is the the SVD in order to solve this system of linear equations or to minimize the squared error that was correspond to the system of linear equations that minimizes the contradictions for that system we could do that do not need to iterate here you can do that in one shot okay so in this way we have our estimate of P which is V 12 this corresponds to my estimates of a B and C so we directly get the matrix and the estimated elements are the elements here of my vector a a B and C right so what I have done now I have computed these projection matrix what I need to take into account if I want to do that so this matrix M is of rank 11 May 11 unknown parameters you know to have that I need to have at least number of points which is larger than 6 and I don't have any gross measurement errors in there so only if my noise in there if I have measurement noise in there I may be off but also need to take into account that there are some constellations in which I can't compute a solution and one of the critical points for the for the DLT especially in arrow photography is if I have non control points on the ground and I want to use those non control points on the ground in order to estimate the parameters of my camera and they are all lying on one plane all the control points on the ground lie on one plane then I actually can't find a solution and we can actually show that okay so I have all my elements in here this corresponds to this matrix where kind of those are those large twelve dimensional vectors as we defined them before and if I now assume that they all lie on a plane for example z equals zero take any plane but you can directly see that if Z equals is equal to zero so they all on a ground plane or on a plane then all those elements here will turn into zeros and this holds for all points so this holds for all elements of the matrix all those columns will turn into zeros and then I have a reduced rank and can't find a solution so the important thing for the DLT is that when all points are supposed to lie on one plane so even if you take a different plane another that equals zero plane you can actually observe the same effect but he can nicely see that those vectors are those those vectors over here within this matrix turn out to be zero and they can directly see that has a reduced rank but it holds for in those points lie on a plane they're actually more complex configurations if they like on a twisted cubic curve all the points and the projection Center but there's something you can't see that easily and this is something which rarely happens in reality but the points should not be located allocating the plane is something that you can see that you can leverage of which happens in real-world problems so if you fly with an airplane for example taking aerial images and you want to estimate the parameter of your camera and where the image has been taken from six non control points on the ground then those six control points are not supposed to line on the plane even if you're very near a plane you get in the stable solution okay what we have now we have P we have P but what they actually wanted to have was kr and x0 right we wanted to know what are the intrinsic of our camera where the camera are looking to and where the pinhole 3d world so what we need to do now is we need to decompose P to estimate those matrices here to know what for example the camera constant for our camera is okay how are going to do that let's look to how our production matrix look like so P is known but not the individual elements so it must be K multiplied with r multiplied with this matrix we have V a identity matrix over here 3 by 3 and then the translations over here and we can actually write this P as some home ography 3x3 matrix h just ignore than 0 don't we don't leave that 0 here for the moment and a vector at 3 by 1 vector ok so then this element over here corresponds to K times R times the identity excuse me H of this equation over here and this H is minus K times R x0 the head is missing you sorry okay so the first thing is how do we obtain the projection center that is pretty easy because K times are are exactly those elements of the matrix so these are the nine elements I can just pick from my matrix P just picking those line elements of our matrix P I can use this in order to compute the projection Center just by saying okay x0 is simply minus this matrix H inverted times H so I basically take my projection matrix on our rubber grip which has one two three four five six seven eight nine one two three elements those over here directly form H infinity and this was my vector H so the only thing I need to pick this out I invert it which gives me H to the power of minus one finishing and I have my vector head over here H then I can use those to you know to compute the projection Center so that's something I can obtain easily from my protection matrix but clear okay yep so I only obtained a solution for P if the if the points will lie on the plane so I assume they don't lie on a plane so I can actually easily check and assumed to have computed the solution where this holds this block is invertible is you can see that from the fact that also if you know what that what it means what those elements are this is a rotation matrix which is always invertible and it's the camera calibration matrix which has on the main diagonal the camera constant which is not zero otherwise the projection everything would be projected into into the same point so the image would consist of just one single pixel in 0 0 0 and then we have all other elements are just the transformation of the system which is invertible and we have the the the shear and they kind of the shift in the in the camera constant so these all form an upper triangular matrix those elements can't be 0 on the diagonal so like an invertible matrix ok so that was easy this can be a little bit more difficult so what I now want to do is I want to actually decompose a matrix H into two matrices but an invertible matrix now when decomposing two matrices where one is an upper triangular matrix and the other one is the rotation matrix I guess you should have heard a technique which does exactly that and two famous ways for decomposing a matrix and one of them is one to pick so what are the two key ways for decomposing a matrix so let's case one which I don't want to use here because use a lower triangular matrix so let's knock triangular matrix QR decomposition that's illegal basically decomposes so if you haven't arbitrary matrix whenever a and this is whatever composed the matrix M and T in this example then this is a rotation matrix and this is a fine jeweler matrix so this is what I have rotation matrix triangular matrix my matrix H however it the other way around k and r so we would actually have to need a vet swapped what I can do in order to fix this I just take the inverse of H course the inverse is the inverse of all matrices which is the inverse of those elements the individual elements swapped and for the rotation matrix it's simply a transposed matrix and then K to the power of minus 1 then I exactly end up in this form of having a rotation matrix and my triangular matrix and K is a triangular matrix that should be clear of course all the elements at the bottom the large triangle are 0 so just by using this trick using the QR decomposition and of using the inverse I can actually break that down in this way compute R and compute K so what I finally need to do due to the homogeneity of K again need to do a normalization which I can just by dividing it through the third element so the ninth element of my matrix this gives me my calibration matrix so this is just based on the QR decomposition of a matrix that it decomposes the matrix into a rotation matrix and into a triangular matrix and I can exploit that you know to get this decomposition into rotation matrix and my calibration matrix the only thing laser if I do that we we said we have their tutor if you remember the lecturer last week there are two ways for defining the coordinate system of my cam of the image plane for a camera one was with a positive camera constant and one was with a negative camera constant you remember that correctly so if you remember that at all so we said we want to use a negative camera constant but this approach has no idea of my camera constant should be positive or negative so basically don't know what I get however the nice thing is we can actually easily fix this by introducing an additional rotation matrix which just does the rotation by 180 degrees by 100 degrees so what I simply can do is if my camera comes with the decomposition which ends in the positive matrix C I can simply add a different rotation matrix and rotated by 180 degrees just just this matrix over here and I multiply K as well as all with this matrix and then obtain the same result it's kind of identical of flipping the camera and moving the image plane to the other side it's just kind of two possible ways how this prediction can work and I just I need to define the coordinate system and then do this transformation the same way otherwise the camera would look into the wrong direction but mainly projection these projection equations do not know something about the field of view of that camera they just look into where how where they bear the point projected to given my the pinhole of my camera and where the image plane lies so if I flip it to the other side it could also flip the camera this is just the thing we need to take into account here okay so this was a DLT so let's summarize DLT in a nutshell so if you want to apply that what you need to know need to know your six points in the world which should not lie on a plane and you need to know those coordinates so the first thing you do you start with your matrix P which you don't know which you want to estimate and need to vectorize that next thing you need to build your SLDS system of linear equations or linear system of this matrix a which consists of the XYZ coordinates of every point and where the x and y coordinates of this point is mapped to into the image plane to set that up and this gives you your matrix pm for which this should hold once you build up your matrix M you apply the singular value decomposition on this matrix M ok compute the SVD which gives you those elements and then you take look at your matrix V take the eigenvector which corresponds to the smallest singular value and this specifies all those elements of your projection matrix there's actually something yeah should be swap but doesn't matter so the elements in this matrix V corresponds to those elements in your projection matrix third step then you have your projection matrix if you need more so if you want to know what is my camera constant Watson principle points what are the orientation of the camera you need to decompose this matrix P in the using this one first and then the QR decomposition so what you then do is if the individual parameters are needed you compute you to invert your the matrix H infinity so this part so this block over here this select you can directly compute the location of the pinhole and with the QR decomposition you can directly estimate the rotation matrix all spells the camera Kirby the the calibration matrix and this then encodes all the intrinsic parameters of your camera so kind of to summarize we discussed this what we now have we have a mapping from P to my camera constant rotation matrix and my projection centers in both ways so if I have load three elements I can trivially compute P and even if I know P we now have a way to do this computation so if direct mapping in both ways from P to the individual parameters in the other way around we are free to choose a sign of my camera constant so we can define our coordinate system in the way we want it if it's the image plane is in front or behind the pinhole and completely fine we know how to do that just by introducing this additional rotation the points are not supposed to lie on a plane and even if they're very close to a plane make it an in stable solution and what also should be clear is that the solution is not statistically the optimal solution so it's a pretty good initial guess but it's not statistically optimal why because I don't take into account the uncertainties of the my image point how I measure them in the image and the uncertainty of the points in the 3d world I can take that into account then I can't apply the SVD in the form which was used in here then you need to do a standard gauss-markov model and drive those parameters so kind of to summarize the key things what happened so far we presented a method to estimate the intrinsic and the extrinsic parameters of a camera given six control points or at least six control points the direct solutions we don't need to iterate that get our solution immediately we don't need any initial guess for doing that and we have kind of our nutshell handbook on how to solve that so that with four steps vectorizing computing matrix M singular singular value decomposition and retrieving the individual parameters and that's it that's all about the DLT but the key things to remember that's for the uncalibrated camera so we use this if we don't know the calibration parameters of our camera if we know what the calibration matrix look like so if we know K then we can do that better and this will be the topic of next week on how we actually do that the DLT is something which is a very central element in photogrammetry therefore the homework assignments will actually ask you to compute the full DLT for an example if you can go through those individual steps and actually do those computations yourself kind of rather important element and of course it's the standard way assuming that you don't have any nonlinear errors for estimating all the camera parameters from known control points need at least six you can have more obviously are there any questions about the DLT so far key things is what does it do what does it do it should be answer able to answer question what does the DLT do what can i compute with it which situation can I apply it how does the procedure works how many points do I need okay okay so making a five-minute break now and then we look into an alternative way for calibrating a camera if I don't want to know 6 points in the world if I use something kind of calibrating a camera before I take it to the field and do experiments but basically taking a checkerboard pattern moving a checkerboard pattern in front of a camera taking multiple images and then based on this checkerboard pattern estimate the intrinsic parameters of my camera we will rely on the technique that we have seen here so some of the steps in there will use exactly the same steps that we have used here but a few elements will be slightly different okay so five mile break and then we going to continue you

Channel: Cyrill Stachniss

Views: 33,612

Rating: 4.9488492 out of 5

Keywords: robotics, photogrammetry

Id: ywternCEqSU

Length: 52min 40sec (3160 seconds)

Published: Thu Jul 09 2015