Epipolar Geometry Basics (Cyrill Stachniss)

so ha I want to look today into it people are geometry and give you an idea what a people are geometry is it's something that we use to if we work with image peers in how to tell give us some information how about a point from image number one for example is an apt into the image of image number two and this simplifies us the third for corresponding points for example so again a camera appear maybe a stereo camera as the one shown over here but this is just kind of one possible examples there are other ways of camera appears for which that actually applies and so what we typically have is we have one camera sitting in the projection center here the second camera sitting in the projection Center over there and the two rays intersect at one point this was a cope linearity constraint that we have been exploiting in the when computing the essential matrix as well as the fundamental matrix and this also will play a role in here and what we typically are faced with in reality that we have an image pair so two images and we look into image number one and maybe we find a feature point and say oh this is a promising feature point let's see if I find this point in image number two and then I need to search for this point in image number two or where should i search for this point do I need to search the whole image or can I reduce the search to in the second image and it turns out that a people our geometry is actually helping us with this so given a point as seen in the first image and we have pixel coordinates at this point only we want to find where is this point in my second image and for this I can exploit the so-called a people are geometry which takes into account the stereo pair the baseline vectors where points are mapped to and that the the direction rays also lie within planes and Hubble's plane from the 3d world I actually mapped in to the so the applaud geometry is typically used to describe the geometric relations in image pairs and one of the key things is as I said it enables an efficient search when I look for corresponding points and it actually turns out if I have a point an image number 1 1 and want to look for a corresponding point in which number 2 I can reduce the search from A to D search space searching the whole image to actually a 1d search so I just need to search one one specific line so it reduces the search space from 2d to 1d this has several advantages the first one of course it's faster that's obvious well the second one is also it reduces the possibilities for making data Association mistakes because you may have multiple objects in an image which are self-similar looks similar to each other and if you then have one corresponding point of one point in one image you want to look for a corresponding point in the other image you may have multiple objects which look identical so they will generate very similar feature descriptors but through the restriction of the search from the whole image onto a lion called ap polar line as we will learn later on will allow us to reduce the search space and therefore may result in the fact that only one object of the self-similar ones is actually intersected by this EP polar line and has definitely the chance for reducing the possible mistakes that we can do when making data associations so this is a figure over here that will actually pop up pretty often in the course here so we have our image number one over here our image number two over here these are the projection centers of Prime and or double Prime and this is my point X I which is my real-world point in the real world so this point X is mapped over here in my image number one and this is projected to this point here in my image number two and I may have a second points Y over here which is mapped to a different location and a different location over there and what I now want to do is I want to go through the individual elements of elements of this epipolar geometry drawing here and then we can derive certain properties at those points what actually has to discuss them so the first thing which is important is the EP polar axis the people that access it the line connecting the two projection centers so it was like the baseline that we had before it is something we call the EP polar axis and then we have a plane which is spanned by O prime Oh double Prime and let's say our point X that we are considering here as a point we are seeing in both images and this is actually a plane the so called a people are plane so if I have a corresponding point this corresponding point will actually generate an a people are plane but different points in 3d ball will lead to potentially different a people or planes that that we may have and then we have certain points which are the so called EP poles and a people in every image and the AP poll is the projection of the other cameras projection Center into the image so I'm here projecting the projection center of camera number 2 into the image of camera number 1 and the other way around so this gives me my so-called AP polls and then we have something which is called an EP polar line and this ap Pula line is the projection of the EP polar or the intersection of the AP polar plane with the image plane of an image so let's say I have the AP polar plane which has been generated for this point x over here for this point in the real world then if I intersect this plane with the image plane one I get this a people align over here and the same over here I get this a people align over here which is intersection okay continues like this with intersection of the a people are playing with this image plane over here and this is an important element because as we will see later on this will be important for finding these potential correspondences and we can see actually why is this the case so let's say I have this point x over here and I want to find other corresponding points in my image number two so if I just have this image here I have basically a direction vector to this point x over here but I do not know is where is this point actually on that line I just know it's somewhere on that line between from here to here somewhere over here so if I take this point now you and slide this point you on this line and predict this point you into the other image this point will actually slide along this a people align cursor my moving X closer here this a people align will move closer to the AP poll if this point goes further and further away this line goes further in this direction and so wherever this point X is I know it's on this line this this line here will be projected to this line over here in the image which is exactly the intersection of the e people are playing with this image plane so I can reduce the correspondence e of the search to this EP polar line and this is what it in the end will actually boil down to okay so the EP polar axis just kind of defining the important elements that we discussed a bit further is the the given through the basis which is the connection of the true protection center so that's pretty easy and pretty straightforward the you people are plane is a plane defined by three points by the true projection Center that's always the case and then by the point in the 3d world which is defining this a people are plane so it's actually very similar to the Koch linearity constraint that we discussed for deriving the the fundamental matrix before and so why is this thing called the EP polar axis if I choose a different point in the 3d world then those two points X Prime and O double prime are always part of the AP polar plane but this point X not anymore so if I take Y I still have the plane which connects these two points but a different point in the real in the 3d world so I can generate this plane by basically rotating a plane around this axis so I have my two cameras the connection like this line between the two cameras and then the e people are plane I'm Ken rotating around the axis are all the planes in the 3d world which have this vector here in common so every point that I have basically now if I if is the exactly horizontal line it depends basically just on the height of that point which will give me define the a people are plane and I can express all the EP polar planes by a plane rotating around the axis and therefore the thing is called the AP polar axis and so I had the AP poles which were the projection centers of the the projection set of camera number to projected the image of camera number one gives me the first AP poll and the other way around so the projection center of camera number one projected into image number two gives me the other AP poll and then I have this EP polar lines which are basically the projections of the of the race which pass from camera number one to the corresponding point projecting this line into the camera image of image camera number two and of course the other way around this gives me the two AP poll aligns so I can also write things as the AP polls is the intersection of the of the of the bases with the image plane number one and the no I'm sorry the AP poll is the [Music] line the EP polar axis which is intersected with the image plane of image number one and this is the image plane of image number two so this is basically the intersection of the line with the corresponding plane and also the EP polar lines as I said before I can express through an intersection of the EP polar plane with the image plane of image number one for the image plane of image number two and let's basically be the element that we have here so this was if people are plane this would be image plane one image plane two and the corresponding AP polar lines and the EP polar line is the thing we are typically looking for if you look for correspondence YZ or this is kind of the key element in here and also the AP poll is an important point because this is just the intersection point of all epipolar lines within an image so in the EP polar plane under this somewhere before distortion-free lens are the two projection centers the point that I'm considering at the corresponding point occurs EP pull-up line depends on that point the EP polar lines because it's the intersection of the e people are playing with the image plane the EP polls itself because the projection centers lie in the plane and this is the intersection of those points with the image planes and the image points itself so where the points are connected and these are all the elements that look in the EP polar plane so you can see the EP polar plane is a constraining element that we can exploit when we look for search for certain things and this is especially important for the task of predicting the location of a corresponding point in another image and that's the point where this is important for us because it allows us to restrict the search space if I look for a point in the other image so in the end this reduces me the search space from A to D search space of one research space so if my task is to predict the location of point that the where the point X is projected into image number two assuming I know where there's an image number one I know that the it must be the a people align a a the people a plane is defined through that point given this point X prime so what I know from the first image so the intersection of this a people are plane with the image plane of image number two where I'm looking for the correspondences will leave me the a people are line in image number two so I just need to search along this a people align in image number two in order to find the potential points I'm reducing from searching the whole image to searching along a line so the next part I want to look into computing the key elements of this EP pulaar geometry and these are important quantities that we have described now but we haven't really seen how we can actually compute them so can we actually sit down and write down the individual elements and we want to look how we can actually compute them based on the projection matrix P or based on and based on the fundamental matrix so the properties that we know from our camera and how can we describe the different elements of the people our geometry so the first thing is the the AP polar axis B and again the EP polar axis can be directly obtained if I know my projection vectors and here's explicitly expressed no more genius vector expressing that we only know the direction but we do not know the length of this axis and so once we have the projection centres and we know we can obtain the projection centres based on our projection matrix then we can also compute directly the AP properly any people are excess sorry vehicle access so the next thing we do is we want to compute the EP polar lines because these are interests things that we are to be interested in and so if we if you remember we are now here in the image plane and if you look into homogeneous coordinates you may remember that the simple test if point lies on a line in the 2d world is simply by computing the product of the two vectors with things are there so X is our point transposed times the line must be equal to zero this is exactly the case when the point lies on the line so we actually get this expression X prime transposed L must be equal to zero so this is the constraint that this line must fulfill if it is the EP polar line for the point that I'm observing actually in image number one and now we can exploit the Copelan arity constraint that exists that be that we actually exploited for computing the fundamental matrix at this point the the vector X that is defined through this x1 and the vector which comes from the other camera must actually form the plane which is already people are playing so and just by comparing this to the köppen arity constraint that we need right through the fundamental matrix we can actually see that in this equation this is the same part this is the same part so this part would actually also be the same so it means the line the e people align described in that image in image number one it's nothing else than the fundamental matrix multiplied with the point generated in image number two so this is the e people align in this image number one so just by comparing those two expressions I can actually see how the line equation must look like and they can compute the line in more genius coordinates just by multiplying the fundamental matrix with the point stemming from the other image that means the EP polar line can very easily be obtained for just using the fundamental matrix and the image the point in the other image and here the important thing is so this is a fundamental sorry the a people align in which number one and the only thing I'm exploiting is the fundamental matrix and the point in image number two so knowing the point image number two gives me the e people align and image number one and obviously I can do exactly the other way around I can compute the e people align in image number two only using the fundamental matrix and the point in image number one so this is the other important thing that we need to do so we have the same for the fold for the other point so I can express the the line if you pull a line in image number two transposed gives M times the AP Paula the point in image number two must be 0 and so the only thing that I now need to do I need to have the transposition operator so I know here that the from from the X from this constraint the people are constrained I can then result that L double prime transposed must be X prime transpose equals F so the other side of the equation and so that the result of this is if I want to compute the people aligned it's just F transpose times X so I need to transpose the fundamental matrix and take the corresponding point so it I have the kind of symmetric part I can compute the a people align the image number two given the fundamental matrix transposed and the point in computed in image number one so and this is this reduction from the 1d from the 2d space to the 1d space so this is our prediction where the point must be in the other image and just by searching along this line I can look for the other corresponding point and this is kind of the key thing which speeds up our search when we look for correspondences so the image points must lie on our a people Alliance and what we basically did so far we just exploited this cope linearity constraint using the fundamental matrix and then can derive symmetrically and in the AP polar lines in image number one or an image number two just using the fundamental matrix and the image coordinate in the other image and this was very easily arrived through the constraint that we the multiplication of a point in 2d and the line in 2d defined homogeneous coordinates must be zero if the point lies on that line and this is the constraint that we actually have exploited here so now we can we know how to easily get the a people are lines just by exploiting the fundamental matrix the next thing we want to look into the AP pulse again remember the AP poll was the production center of camera number 2 projected into image number 1 and the other way around and this obviously is very easy to do we just need our projection matrix and the projection center of the other camera and this is something that I assume to know if I know my the matrix P from the matrix P prime I can expect this one and from the matrix P double prime I can express this projection center and then I can directly compute the AP pools so also computing the AP polls is something which is straightforward to do and these are the points as I said before the EP pools here and there there are the points to which all AP poll ions pass and so that's kind of an important thing that we actually need to look into so all AP poll aligns path through that AP poll that means for all lines l1 the EP pole transpose x this line must be 0 and the same time for all AP poll aligns image number 2 the AP poll of this image number 2 transposed with the AP poll Alliance which are generated must be 0 ok so this is an important property that we have this means the AP poll in an image multiplied with the fundamental matrix and upon the other image must be always 0 so given through this equation so the constraint that we derived that all ap polar lines multiplied with the EP poll gives me 0 this minds this means that the EP poll here will bring this expression always to 0 all right so independently on where the other point coordinate is because we know that all a people aligns path through the EP pool so at the AP poll it brings everything in here to zero so with this equation this means nothing else that the AP poll is in of the first image is the null space of my fundamental matrix transposed over here so the the null space of the fundamental matrix transposed is the e people the first image and the null space of the fundamental matrix is the AP poll in the second image and this tells me something about the eigenvectors that I have of F so the EP polls are the left and the right eigenvectors and of the fundamental matrix and span up the null space of that fundamental matrix so they occur they are item vectors of F and they are those eigen vector that corresponds to the eigen values of zero and so these are important properties that I can exploit when making considerations about where points have to lie what is the null space of this fundamental matrix and do certain investigations about these matrices and the whole geometry involved this actually brings me to the end of my very brief and rather informal introduction about a people of geometry so what we used with this epipolar geometry we used the image pair and try to derive what can we say about the geometry just by knowing that these are straight line preserving cameras it means uncalibrated cameras we discussed kind of epipolar geometry and what the key properties are there such as the e people if you polar lines if you polar axis and AP poles and the key thing why this is important is that it allows me to look for corresponding point than the other image so if I know the relative orientation so fix the fundamental matrix and I know an image in camera number one I just have to look along one single line in camera number two to find the corresponding point on that line I don't need to search the whole image and this substantially reduces the workload that I need to do as well as the likelihood of making data Association mistakes and therefore if people are geometry will always pop up and you look for correspondences or if you want to constrain certain things in image what must lie in which in which part of the image about which points you can say can make certain expressions like about the AP polls which from the null space and these are all things where a people are geometry plays an important role thank you very much for your attention

Channel: Cyrill Stachniss

Views: 7,392

Keywords: robotics, photogrammetry

Id: cLeF-KNHgwU

Length: 24min 35sec (1475 seconds)

Published: Mon Apr 20 2020