Photogrammetry I - 14 - Homogeneous Coordinates (2015)

Oh what we are going to do today is to look a little bit into projective geometry or especially homogeneous coordinates I know that most of you or should have attended that already or have attended one lecture on that last year with what Casas but as the homogenous coordinates are kind of central for the remaining part of the course I thought of giving kind of whatever 60 minutes something around 60 minutes rapper or introduce them again wrap up and present you the most important component that you need to know for understanding the remaining part of the course so this is just kind of seize it a little bit it's a summary of the most relevant things that we are going to use here so the first thing is why are we considering homogenous coordinates and why is this something which is important and especially simplifies our life so we all started or most of the activities that we do especially from now on assume a camera which we use in order to take pictures and we want to analyze those images and then do inference about what we see what the geometry of the scene looks like maybe we have the cameras and other things in order to do that we need to model for our camera and the standard model is the pinhole camera model so just as a reminder that was kind of this quite simplified model so should be green you have a box here which is your camera there's a small pinhole and you have an object in front of your camera and save this tree and then we'll raise from the tree pass through that pinhole and generate an image here in the back of the camera so this ray goes somewhere over here this one down here the one in the middle here so that we get the tree which is then standing upside down something like this and the important thing is that here all rays passed that's central for this point the single point and all raised from an object to my image path through this boat and this concept that every ray go through this pinhole is something which is very important and that will actually exploit in programa tree in the program free course or until photogrammetry to actually and what we are now want to look into this kind of what we see what we get here my our image is actually a projection of the 3d world onto this 2d plane me do you understand the mapping to this plane in order to appropriately describe what we see so for example if we have a certain interest interest point somewhere in our image what do we know about where this points lie in the 3d world and if you can look to to an image that's the standard image taken from the house and we want to look into certain properties or geometric properties that hold in the real world the questions do they hold in this image so for example if there's a line with 3d world is this line actually met to align in this image what do you think so if we have a line in the 3d world it's this line mapped onto a line in this image so you can just let's say you randomly pick a line let's say whatever this line over here with shooting reality is very likely to be aligned with something then you can say to to your background knowledge how the roofs are constructed or that this is probably one line over here or the bottom line of the house and this seems that these seems to be more or less aligned also this is probably a straight line in reality in Smet - a straight line in this image so this for this image we can say this is a straight line is a straight line preserving mapping so straight lines in the real world are prisons in our image there are other things what about angles in the real world or angles in the real world met two corresponding angles in our image that is true because here at these piles over here so that these are all 90-degree angles and they are not mapped to 90-degree angles in this image so we are not preserving the angles that we have in the real world what about size of objects so the size of the object differs depending on how they are away from the camera there's basically determines how large the object is in the image so two object on the same size being at different distances of the camera are of course met at different sizes in my image so we don't preserve the length it is exactly just a summary what we have actually discussed here so we at least this image was straight line preserving so straight on his map to a straight line but the two other properties we said links preserving or anger preserving this does not hold for this mapping thus the straight line preservation of straight lines is always true for say standard cameras typical cameras so if we use wide-angle lenses especially or fisheye lenses you see very strong as tour foods you can perfectly see that on the horizon line that the horizon line is typically not a straight line any more or any other straight line that you see or picture through for example fisheye lens or wide-angle lens but for kind of mobile lenses or for the pinhole model we can assume that they are or they're very close to straight and preserving okay so if we preserve straight lines what happens about parallel lines parallel lines lines which are parallel are two lines so we'll go back to my house of you identify a few lines which are parallel or are they still parallel so if the plane which the lines lie is the same Orion has the same orientation that my image plane that's true and if you met some whales there this is not the case so parallel lines may intersect they may not stay parallel so I just indicated that by two things which are parallel so here with the rule starts in the bottom of the house for both sides you can see those parallel lines intersect they don't intersect inside the image but if a kind of extent that they intersect somewhere outside image okay so parallel lines in the real world don't necessarily stay parallel okay okay what we can do however we can actually rectify images so we can do a rectification of those images so for example if I want to measure certain elements on the facade I can actually rectify the image as they the image plane would be the same orientation than the plane for example this wall if you do that for this wall they can even do it for therefore the ground floor and then actually can do measurements for these relative measurements in those images so for example here seeing the tile so those images are generated from this image knowing something about the geometry of the scene where the cameras I can actually generate those rectified images but I can do the measure and then in the plane so here on the ground plane on this side of the wall or on the other side of the building but I can't do this in any part of the image but I can do certain rectifications with respect to certain planes that I have in my image so I can do that and then this allows me to derive some information about the geometry of that scene for example or do measurements in there so central questions that we are typically addressing and this all kind of the motivation of why we're using homogeneous coordinates later on or introduced them today and use them so central questions that we're interested in this deriving a relationship or a geometric relationship between the objects that we see in the scene and the objects we have in our so if you see something in an image what can we infer about the geometry of the scene or what kind of images will be generated if we know something about the world also kind of often we can't or depending on kind of what observations I have I mean for some parts of the geometry I may not it for some other quantities for example for just a single image inferring something about the deaths the PD you require some background information some background knowledge so maybe I'm also fine with the relationship between the points in the image and the Rays to the object the ray going from the image here to the object and just know that the object basically so long as rain I may also be interested in estimating where the camera is so given I know something about the scene let's say given I know the geometry of that tree and where that tree is gonna take the picture can actually derive from the picture the position of the camera and that's a for something that we will do in the remaining part of the photogrammetry one course so starting with that in kind of next week how we can actually estimate where the camera is if I know the geometry of that tree just from the picture that I have okay that's what we want to do is but the problem that we have is that the camera generates a 2d image from the 3d world so projects the 3d information onto my 2d image plane and this causes what we call those in the beginning a loss of information so we lose some information we can restore some of this information or estimate some of the lost quantities under certain circumstances so for example if I have multiple images there are multiple images of the same object I can infer something about the depth information up to a scale typically work depending on how much other information I have if I know some of the locations of the cameras from where they have been taken in the an external coordinate frame I can even resolve some of those ambiguities I typically need to know some details about the camera if I know the size of certain objects I can also infer something about the geometry of other things that I see in the world so if I have more than a single image I can actually do more that's something which we will see directly in the beginning of the photogrammetry to course when we look into using stereo cameras or two cameras or two images taken from different locations potentially from the same camera maybe from a reader stereo setup so if I take one image of you from this location in the second image of the same scene from this location and I can actually infer something I know I can estimate for example what is the movement of the camera it's one thing I can actually estimate or I can estimate what is the geometry of the scene or I can actually do both things at the same time and that's something we will look into how we can actually recover this information but the most important thing what we are going to explore this basically what I've drawn here or just illustrated by this effect so this is my image I have the Rays going from the image to my pinhole into the real world and I can just measure those rays so one of the pixels one of the points in my image basically represents the information along this array and I don't know where along this ray this information is only if I have multiple images taken from different locations like in this example over here if this is chemic location of camera one and camera two and by triangulation if I know the corresponding points between both images I can actually estimate the depth information and what we are want to look into today is how can we actually describe this projection or the fact that I have a point somewhere lies along this line in a very nice way so it's kind of try to set the focus and if we now go back and to the previous example that we had we said straight lines remain straight and but parallel lines may not remain parallel one of the examples they can actually also very nicely sees is this classical images of train tracks so those train tracks are actually parallel but they intersect somewhere it was so called the vanishing point so somewhere along the horizon there our vanishing point where those two parallel lines intersect never have two different parallel lines let's say two parallel lines which move in this direction they have a different vanishing point which may lie somewhere over there and this vanishing point results from the fact that these peril lines in the real world don't stay parallel due to the projection onto my image plane this basically everything yeah is set so all parallel lines which are mapped intersect at the vanishing point and the vanishing point has to be something described as at a point at infinity so it's infinitively far away and there my power lines actually intersect and every direction in the sense of FF 2 or multiple parallel lines if they have a different orientation look into a different direction then they have a different vanishing point one direction I expect to get one vanishing point and the key thing is kind of these points at infinity and as soon as we have that we may also think how can we actually describe points at infinity of course if we are in the Euclidean space and one described a point at infinity we could actually set the whatever XY that coordinates to infinity and then it's a point which is infinitively far away but what I lose if I do that is actually directional information so if I have a point or say okay the XYZ coordinates is or the X Y coordinates is for example infinity I know this point is infinitely far away but I basically lose completely track in which direction this point is so I can't describe the fact like in this direction over there the point is infinitely far away this is one of the limitations that we have in this is one of the motivations why the Euclidean geometry may not be the best framework to use if we deal with cameras Nvidia loose projections and this is the reason or motivation for projective geometry where we say okay there's a different formulation for that rule for describing the world and which is well suited or better suited to describe some of the relationships we we haven't we want a model and one of the things actually poison infinity at a certain direction so it's important to note that the projective geometry is just that same alternative framework to describe our objects and transformations in the real world and it's not that we can do better things with this technique compared to a fleeting bulb the only thing is that thing typically gets easier so the occlusion geometry the mass can get pretty difficult if I want to do all those computations and this is not the case for the projective geometry we use projective geometry and one of those ways to actually exploit that is homogeneous coordinates the mass gets to be least simpler and the simplification of the mass leads to better insights that we get about the relationships that we have here the important thing is there's nothing which is fundamentally different in the sense that it's more expressive so we can still say we can give a direction vector and then have whatever a parameter which runs towards infinity and represent that point in the Euclidean world but it's simply a much more complicated way of describing that and this led to the introduction of the who genius homogeneous coordinates to be abbreviated as HC on the slides because the word simply that long this is one way for improving the situation and which allows us to simply to end up with having simpler equations as if we would formulate them in the expedient world or the Euclidean kind of framework and two of the things which are very nice and which is very useful for us are those two points down here first thing is that we can actually describe points in infinity with finite coordinates so we don't need these infinity values somewhere we can use finite coordinates and still represent points on infinity and this makes a lot of the computations much easier there's one point much easier to grasp that we can actually represent the fact that there's something along these lines potentially in it and infinity and the second thing is which is actually pretty useful if you work with transformations is that using these homogeneous coordinates we can represent different types of transformations like rigid body transformation find transformations or projective transformations using a single matrix and this simplifies a lot of the activities that we do because we can simply chain those matrices and just multiply those matrices and this way chain the transformations which simplifies most a lot of the computations that we do substantially the second of one of the two reasons why life gets easier if we do the computations in this space and this allows us to get better insights about what we are doing so it's a framework kind of roughly said so is everyone clear why we are going to do that using homogeneous coordinates so the our goal is to simplify our life we can do the things in the appealing robots more complicated ok if given that we now have different things of things geometric objects like points lines or points and they can be expressed in repeating coordinates or no genius coordinates I want to introduce a slight notation that if you will use throughout the whole course which goes back to what went first and also the book that he wrote so you uses exactly the same notation so one of the things that we have if we talk about a point a line or a plane we can actually use this kind of graphic letters over here and this represents kind of the object itself so the entities of the line or the plane itself these guys those letters over here if we say that we have a point expressed in homogeneous coordinates we use kind of this type of font so long selected font non italic and if we go for your cleaning coordinates you have that in an intel equate and this at least also for me it really it simplifies life if you stick with so you thought will find this notation through the whole remaining part of the slides of the photogrammetry 1 & 2 course I'm not changing that but whenever you see something in this font we endear Klee involves whenever something in this font when the homogeneous coordinates whenever something is lowercase it means we're in the 2d world whenever something is uppercase or capitalized characters women 3d balls also used to actually simplify Turner's so simplify the understanding of those equations okay it's kind of clear also the difference between whatever the different types of coordinates and the point itself so we can talk about a point in the space and the point can be represented in you homogeneous coordinates or and Euclidian coordinates and therefore this distinction of giving the point of different font we just described be the quality of the of the of the objects of the point and not its best special configuration expressed in a certain coordinate system so it kind of clear okay so let me go directly to Virginia's coordinates and look to the definition what homogeneous actually means so what homogeneous means so representation of X of a geometric object is homogeneous if x and x multiplied by a scalar represent exactly the same thing if the scalars unequal to 0 so we have this relationship over here x equals the scalar which should not be 0 and X and this represents exactly the same thing this is something which is super genius the clean role this does not hold this vector the cleaning world this is this equality doesn't hold can make it whatever can make a very simple stupid example of you in the opinion walls and you have a vector whatever to 3 and then you can actually say is this the same thing then whatever for 6 which we can express as 2 times 2 3 and in the Euclidean world this is not the case obviously because 2 3 is different than 4 - so this is empirically involved you cleave D and this is not true this does not hold ok so this the homogeneous property does not hold okay how are we these kind of this things we infinite for the first time stably somewhat counterintuitive so how should such an object actually look like which has this property how homages coordinates of an n-dimensional object looks like is typically that we say okay we simply use one dimension more so we add one dimension more we go move from an n-dimensional space to an N plus 1 dimensional space by simply adding one new dimension so if we have our point in the 2d world so it's small its 2d as this one so we indeed feed in world so you've just given by X Y so this is what I haven't deeply involved when I express this in homogeneous coordinates I add a new dimension over here I copy the first n dimensions and add a one over here that's kind of the transition from the euclidean growth to the homogenous coordinates what it does how do we actually interpret this thing in homogeneous coordinates so to have this kind of equality if I multiply that by 2 for example it's 2 X 2 y 2 why should that be the same so all those vectors in MA genius coordinates represent exactly the same point in Euclidean coordinates if I take the last component over here and divide all other components by the last component so X divided by 1 Y divided by 1 if this gives x and y we're talking about the same the same point so this case it's clear it's X divided X divided by 1 Y divided by 1 will give me a vector X Y just exactly this vector so it's the same object if I now what multiply this vector by 2 so if I would have two times XY 1 is 2 X 2 y 2 for now on a map back to the Euclidean world I kind of take last component and divide all n components by this component so this is 2 X divided by 2 - why / - and maybe yeah and this is X Y so this is the same object as XY 1 which would be indeed controlled again XY ok so just adding this multiplication of 2 so this is the scalar vector doesn't change the object itself similarly the same thing if I have any mo genius coordinates something u v w say it's the same thing it's moving W out of the vector so it's u / WV / w + 1 and since the homogeneous properly I can simply skip that I obtain this vector so I have one here or something which is sometimes called normalized and then u divided by W and V divided by W corresponds to x + y what we basically do is we add if you go from the York leading role to the homogeneous coordinates we add one additional dimension and by default we set it to 1 if one represent the same object that the first n dimensions would represent in the feed involved and if I then I can do all my computations with my in my homogeneous coordinates and I want to move back to you clean coordinates I simply have to divide all n components by the last component so again there's another example those exactly which was shown pretty similar both was shown before so this is my element in homogeneous coordinates this is a corresponding element Beauty involved okay so that thing is which kind of numbers are allowed in here in noumic in the homogenous coordinates are all numbers allowed Canada all combination of numbers in here and this is not the case so what in the homogeneous coordinates we require that not all elements are zero at least one element must be larger than zero so that is one requirement so we can express this if we are here in the two dimensional worlds and homogeneous coordinates we have our third dimension then a constraint is that none of those elements not all of the elements are zero at the same time okay and if I want to directly move back to your cleaning coordinates as I have to divide you divided by out my third component I can only met that directly back into the Euclidean world if W is unequal to zero otherwise that would divide by zero thank you so in the you clean coordinates it's perfectly fine to have a zero down here but if it is if this last component is zero I cannot directly map back to the Euclidean vault at least was finite coordinates of course I will divide by zero and that would be kind of infinity here this is exactly the way we will see later on how we can actually represent points at infinity this value is zero we can still represent a direction with those two components but represent that the point is actually infinitively at infinity is a infinitively far away and at that point there's kind of not the direct relation between the point in homogeneous coordinates and in Euclidean coordinates because in Euclidean coordinates due to this division those values would approach infinity and therefore we won't have finite coordinates in here so we can represent points at infinity in the Euclidean world but only not with finite coordinates so just needs example if I have points on the projective plane which is kind of the space corresponding to the projective space corresponding to the r2 so it means all points which are expressed in you clearances XY then what belongs to the projective space these are all three valid valued vectors like X Y and one just as an example just at the component all points at infinity just X Y and 0 get this zero over here but what is not in there and doesn't belong to their spaces is 0 vector 0 0 0 because our constraint would be violated that clear so the only vector which is not part of that space actually is the vector where all components are 0 so the first end end the last one at least one of the components needs to be unequal to zero and again this is the reason that not everything can be or if W is 0 then I have the problem of mapping back to the original space in am a genius coordinates I can still live with a 0 here and those two elements give me the direction in which direction the points in infinity but if all of them would be 0 I would have no direction and no point I could set to infinity so you can also maybe this becomes clearer and this a kind of illustration so what we have here this planes of the grey plane is kind of our to a regular r2 world so so far we were living in this r2 world and now if you go to these homogeneous coordinates we have one additional dimension so we can actually see this as a three dimensional space we just have we just add this new W dimension which is my 30 in this example my third dimension and a point in the Euclidean vault with x and y so here is an e4 Euclidean Vickers the figure uses slightly different notation but what I wanted to say is that this point in the 2d world corresponds to this line in the 3d world language starts at the origin and passes through this plane and we can also have a point or since somewhere over here if I do the normalization so that the last component is one which basically means I divide all the components through the last component what happens actually is that I move this point here back to exactly this point so every there for this reason every point which lies on that line corresponds to the same point in the 2d world so if I kind of do this divided by W what this actually means that somewhere along this line I kind of move the point back over here okay is this figure does this figure support listening yes please negro so just having this point in here is not explicitly part of the things so the things i can extend this by another dimension then go to be to the next dimension then i use this all three so it's still part of the figure but so this point is not part of the space because what I'm doing a basically representing lime starting at this original and if I if the point lies at the original I can't have the line in both points on the same line and could have any arbitrary line therefore this one is not part of the of the projective space so if I'm kind of if I visualize this the whole thing as r3 then of course I would have this point in there but the projective space explicitly excludes this point any further questions okay what we can do in 2d we can of course to exactly in the same way also for for 3d the only difference is we have don't have three we have actually four components and then we can move from the four components and you know genius coordinates back to our three dimensions in our Euclidian world the rest works exactly the same way oh these are capitalized characters because they represent object in belonging to the 3d world I can also define the origins of the Euclidian coordinate system and they are given by 0 0 1 this exactly this example over here I've hoping the three evolve 0 0 0 1 this would be always at that point which lies within this grey plane so here so just edit this figure to that slide it's a bit suboptimal having the same points over here so the best thing we actually were to ring would be to remove that to not cause any confusion because these are the Centers of the clearing coordinate frame pressed in homogeneous coordinates and they have to have a 1 in here because this is the center of this plane okay so if you're comfortable with points in this world in this kind of representation you all find with points homogeneous coordinates yes no so far the difference well the difference is just the only thing we say it's we just have points along this along this line every point along this line corresponds to exactly the same point here in homogeneous coordinates in Euclidean coordinates how we transform those points something we'll discuss later on if you use our transformations if you want to do it okay so if you if you're happy with points we can move two lines and see how can we actually represent lines and if you look back how we can represent lines in our are you clear involved there typically three ways and how we do that there's something which is called the hassle normal form on Germany has a form where you buy the basically specify an angle Phi and then a distance and then with those parameters x and y you can vary as long as this equation holds the point XY is on the line it's one way for representing a line the other thing is the intercept form or accent a platform where you have your x and y over here and you have two parameters x0 x0 and y0 equals to 1 or minus 1 equals to 0 in which we can use or what's called the standard form or implicit form is you have this expression over here so ax plus B Y plus C must be equal to 0 so the kind of important thing in here is that we have always a system that we have always an equation and we've always an equation which something is equal to 0 so I have a component with X I become was why I have a constant component and I have zero over here same thing over here I have X some coefficient Y some coefficient constant part equals zero and the same exactly we're doing we have in here so the key thing the key inside in here if you look how the lines are represented or in the in the occluding world is that they are all form linear equations which are equal to zero okay so this is clear because we will exploit that in a second so whatever representation I use you know to represent the line can be written as an equation which equals to 0 and we have linear components in the X and in the Y okay all fine everyone accepts less this ok now let's see how we can exploit that in order to represent lines in homogeneous coordinates so this was a point in homogeneous coordinates X Y and 1 if you actually go back you can see that we have an X a Y and a constant component ok so that's all point U and then we can actually represent provide three different vectors L ones representing has a form once with the intercept form and once with a standard form where the first second and third dimension takes those parameter values just basically took those equations part which corresponds to X the part which corresponds to Y and the constant part and actually place them in here the part corresponds to X to Y in the constant part what I now can do is I can simply take this point and compute the dot product with one of those representations so I say X dot L they were computed that would give me x times cosine Phi plus y times cosine Phi minus D that looks like pretty similar like this one the only thing I'm missing is actually zero so what I can do is if I have this kind of representation I can now use this to formulate condition under which the point lies on that line which is X dot product with l equals zero if this holds then the point X C lies on the line which is represented in this way they can do this exactly the same way for the two other forms the key inside here is x times well or X transpose L alphas post x equals zero is the key condition for formulating that a point lies on that line of appointed homogeneous coordinates this is a representation of a line homogeneous coordinates and by multiplying the point with line equals zero I'm expressing that the point lies on that line so all points for which this equation holds are actually points which lie on that line so they define the line L so this is kind of clear what we have done okay we start with a point homogeneous coordinates we take the equations that we know from our Euclidean geometry and how we can represent lines we just pick one out of them and then I say okay we have to combine the point and the line in a way so that we can have an equation which is equal to 0 because then we have the correspondence to the Euclidean world we obtain that by saying X transpose L equals 0 or the other way around okay okay so but we can derive from that is you know our in our homo gene in the homogeneous coordinates we can define a line in the 2d world as a three-dimensional vector l1 l2 and l3 and again I have the same constraint that not all of them can be the null vector because then it would hold obviously for every X Y I put in there and this corresponds to the representation of the Euclidean world which is the standard form over here okay now what we can do is we can actually very easily test if a point lies on the line by just taking my representation of a point my representation of a line I simply do the test if X transpose L equals 0 then the points but the point lies on that line so if I have a point in homogeneous coordinates of a line no Medina's coordinates just by can be computing the dot product and testing if the result is 0 I know of the point lies on that line which is nice everyone happy with this test I don't have everyone understood why we ended up doing this test so we look to the equations we know from the Euclidean world said how does the point look like in homogeneous coordinates how do I need to build up a line homogeneous coordinates and I just need to multiply them and if the result equals 0 exactly have the equation I know for my Euclidean volt okay so the next thing we can do so far what we did we checked if a point lies on the line next thing we can do is we can simply take two lines and what be interesting do they actually intersect so I know I have a line L and the line M I want to intersect those lines and they should intersect at some point X this is what this means so L intersected with M will give me the point X the question is now how do I find such a point how do I find either section of two lines and I will exploit what we just learned so we learned before and we want to test if a point is on the line I can do it in this way the question is how can I now specify that two lines intersect because they should intersect in one point what can we do in order to formulate that exactly it's as easy as that so we say our power point where they intersect transposed owl should be 0 and X transposed M should be 0 just give me a very simple system of linear equations which I could solve we can do exactly that say L times X and M times X should both be zero so this boils down to actually solving this system of linear equations so how can I solve such a system of linear equations actually one of the things that you can do this actually the Kramer's rule which is very easy to use for small system like two by two systems which actually says if I have a so does everyone else grammar rule or should I explain that again everyone knows that at least a few people playing it okay just to explain how that works and the system ax equals B and I can say then the first component of X of my solution X is given by the determinant of a matrix a I plane by the second divided by the determinant of the matrix a a is this a and a I is the matrix a where replace the eyes column by the vector B so the next we even do a small example of the blank but so do an example in the blackboard know how apply that so what I basically have I have L 1 l 2 M 1 M 2 times X 1 X 2 equals minus L 3 minus M 3 so this is what this is what I want to solve let's go to a concrete example let's say L equals 1 2 3 so I have to continue with the white chalk anymore and M is 2 0.53 for example so what they need to do is X but the Krammer will send me X 1 is that student of a 1 divided by determinant of a which is the determinant off a 1 I take the first column replace it with those elements so it's minus L 3 minus M 3 and L - M 2 divided by determinant of L 1 l 2 m 1 M 2 and now I can actually fill that with numbers so this is a determinant of minus 3 to minus 3 0.5 divided by determinant 1 2 to 0.5 I never know compute that I get the result of minus 1 point 2 8 6 in this example it can be exactly the same thing for X 2 except that I compute the determinant of a 2 that hey over here so it exactly the same result except that I don't replace the first column of a second column over here and then in this example you should get for x2 minus zero point whatever eight five seven if you compute that now okay that's the way how we can actually use how we can actually solve that system so this system can be solved through the Cramer rule by X X which corresponds to my x1 over here is the one divided by the tooth of this determinant of a1 the determinant of a and same for y was determined of a 2 determinant of a and the determinant can simply be computed using these equations let's take this guy times this guy plus this guy a - this guy times this guy and then just give me exactly those equations and you can actually compute that so the interesting part comes now when I say what I've written in here so x + y the solution can be expressed if I write this as a vector all right XY is 1 divided by D 3 D 2 D 1 D 2 I can write it in this way right this one actually get this result I can also it but I'm not gonna do this why not simply extending that a little bit adding D 3 over here and 1 over here right now can have my 1 over here my my homogeneous coordinates what I can do is the genius coordinates I can drop any vector that I have before that equation and throw that away so came up on X Y corresponds to those elements T 1 T 2 T 3 and there compute exactly in this way and this is by coincident exactly the way how you compute the cross product how you compute the terminal to leave all gives you exactly the cross product over here so the interesting thing is now if I want to compute Li there section of two lines m1 and m2 that basically boils down to computing the cross product of the two lines and this gives me directly the intersecting point just by applying the Kramer's rule not on the numbers but just with the variables in here I can actually make this transformation so you can create my new vector which consists of X Y and one it put the numbers in here get 1 divided by d3 out drop that because of the Medina's property and then these are the component the remaining component of my vector and this so this solving this equation our cross product M gives me directly my intersecting point X so sort of requiring me to solve this system of linear equation just can execute the cross product and I have my solution so it's guys that's it actually that's pretty simple so if a point X is defined as the intersection of two lines I can actually compute this in this way so it's a very very simple way for computing the intersection of two lines using homogeneous coordinates ok what else could we do with lines and points you take two points and for the line through those two points so next section activity we can do which we may often need and happily we also have an easy way for doing that so consider we have two point a point X and the point y so this x1 x2 the coordinates of the of the first point and y1 y2 other coordinates of the second point and I want to compute the line L consisting of l1 l2 or l3 in the image aeneas coordinates then like to actually connects both of them how can I do that so i1 one line which intersects with two points that's not what I want to do so consider the solution we just arrived so far for the for computing of intersection of two lines what we had with very similar situation we said we want to have two lines and they should intersect in one point we formulated system of linear equations to solve that problem now we have the situation we want to have we have one line in two points what you can do is we can actually exactly the same thing we can simply say my line should intersect with the point X and my line should intersect with the point Y and again for muriatic SEC is the same system of linear equations solve that we're happy to do it's basically exactly the same thing so what we do is we say okay X transpose L should be zero and Y transpose L should be zero because these are my two points through which I want to fit L so I can set up my system with a grammar rule I have my l1 and l2 here I think than what a determine of what also l3 sitting in here but I ignore that for a moment so I can simply just as we did it before apply grammar rule saying l1 is d 1 divided by d 3 l 2 is d 2 divided by d 3 SEC the applying thumb rule as we did that before the only difference compared to what we did before is kind of we have the L sitting in here as a vector but it's a bad that doesn't hurt us at all because basically exactly the same trick as we did before we say ok we have L 1 and L 2 so we add in a third dimension L 3 which is the third dimension SL 3 times 1 which is d 3 divided by D 3 so I set up my vector L saying D 1 divided by D 3 D 2 divided by D 3 and L 3 D 3 divided by D 3 the first thing I can do is as in D 1 and D 2 I have my l3 in here as well right so I can actually move that out move l3 divided by D three out of the vector I have that here and then thus the remaining parts form this vector over here and again this vector looks pretty much like a cross product never inspect that we said actually realize that this is given by the cross product of the back of the points of the three coordinates of the points x and y there's basically more less exactly the same on what we what we did what we did over here mmm is 1 divided by D are the only differences we move l3 divided by D 3 out and then the homogeneous property I can actually drop that factor because it's just my scalar lambda and then the only thing remains m1 + 2 + 3 which will have three components resulting from the cross product so I can use the cross product of two points in homogeneous coordinates in order to obtain the line which connect those two points also something which is very easy it's kind of to summarize what we have seen so far we can very easily sorry test if a point lies on a line and this hulls if this condition is satisfied we can also say if we have to lie to on a computer section of two lines we can do that very easily and if we want to have two points and connect those points for a line you can also do this exactly in the same easy way so it seems to be a framework which allows me to do some of these geometric operations quite easily the quite easy an intuitive way which is nice was one of the reasons by using homogeneous Mintz is popular but all the things which is very important facet as we work as cameras or actually points and lines infinity so we said before that we want to be able to represent points which are infinitively far away but in a certain direction and just it just told you that homogeneous opponents offer off the possibility to do that by setting the last corner to zero so we are still maintaining the direction the first n dimensions and putting the last one to zero move this point infinitively far away so if this thing which is called points at infinity or X sometimes written like X infinity which is the point whose two coordinates here and they should not be zero both of them cannot be zero only one can be zero and last component is zero and it allows us to represent a point at infinity by not easy by only using finite coordinates so we don't have any infinity sitting in there this is kind of one of the big advantage because it's the same point in time we can say we have a point which is infinitively far in this direction I can main obtain this directional information and this is a very very useful thing for camera speaker because what the camera basically does every pixel corresponds to directional but I've no idea where along the line this point edge case and I can represent points which are infinitely far away and of course if I want to map back to you Klien coordinates if this last component is zero I will get I will get non finite or infinite coordinates in the Euclidean world because I cannot explicitly write explicitly represent that with finite coordinates in the Euclidean world okay so let's have a look to those points infinity we said we have this parallel lines which intersect in infinity let's look how intersections at infinity could look like so all lines L for which this equation holds so L dot product that's actually written in a nice way so this would actually be written in in a different font because these are the objects and don't have the dot product between the objects so this should be in homogenous coordinates equals zero that means that all lines for which this equation holds they actually path through the point in infinity as the last component of this point X is zero I can ignore the last component of L because anyway set it to zero so I have my U and V the dot progress cosine and sine of the direction of the line if I you see has a form should be zero so independently for the equation of L what the last component is all of them meet at that point infinity so any line is cosine Phi sine Phi and here an arbitrary value is a line which intersects at this point infinity if you now look to this equation over here and look look to the interpretation of the hasser form the v specifies the the directions of a the direction of the line and this just specified what specifying the distance so this doesn't matter the only thing which matters is the direction and every line for the first two components look like this that means all ions which look into the same direction meet it's the same point at infinity this is exactly what this picture of the vanishing point was actually showing us that all parallel lines meet at one point in infinity you can also see that by saying okay let's simply intersect two parallel lines with the framework of intersecting lines on what we learned by cross product so we can say okay let's simply say L M over here we said okay we have two general lines but these lines should be parallel so the first two components should be the same so of a a B B and then C and D down here and what this gives me is give me the intersecting point do a little bit of math like this and the important thing is this expression known here a B minus a B gives me 0 so I obtained this expression where some value over here and some value over here and this only depends or this depends on the directional components and but also on the the distance between those points okay this gives me the direction but the point is at infinity because there's a zero in the last component so all parallel lines needed infinity and this something what you can kind of illustrate they see here if all parallel lines that would have another parallel track here another parallel track here they will actually meet at the same point at infinity as long as they are parallel if you would have different parallel lines like those bars over here they simply meet at a different point in infinity we meet somewhere over there they all meet at one point in infinity so this is kind of clear we say intersect parallel lines intersect at infinity so it's not intersection between parallel lines in this framework because we have expect an explicit representation for points at infinity it's nothing which I can arbitrarily introduce is something which naturally comes out from the equations of how to compute the intersection between two lines or parallel lines okay okay infinitively distant objects or something if I say have it I said I have my infinitive distant point just this form over here and I can also have an infinitely distant line or show in the second one looks like there's called the IDEO line this adia line is the line which has zero components over here and one down here and well this line is this is line where all points all infinitive distance points lie on so how to compute the if a point lies on a line just by the dot product so we take this point independent of U and V with this vector over here the first component will always be zero the second component will always be the Euro the third component will always be zero so and therefore it is zero and therefore all for all infinitely distant points lie on that line so all points of infinity independent of the direction lie on this line so the only interpretation or you can see it is actually there's gonna be your horizon line at infinity for all points at infinity lie on this IDIA line it's exactly how you can see that the idea line is given through this equation so independently on which where the point is at infinity it will always lie on that ideal line you can actually see on to interpret that as the horizon line the infinitive leaf our horizon we can actually do the same thing else for 3d objects and say okay we have a 3d point which now is four components over here that we can instead of a line we have our plane is just as one component more again something we can do in exactly the same way how we did that before can do exactly the same thing a point lies on the plane in 3d if a dot product x equals zero so all the the derivations work exactly in the same way as we have done very soon I know Lagos way and then we can also fall points at infinity we have also kind of have this idiot plane we can actually illustrate that so this is the person you're sitting in with sent off our coordinate frame we have our lines and here all the different lines all the things that infinity so this is kind of your horizon line this was the ideal line going around here and this is kind of them the plane like whatever it's the sky you can actually see that this is ideal plane so this is kind of the the analog II to the things that infinity but but we are mainly using here actually the points that we have in infinity so being able to represent points in infinity though one of the big advantages that we have the second big advantages transformations it was the second point I had in the kind of positive list for the homogeneous coordinates that was that we can very very easily deal with transformations so projective transformation the transformation the other space that we have is an invertible linear mapping which maps with this h the mapping which maps X to some other X Prime and one of the key over is called the fundamental theorem of projective geometry tells me that every one to one so one to one is an injective function straight line preserving mapping is a projective space PN on itself is in the home okra feed for any N larger or equal to stool but it basically implies whether the important thing is that all want to warm straight line preserving mappings and those were the mappings I'm actually interested in for my pinhole camera model our linear is a linear function so such can be expressed with a single matrix if we use homogeneous coordinates that's the reason why I can actually describe then in this simple form because everything can be put into this matrix H so the general projective mapping has exactly this form and what I now want to do is I will look into this different instances of age which are possible and see how we can describe different types of transformations so the most easy form is a transformation let's express it Tran simple translation through this matrix H and this is also one of the big advantages compared to the Euclidean space where I can perfectly express my rotation with a matrix but another translation I always have this plus a vector if I move to this n plus 1 dimensional space I can do this much easier by setting up our matrix H ok so it's an arbitrary scaling factor which item here because this is part of the homogeneous property and I have a matrix so this is now for the 3d world of 4x4 matrix where this is an 3 by 3 identity matrix this is a 3 vector this is a 3 vector transpose and the one over here so exactly as it is written over here then this means this is this is a rotation which is expressed typically here but it's an identity though there's no rotation involved I have a vector which translates along X Y and that now build up this matrix H identity here 0 here 1 here and here we just have a vector of how much I'm gonna translate in X how much do I want to translate and why and how much do I want to translate in that then this mapping actually performs a translation of the point in by TX T why it is that in the corresponding dimensions so this point is translated through this transformation gives me the new point so X prime is a translated variant of the point X my thing is this is just a matrix product so I can actually add a second matrix over here let's say I apply H twice H H X will be simply two translations by sekhti the same distance DX dy DZ other things which are important rotations I want to express a pure rotation I put here a zero vector of dimension 3 because this what I have don't have one or any translation and I replace my identity matrix which was sitting over here by our standard rotation matrix so standard rotation matrices as you know them Express the 2d or in 3d rotation about the x axis y axis that XS I don't want to go to the details you've seen that already you just take your standard rotation matrix and actually plug it in here that's it and then you have kind of this type of transform which is as three parameters and these are the rotations around the x axis the y axis and the z axis now we can actually combine those and look into the translation and a rotation with something which is typically called a rigid body transformation which is six parameters which involve translation and three degrees of freedom for the rotation now if my rotational part over here and my translational part over here and then I can step-by-step editors with parameters and make new transformation which have different expressiveness powers of expressiveness so what I can do is I can now add a similarity transform which is a rigid body transform plus a scaling with one degree of freedom in terms of scaling I can do that by simply adding a scalar M over here so basically scaling my whole system make it larger and smaller rotation and in translation the important thing in here is this scalar is not supposed to sit in front of this matrix because then would be eliminated through the homogeneous property I'm only allowed to skin the first three dimensions the key trick is to not scale the one down here because that this would be scaled this would simply go away through the normalization so the similarity transform is the angle preserving transformation we also looked to affine transformations where we have in addition to the similarity transform we actually have three scale parameters not only one in the different directions and we have a shear for three shear parameters for shearing the system through the shear we don't have an angle preserving mapping anymore but we are still have the the fact that parallel lines remain parallel so with the affine transformation we have an angle it's not angle preserving anymore but things which are parallel before our parallel afterwards so this matrix a here is a 3 by 3 matrix which takes 9 it takes 9 degrees of freedom this takes 3 degrees of freedom which gives me 12 degrees of freedom so this matrix a is a general matrix no constraints on the matrix and the only thing which is left now here we have some three degrees of freedom left is the are those zeros over here in the next step in terms of journaler turner general relative from affine transformation is then the projective transformation the projective transformation eliminates those zeros over here so here we have a general vector a which are three additional degrees of freedom so it's affine transformation plus the three parameters and this is these three parameters are those parameters are part of the projective transform which are the reason that para lines may not stay parallel this is what happens in the projection this is what we have seen in the in the image I showed in the beginning of the house may have your peril lines and they're not parallel anymore because through the projection of the pinhole model we have elements in here which are nonzero okay there's also small lists for 2d what kind of different type of transformations we have how the original figure looks like how the transformed figure may look like and what the a how the H matrix looks like at the individual parts and written in this compact way then you can see here the most general one which in 3d has 15 degrees of freedom in 2d has 8 degrees of freedom that those parallel lines here are not parallel anymore through the projection can do the same thing for 3d and what we basically have we have this type of hierarchy we have translation and rotation which is one or so rotation one parameter and in the 2d world three parameters in the 3d world what translation by XY or XY is that I combine those I get the rigid body transformation with six degrees of freedom in the 3d world and three degrees of freedom of the 2d world if the similarity transform which is the rigid body transform plus the scaling one scale factor I may have the affine transformation which has different scalings and the shear and then we have the projective transformation which is all the degrees of freedom so 15 for the 3d ball we have a four by four matrix and the only degree of freedom which is missing is due to the one in the lower right corner which is the homogeneous property therefore I have only 15 degrees of freedom although of 16 elements but one element is forced to be one more depending on the scalar vector can always be normalized to one so all these different types of transformation I express can express with linear mapping in this projective space which is really nice and since it's a linear transform and it can use the matrix to actually encode that I have the very easy way for chaining and inverting transformations so if I have this transformation that I want to get the inverse transformation I just need to invert my matrix H by computing the inverse of my matrix H I directly invert this transformation so just need to stupidly invert my 3 by 3 or 4 by 4 matrix and have the inverse of my transform and I can also easily chain them up so if transformation a 2 and a 1 you can execute them one after the other of the vector X and get my new result remember them matrix multiplication is not commutative so if you swap them you may end up with a different transform so this doesn't have to be but the ability to change those matrices really will simplify our life if you have a large set of different number of transformations are gonna execute one after the other it's just a product of matrices to can very very easily write down and compute even can combine all of them into one matrix and then one matrix encodes all the transformations of the individual matrices and also being able to invert them the transformation just by inverting a matrix simplifies lives essentially and this brings me to the end of my whatever tour through projective geometry and homogeneous coordinates and the kind of key things as a take-home message is that the modulus coordinates are just an alternative way for representing geometry and they have a few advantages the advantages are that they can simply simplify mathematical expressions we can explicitly represent points at infinity with finite coordinates we can very easily define the transformations that we have seen here so the projective transformations in matrix form so we can invert them easily chain them and train them in a very very easy manner this substantially simplifies the mathematical equations that we to believe face and have to have to deal with we can represent points infinity yeah that's what I said and how homogeneous corners actually realized that is that they add an additional dimension that this additional dimension is used for normalization for normalizing the other components if I map back to the Euclidean world and by setting this normalization to zero I have the chance of representing elements of geometric elements at infinity and the only and what homogeneity or homogeneity is actually defined through this equivalence I'm an object is homogeneous if x equals a scalar times X is the scalar son equal to zero some somewhere along in the space I can see this I add a new dimension and as somewhere along this line in this n plus 1 dimensional space where this space intersects with the N dimensional space where the last I mention is one this is actually the corresponding point coordinates so being familiar was using homogeneous coordinates in those transformations something that you will need to do through the remaining part of the course and through photogrammetry to of course for at least fifty to eighty percent of the course material so make sure you are familiar with that there will be an exercise bottom of genius coordinates as well so they can kind of familiar with using them if this is not already the case there's also what confess the script gives you a very good summary about what's happened here I'm used exactly the same mutation as in this script so I can strongly recommend it that's it for my site there are now questions thank you very much for attending and from day after tomorrow we will start with looking more deeper into cameras into intrinsic parameters extrinsic parameters of cameras and how we can actually describe the mapping from a point in 3d world to my image thank you very much

Channel: Cyrill Stachniss

Views: 25,468

Rating: 4.9029126 out of 5

Keywords: robotics, photogrammetry

Id: ZNB6SpEBnBQ

Length: 80min 2sec (4802 seconds)

Published: Thu Jul 09 2015