9 Stereo Imaging

you know a last lecture we were discussing about the imaging geometry and we have already seen that given a three-dimensional point okay in a world coordinates how do we get the corresponding point on the camera let's say that by applying the perspective transformation how do we get the camera coordinates and then we had also the scene or studied the general case in the last class where we had made the coordinate system for the camera and the coordinate system for the world to be different we had considered that the camera is located at some other location okay whereby we had considered some translation some angle for fans some angle for tilt and also the some distance with respect to the gimbal center of the camera so considering all these things we had made a generalized expression okay whereby from the world coordinates we can get the camera coordinates now in the last class we had also seen that if we are trying to do the reverse process that is to say from the camera coordinates or rather given the camera coordinates if I want to calculate the world coordinate point we were finding a difficulty and primarily that difficulty was because the world coordinate to the camera coordinate transformation was basically a many to one transformation and as a result of that we were losing the depth information pertaining to the was coordinate point now let us see that what arrangement we can make to recover that depth information okay now we go back to our earlier discussion okay the earlier imaging human rearrangement which we have seen so we had a camera plate like this and we had the XYZ coordinate system something like this this was our XYZ coordinate system and we had the optical center somewhere over here and then if we had the image point over here let us say that the image point is ti corresponding to some object point let us say P and P was having a coordinate XYZ okay and P I was having a coordinate X Y with respect to this imaging plane and we where all the time assuming that the coordinate system for small XYZ and the coordinate system for capital X Y Z let us consider that both the coordinate systems are the same now we once again consider the focal length to be lambda as we had considered before so lambda is the focal length and now if you remember correctly the expression which we had got was something like this our small X that is to say this small X for the camera coordinate was equal to lambda capital X by lambda minus Capital Z and small Y was equal to lambda Y by lambda minus Capital Z now from these two equations we get capital X equal to lambda minus Capital Z into small X divided by lambda and Capital y is equal to lambda minus Capital Z multiplied by small Y divided by lambda now from this first equation and the second equation okay what do we see our capital X is unknown Capital y is unknown Capital Z is unknown if somebody has supplied the camera coordinate point small X small Y okay and we would like to determine the three-dimensional point the corresponding three-dimensional point P then what we have from here is capital X unknown Capital y unknown Capital Z unknown so there are three unknowns and we have got two equations so obviously we cannot solve uniquely for capital x y&z so what do we have to do now what kind of solution should we look for if I introduce a second camera plate okay let us say that at some other distance I introduce a second camera plate and then for the same point capital P okay I obtain the corresponding image let us say now in the second photographic plate second camera plate the image is obtained at some other point P I Prime and for which we assume that the corresponding coordinate is X prime Y prime the corresponding image point in the second plate is X prime Y Prime in that case I will be obtaining two similar equations like this is it correct I will be obtaining two similar equations involving X prime Y Prime and capital X capital y and capital Z because capital X capital y and capital Z remains the same because I am considering the same point in the three dimensional space and just considering it's engaged just in two different place okay so now I will be obtaining how many equations I will be obtaining four equations and I have got three unknowns so it is possible for me to have a solution so what I should do now is to add a second view in such a manner that I can get we get a solution for capital X capital y and capital Z so let us consider some imaging arrangement like this so we are considering a point P in the three-dimensional space and here there is a camera okay whose focal length is equal to lambda this camera is having a focal length equal to lambda so this is a point in the three-dimensional space so I am considering o to be the origin of the XYZ coordinate system for the first camera and I am considering I to be the photography plane of this first camera and what I am doing is that this focal point which I am calling as see I am joining this capital P with the focal point C and if I extend the Ray PC okay the point where it intersects they photographic clean will give me the the point P I which I get okay that point P i okay gives me the image point corresponding to the object point P for the first camera is the correct just leads individuals okay let me repeat whatever I say I have considered a photographic plate over here okay for the first camera where the coordinate system is x y and z and along the z axis i have considered the axis of the camera to be along z axis and i have considered a point c on the z axis okay such that the distance OC gives me the focal length or other C is considered as the focal point for the first camera I have considered a point P in the three-dimensional now if I join point P with C and extend the Ray and if that Ray intersects the plane the plane of the first camera at the point P I then at the point P I I get the image for the point P is it okay again I have considered a second camera and the coordinate system for that second camera is assumed to be X prime Y Prime and Z Prime and I have considered a point C prime along the Z prime axis such that the point Z prime is the focal point for the second camera this is the plane of the second camera and are considered C prime to be its focal point and Oh prime Z prime to be the focal length now if I consider array from capital P to C Prime and extend that Ray the point P I prime where it intersects the second photographic plate will give me the image of the point P for the second camera is that alright so just make a note of it that for the first camera I have considered a coordinate system XYZ and for the second camera I have considered a coordinate system X prime Y Prime and Z prime questionable after that we have three different coordinate systems now we will make some simplifications because it is really difficult to deal with three different coordinate systems now there should be some relationship between all the three coordinate systems that I consider so definitely this XYZ coordinate system and X prime Y prime Z prime coordinate system they must be somehow related alright and even the world coordinate system with respect to which I am defining the point P okay that also will be related to the two I mean either of these two cameras coordinate system so definitely for the ease of our analysis what we will finally do is to represent the world coordinate the second cameras coordinate everything with respect to the first camera coordinate system alright so that is what we will do instead of considering three independent coordinate systems so now let us say that the relation which I have between the coordinate system X prime Y prime Z Prime and XYZ is like this X prime Y prime Z prime will be related to the XYZ coordinate system in this manner X prime Y prime Z prime will be the multiplication of a matrix R okay which is a three by three orthonormal matrix indicating rotation so R is a three by three matrix which indicates rotation and T is a three element column vector okay T is a three element column vector indicating translation all right so this is how X prime Y prime Z Prime and X Y Z will be related to each other and now let us assume that the coordinate of this point P with respect to the coordinate system of the first camera is small x0 small y0 and small z0 so instead of writing this as capital x y and z with respect to the world coordinate okay I will now write the coordinate of the point P with respect to the coordinate of the first camera as x0 y0 and z0 and i am making an assumption and that assumption is that both the cameras that is to say the camera number one and camera number two they are having exactly identical focal lengths so I am assuming that oh see the distance of OC is equal to the distance of Oprah MC Prime and that is equal to the focal length lambda so mind you my basic assumption which I am making is that both the cameras are having exactly identical focal lengths alright and x0 y0 z0 is the coordinate of the point P with respect to this coordinate system XYZ and I have already defined the relationship between the X prime Y prime Z Prime and the XYZ coordinate system so now just consider this first camera okay and assume that the coordinate for the point II I is X I Y I and assume that the coordinate of the point P I prime with respect to the X prime Y prime rate prime coordinate system is X I prime Y prime but with respect to the X prime Y prime Z prime coordinate system of course I can convert this X prime Y prime Z prime coordinate system back into the XYZ coordinate system okay so let us first consider the first camera only where I have got a point P I as the image point and its coordinate is X I Y I I know the point x0 y0 z0 in three-dimensional space the focal length lambda is given to us so now what should we get as the relation between X 0 y 0 X I Y I and Z naught I will get a relationship in terms of lambda the seedings in the same way okay as we did for the imaging geometry so that means to say by considering the similar triangles once again okay we get this relation for the first camera for camera number 1 this is my camera number one for the camera number 1 I get X 0 by X I equal to y 0 by Y I and that is equal to lambda minus Z naught divided by lambda I call this as equation number 1 all right and for camera number two for camera number two if I want to make the representation with respect to X prime Y prime Z prime coordinate system okay then I should better represent this point P with respect to the X prime Y prime Z prime coordinate system so let us say that the coordinate of P ok coordinate of P with respect to P with respect to X prime Y prime Z prime coordinate system is X naught prime Y naught prime Z naught prime so the point P with respect to XYZ it was x0 y0 z0 and the same point p with respect to the X prime Y prime Z prime coordinate system is X naught prime Y naught prime Z naught Prime and considering this again by applying the same similar triangle principle for the camera number two okay what do we get we get can anybody tell me X naught prime by X I prime becomes equal to Y naught prime divided by Y I climb and that is equal to lambda minus Z naught prime divided by lambda all right so now this is my equation number two if I want to solve for the coordinate of the point P okay I have slight difficulty difficulty because for camera number two I have represented everything with respect to camera twos coordinate system and for camera number one I have referenced everything with respect to once coordinate system so all that I have to do is that coordinate transformation and I have already defined this relationship for the coordinate transformation but one thing that if I assume the coordinate systems relationships to be like this that means to say with respect to XYZ the X prime Y prime Z prime coordinate system can have translation as well as the rotation then I think we are unnecessarily considering quite a complicated imaging situation because after all what do we need we need to take the image of the point P from two distinctly different positions so why should we consider such a kind of general case where the two cameras coordinate systems may have some rotational relationship as well as translation relationship between them let us consider some kind of a simplified geometrical arrangement between the two cameras such that we consider only translation okay no rotation we consider only translation and that cue the translation we will consider only in only along one particular axis that means to say that I am considering a relationship of this nature X prime Y prime Z prime is equal to I I being the identity matrix X Y Z plus Delta X 0 0 or I consider the camera arrangement such that X prime Y prime Z prime is equal to identity matrix product XYZ plus 0 del y 0 or I consider X prime Y prime Z prime equals to I product with XYZ plus 0 0 Delta Z so I will try to consider either of these three kind of imaging arrangements that means to save watch if this is the point I mean this is the plane of the first camera and this is the plane of the second camera ok then if this is the z-axis of the first camera and this is the z axis of the second camera then all that I am considering between these two cameras that is camera number one and camera number Q is our displacement by an amount Delta X in the X direction so if this is the origin for the first camera oh and this is this is the origin o prime for the second camera and if the distance between them is Delta X okay and if this happens to be the z-axis for the first this happens to be the z-axis for the second camera then I get this Delta X displacement so in that case I am considering my first model model number one where I have assumed that along X there is a shift Delta X between the two cameras but along Y there is no shift along Z there is no shift and that is how I happen to consider the two cameras plane to be the same because they belong to the same Z I have considered the Z displacement to be zero or the other alternative could have been where I consider one plane over here and another plane over here okay this being the origin of the first camera this bring the origin of the second camera like this and I have a displacement by an amount Delta Y between the two cameras all right so thereby I will be having the second model that is a displacement Delta Y only in the Y direction for XM z the displacement is zero or I can have a third arrangement and what is the third arrangement I have got my ex plate over here I mean I have got my first cameras photographic plate over here and I am God the second cameras photographic plate somewhere over here alright and this is the z-axis for the first camera and this is the z-axis for the second camera and the two origins are just shifted from each other by an amount Delta Z okay so that is the third arrangement which is given by the the question now the first one and the second one okay where we had considered the same plane okay but the cameras are just laterally displaced so this kind of studio camera model is called as lateral stereo geometry lateral means the two cameras are just laterally shifted and the other arrangement which I have okay whereby I had the separation in z-axis there what I have is that is to say where the shift is by an amount Delta Z in the Z direction my camera displacement is along the axial direction of the camera and this is called as the axial stereo model alright so we will first consider the lateral stereo geometry where we will assume that where we will assume this model that is to say the two cameras are just separated by a displacement Delta H let us now see the situation by considering this kind of an imaging geometry so we have considered two cameras the first one is having the coordinates X Y Z the second having a coordinate system X prime Y Prime and Z prime their distance of separation is Delta x over here alright now I consider and I am assuming CL to be the focal point for the first camera and CR to be the focal point for the second camera now the first camera which is on the left hand side I am calling that as the left camera and this camera which is on the right hand side I will now call as the right camera all right now I consider an image point P L on the Left camera and I join this PL okay with CL and extend this ray supposing somewhere over here I have my point P having coordinate x0 y0 z0 defined with respect to the first cameras coordinate system X Y Z okay with respect to the first cameras coordinate system it is x0 y0 z0 now i just extend this ray with the points ER and supposing I get the image at the point P R okay I am assuming that the point P L is having coordinate X L y l and the point P R is having coordinates X are Y R and this X are Y R coordinate please make a note that the coordinates of both PL and P R I am now representing with respect to the origin of the first camera so this X are Y R as well as x ly L both are defined with respect to with respect to the XYZ axis so please do not misunderstand that this x ry r is defined with respect to X prime Y prime Z prime it is not X are Y R is defined with respect to XYZ only x ly L is defined with respect to XYZ even the point P okay defined over here is defined with respect to the XYZ coordinate system so now for the first camera over here what relationship do we have just applied R say given the relationship which we have video study can you tell me what relation to make a Prada left camera anybody X 0 5 X L correct is equal to y 0 prime y L and that is equal to lambda - somebody will say steal Oh actually CL close to the total rate lambda so this distance is nothing but our lambda so this is equal to lambda minus Z naught divided by lambda all right let me call this as equation number 3 now and for the second camera can anybody give me the relationship for the second camera like to say for the right camera can you change the relationship now you have to build up for a transformation because remember that the point itself why are the point P R because a component is salvia has been defined with respect to the coordinate system of the first camera so you have to do a shift off or you all right now this is taken as the positive x-direction and I have moved by amount Delta s over here so for the Delta X value that I put is negative but considering a shift Delta X between the first camera second camera very simply what we are going to have is X naught plus Delta s divided by X R plus Delta X and that should be equal to Y naught divided by Y R because I am NOT considering any ship for the Y and that is equal to lambda minus Z naught divided by lambda and this is my equation number four so now out of this equation three and four okay what do we see which are the known quantities I will mark all the known quantities by red lambda is known to us if somebody gives me the point x ly L this will be known to us if somebody also tells me that corresponding to this X ly L point corresponding to the point P L on the Left image if somebody happens to tell me that what is the corresponding point for the right image x ry r then even x ry r will be known to us and Delta X is obviously under our control because we are shifting the two cameras with respect to each other so Delta X is known to us so out of these two equations okay what I have is that I have got x0 y0 and z0 as unknowns but really speaking I have got not just two equations but I have in fact two equations for each one of this so that two plus two there are four equations than alpha P our knowns and it is possible for me to find a solution and at first let us attempt to find the solution for Z naught from these equation let us try to solve for is it not okay so what do we get in the process if you solve these two equations for Z not the ultimate expression which you get for Z naught is like this you get Z naught equal to lambda plus lambda into Delta X divided by X L minus X R plus Delta X simply solve these equations okay and you eliminate it's not and why not from these equations all right and you will get this expression by eliminating X naught and y naught from these two equations you will get Z naught equal to lambda plus lambda multiplied by Delta X divided by X L minus X R plus Delta X now let us look at the physical significance of this equation this is a very important and quite an interesting equation that we have got now here everything on the right hand side is known to us lambda known to us Delta X known to us I know what is Excel I know what is XR because somebody has given me the corresponding points TL and P are Delta X is known to us so I can determine the unknown depth Z naught which was originally lost for a single camera view now what is the physical significance of this equation let us examine the this tab that is to say lambda Delta X divided by XL - within bracket XR plus this that is X L minus X R plus Delta X can anybody tell me that what is the physical significance of these denominator term you think oh wait they're walking the significance of this denominator term again look at the Stelios unity over here all right and you have the point accel yl over here you have the point x ry r over here okay and what is the signal physical significance of XL minus XR plus Delta X obviously if you are considering this XR plus Delta H star all that you are considering is that as if to say you are translating this point P R to somewhere over here okay by having by considering XR plus Delta X you are translating this point P R on the first kuvira's photographic plate and then you are as if to say finding out a relative displacement between the corresponding points in the two views because this is having a x coordinate equal to XL the point P L is having x coordinate equal to excel and the translated point P are referenced on the first images photographic plate is having a coordinate X R plus Delta s so the difference between the two is XL minus XR plus Delta X so the denominator term over here gives you what it gives you the relative displacement now can anybody quickly tell me that what is the relative displacement in the point P is situated at infinity yes the correct answer is that if I consider this point P to be a infinity then if I translate this point P R to the first images first were cameras photographic plate the point P R will merge with the point P L that means to say there is absolutely no relative shape if you are considering some object which is at infinity at infinite depth from you okay and you are trying to view that object from one position over here and then you go over to the second position to view that particular object okay you will find that there is no displacement that has really taken place whereas for the objects which are relatively near there you will find that some displacements have taken place between the first view and the second view so if you so you find over here that the unknown depth Z naught okay becomes inversely related to the displacement between the corresponding points in the left view at the right view if the displacement is less the depth is increased from this relation it is very clear that if X L becomes equal to X R plus Delta X the denominator term becomes 0 and we get Z not equal to infinity okay whereas if we make this XL minus XR plus Delta X the denominator term if we make larger and larger okay we will get smaller depth or to say it in the reverse way that for the points which are located at relatively smaller depth for them the displacement the relative displacement between the two views will be more so in order to determine the unknown depth what we have to do is we have to find out the corresponding image points and PR so that is the first task for us that we must find out that for a given point P L on the left image what is the corresponding point P R on the right image so this process is a process of correspondence finding out the corresponding point and after doing that we have to determine I mean we have to determine the Z not the unknown depth by making use of this relation rather we have to measure the relative displacement so it is the relative displacement which gives us the idea of the depth all right any questions efficient examples in seven here actually the previous with respect to a 0 K x1 information yes with this big book Israel may sound why on is that respect to the XYZ coordinates is just that system yes he may transform it to inspire viruses yes you get this equally we are taking the first two x axis yes yes so we first few pages we can label all right the confusion which is asked is that whether it is positive or negative whether the relationship which I have a rickshaw love their diets and itself plus Delta X whether it will be plus Delta X or whether it will be minus Delta X now I have considered this Delta X to be in sign I mean I am NOT considering only the magnitude part of it in this case Delta X has become negative with respect to this o Delta X has become negative so I will substitute negative values over here so that is how I am keeping this relationship I am maintaining this relationship I am NOT considering the magnitude of Delta X if I had considered the magnitude of Delta X in that case in this particular case we would have got X naught minus Delta X but I am NOT considering that I will just substitute the negative value of Delta X for this particular case because my positive x Direction is along this okay so this Delta X which I am considering is negative any other questions any other confusions why do we get this dissonance for innovation I mean ourselves why do we get this displacement now it is very clear from the geometry you are considering a point from two different positions okay so you have for the first one your point of focus is over here and for the second point your point of focus is at some other place okay the two viewing distances are differing now that is why just see the Ray which comes from point P to this point and the Ray which comes from some other point let's say we have a point P Prime the Ray which comes from point P prime okay that will intersect the second view at some other points so obviously there is a relative shift always they there is a relative shape with reference to the I mean with respect to the depth of the point in three-dimensional space alright because the two viewing positions are different the only case where we do not get the disbursement is when the object is at infinity when the object is at infinity then taking the two views okay will not do much of a change because we will be just seeing the same image virtually I mean for the same I mean father an object which is at infinity even if we change the waving positions okay it's not going to change our displacement taking care of our intense correct but just see from this relation that if I have the point Z naught at infinity then this excel becomes equal to X R plus Delta X so I do not get any shape any other question all right so I think we will consider the further aspects of Serio geometry we will continue this discussion further in our next class thank you

Channel: kashyap B

Views: 19,944

Rating: 4.8545456 out of 5

Keywords:

Id: hab07nMeUzA

Length: 50min 8sec (3008 seconds)

Published: Tue Jul 02 2013