Lecture 13: Fundamental Matrix

okay so we are going to you know talk about new new work new materials and this is in connection with so you can edit this Henrico okay so we are going to talk about new material which relates to what is called fundamental metrics and this is a relationship with the transformation between two images and we have been talking about for some time and as you just heard that we can relate to images by a translation translation X translation Y or we can have rotation or we can have rigid transformations which include a translation in the rotation or we can have similarity transformation which is scaling in the rotation and now fine which captures lots of these things at the at the top and which has these six parameters so we have the affine transformation as the rotation and scaling and that matrix which has four parameter than your translation you have six parameters and then more general case is a projective transformation which has eight parameters and a fine and special case of projective and we also looked at the linearization of projective which is approximation of projective transformation like pseudo perspective and bilinear and so on okay so today we are going to talk about new transformation between two images which is called the fundamental matrix and let me first make sure the audio is working and also I'm online okay good so this fundamental matrix is very important concepts and it will be used in lots of different places and but it is very intuitive very simple so you need to pay attention and I will make it interesting for you guys so phenomena matrix again is like a transformation between two images and once we can find the phenomena matrix then it has lots of application including the that we can do is tell you when you have two images we can do the depth recovery you know from two images using a stereo and we fundamentally this is also use in structure for motion that you can recover 3d information from our sequence of images and it can use also in human action recognition and many many applications of phenomenal matrix it's a very important concept so for example stereo you have these two images when the left and right image we can come up with a depth H pixel that how far that pixel is from the camera and this is another example and this is that recovery of this and then in visualization of that image from different viewpoint so important step in stereo is to be able to rectify images so that we can find the given a point in the left image we can find its mate in the right image along the same row and that is will simplify a lot and that can only be done if we can if we can take any operative image is taken from different viewpoint and and align them so that they are rectified now and due to rectification then this row will correspond this row and so on the matching will become simple and it's a very important step in this stereo so for that we can use fundamental matrix to help us to do it and also as you remember I showed you this video Microsoft photos and where they have a method which will recover 3d informations and you have seen this so I'm going to skip a little bit and you can you know you have a link so you can you can watch this videos available so they will take different pictures off of some building some area and then they will reconstruct 3d and basic operation there is to be able to come compute the fundamental metrics and if you read their method in detail you will find out that that is important that they need to compute them okay so fundamental metrics started from a paper from a British scientist called longe and Higgins 1981 and he actually talked about what is called essential matrix which we have been talking which you're going to talk about and which is the matrix relating the two 3d coordinates of left imagine light image and left camera right camera and then in 1992 richard hartley and all we have four drawers the this one is an Australian researcher actually he was working in US and GE and this is the French researcher they published this paper simultaneously in two different conferences about this fundamental matrix and both are given a credit about that then there's a lot of work about phenomenal metrics and one of known work is saying using who is not Microsoft Word student of for Jerez have done robust estimation of the phenomena metrics so the interesting thing is that phenomenal metrics is so crucial is so important in computer vision there's actually I found this YouTube video there's a song about phenomenal matrix so just make you makes you interested in this meta stuff so how do we play this thing so I thought to show you that control and then this hmm okay now it's opening oh great okay how do I go to the thing to do the whole thing so instead of our do here [Music] see [Music] [Music] [Music] so Richard Hartley is from Australia so this is the famous opera in Sydney [Music] [Applause] [Music] okay so you like it okay so that's you know that's the science is good so let's go to back to the lecture and and all these things they were talking about in this actually we will talk about all the a peoples and their people align and all these in the song okay so so I'm going to you know cover little preliminaries which will be used in deriving this fundamental matrix very quickly so that you feel comfortable so one idea is of called linear independence and most of you I think must be familiar with that but just for the sake of completeness I want to cover those so then other is the rank of a matrix and matrix norm and singular value decomposition and vector cross product and prevent a red cross product of matrix multiplication and rancic actually there's a song about n Sanga rancic also I'll play that for you also so so the as you know that if we have a set of vectors these vectors say we want to VN they are linearly independent if we can write down like this where we have the coefficients a1 in and not all of them are 0 which means they these all these vectors are linearly independent so we cannot try to one vector in terms of other vectors and that's a definition of that and that's a basic idea in linear algebra is used a lot so that's one thing what you need to know second thing is the rank of a matrix so this rain can be a column rank or the row rank and it is basically if I have a matrix how many if I want to look at the column rank how many columns are linearly the pendant that'll become brain if there are five columns then five the rank and then we have a row rank which is also how many rows are linearly independent if all of them are independent then that will be the fund rank of that matrix so and the rank in a way give you a dimension of the space and the column rank will be the dimension of the column space and then row rank will be the dimension of the row space so here's examples there's a three by three matrix the maximum rank it can have is three number of rows are number of columns but this R in this matrix has either you drink two and what you do is one way to find a rank is you convert this in the row echelon form you must have done this because in elimination they're called substitution how you solve a linear system so you take this matrix and try to make this upper triangular matrix you make all zeros here know that diagonal and so that it becomes the row echelon form so we are going to do the operation we take that first row multiply by to add to the second row so then this will cancel with this and then we will get like that then we will have another operation we are going to take the third row and - it and then add to the first row and then this will give you this will become 0 so then the final operation will be we'll take that second row and third row will add up so this will become 0 so now we have this kind of opportunity X and since the last row is zeros then the rank of this is true ok because 0 is not you know one of the vector linear independence so there are different ways to find the rank of matrix so you need to understand the maximum rank is maximum number of columns or rows but most of times rank is lower than that that's one concept so the second concept is called singular value decomposition so I can take matrix say a and doesn't have to be square meters and M by singular matrix I can break it into three matrices or one and the gamma and Oh two three matrices this is also called factorization of matrix M factorizing this matrix and three matrices when I multiply all these three then I get in this matrix okay so now this is M by M and this is M by N and this is n by N and this is n by M so that is called singular value decomposition and MATLAB has one CH and they give you SVD singular value decomposition of a matrix and it's also very useful okay so this is always a diagonal matrix which will have the singular values or the square root of the eigen values and these are the orthogonal matrices which means you can take a row column and multiplied with itself b1 if I if I take a column allow the other column and b0 that's called orthogonal matrices so the next concept is a norm of a matrix you know like you are used to norm say l1 norm of a scalar which means absolute value is a norm or you can take the squared Euclidean norm you can take a square and under root that's the you clear now so similarly there's a norm for a matrix matrix has many numbers it has rows and each row has some different elements so one is the l1 norm which is shown here what we do record the each row of a matrix take the absolute value add them up we get one sum it at the next row take that all the absolute values of the that row sum them up because even a number we have these numbers and we pick the maximum one and that's some the norm of a matrix and so l1 norm because we are finding the absolute value similarly there is the Infinity norm which is the which is shown here that we look at the absolute values of these elements and that in the matrix and pick up the maximum one whatever the maximum one is some of that and we picked the mixing one so these are different knobs now the next thing is the that how we can express the vector cross product in matrix multiplication so as you know if I have two vectors and B then I can find out the cross product of those two vectors by using that ijk XYZ the surface vector is the second vector and the way it works will find actually determinant of that so we will first element will be to move this one this one come a Y B C - the Z minus B Y which will be this element first element the next one will be this one and this one then we'll multiply these two ax B C - AZ BX and with a minus sign because plus minus plus much is shown here then the third one is we remove this one and this one then X B Y minus a by B X which is shown here so that is the cross product of two vectors which you have learned okay so now what we want to do that we can show that we can represent a cross product as a matrix multiplication with a vector so we will make of matrix from vector a which is this one like that and this matrix is anti-symmetric which means on the diagonal we have all zeros and then this element if this element is AZ then this will become minus AZ if this is a y this will become minus a1 so these are negative of each other in the diagonal element of 0 this is called anti symmetric matrix so we take a vector a which has three components we made a three by three matrix like that now we can take a vector B multiplied with this we will get like this and this is exactly same as here and so which means is saying you that the crossbar that can be expressed as a matrix multiplication when matrix is formed from the first vector as the anti-symmetric matrix and you can verify if I multiply this with that I'll get first element multiplied this was second I'll get this one and so on so that is another concept which we are going to use okay so the next thing is called random sampling and consensus and this is called 'rain sake and I have a song for this also and this is also very important concept which is used a lot that's why people have written songs about it so so let's listen to the song first see how hard I'm trying you do it [Music] [Applause] [Music] [Applause] [Music] [Applause] oh you've got an idea so I'll give you a link so you can you can watch it in know the whole thing okay so so again the the idea is very simple and intuitive so one thing you will notice in science that the things which really become popular and the things the method the concept which becomes very useful crucial at the end they are very simple very intuitive that's a beauty of the science that at the end you can explain things very easily and the concepts are very complicated at the end they will not become popular because the nature is very simple at the end you can describe everything with simple terms and so rain sake is another very simple intuitive idea which is used for example if you have a set of points which is shown here so you have five points we want to fit a line to that okay so this equation of line y is equal to MX plus C and typically what you will do you will do least square fit as you here we have been talking about that and but you can do using what is called rain shake so drain SEC is stand for random sampling and consensus okay so we are going to do the sampling of the points we are not going to use all the points available to fit the line but we are going to use the minimum number of points required to find the equation of line in the equation line minimum of our point is two because we have slope and intercept okay so so there's another method which we actually talked about that also Hough transform but but let's focus on this so this is your typical Lisa scarf it that this is the equation of line and then we minimize this the square of the error we come up with these parameter MHC so that this at least as far as carly's perfect okay so now the what we do as we've been talking about we get the money question for one point so we have say n points we get any questions and we can write down like this matrix and vector form and we have two unknowns which is M and C and this matrix is not acquired so we can multiply by transpose and get a pseudo inverse and the survey if you find it's very simple you have done many many times God Lisa scarf it so now the rancic is the method which looks at differently and so random sampling and consensus so the idea is that we'll select randomly two points from the endpoints you know and we will find fit a line and we will find the error of that line from the rest of the points and we will note that then we again randomly sample another two points fit a line and find the error from the rest of the points and note down that so we keep doing that whichever will give you a minimum error that's a solution so this approach is very good in presence of in the oven noise then you have in this case when you have these points say some of the points you are trying to fit here is a point here it is very simple it's an outlier which was just a noise but now this will define the whole line because it will try to fit that point also and that will be a problem but when check what's going to do since it requires only two points so we're going to select say randomly two point three point number three and point number five put a like and find the error of course the error from this other point would be large but the error from the rest of Pineview is small when I'm you select another two points say one and four then five fit a line find the error and keep doing that at the end it will find the line which fits most of these people most of these points and it will get it after then why's that analyze appoint the outlier and that's the idea of the rain check okay it's a basic concept and you can use for anything and we're going to talk about that how we'd use this in the computing fundamental matrix okay so that's that so now we I think are done that in the preliminaries you know the ransack the norm of a matrix SVD the rank linear independence and so on these are all going to be used here and then will come out very nice beautiful result then far which you can write a song okay so so the what we have here we have a 3d world like this one we want to take a picture you take two pictures there's a one picture from left camera and this is the image plane of the camera this is a camera Center and we take another picture right camera in this image plane of light image plane of light camera so then what we do that if I have a point P here then image is going to be formed in this left camera here which will be XL and the image of this point in the right camera form here which is our XR and these are the center of camera CL and C are left camera right camera and one thing you notice that all the points which lie on this line will have one image which is this one because it's a ray of light hitting so all these mines we projected in one point now on the right camera all these points will be projected differently each point here another point here and so on okay so now this line which is actually projecting of a one point the left cameras call a peephole online okay that's as the name is called a people of light because in one camera it's a line and another camera it is a point and that's a geometry there it is so that's one thing which you need to know second thing is that the image of the camera Center which is here say right camera in in the left camera is called e-l which is called a peephole okay and the image of the left camera in the right camera which is ER and this is a peephole so it has a name so what we are doing that we are finding the image of the camera center by connecting the camera center with the camera scent of other camera so that that's the way always when we find image we take a point and draw a line from that point to the center of camera then then wherever it hits the image plane that's the image similarly as we did we draw a line from the point the center of right camera it hits here then that's image we take this point right line from here to this one it's image and so on so so we have a peephole and we have a people on line okay and then this is called a people or plane so we have a point P CL and C R and we have these three points they make a a people a plane and this vector which separate from the separate the two cameras CL and C R is called translation vector T vector okay so that's very simple very intuitive two cameras and we are looking at how we are taking two pictures that's first thing now we are going to look at little more detail and we are going to look at these vectors so vector from CL this that point P will call PL left vector the vector from CR to P will call right vector and this vector we already talked about translation vector okay so now you see that these vectors lie in the plane okay the vector T vector PR and vector PL minus T which is this vector these three vectors now in the plane as you see now when the vectors lie in the plane then we have the coppa linearity constraint which means you can take any two vectors find cross product and then find a dark pack of a third vector that has to be zero that you must have learned somewhere so that is what this is saying I take two vectors T and P find a cross product and find the dot product with the third vector it has to be zero because they are on that all all on the plane because you take two two vectors find the cross product the third vector that the cross product vector has to be perpendicular to that plane yes and if you find the dark particle vector which is perpendicular plane and the vector which is along the plane it has to be zero because 90 degrees zero okay so that's what this is saying so now also we can see that the PR vector which is in this vector we can write down in terms of the PL which is this vector in the translation and rotation this is this is the relating the two coordinate system of this left camera right camera we take a PL minus T and rotate and we get PR which is shown here it's pretty simple transformation so that's one thing now we can further manipulate this and we can for example take the rotation matrix and bring it on other side which would be inverse R cos 4 transpose of this rotation matrix so this will become transpose we have PR here and PL minus T okay then we can take a transpose on both sides so this will become P R transpose and then R and this whole thing will become PL minus T transpose okay there's a pretty simple manipulation of these vectors the transpose and so on yes hmm yeah so see that what the idea is that mm-hmm the if we take the practice this matrix here so we started from here okay so the we have to take another side will have the R inverse okay so the as you see the rotation inverse of rotation matrix the transpose of the rotation matrix as we have talked about if you rotate clockwise and if you turn it counterclockwise so therefore this is this is true okay so the other thing is that now we are taking a transpose if you have two matrices a B transpose of that will become B transpose a that's what is this okay so now given this thing we are going to put in in this this our original constraint here that PR is given by this and this relationship so from here we found out that PL minus T is given by this so P I PR transpose and then rotation matrix R so instead of this one we will replace by this so we have now that PR or transpose and then T cross product PL so this is fine we just keep it here so we have done all this manipulation for this vector and we found out that this we can replace by that so we have now these matrices yeah yeah with respect to second yeah that's right so so because you can relate any two coordinate system by rotation translation that's what this is saying so now we have this okay now again PL is the vector which is from here to here T is this vector or the rotation matrix PR is this vector we know meaning of it everyone so in any question you it's important you need to know what each thing represent then you can you know you know visualize and you can really have meaning if you don't know they're just symbols then it's very hard you have to memorize and it doesn't make sense okay so we have this one now what we are going to do is we will introduce this idea of these cross products so remember that we have two vectors T and P L is a COTS part of that as we say we can write down as a matrix multiplication so we will find the matrix S which will be from the T vector and multiply with the PL vector so which is this one this is the matrix in time symmetric matrix the zeros on diagonal and then DX dy and they are negative of each other that's good so then there are two matrices our matrix with the rotation matrix s matrix which is the translation vectors anti symmetric matrix we multiply these two matrices is called emai tricks and scott essential matrix and that is what was discovered by an unguent agent in 1980 and there was the first paper in Nature which is very famous general for science and published that that you can do that it's very because what is doing it's capturing this transformation between two you know cameras in the 3d by a very simple equation which is saying that TEKT appear or transpose that which means the coordinates of the point P in the arc light camera coordinate system coordinates of the same point in the left camera system multiplied with the essential matrix which contain rotation and translation between these two cameras it has to be zero it's very nice to serve very compact and that that's why I became famous called essential metric so so far we haven't been 3d PLP are they are all 3d so the next thing we are going to do will be in the image plane and that's why it's called from the new metrics this is called essential metrics and next one is called phenomenal metrics and that is again very simple now when we apply camera model we did this last lecture that we take a point in the world say left camera multiplied by camera matrix we get this image which is shown here and we take the right camera metrics and multiply by the coordinates and 3d right camera we get the one is this one is the left is from the right camera okay and then we have this relationship of these the essential metrics and these 3d corners like that so we are essentially going to use these camera models to find the PL from here we just find n words of M which is shown here PR from here find the inverse of M or like here and so then we are going to take the PR and find the transpose of that again we have two vector and the metrics to transpose will be this transpose and then this transpose so we will have this so now we have PR transpose in terms of this and PL in terms of this we are going to substitute here okay so for PR transpose will substitute this and PL will substitute this and that will become like this this is the PR transpose this a here and PL from here and then we have the essential matters e left camera matrix right camera matrix we have three matrices multiplied together become one matrix and that is called fundamental matrix bingo yeah that's again a very interesting result so what this is saying that I take a point and left camera multiplied with fundamental matrix then find out you know the corresponding point in the right camera multiplied with it it has to be 0 let's call fundamental matrix and strength and we just drive it very simple geometry okay and it is a lot of big consequences because it is really you know have big impact and then defeat so so that's it no so now the fundamental matrix is a 3 by 3 matrix which has its nine elements and it is X the image in the right the X transpose I mean the left and has to be zero so now as you sign the picture that if I have a point in the left it will be projected a line in the right and so therefore if I take a point multiplied with fundamental decks this is actually a line and multiply that with the X transpose then become it has to be 0 so that is what the fundamental in Texas so now this is has properties it has nine elements and it is of Rank 2 because the the S matrix is offering to that matrix we we caught that anti symmetric matrix as you remember here from the translation it is rank to rank up this is 2 if you find the eigen values of this and actually you should do that then one of the eigen value will be 0 so if not all three eigen values are non 0 the rain cannot be three if one of the eigen values 0 so rank can be maximum true so this actually has ranked 2 so so that's one thing we are going to use that actually so then other thing to look at is that this fundamental matrix is capturing the rotation translation between 3d coordinate system and also some internal parameters of the cameras so if you are add all those together there on there are actually seven degrees of freedom okay now 7 you know variables there which you can chain but actually nine elements the way it is and so as I told you this is a history started with lunch antigen in 1981 in the paper in nature journal about the ascension matrix Hartley's and for jobs came up with simultaneously one adding in CPR eleven EC CV conference in nineteen ninety two phenomenal matters idea and sayings and using has come up with the robust way to estimate that so we have talked about how to drive fundamental metrics now we're going to talk about how to actually estimate given two images okay so again the fundamental metrics capture the relationship between two images and there's no approximation now okay so all other transformations when we talk about a fine projective and all the that approximate well projective is approximation when the scene is planar and the perspective projection our fine is approximation when the the orthographic projection but here this relationship fundamental is no no no approximation so it's full it is perspective it is full 3d and all the same so that's you want to note so now the using this fundamental the metric relationship we have this that we have a vector in say left camera the corresponding point and right camera and then we have these nine unknowns so our aim is to find these nine unknowns given two images say when Harris pine see our hands find here find the correspondences and then estimate these nine unknowns and that will do a lot for us so how do we do it so we can you know multiply this with the matrix so just become X I prime F 1 1 y prime F 1 2 and F 1 3 which is shown here multiplied with second or we'll get this butterfly third row we'll get this then we multiply this with that and we'll get a one in question so this X prime multiplied with this all these elements wipe I will be multiplied with this and so on we get one equation so which means if I have a one point in the left and we have point corresponding the right so the left camera is X I Y I the image point and the right camera is X I prime by ax prime then I can get one equation and I have nine unknowns okay so I need more points so that is what we have and so this is a question in which we have F 1 2 F 9 r f 1 1 F 1 2 F 1 3 and F 3 3 there are nine 9 elements and these image coordinates and left and the mechanism right x and y and x by y prime so so one equation for one point and so we are going to do again like Alissa scarf it you know that we have the one point one equation so you give you this row second point the second or third row and your end points n rows and F is the nine unknowns here and this has to be 0 so that's again a homogeneous system so this cannot have a unique solution okay so therefore we can pick one of them opportunity so we actually have to find a ton nodes ok so that's what we are going to do so the computation of phenomenal matrix there is this algorithm from hotly it's called eight Pinal Kaka as we said there are nine unknowns but some ammonia system we can find only eight the ninth element we will select operative one okay which is fine so therefore his algorithm needs eight points correspondences in two images and go through an estimate the fundamental and listen to this very carefully these steps are important but at the end it's very nice so the objective is and this is very famous paper no it is called in defense of eight point algorithms title is that that you know and pay me so we want to compute fundamental matrix such that no this is certified that's our you know constraint so so what we are gonna do so we are given eight points they are given the points in eight points in first image second maybe we know their correspondences there are no coordinates and left image coordinates and right image so first thing we are going to do is normalize the coordinates and by applying some scaling and translation so that we can have both in the same kind of scaling format so we will apply this in the transformation matrix translation and kind of scaling for the left and right and that essentially means they these are the scaling and these are the translations and that means that protect the points and the left image find the centroid and subtract the centroid from all other points and normalize these branzino and one will do the same thing in the right camera find the centroid of eight points subtracted that centroid from all the points and normalize the coordinates between zero and one that's it so then we will come up with a solution of that system I showed you by finding the eigen weight vectors of that a matrix and he somehow showed in that paper that if you take the eigen vector corresponding smallest eigen value that is the solution of this F vector which is the fundamental metrics you know corner detector Paris corner has come you get lots of points then you find the correspondences and once you know correspondences then do this so we are assuming I pines are known their customers known them your different of inventors okay so so with this when you will find this F vector but we are not done yet okay so now when you cut the F vector which is which is this vector here which has the nine unknowns then we are going to take that and make a matrix out of that because that's what it was and then we will normalize it and that's where we need to use this norm okay and so this is a norm which means we'll find the norm and but divide each element in this matrix by norm and we can use L infinity norm L 1 norm and so on so now is the important thing remember that we had a constraint that the S matrix is the length too so you wanna make sure that this n if the matrix kernel matrix is product of s matrix R matrix and cámara matrices so if the one of the matrix s matrix a rank to the rank of the phenomena this cannot be more than 2 that's that's another fact so therefore rank of an element X is 2 ok so what we are going to do that will take the fundamental in X and find the factorization of that and three matrices you the singular values and V and these are the singular values if the rank of this phenomenal X is to the third singular value has to be 0 and if it is not 0 will force it to be zero to satisfy the fundament matrix because as you as you know that we cannot find a unique solution from HD system because as a F is equal to 0 there's no unique solution so we found a solution here and we want to force it to be a rank 2 so we will make the say third eigen value a third singular value to be 0 then we multiply back you and this and we'll get F which is better than the F we have yeah so that is very important step so then last step will be real nom de Namur lies because remember of anomalous coordinates they apply the T matrix to the left and T prime to the right and we'll come back and apply those as we did before here and then that's at we get a fundament matrix now this is the basic a triangle Gotham which was proposed and you know which is a you know very good contribution now after that people found out that it's not enough you cannot get a good fund America estimation just using that so then there's an extension there's a robust estimation phenomena like by Zen using and which you use on the top of this then you get really good or novel methods okay so again the idea here is very simple and it's very intuitive so let's say as you are asking you know you can have lots of points and left them in data point item is how do you select which eight points to choose okay so this is where the rain check will come okay so suppose this is the these are the points so what then you saying does they're taking them a divide in the 8x8 grid like this okay and then randomly select eight grid cells and pick one point from each other set okay so you get eight points you apply the Heartless algorithm right on the previous one on those eight points okay so and then you estimate phenomenal metrics using that then you select randomly select other eight cells and get the one point from itself apply the eight Pinal Gotham for by hartley and compute the fundamental matrix and you keep doing that though that's what the range sacristy now then you find out given the phenomenal matrix you find out how good it is by finding the error from the other points which you are not used because there are lots of point which you only use 8 points as we are doing the fitting a line with in fitting a line we just selected 2 points but there are lots of points you find how much the error from the rest of minds that tells you how good is a fit same where here there are lots of points so we selected only 8 we find a solution then we are going to look at the error of the rest of the points which is the error from the left and from the light we sum it up and find that inner and so we let error for phenomena this 1 2 3 4 lots of errors and we will find the median of that and we'll pick the one which is you know is that select the best according to that residual okay which has at least residual which is a median error of these all the points meaning points which we are using and that will give you the candidate phenomenal metrics and now we want to also look at that but if that error determined if the error is very large then that will have lots of outliers so we don't want to use that we will remove that and the the at the end of it will select using this criteria the one best fundamental metrics from lots of these which has the basically least errors as we didn't filling a question of line now using this this as the as the point as the fundamental attacks then we will compute the fundamental metrics with the weighted least square fit where we will assign the weight which will portion of this weight and using all the points so that is basically the the algorithm by then using which is robust to outliers and tried to use all the points not necessarily only eight points and the question is which eight points that's why it's dividing it by eight and selecting randomly and it turns out this map this estimation is much better if you use only eight final Carter okay so that is basically the phenomena matrix computation and also the derivation so if you do that then if you have these two images now if you want to do say stereo between these two images then you won't be able to do because these are very different viewpoint you cannot take a liar over here and go to exactly same row and find the correspondences so once a phenomena is known then you can rectify these images and these are the epipolar lines so therefore the match of this line this point will be on this line exactly same match of this line will be exactly same as you are showing yeah and that is the advantage application of the fundamental metrics that can help you rectify images and this is another example see there's a big rotation but we can find the people on lines that if I want to make this point this point Pines around here I'll just go correspond a people line like this one okay I want to find the corresponding here not for like first one there so that is the example there's another example here again these are the people on lines for these two images these are popular lines for these images and funnel metrics help us to estimate that in addition to all other application of that okay so that's that's the fundamentals so what this is doing is essentially that we want to normalize the XY coordinates between 0 & 1 are 0 & 1 and we want to make that the origin at 0 so which means you will take the set of points you will find that mean and subtract the mean from each point and that phenom lies between 0 & 1 and then you will also a scale now actually that will you know make its origin and then you scale them between 0 & 1 so this is that you essentially are changing the origin to dxdy which is the mean of these points and you are scaling in x and y these amounts so you find the range of x range of y so you divide the range by every point so it'll be you know maximum minimum certainly it will be scaled between 0 & 1 and similarly you do for the y okay and that will scale if you have set up numbers you want to scale between 0 & 1 you find a maximum you divide the maximum everybody and you know it becomes you know one of scaling and you do the scaling x is scaling y second thing you want to do that you want to put we want to make the mean of the points at the origin so you want to subtract the mean from all the points so they are with respect to that and that is the the translation here so this is just a formal way to do the normalization but you can write a program to do it okay yeah [Music] lines are used for the sea line is like a line row in an image so you have an image say you have 256 rows and 2 fruit each column their metrics number huh so now you have a corner point in row number 50 and column number 30 so you have corner point you want to find a match of that point in the max image right image now you can go search in every room there is a match for that where it's exactly same cornica will be a lot of computation there'd be some ambiguity and so on so the best way will be that if you go to the right image you go run over 50 years was the point is in the left image and finds mates okay so that's the idea of Rho and Rho is a line so as we were talking about here that the the epipolar you know geometry here here so see that now the I have a point here this is the image of this point now I want to find where is the match for this in the light image so I can go in the whole image a lot of computation and I make it confusing son but if I know that it has to lie on this line that's a people align so I just search here and I find out actually this is that point that's the idea okay

Channel: UCF CRCV

Views: 35,062

Keywords: Education

Id: K-j704F6F7Q

Length: 59min 9sec (3549 seconds)

Published: Sun Nov 11 2012