OpenCV Lane Detection Tutorial using Python - Intro to Self Driving Cars - with Code

in this video we're going to see how we can do lien detection which is used for self-driving cars it's a simple Lane detection application not using machine learning or AI a simple Lane detection using image manipulation from open CV functions so it will give a good hands-on experience on using some of the open CB libraries but at the same time give us an introduction into Lane detection now we are not using AI or machine learning and there is no weight training going on on the lane detection so if you're thinking about this on a production level this library or this function will not work out however for anyone who is getting into Lane detection or starting Lane detection or working on self-driving cars this will be a very good introductory application so let me go into the command prompt and see and just give you a quick demo of how this function is going to be looking like I have this window open up here and it gives out the result so you can see the lanes are being detected by the open CV and it draws these red color lines on it showing the links that it plays with the lanes now this function will work out if the Lane are in a straight line but if the lanes are converging or diverging or Road you know they're rotating and if they're curved it this is not going to work out it's only going to work if it is a straight line no with that being said we'll go into the code and see how it runs I'm going to comment everything for now and we comment everything to the point of this and I will slowly and slowly open up each commit and each application age function in each line of this Python application and show you what it is doing so the first thing we want to do is input the opencv library where we are going to be using all the application all the advanced image manipulation applications the next is the numpy and numpy we are using mostly for matrix manipulation as a little bit of matrix just to read the images I'm not sure it's not nothing nothing too crazy just to read the images and give out the information of some patties and matrixes that's about it now most of the heavy lifting is going to be done by OpenCV now let's go down all the way down and this over here if you see the opencv weight key function release and destroy these are almost an essential part of an open CV Python application weight key is something which you need in order to show you run this imshow function which we'll see later and you know if you if you have an IM show function you've got to have weight key without weight key I am show function cannot run the next step once once everything is done once your application is completed done you want to release the cap and capture is nothing but the variable that is gonna hold your video now it could be a video file it could be a webcam it could be a live screen whatever it is but capture is what it is holding on and you want to release before you are closing your application once you are done with that you are destroying all the windows that open C we might have created and this also accepts the program so we now start approaching the or you know running the application one by one one step at a time so the first step in this Python application once you take care of all the importing of libraries and everything the first step is to read the video file and the video file is test dot two one and test one mp4 and I have this folder here the Lane detection folder and you can see the Lane detection Python file and the test one mp4 both of them are in the same folder hence I'm able to simply write test one dot mp4 and this is how it's also uploaded on the github code which is in the description below and I don't need to provide but if you were to put them put the video in a different folder or a different directory you want to provide the directory file name or path name and we are storing that capture into this cap variable once you have defined that you want to start reading the frames from the video and video is nothing but a sequence sequence of images a sequence of images placed a one after the other and that's a video so now you want to start reading the images so you're using this vial loop and this vial is going to be reading the images as long as capture is still open and it's still supplying those images from our test one video and you're using cap dot read in order to read the frames the images from that video and saving it into this variable called frame and this is the original this is the main image from that video where there is where you can see it's at Road kind of car which is driving on the road and that frame is being saved here the first image manipulation we're going to be doing is canny image and you're gonna do canny edge detection and canny edge detection is a very old edge algorithm edge detection algorithm and it allows to read the image and detect edges in it so we'll see how it works and we'll come back to the top of the application and we will uncomment this portion which is the canny edge detection function and you can see it takes the argument image and it comes down and it sees whether the image if there is an image or not if there is no if the image is not there then it releases the capture describes all the windows and it exits the program so this is just a flag to perceive whether the video is still running or not you may you may or may not need this but just to be you know not to create any more errors in the program so the one this check is done now it goes into starting to manipulate the image the first step is to convert our colored image from BGR to gray using this convert color function of OpenCV you're providing the in original image and you're converting the BGR to gray OpenCV saves color images in the BG r format unlike the RGB format which we normally know and many of the machine learning applications use RGB but OpenCV uses BG are you converting BG r to gray and that is what can eh deduction needs and you're saving the gray image into this gray variable you're now defining a kernel size of five and what is this kernel you are going to first blur the image before passing into the canny edge detection and you're doing blurring in order to reduce the noise level in the images because if the image is crisp and sharpened and you're going to be running an edge detection and that it will cause bring up a lot of noise a lot of noise that you're not interested in so you're going to be providing this kernel image and this kernel and the size of five by five so any image or any object in the image which is less than five by five will simply be erased by this crossing blur and once that noise is removed that initial level of noises removed you're going to do a canny edge detection on this and this canny edge detection is going to be done on this gray image and it saves into the scanning variable and it simply returns the can e-file this image back to our main function so let's go back and go down and see so we have this canny image shown now let's see how this image looks like we'll right cv2 i am show canny image karma canny image so let's see let's save this and come back to our command prompt and run this so you can see that it's detecting all the edges in the images and it's detecting every image it's detecting the the trees the cars the drawers decide everything being joined it's too much of information we are not interested in we are only interested in these two things so let's see how we can take care of that go back to our code and we are going to provide we're gonna let the function or the program know that we are not interested in the all the edges per hour at present of the image we are only interested in the location where the lanes are and by human knowledge we know that the lane is going to be present on the lower half of the image because that's where the road is present so we are going to simply crop that much region and then do our algorithm search on it so let's uncomment this and this is going to be done by region of interest so we're going to be creating a region of interest in our caning image let's go up and see where the region of interest is and how what it does so the grecian of interest function takes the canny image it takes the image it obtains the height the height is given by the shape variable of the can image and 0 0 element of the array gives the height the first and then the second element gives the width and there is also a third element which gives the channel size but in our case we are only interested in the height and the width we are also creating a mask because that's what we are interested in we want to remove everything else except the lane on the road the third step is to create a triangle and we'll see why we're creating that triangle but just to see what the triangle list looks like so the triangle has these coordinates it takes in three quadrants we're using numpy for this and we are giving three different coordinates of their triangle so basically the three points of the triangle the 200 by height 800 by 350 1200 by height so these are the three points of the triangle so I'll show you what this triangle is basically it help us make look like this fill poly is making basically masking out everything else except that portion which is being defined by this triangle and once we are done with that mask we just do bitwise and so that we can remove the external feature and only keep that much region which is needed for her which is being defined in that triangle and where the lanes are supposedly going to be press it once we do all that we just return the masked image so let's go down and see what this triangle has done for us so we will instead of calling the CV - I am show here we will call it in this and let's bring it back to our format and we will now run cropped canny instead of the canny image will do cropped canny because we are interested in to see how the triangle is affecting our program so let's run this and you can see that it is only running that region you can I'm not sure if it's visible but there is a triangle coming up here which whose tip is here and the two edges are here and it's trade removed everything else and only kept the points which are interested to us let's come back here now once we know where the leaves are present and once we have done the lane detection it's only a matter of identifying the lines present in that image and then drawing the lines on it so these are three or four functions that we are going to be doing using in order to do that so first is hope lines and hook lines is algorithm hope line transform it's a algorithm which is supposedly supposed to run on a canny edge detection and it gives an approximate presence of lines in an image we are going to be using Hogg lines and average lines to obtain the location of these lines once these two lines are detected then we go and display these lines on the image so let's go back and see what hook lines dance for us not sure how exactly it is pronounced but Hoch lines is algorithm it's a very neat algorithm if anyone is interested leave a comment and I'll go make a video of what Hoch line does and how it works it's a very nice very neat algorithm and so there's something interesting going on what they have done in order to reject lines in an image so this is the function and OpenCV has that function already so you're using hog lines and you just preparing the cropped image and defining where the lines are so you're just returning the lines but those lines are not it is just a probability of where the lines are so they're not strong so you want to prove once you are done with hog lines you do average slope intercept so this is where the average slope intercept ends and it takes those images those lines and our price to obtain the exact slope of that intercept so it it can identify the exact location of the lines so out of all the lines that are present it it's only interested in the lines that may represent a lane and that is what we're going to be doing in this function function so it's it's a pretty much function out here which is going to be using the slope and we're using another function called make points and it goes through the lines and it goes through each line one by one and it tries to identify which line is important and which line is not and you actually after that you find the slope of these lines and you're using this make point function which is out here and it defines it uses the slope of these lines and it we are defining the slope of this line based on the image sizes the image size is given by here the y1 y2 x1 x2 these are nothing but the image size and replaced on that we are defining that the line should be so and so big so in this case we're giving it that it should be three by five the intercept so these are basically the ways that you are as assuming your lines are going to be now again these are our assumptions if you were to give a different image or a different scenario or a different video file altogether then you would make you would have to change these functions accordingly so these are some of the things that you would want to change and even in in the case of the triangle this is the triangle that worked out for this particular in a video file if you're running on a different video file then you would want to change the coordinates based on where your lines are and I would say even the added weight function will also need it so once we are done with detecting the lines once we have known where all the lines are we go into the next line which is the display lines function and we'll say display lines display lines is nothing but a simple OpenCV function of drawing lines on an image and you're using the image the image if you want to draw the lines but you're also using the lines that we detected in the original in the previous function with the slopes that we've obtained and you're just using a line image this is a blank image that we're using and if there's no lines then they should be no it should not draw anything but if there is a line then it draws a line and you're drawing it on the line image and you're drawing it on the coordinate x1 y1 and x2 y2 this is the coordinate of the line and the line is represented by the starting point and the ending point this is the color of the line and this is the thickness of the line so BGR so it's B's which is blue is 0 G which is green and 0 R which is red is 255 that is why you see a red colored line and you're just returning the line image once you have done with red returning the line image you are going to add some weight and the reason we are doing that is because the line which we are drawing it is on top of the image and you want to give some weight to that line so you come back here and you uncomment this function and so that you're providing some way to the image so that line the line is visible you're basically adding these two images the line image is the image of that we created in the initial function from the previous function sorry and this line image is just drawn on a black coloured image which we had created here for using the zeros like and this line image has been created on that black colored image and we are now adding this line image with our frame image the original colored image so we'll just rat adding these two but we're using this add weighted so that we give some weight to the line image as well so that it does it's not fully more company much and it's not visible at all so we give some sort of weight and this is some type of emerging or addition of two images a function between in open CV once you've done with all that we go back and we return the image into this combo image and we simply display we can uncomment this guy and save this and go back and run our code so this should show how it was earlier in the demo and you can see how the lanes are being detected assumedly there used to be a lane here previously that's the reason it's causing some error initially but now when there is no image though no lanes present then safely detecting the lanes on the road so you can see how this function works on a straight path since you're using hook line transform and hook line transform is only able to detect straight lines hence this function will only work on straight path but if we were to run on curved roads or roads which don't have full perfect lines or lanes then this algorithm will unfortunately not work but you now we know how Lane detection is runs and how we can use it for self-driving cars it is a good starting point and we'll see in the future how we can develop this more algorithm further if you liked this video leave a comment and subscribe to the channel and if you like those journey if you like such programs such technical challenges leave a comment and I would be happy to run into them until then take care bye bye

Channel: Misbah Mohammed

Views: 26,031

Keywords:

Id: CvJN_jSVm30

Length: 20min 22sec (1222 seconds)

Published: Sun Jun 07 2020