Motion Detection Made Easy: Optical Flow in OpenCV Python

so now we can see the results here of the dense optical flow where we can see when we're moving around in the in the environment here or like in the frame we can see these errors here representing like how much motion is there in the image frame hey guys welcome to a new video on this computer vision tutorial so in this video here we're going to talk about dense optical flow so in the last video we talked about sparse optical flow where we just took different kind of points or like good features in our image and then we're tracking around those points or objects um around in the image frame but now we're going to talk about dense optical though where we're going to look at the whole image plane and we're going to see like how motion is actually changing from frame to frame um with our camera but first of all we're going to join the disco server i'll link to the down description here you come join the channel chat raise about computer vision deep learning ai and so on you can also become a member of the channel if you want to support the channel with a small monthly fee and everything will go to create more and better quality content here on the channel so thank you guys so first of all here we're going to start in the opencv documentation again because this is act like just one method or like one function that we're going to apply and then we're actually just going to draw or like plot what we actually get as the output from this function here so this function here it computes a dense optical flow using the conor fanobx algorithm as we can see here we can see all the different kind of input parameters it takes how we can set it up in python and we can also see like the data types and stuff like that that we need to pass into this function to be able to have it work so here we see that it takes an input array uh two input right here and also takes like this output here which is the actual flow so this will be the flow uh which will correspond to the change of motion in the image frames so we need to pass in the previous image frame and we need to pass in the next image frame to actually be able to see a change in motion from the one one form from the previous frame to the next frame that would load in from a webcam or like any other camera that you're using then we need some other different kind of parameters that we need to set as well we're just going to play set them as default um in this video here but you can play around with them if you want to change some of them for your own applications or you won't get like uh you're not getting like that good results you can play around with them as well but down here in the description we can read what these parameters actually does so we need to make sure that our images that we load into this functional like pass into this function is 8-bit single-channel input images so we need to first of all we need to convert our images to grayscale images before we pass them to this function because it only works on 8-bit single channel input images and then we're going to do the exact same thing for our next image so we're going to pass in two images and then we'll get back a flow which we can see here which is a computer computer flow image that has the same size as the previous and type so we're going to have this um 32-bit floating point floating point data type here with two channels and then we can actually go in and draw this flow here so we'll get the errors and so we can see how much is the is the motion uh how much motion is actually changing from one frame to the other so here we have these different kind of ramblers you can just go and read through them if you want to we're just going to use them um in this video here but we can see we have this pure scale where we have this parameter specifying the image scale below one to build pyramids for each image so we also talked about that um in the previous videos these pyramids how we actually find these um optical flows so we have appeal scale here of 0.5 it means um that a classical pyramid here uh where each next layer is twice smaller than the previous ones we also have some levels window sizes so like everything window size we want to look at when we actually like sliding through our our image and we want to to capture that motion in our image or like from the one frame to the other frame we also have a number of iterations here that the algorithm does at each pyramid level we have some poly end poly sigma and stuff like that and we also have some optional flags down here that we can set if we want to use like initial flow or like fauna back excausion we can read what all these things here does we can even go in and find the algorithm how to implement it but in this video we're just going to use some default parameters use this method here and see how we can actually set it up in code again you can download the code on my github i'll link to a down description here you can just go into github take the code copy paste it run it on your own computer and just play around with the parameters you only need the python script and just run it and everything will work so let's now jump into the code here and see how we can actually set these things up and how we can use this function that i just showed you in the opencv documentation first of all again we're just importing our modules numpy opencv and time so we can see how many frames per second do we actually get when we're running this algorithm here and creating this dense optical flow where we're looking at like how motion is changing um in our images or like how the environment is changing in our image frame so first of all here we're going to have a function and we won't go into details with these functions so we're going to have a function to draw the actual flow where we just pass in the flow and we know that we get back at 32-bit floating point number with two channels which will correspond to the flow so this will be the errors that we're actually going to draw and see and when we're going to run the program at the end of the video so here we're actually just going to draw the lines and the arrows and then we're going to use these polylines as we did in the last video where we're just tracking around objects tracks like be able to draw lines and from our from our trajectories in the last video where we tracked around different kind of objects we're going to do the same thing in this video here but we're going to do it for the whole image instead of only four specific points so basically the idea is the same here we're just converting to the correct colors and then we're just drawing these lines and the circles for these points that we actually detect and we get from this flow function later on so this is just how we can draw the flow and also down here we're going to draw the hsv so we're going to draw a color image besides it so we can actually see the whole image or like what is actually changing in the image from the one frame to the other frame and then we're going to have another frame where we have these errors so we can actually see how much not not even like what is changing in the image frame but how much is it changing in the image frame from the previous frames to the next frame so we won't go into details with these we're just passing the flow we correct it to the correct color and then we just convert it back and forth between hsv bdr and rgb and up here we're drawing our actual like errors on our on our image so down here we're going to open up our webcam so we have this cap here where we just open up our video capture at the zeroth element so this is my webcam and then we're just going to read in the first frame here from the webcam before we're going into our while loop because this will be our previous frame so we first load in a frame that will be the previous frame and then we will load in a new one so we have two frames and then we can look at how the motion or like how the environment is actually changing from the from the previous range to the next frame and that is those two frames that we pass into our function we'll get the output which is the flow and then we just draw the flow with the functions that i just showed you up above so here we're just going to convert our colors first of all from pgr to grayscale because we know that we only need like it can only take 8 bit single channel images so it's grayscale images where we have values between 0 and 255 for the pixel intensities so down here we're just going into a while loop so while true here we're just going to run a program until we hit q and if we hit q at any time in this while up here we'll just terminate the program release our webcam and we'll destroy all the windows that we have opened up with omcb so now we're going to read in the new images and then later on we're actually going to when we read in this image we're going to set it set that equal to the previous frame later on but now we're going to read in a new image and then we're going to convert that to grayscale as well and then we're just going to start a timer so we can act like time how long does this algorithm actually take to run so we can see the number of frames per second and that we get by creating this dense optical flow and tracking around everything that is moving in the image frame so down here we're just going to use this function that i showed you in the documentation and what this video is all about so we have a flow here which is equal to cv2 dot calc optical flow with this far enough far and back algorithm we just pass in the previous frame um the new frame and then we have these parameters here which is just default you can go into the documentation as i just showed you so these are just the parameters that you can set play around with them we had this 0.5 here which they just showed to show that's the default in the documentation we also have the windows sizes and and so on that we want to look at play around with these parameters if you want to change it for your own application or project but this works like in in most cases or like in general because these are just default parameters and then when we've done that we can actually actually have the hole here so we have a whole flow so we have the hole a dense optical flow actually like stored in this variable then we can just set our new frame equal to the previous grade frame so we just keep updating every time we have calculated our flow we will then update our previous frame to the frame that we just calculate the flow for so just keep updating it so we get the newest frame and then that frame will be the previous frame so we're looking frame by frame how the motion is actually changing um from those frames so we're going to end the timer here we're just going to calculate the number of frames per second so it's just one divided by the end time uh subtracted by the start time we're just going to print out the number frames per seconds just for so we can see like how fast it runs then we're just going to call this in show so we're going to show the flow here the actual flow where we can see the errors and then we're just going to use this drawer flow function where we just pass in our grayscale image so the image where we want to actually like draw and draw our image and then we flip the flow here that we want to draw on top of that image so this will be the errors and the polylines as you specified up in that function from ohms v then we also have this flow hsv here so we actually like have an image where everything is black if there's no motion tracked or like it in the image from one frame to the other and if there's actually like some some differences we're going to look at like in what direction is that motion and then it will have a corresponding color regarding or like for that direction of motion and also how much it is actually changing in the image frame um so if we had q here we will terminate we will just break will terminate release a webcam and destroy all the windows so everything we have to do is actually just have these functions that is drawing what we wanted to under on the actual images so we can see what is going on so these will be the errors and down here we're just going to have this 80s wii so we can see like how the motion is is actually changing from one frame to the other and we're also going to have um hsb colors for the directions of our optical flow so this is everything we need and then we just call this function from opencv it will run the whole algorithm and it will create this dense optical flow that we can then use so this is everything and we can now run the program and see the results so down here it will just load in everything we can see we get around like 40 30 40 fps and then we get the images up here at the bottom so over here to the left we can see the image with the with the with the dots here and then if it if it tracks a motion in the image frame we can actually see we will get these errors and depending on how much we're changing uh from one frame to the other frame the magnitude of our error is actually changing too so we see here if i'm starting to move faster the errors will actually become bigger or have a greater magnitude so we can see the faster you have you're moving uh the greater the magnitude of the errors and if if there's only like small changes we can see the errors is also smaller and we can also see the direction of the errors here which will correspond to the direction that the motion is actually detected or like the optical flow is detected in the image frame and it will do this for all the pixels in our actual image frame so this is what dense optical flow is in the last video we just tracked different kind of corners in an image frame and then we will act like dragging those corners around storing the values and then we didn't do anything else with that but in this video here we have this dense optical flow so we can track motion in the whole image at the same time and over here to the right we have this flow hsv so we can see here if there's no motion in the image everything is just black we can only see my mouth here now is moving so if i'm moving to the right here in in the real world we can see that the color actually changes to red and then when i go into the other direction it actually changes to blue so we go red we go blue and then if we go up we start to get green and if we go down we get like purple purple or blue or like darker blue so we can actually see that these colors here alter the text in what direction are we actually like moving from one frame to the other so if you're moving around here we see like all the different kind of things how is it moving and what direction is moving how fast this is moving so we can see that the color is actually brighter the faster we move as well so that will correspond to the magnitudes of our errors in the left image so this is actually really nice and really cool and we can use it for a lot of different kind of things as i said in the last video where we talked about like if we for example having camera camera on top of like a highway and we want to track cars cars around or in the case here with dense optical flow if you for example had some fluid or like some objects moving around in the image frame and you want to have the whole image frame and you want to take motion in the whole image frame but not only motion but also how much or like the magnitude of the motion changing from one frame to the other frame so it can be used for a lot of different kind of useful uh real life applications so i just think that this is really nice and it's it's actually really cool and we're definitely going to create more videos on channel here and create some projects where we actually using these different kind of things for um for some really nice projects because again these are just two functions to actually draw these things and then we just call one method or like one function from omcd that does everything for us we just need to pass in two images and then we get the flow out here by running this algorithm uh where we get up to like 30 40 frames per second on a standard uh cpu we're maybe also going to do it here on the tpu in the cooler cuda omc tutorial where we're using the gpu to speed up some of the processes here because that is way more efficient so thank you guys for watching this video here and rented subscribe button notification under the video and also like this video here if you like the content and more in the future i'm currently doing this computer racing tutorial where we're talking about like basic email operations how we can calibrate cameras use their revision to estimate depth in images now we're starting to talk about like point clouds how we can use point clouds in computer vision and also optical flow so now we're talking more about real life applications how we can act like uh see motion in images but also 3d information about like in our environment how we can get that from only using cameras and stuff like that so i'll link to that tutorial up here or else i'll just see the next video guys bye for now

Channel: Nicolai Nielsen

Views: 21,582

Keywords: opencv python, python opencv, opencv, optical flow opencv, optical flow object tracking, object tracking opencv, opencv trajectories, opencv optical flow, how to track objects in opencv, opencv object tracking python, object tracking python, opencv tutorial, the coding lib, python, opencv for beginners, computer vision, trajectories python, track objects in python, opencv track object movement, opencv object tracking, opencv object trajectories, optical flow computer vision

Id: WrlH5hHv0gE

Length: 15min 4sec (904 seconds)

Published: Wed Sep 29 2021