Build a computer vision timer tracker | Opencv with Python tutorial

welcome to this new video I'm Serge of a computer consultant developer and course instructor I build completion solutions to help companies improve their process efficiency reliability and scalability today we will be building from scratch a fun project to track how long we are in front of a computer or a monitor and the idea of this project was born from some researchers that I've been reading about how the eyes are affected when we are a lot of hours in front of the computer specifically there is what is called competition syndrome or digital eye strain and it's it's proved that after certain hours like the eyes are affected by the screen and this should be of course common knowledge and I will not get into the details of the research because that's not my field but I want to build this one project where we can track with the camera automatically how uh how long we are in front of the computer so the camera keeps checking if there is someone while you are doing our work or watching some videos or doing anything and after a certain time we get an alert telling us you have been in front of the computer for Thought time there are already apps that do this I use some of them that do this not without checking like if there is a face in front of the computer but they do this just checking if we are using the keyboard the mouse and so on today we will do something more advanced checking even if there is the phase these are fun project so uh it's we'll do everything from scratch so even if you are a complete beginner in Python you can follow this because everything will be done uh from scratch or let's now get right away into building this code so I'm going to create a python code this is a python file that will write from scratch I'm going to use two libraries media pipe and opencv python to install this to install these libraries you can use the common prompt uh peep install openc Dash python to install opencv peep install media pipe install the media pipe Library if you have problem with python installation of Library there is a lot of material online to understand how to install python like how to use I'm using pycharm for the for the idle there are a lot of like editor where you can type your code so we will skip all these steps let's now get right away into the code first of all we want to use opencv to extract the frames from the camera in real time before performing our detection so let's import CV2 which is the opencv library now what do we have to do we have to take frames in real time from the camera so for this we get a capture object so let me put some commentary frame from camera capture object cap equal cv2.video capture zero so this is a capture object zero is the index of the camera zero means let's take the first webcam if you have more cameras if you put one it will load a second to the third and so on now let's test this quickly so if we want to make sure that this is working red underscore frame equals cap dot read this is so once we choose the camera with this cup we want to take a frame from the camera so this is taking a picture from the camera so take frame from camera red is telling is false we are not getting any frame it can be if the camera is disconnected if it cannot the camera if the camera is already already occupied like summary reasons it's true or false frame if we're able to get the frame it's the frame that we can display so let's Now display this cv2.m show the window we can call the window as we wish let's say frame and then what do we want to display we want to display this Frame now we want okay let me put a comment display frame uh well after we display the frame we need a weight key event k equals cv2. wait key zero this is the concept is very simple when we execute the python code is executing line by line after we show the frame the code finishes so it shows the frame and the code closes itself so let me run this quickly so that it's more clear so we are you see like there was a frame for a moment and it closes so we need a weight key event that keeps everything open on hold until we press OK that's why very simple it's called weight key white key zero is freezing the frame so let's now run this one I'm using uh this is like the the webcam that from my laptop so we're taking the frame that's it like very basic we are taking the frame nothing else now consider we want this to be in real time we need to take the frame in a loop so we took a picture we don't want a picture we want this in real time so what do we do we take the frames in a loop while true and we put everything inside a loop so if I run this again now it's showing what frame because it's freezing on the line 12 cv2. weight key zero if I press a key it's taking another frame if I press a key it's taking another frame so each time I press a key so you see I press a key it takes a new frame we want this in real time I don't want to press the cage time so I will say wait key one so with k0 is freezing a frame width key one is waiting one millisecond and going to the next so it's not freezing anymore and that's how we've opencv very quickly and easily we get the frame real time so this is directly from python we're getting the frame in real time or directly from the python code now with this that we have right here we can do whatever we want we can draw something on the screen we can perform detection uh we can do any image filtering operation we can do anything we want now let's stick with our project once we we want to do with our project is to detect the face if there is the face in front of the screen then we can count the time so that we can count how long we're in front of the screen if it is not phrase no on a face we don't do anything or we will see later if we want to reset the timing and so on we will use phase detection so this basic phase detection is there are different libraries that can perform face detection um it's phase detection it means that not that it's facial recognition telling who you are and so on face detection would be simply surrounding the face with a bounding box or even more basic telling if there is a phase or not a phase we have the lib library for example for python or a newer one is Media pipe which is very fast very good so we'll use media pipe import media pipe SMP this is the library that will perform face detection so let's load phase detector face detection equals MP dot Solutions Solutions dot face detection dot phase detection well this is like so quickly so easy with this one you we can detect if there are faces on the screen or not very very easy where do we detect and how we detect if there are faces even this is simple we are taking a frame so keep in mind that when we work from cameras or videos with computer vision we are working with single images one after the other this makes the concept very simple because we check the image is there anything I mean we process the image then we go to the next image because a video is nothing more than frame after frame usually we have 30 frames in a second or 60 frames or more if you have very Advanced uh camera so let's now usually webcams have like 30 frames or 60 frames like not more than that let's write here perform space detection so first detection will be a result dot results equals phase detection dot process and we want to analyze the frame okay one very uh short technical thing is opencv uses the BGR color format so blue green and red while standard anywhere we see RGB so the red green and blue so before processing the image with media pipe which wants to want the frame in digital format we need to convert the format RGB frame equals cv2.cvt color we want to convert frame and then cv2.color underscore BG R2 RGB so this is the function to convert the bigger format to RGB format so that we can process this one with media pipe now the results is going to give us faces if there are faces otherwise it's going to give us none so let's now print results so that we have an idea of what we are getting from this so let me go let me take out the face okay um it's my mistake I need to uh we need a results uh is an object which contains inside information so we need to enter inside the results to get that uh so we need to accept results dot detection and let's now print this one the detections results dot detections plural let's print this one and we'll see a lot of output right here let's now go out so you see if I move my face outside we see known if my face is inside uh we see the output let's analyze quickly this the output that we see right here is the position of our face on the screen where we have the X position the Y position so the top left point right here then we have the width of our face and the height of our face so that we can draw around that rectangle we call draw a rectangle now I will not do that we don't really need to draw the rectangle for this purpose even though later we might do that let's see if the tutorial doesn't take too long what we want to know if there is Phase or if there is no face like that's what we want to know so uh is the face on is the face or detected if if results dot detection then print face on the screen or like face let's say looking at the screen else face looking at the screen else print North Face so the idea is that we check always in real time if the face is looking at the screen we do something if the face is not looking at the screen it means that we took a break so ideally our eyes are not looking at the screen but they're looking somewhere else so let's run this one just to make sure that everything is working correctly or detection again so if results dot detections so this is plural results Dot detections okay face looking at the screen let me cover this one no face face looking at the screen of course I'm cheating right now because I'm covering this so Facebook at the screen no face we have like the major part of our app already built because this the computation task is understanding if we are in front of the computer or if we are not in front of the computer also I can I'm not able to close this if I press any key so let's also add a key event because we are showing the frames in real time but we are not able to if we press a key nothing happens so let's add some event if key equals to 27 27 is the ask on key on the keyboard we want to quit Also let's complete these ones we quit cab dot release this simply saying uh let's release the camera so if you need to use the the camera with other software like Zoom um Skype or any other things the camera is not occupied by the python code so Capital release is for that reason cv2. destroy or Windows to close like any opencv windows that might be open so results dot detection phase looking at the screen now face let's now add what it's uh our goal the time tracker so track time how can we tag the track the time we need to we need to understand when we are starting so starting time and then we will check elapsed time um at each moment so let's do that for starting time we have a Time library in Python so important time uh time Dot time so this is the starting time and we can check the elapsed time that we are in front of the screen so let's say face uh we are in let me think we have the starting time we need to check the elapsed time so um elapsed time equals starting time minus oh sorry uh the time now so time dot time minus starting time and we can print elapsed let's let's show last time Dot uh format elapsed time and let's run this one so we see a lot of time two seconds three right here now right here we see 6.0921 and so on these are seconds we have like the like we have a huge approximation that we absolutely don't need we can just stick with the seconds will be more than enough so we can say we can want we want an integer of this so we have a long float number let's use only one two three not one point zero seven four and so on there is a type of ercv2. destroy all windows okay so we have now the elapsed time I want to make this graphically uh just a little bit more nice so let's put uh I want to to have a screen where we can see this so let's um let's do it this way so I want to show the elapsed time in real time on the screen instead of printing it with python so let's Now display the elapsed time on the screen so right here after lapse time we can say seaweed soup so draw a labs time on screen cv2. put text this is the openc function to draw the text where do we want to do the text on the frame what do we want to display on the text we want to display let's display just the counter of the time so this is a DOT format elapsed time so this one let's say seconds and then uh where do we want to display the text we need to give an x and y coordinate let's display this at the top left so let's say it starts from 0 0 and for the Y so for the X we need to go like this for the one we go down so we go to the right and we go down so let's say 10 pixels to the right and then 50 pixels down now all what font do we want cv2.font hair shape plane or we have a few fonts on opencv uh to be honest I don't like any of them there are no fancy fonts so let's just use some I'm taking just a random font that I know of the size of the text let's say two or let's say three let's make it very big three the color let's make it yellow for example so to make a color we need to give like three values BGR blue green and red from zero where is the absence of the color to 2055 that is the maximum of that color let's say 15 of blue 225 of green 2015 of red thickness of the text let's say two pixels let's now run this one I want to make sure that everything is working correctly it's not font hair shape plain so there is a typo on the font line 28. font hair shape plane a one second to Second so we have the timer in real time that it's running right here timer so the timer is working we can make this look more nice what we could do for example is I I could put a square below the text so that it's more visible so I will quickly do this change so let's also draw a square so I'll draw a rectangle draw a rectangle see to that rectangle I will make this uh let's put the rectangle on the frame to do rectangle we need two points top left zero zero two let's cover all the width of the image so let's take the width of the image um height with channels equals eframe dot shape so this is how we get the information from the image so from the width to let's say around 70 pixels I will put this just a bit below the text so from from 0 0 to the full width and 70 pixels down the color of this rectangle let's say full black or it could be yeah let's say almost completely black let's say 10 10 10. minus 1 to make to fill the rectangle with the color let's draw this I mean let's run this and see if everything is working correctly it's working correctly that's exactly what I wanted now it's the key of our program let's say that we want an alert when we reach one hour um so we should understand and check in real time when we reach this hour so it will be if elapsed time is greater than how long is one hour so if the elapse time is greater than 3600 seconds then in this case we could display an alert oh reach it maximum time um show alert oh now for testing purposes 3006 on the second is too much let's say maybe 15 seconds so that we can test list this one so if elapsed time is greater than 15 we what can be the alert so I I thought about two types of the alert the first one could be let's surround all the all the image we've read bounding box so this would be an alert so we can do a red rectangle CV2 dot rectangle rectangle so let me copy this so we draw the rectangle from 0 0 to the end of the image from height instead of -1 which will fill this with the color let's say let's say for example 10 and let's make this red so 0 of blue zero of green 2055 of red we're doing a rectangle that is surrounding exactly the entire frame we're doing that with red color let's say it will be 25 it's more nice I don't like the full red 2025 and 10 pixels is the the thickness of the Border so let's now test this one so I'm now going to wait for that this switch is 15 at 50 seconds we should see a red rectangle uh surrounding right here so no 15 16 okay that's the alert that I wanted now this is not enough as an alert because let's say that you are using the computer you are doing something else you could put an audio alert which or it will just you're doing something else there will be an audio I don't want to do that I want to be something more fun let's say that the window will pop up and you will not able to put it down so that's the alert that I like so we can put this property let me do that with opencv uh cv2. set window property let me double check to make sure that this function is correct so set Windows proper wind of property uh I want to put this one where on the frame so I want to set which window frames so I want to put change the property of this one and I want to put this window on top all the time uh c v Sue Dot W and D prop top most underscore comma one so that is that whatever we are doing the program is open running in the background when it reaches like the the threshold time of 50 seconds the window is going to pop up in front of us and I'm going to test now this one so I'm running this current file let's let's suppose that we are doing uh we are working we are doing a work project so this is me doing something else it can be any or any program could be opened let's say I'm opening the calculator and now you see this window pops up in front and it doesn't matter what I do with this window will not go down so this is an alert and it's good because it's disturbing so it's forcing you somehow to stop the working now it should probably close yeah it closes but uh if you want to be even more mean we we could do that it will not close even if you press like the X but we will not do that now but it's possible like to to just keep it on um all the time so uh to this one at this moment I will just add one thing I mean a couple of things let's do this so the track time let's say that here we have the the settings of our program let's say uh settings settings at maximum time maximum time equal let's say 15 seconds [Music] um so that we can change the value right at the beginning of the code now what else can we make to improve this program this is not enough right now because we want also the program to understand when we are not looking at the screen otherwise there is no point if it's just checking and adding this all the time so if I move so let's say that this program is running uh we leave it doesn't show this but when we come back it's still showing the same time ideally when we leave the computer when we go somewhere and come back we should see our res reset of the counter so I will make this now very simple because this is only an exercise so if so if there is no phase we can reset so if OK let us keep the printing of face let's reset the counter if there is no phase we can reset the counter by resetting the starting time as simple as that so when we leave the counter will be will will be initialized again and will start from scratch and this is of course very basic this is an exercise it's not the ideal that if you live for one second you you will restart the counter you could put a function for example uh that will check if you are a minimum of let's say 10 minutes away we will start the counter otherwise it will not restart the counter so it can like if you do this program properly there are many things that you need to take into consideration it's not just an if statement if you are far reset no so no let's just check that this is working so I leave I come back and it starts from zero seconds this is a very very basic program that we just created all the code that it's here it's available to download down below for free so I recommend that you follow everything from scratch and then you type this from scratch because that's how you learn the most but you can also check the blog post you will see the read article about this video plus you can uh download the code as I said this was only a fun project just just for for you to play around with computer vision to see how and what is possible to do with computer vision if you want to know more about this and if you're interested into getting more involved and learning about detecting objects more advanced stuff for competition I have a blueprint it's a crash course that you can check it's one hour where I teach you like I show you like the main path to learn computer vision to detect and track any object to use that on a more let's say on a more professional level and Industrial environment if you are interested in other Solutions you need solutions for companies we can help you do that we offer consulting services and we develop software on demand uh I will be releasing soon new videos about new fun project and also industrial projects if you have any questions put them down in the comment below this is all for this video and see you in the next one

Channel: Pysource

Views: 5,120

Keywords:

Id: 7BQcpXTcyWI

Length: 31min 35sec (1895 seconds)

Published: Tue Mar 14 2023