Object Detection Raspberry Pi using OpenCV Python | 2020

hey everyone welcome to my channel in this video we will run object detection on raspberry pi 4. now we will also create a function that will allow us to get information of specific objects as well so let's get started so the code that we will be using is from a previous video that we did on object detection and we use the mobilenet ssd to detect different objects so some of you have been asking where can i access your code i am unable to access the code i have been receiving a lot of these uh comments so let me show you how you can access the code of any of these tutorials so you will go to start learning or you will go to learn once you click on that by the way this is my website you can go to murtazahasan.com and then you will select which of the projects that you want to access so here we have the object detector in the opencv project so we will click on that and uh if you haven't registered on my website then it will say enroll and it's free to do so you can put in your username and password and then you can just start the course so once you do that we will go to the object detection and in the object detection we are going to go to code and file so over here you can see that we have the code so we can copy this so this is the code without the nms which was basically non maximum suppression to remove the duplicates but i will show you an easier way to how to actually implement that as well so here we are in the same environment that we used in the previous video so what i will do is i will create a new file and i will call this object detector so if you have been following my tutorials on raspberry pi i'm always trying to make modular code so that you can remove it and you can add it very easily so i will use the same concept here and the concept is that we are going to write the complete code on a different file and then we will create a main file through which we will access it so this main file is basically from the one that we did before and we are not going to use this so we will just paste the code that we just copied and here we are going to make sure that first we have all the files so we have ssd mobile let's dot pb txt and then we have the frozen inference graph dot pb and then we have the coco names so these are the three files that will be required and they should be in the same directory as your module let's go through the code that we have done before so that everyone is up to speed so the first thing is we will import our library which is opencv and then we have the threshold parameter this is the confidence level so if the object detected is has this much confidence that it is an object then it will be used otherwise it will be rejected then we have our webcam object here we are creating our webcam now the thing is that i'm using one because i have multiple webcams you should be using zero if you have only one and then we have the size do we need the size yes okay let's change this to 640 and we will change this to 480 this is the height and for the brightness we are going to remove that then we have the class names in which we have all the classes so we have it in the coco names file so if i open that you will see all these files all these class names that we have in this file so we are extracting that using this method and then we have the path to this these files now you might be wondering why am i showing all of the code in our computer on pycharm why are we not doing it on our raspberry pi now the thing is that whenever i'm writing the code i'm first writing the modules in what you call pycharm and then once it's done i just take it to raspberry pi and run it over there so we have to do minimal changes in raspberry pi because it's always a little bit harder to work with raspberry pi rather than pycharm so when we go to raspberry pi we will have to change the path to this and it has to be the complete path even if it's in the same folder we will have to give the complete extension so that should not be hard and then we will just pass it to our dnn detection model and that should be good and then we have some parameters that we are using and here we are starting our while loop and inside that we are first reading our image and then we send this image to our uh detect method uh in our net so this will give us the confidence values the bounding box and the class ids so we can print that out um actually let's just remove that and then we are using uh this information to actually display a rectangle around it and then we are putting the text uh in terms of name and then we are putting the text in terms of confidence values so this is pretty much our code and at the end we are just displaying it and we are giving a delay of one milliseconds so this is basically our code that we have written before so now we have to make this a module now to do that the first thing we will do is we will go down and we will write that if now we want it to run by itself and we want other uh what you call scripts to be able to access this so therefore we have to know if we are using this or not so we will write that if we are using this file so the name of this file is basically main then we are going to run a sample so that sample will be that we will first create our what you call capture object and we will cancel this here and we will paste it here and then we are going to say that we want to run the while loop so while while true we are going to get our image so let me remove the while here and then i will take this and i will put it down here and once we have our image we are going to send it to our function and here we will declare our function so we will call it let's say get objects and then we have to have the input parameter as the image and over here we are going to write that let's say our result is equals to uh get objects and we will send in the image and once that is done we are going to display this and we will cut it here and we will paste it here and at the end of the day we just want to return our image so here we will say return and we will say image so this pretty much should make it a module so that we can access it from other scripts as well so yeah let's put this down as well and let's bring it here in case we need it so i think we should run it at this point to see if everything is good let's run it and yeah that works so here we have our cup we have a mouse we have a keyboard we have the remote but again you can see that it is overlapping it's giving multiple boxes especially in this case you can see mouse and mouse so we need to remove these multiple cases now i did a video on that and we use the method nms which is non-maximum suppression now the thing uh is that the net.detect method actually comes with this functionality the reason i went ahead and did it manually was because if you are creating your own uh method of object detection so that you can understand how actually the nms works and if you want to implement it in your own method then you can use it too but if we are using just this dotnet dot detect method then we can simply write here nms threshold oops uh nms threshold and we can write for example 0.2 so the lower the value the stronger it will have an effect so let's run that and see so you don't have to change anything else you just write the threshold value and that's it so here you can see now the mouse is not being detected twice and we have the cup we have the mouse we have the remote uh being detected as well so that seems good and i think that should be enough but now the thing is that we are creating it as a modular function and sometimes we don't need all the capabilities or sometimes we need to block a few things for example if we want to reduce or if we want to increase our frame rate we could just say that we don't want to display anything right so that will help us speed up the process a little bit so we can write here that whether we want to display it or not so we can write here draw is equals to true so by default it's going to draw but if we write false it will not draw so how can we add that functionality so what we can do is we can simply go into our for loop here and we can say here that if draw and then we will do all of this okay and there you go and if we run this now oh then it is drawing and if we go down and if we write here false then it will not draw anything so it's not drawing now okay so that is good now you might be wondering why didn't i put this above the for loop why do we have to loop now the thing is we want to add another functionality and that is that we want to send the information of the bounding box and the class names as well because when you are actually running some sort of algorithm you want the actual information you just don't want to display it right so what we will do is we will return our bounding box and our class name as well so here we are going to declare a new list we will call this object info and we want to return this object info okay and there you go so here we are going to write that object info is equals to not equals dot append and we want to append uh two things one is the bounding box and the other is the class name so let's say we will do the box first so we can simply append this box we will write here box and then the second thing we want to append is our what happened there the second thing we want to append is our class name so for the class name basically if you can see this is the code that we wrote earlier so this is basically the class name so we will copy this we don't want to we don't want to uh okay so we will copy this and we are going to create a variable out of this so we will write here that this uh oh actually this is supposed to be outside okay let me just write it here and then we can do that so we will write here that our class name class name is equals to class names class id minus one so this will give us the name and then we can write here class name and then we can just simply put it here okay so as i was saying before that this is supposed to be outside the if draw so it should be here so whether we are drawing or not it doesn't matter we should send our object info and we should receive it here so we can write here objects info let's make this lowercase okay so object info and that should be good so if we print this out let's try it so let's print objects info so if i run this now it should not display uh the bounding boxes but it should display the object wait what happened oh i forgot to put the name over here where is it so here we have to write class name class name dot upper okay and let's run that and there you go so now you can see that it's giving us that it's detecting a mouse and these are the coordinates for that or the bounding box and then if we detect something else uh there you go now we are detecting the remote as you can see i'm on the remote and if i go to mouse it will give me mouse and then if i'm on the cup it will give me cup okay so that is good now what we can do next is what else can we do okay so one thing is that sometimes we don't want to detect everything right now the model is trained on everything so it is going to give you the output of all these uh different objects but what we can do is we can send out the value of the object information with just whatever we need so for example if i just want to detect cups i can just write here cups and it will only detect cups right so that will be very helpful if you're trying to detect uh one um one thing or a couple of things rather than everything that it was trained on so what we can do to actually achieve that is that we can create uh let's say objects here we will create a list of objects and we will say that whatever objects you need you have to write them down here but what if the user does not write anything inside of it so that will be an empty list and if that is the case then we want to display everything right so we will say that if our objects actually the length of our objects is uh equals to 0 which means the user did not define anything then in that case we will say that our objects our objects is equals to class names class names so all of the classes right so we want to detect all the classes and then what we will do is over here in the for loop we are going to write another statement and we are going to say that should we write it down here or yeah after the class name because we need the class name so we will say that if our classname name is in our objects now this object is basically given by the user okay and if it's not given by the user then it's all of these class names so here we will write that if the class name is in our objects then we are going to perform all of these actions otherwise we will not do that so i think that should be enough so if i put here nothing right so we don't have anything let me put this through or let's just remove it it will be true automatically and if we run that you can see it is detecting everything it's starting taking the keyboard cupboard uh not the cupboard the keyboard and the cup and the mouse and the remote the tv so it's detecting everything now what watch what happens when i put let's say a cup so i will put here cup uh this is not the second parameter this is okay we'll have to write true or we will write have to write objects so object is equals to is it object or objects it's s objects is equals to we have to write let's say cup and let's say okay let's just do cup first and then we can add to it so now it should only detect cup so if i put it on remote or anywhere else it's not going to detect if i put it on a cup it will detect only the cup and what if i want to add multiple then i can write another one here so i can write mouse and if i run this then here we have the mouse here we have the cup so it's not detecting the remote or it's not detecting the keyboard but it is detecting the mouse and the cup so whatever many objects you want you can write them here now let's just comment this out and okay so when it's zero it's not the most efficient way to do this but it's not very computationally heavy because we are just detecting a few objects maybe five six seven ten something like that on each iteration so it's not that computationally heavy so i'm going to let it slide but you can try other methods as well okay so this is what we have as our what you call object detection module and i think that should be enough to get us started is there anything that i am missing i i don't think so okay so what i will do next is i will copy all of this code and i will take it to our uh what do you call raspberry pi now before i do that let me show you if you want to run it from another module how do you do that so if i go to object detector and i go to new i will go to python file and let's say i will say uh main module okay let me write the m capital main module okay so here we are going to write uh let's say what is the name of this object detector module so we are going to write uh from imports sorry from objects detector module imports everything okay that is what we will write and then we just have to copy the code over here which says if name we will copy all of this and then we will paste it here okay so now this should be back and this should be back as well okay and this get objects will be already detected because we have imported everything from here so if i run this now it should be exactly the same oh yeah so there you have it so it will detect mouse and cup and it will not detect anything else now again if i wanted to i could remove all of this and i can i can write it like this and then i can run this and there you go so now it will detect everything okay so that is enough now if you wanted you could add more parameters like you could uh define the threshold value as well so here you can add the threshold value and you can add the nms as well so should we add okay let's add this so we will add the threshold so we will put this here first thing is the image then we have the threshold and then we have the objects and everything so the nms i'm not going to add if you want you can add nms here as well or okay let's just add it now if anyone wants to use it so we can add it here so here we will write nms an ms and over here we are going to write 0.45 and then it was 0.5 i think no 0.2 so that should work fine so let's run that no it's running the main module okay we can just copy this and paste it in the main module over here and let's run this and there we have it so it's detecting cup and mouse exactly the same as before now if i wanted to remove nms i would just write one over here and you will see now we will have a lot of duplicates so there you go now you can see we have multiple detections for mouse and the cup is fine but you can see multiple detections for the mouse okay so i will put it to 0.2 and 0.45 these are the values that i found uh work uh pretty well so we will keep it like this and i think now we are pretty much done and we can go on to our raspberry pi and we can run it over there okay so here we are in the raspberry pi and i have some good news and i have some bad news now the bad news is that there is no good news uh actually that's not true the thing is that i was able to run it in raspberry pi but the bad thing is that it took a bit longer than expected so the problem was that the method we are using a detection model that is not available in 4.1 and 4.2 version of opencv so i think it's available in 4.3 and onwards so i had to go back and install opencv 4.4 to actually make it run so if you haven't installed it before then you have to do the whole thing if you have done it before then you have to do half of the steps again so how can you do that you can go to my website mutahasan.com and then you can go to the courses and in that you have the raspberry pi ultimate robot over here you can go and start the course now if you haven't enrolled again you can enroll and then you can start and then over here we have the opencv installation opencv installation on raspberry pi and if we go down and here we have all of the steps that we need to install so you can skip till step eight and you can start from step nine if you have already installed it before so here instead of 4.1 you have to replace all of these with 4.4 so all of these so one is here the other one is here and you have another one here and the last one is over here and this will takes a while it will not happen within 10-15 minutes it is going to take a while so you have to be patient what is this this i think is an html error this is supposed to be and and and that's it it's not supposed to be amp caller i will fix this later on anyways once you do that then again this problem is here as well yeah i think this is the html problem okay i will fix this and once you have done that you can go to your terminal now there is a complete video on this if you haven't seen that you can check this video out in which i have done it step by step it's very simple all you have to do is copy paste but make sure you are changing it to 4.4 in fact what i will do is i will change this to 4.4 here so that everyone can install and then once you have done that then you will go to your terminal and in the terminal you will write python3 python3 and then you will import cv2 import cv2 and then we will write cv2 dot underscore underscore version underscore underscore and that gives us 4.4 so this is the version that i've installed if it's lower than this i think 4.3 will work as well i haven't tried it but i think it will work but i think lower than that will not work so make sure you have 4.3 or 4.4 so once you have done that then we can finally go on to our code so what i've done is i have created a folder on the desktop and it says object detection uh what do you call so it says object detection mobile net ssd and inside that i have all the files required so as you can see we have the weights file and the configuration file and then we have the cocoa names and this is our module so what we can do is we can go to our mu editor so it's in programming so we can click on that you can use any text editor you want but i prefer the mu editor so over here we are going to click on load and in the load we are going to go to computer actually you know we will go to pi and then we will go to desktop and then we will go to object detection and then object detector module we can close the other one okay so this is our code now as i have mentioned before you cannot directly run this what you have to do is you have to change the path and the path needs to be the whole thing so the whole path so we will double click this and over here we can see the complete path so we will copy this we will copy this and we will paste it here actually we'll start it from here and then uh slash again then again here and then again over here okay so that is good and i am again using two cameras i have the pi camera connected as well and i have the external camera connected as well i'm going to use the external camera for this purpose but you can use the regular cam or what you call the pi cam as well so do we need to change anything else okay let's remove our objects here so we want to display all of it so we will remove this and if i go back here i don't think we are missing anything so we can simply hit run and let's see what happens so here you can see that we have the output and it is quite slow but it is able to detect the bottle and the cup and the remote so if we add more stuff it will still be able to detect those as well but it is quite slow so let's try to remove some let's just detect the cup and see if that works so if i write here that we need objects is equals to let's say only the cup and let's run it again and see if it works and there you go so now it's only detecting cup it's not detecting the remote or the bottle so that seems to work fine and if we try with the bottle that should work as well so let's write here bottle and let's run it again and there we have it so now we can see that we are only detecting the bottle and we are not detecting the cup or the remote so it is quite slow and this is expected from raspberry pi so in the next video what we will do is we will try this on jetsan nano and compare how well does that perform so hopefully that will be much better than this and we will have a decent amount of frame rate that we will be able to use in different uh projects so this is it for today's video i hope you have learned something new uh for all the files and all the code you can visit my website and you can download it from there for free and if you like the video give it a thumbs up and if you haven't subscribed yet do subscribe and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 84,901

Keywords: Object Detection raspberry pi, object detection opencv, object detection raspberry pi opencv, object detection, object detector, raspberry pi opencv, raspberry pi 4, pi 4, rpi 4, rpi object, object detection opencv python, object detection python, fast object detection, opencv python, mobilenet ssd, opencv mobilenet ssd, ssd object detector, ssd mobilenet, deep learning, deep learning opencv, dnn, dnn cv2, cv2 object detection, computer vision object detection

Id: Vg9rrOFmwHo

Length: 31min 33sec (1893 seconds)

Published: Sat Sep 05 2020