AI on the Jetson Nano LESSON 54: Recognizing and Locating Objects of Interest in OpenCV

[Music] hello guys this is Palma quarter with top tech boy comm and we are here today with lesson number 54 in our incredible tutorial series where you are learning artificial intelligence on the Jetson nano I will need you to pour yourself a nice big mug of iced coffee I'm gonna need you to get out your Jetson nano gear and I'm gonna need you to get ready to learn some cool new stuff hey as always I want to give a shout out to you guys who are helping me out over at patreon it is your help and your encouragement that keeps this great content coming you guys that are not helping out yet look down in the description I have a link over to my patreon account think about hopping on over there and hooking a brother up but enough of this shameless self-promotion let's jump in and let's talk about what we are going to learn today I do believe last week in lesson number 53 I left you with a homework assignment I sort of showed you how we can use the Jetson inference libraries and utilities combined with OpenCV to sort of come up with some artificial intelligence programs that were the best of both worlds we love OpenCV because it makes it easy to grab an image from the camera gives us a lot of degrees of control over how we do that and also it's great for displaying the image because it gives us all types of hooks and control on how we want it to look so the grabbing the frame and the showing the frame is something that we do in OpenCV but the actual object detection the actual object recognition we do using the using the Nvidia Jetson inference libraries now the one kind of tough thing or the one little tricky thing about that is is they use different format so we grab it in OpenCV in BGR we convert it to CUDA RGBA float32 and then boom we are ready to go now what's your homework at Simon is I sort of showed you how to use the object detection libraries that Nvidia together and your I just sort of showed you how to kind of get some parameters off of that and your homework was to go in and look at a scene from a live camera and then box the things that are recognized and then label the boxes so were you guys successful in doing that I hope you were but for you lazy old guys out there that didn't do it never fear I will do it for you so let's call up our knives visuals to visual studio code and I had better get out of your way this looks like a pretty good view I do believe we are working in the Nvidia folder and we will add a fresh new program which we will call deep learning - seven dot P Y alright so we got a fresh new program and where we want to start we want to start with the code that we ended with in lesson number 53 and so those of you who didn't save your code and you know who you are you guys that did the last lesson but didn't save your code or lost it I don't make me call you out by name but you know who you are but never fear what I've done is you can go to the most excellent wwp tech boy comm and you can search on lesson number 53 and then if you come down here you can get the code that we left off within that lesson so just double chorus single click on the double page right mouse click and copy and then we will come back we will get this out of the way come back over to Visual Studio code and we will paste boom okay let's make sure that this program is still running right mouse click run Python file in terminal I'm running on the webcam just because it's on a wire and easier to move around but you know you guys know that you can do this program on the webcam or the Raspberry Pi camera because I kind of have both lines of code you just got to figure out which one to comment out okay making us wait a little while in eager anticipation okay there it goes and then we'll just kind of point it down here at some stuff and then we'll quit and then we'll come over here and look and see if it found anything yeah look it found a keyboard a keyboard and a person okay the two keyboards are very obvious and then even if it sees my hand it identifies me as a person okay like any part of a person counts as a person so that makes sense all right but now what were you supposed to do you were supposed to come in now in box those things well now to do this what you've got to see is it finds lots of objects and then it has kind of a list of arrays and you have to you have to step through and detections it's all the things that it found bundled together you have to step through all of those things one at a time so I stepped through detections in a for loop where I loop on detect so it gets the first thing detected then next time through the next thing detected the next thing detected okay and then for a given detect I can find the idea of it with ID is equal to detect class ID and then top left bottom right and item B and this was an easy homework I almost did it for you I did too much of it for you I should really make you guys do more on your own okay so now instead of printing these things that we found we will comment that out what do I want to do I want to put a box around it okay I want to put a box around it so I'm going to say see the - dot rectangle C v2 dot rectangle and what do I want it rectangle IMG remember in whoa that was crazy remember IMG is what we read in and then we convert it to frame to do the detection but now we want to go back and we want to work with our original IMG the purpose of frame was just to do the detections okay so we're gonna do it on image and then if you remember how this thing has this rectangle worked works it wants the kind of left top and then right bottom and it wants those two points as a tuple so I will say left which I already found top which I already found and then I'll scoot over and then I need to give it the right bottom coordinate okay and now I need to give it a color I like green today 0 comma 255 comma 255 all right ooh 0 255 0 all right and now we are going to make it a line width not inside the tuple let go outside and then comma 1 and that should give us the tuple and our that should give us what am I doing here yeah this is the color and then this is going to be the thickness of the line and so this should just box things if it works let's go ahead and run this and see if it works just to make sure that we didn't make a mistake in our first line of code this was too easy I've got to start making things harder whoa did that crash I can't believe that crash but maybe I didn't actually run it okay it's loading the library's so far so good so far so good whoa okay it tried to load and then it died okay an integer is required and got a top tuple okay that is pretty annoying okay on this left top those are the things that I put in that our variables and so what I'm gonna guess is is that those are not integers and so even though I asked for the top left right I wonder if I can go back and look at our earlier run we were printing out I thought I thought we were printing those things out ah look at that the location is not an integer and so we need to go in and make these units when we grab them so I'm going to say int make an integer out of no not out of detect that already was an integer but if we go to this detect top is a float so I need to make it an int by doing an int open and then close and then I might need to make this one an int open close bottom has got to be an int why would they give a fraction of a pixel is that not crazy you can't have a fraction of a pixel all right like that let's see this should make rectangles this should make rectangles this time I really think it's gonna work I mean how much can go wrong with one line of code police shouldn't ask I'm probably going to find out but let's see what happens here okay I've got some stuff going on it's at that slow Coco loaded step there got some stuff happening the old GStreamer warning what is this okay hey we got we got something to pop up here okay and then it died it still doesn't like this okay it still doesn't like this rectangle thing even after I fixed that did I fix all of them hey I didn't put in there I put what did I do I put the parentheses around it but it didn't put the int on there okay let's try it again I don't know how that happened maybe I am in fact going insane but you guys probably saw what happened but I don't know what happened okay things are going looking good you guys should have been able to do this this was really an easy homework but we'll go ahead we'll do a little bit more than what I asked you to do today okay so let's see like if I look at the keyboard it finds it it finds both keyboards it finds both mice three keyboards two months and a partridge in a pear tree all right and let's see yep still finding things all right so what's good we have got the things box that it found okay we've got the things box which it found what is not so good we still need to label them and this camera is gonna drive me insane because it never wants to focus it never wants oh there it is okay that's a great focus well I'm very particular about having things in focus I used to be a photographer back in the day and so focus one of those things that's very important to me okay so let's quit out of that and now we are going to do the dreaded put text command the dreaded put text command have you noticed that I have more errors in the old put text command than in any other command so let's see if we can do it this time without errors okay so we are going to say cv2 dot put text all right and then we're going to go IMG and then we're going to label the item and remember we got the item up here so we know what the name of the item is so we're going to put that name on the box and then we are going to put it in a tuple because we know it wants a tuple left and then top plus 20 all right left and top plus 20 because we we don't want it half in the Box half out of the box the plus 20 we'll bring it down a little so it should be all in the box that way nice and neat and then we got to tell it the font and we did define the font up here yeah right here we define the font so that's good so it's going to know a font to use so we'll put font in there and then let's have it mmm point seven-five on the text size and I think I'm going to make it red so I'm going to go 0 comma 0 and I hit something strange there 0 comma 0 comma 255 should be read in the old BGR blue green red color space and then I think I will make the thickness 1 write that now I'm gonna make that one too okay text needs to be a little bit more readable okay so let's see what happens here right mouse click run Python file in terminal okay it's doing its thinking here but I'm enjoying my coffee I am really enjoying my coffee I should do a show one day just on coffee okay so let's see there we go and look at that keyboard keyboard and I've got that dreaded focus problem come on in focus you aggravating thing okay there keyboard keyboard mouse mouse Shazam homework done all right look at that okay look at that okay pointed up there the umbrella the lighting that looks good it's finding chairs and books and TVs and it ought to be able to do that bottle why can it yeah it's doing the bottles they're finding the bottles finding some of the books I find it interesting that it has a little more trouble with the book refrigerator it thinks my expresso was hey you guys that sent me the coffee up there's somebody just sent me a nice big crate of coffee really appreciate that that was really nice and hold on just yeah I'll say that for another day all right so let's leave this back here okay so let's think about this right so now we've just done kind of like on our own what the Jetson utilities do by themselves but then let me show you kind of what we can do now that we can do things on our own this isn't that different than what their result was but let me show you what this allows us to do like you guys who kind of haven't been with me a while on this channel know by now that I hate cats I don't like cats I think that your cat is not a domesticated animal if you have a cat in the house I gotta let you know that cat is sitting there looking at you kind of being nice you'll keep it around but he's thinking about the day one thought one day that guy's gonna die and I'm gonna eat him okay all he looks at you as is food and he's just he's not actually gonna kill you well maybe if you were close to the end he would kind of help nudge you over the edge but when you die he's gonna eat you that's how cats are I do not like cats all right so let's see what I could do here since I never want to see another cat as long as I live I could come up here and right before cv2 rectangle I could just say let's make TK equal to one and then on my line thickness in boxing the items instead of making that one let's make that TK okay no change right no change TK is one but my friend if item equal equal cat then if item equal equal cat then what are we going to do if item equal equal cat we are going to say TK equal minus one like that okay now let's run it run Python file in terminal [Music] okay so now if we look down here things are kind of normal we've got keyboards we've got a mouse we've got a coffee cup we've got a cup we've got an umbrella things are pretty normal but if we come over here and then we can come over here and then what we are going to do can you believe this I got this cat video all right let me let me show you this thing okay so we come here and look at that I don't have to sit and look at that cat it blocks it out now if I make goggles for myself and wore it with a camera I would never have to look at another cat again now I want you to notice something here you see how this cat do you see how this cat is playing with a toy the cat is playing with the toy in this video let's see this video has three million one hundred and two thousand nine hundred and thirty-five views okay my friend my videos get like 2000 views I went to Stanford I got a master's degree I spent my life learning technology I'm bringing a lifetime of experience to share with the world for free and I get two thousand three thousand views and this guy takes a picture of his cat and he gets three million one hundred thousand views of his cat playing with a toy you guys had better go over to patreon and hook a brother up and help me out here because there's people out there getting three million views by putting up silly cat videos you know what I'm gonna do I'm gonna all right the cat has been done I'm gonna do chickens I've got chickens I'm gonna put a little chicken cam in my chicken coop and people can tune in and see if the chicken is gonna lay an egg oh the suspense you sit there and you watch all day waiting for the chicken to lay an egg I am gonna do that okay I have digressed sorry I will get back on topic let's turn the silly cat video back on okay silly cat video okay this is working pretty good isn't it I do not have to look at that cat I need to make something else that blocks out that three million views that this guy has okay let's see here let me see we need to get on to a different cat okay here we go okay that's blocking him out pretty good isn't it oh and his friend as well okay we got another cat and we are blocking that cat out pretty darn well aren't we let's look at this oh yeah you didn't even have to look at that one okay so this is kind of showing you what you can do with this and while this is kind of a while this is kind of just a silly demonstration that I did with cats what you can imagine is you can look at a scene and you can find different things and then when you found different things you can do different things based on what you found similarly you could kind of think about what we did with the image with the facial recognition you could sort of take different actions based on whether it's a known person whether it is an unknown person you could even come up you could train it differently you could you could have like one group of people that you like and one group of people that you don't like and then you could do different things whether the person coming up to you or to your camera or your Jetson Nano is one of the liked people or is one of the unlike people see see there's just a while again I'm just kind of being silly with these examples you can see that by being able to unwind and unpack the data coming from those object detection models you can then go in and start developing your own code and that was really what I wanted to do in the last few lessons you know not just run the the Jetson can routines that allow you to kind of impress someone show someone look what I can do but then not have any ability to go in and start working with it what we've shown you here is a method now you can go out and you can begin pursuing your own ideas based on what the Jetson nano is Jetson nano is saying he got some really neat new lessons coming up hope you guys will stay tuned in also just a reminder there's some good stuff going on over on my jetson Xavier in X lessons that those will also run on the Jetson Nano and so you might just check out some of those videos because some of that stuff might be of interest to you and if you've gotten to lesson 54 and these in this series you out you will certainly be able to run any of the things in the new in the new Judson Xavier in X series on that I sort of backed up so if somebody just started at lesson number one there they could go along and learn the stuff so some of the earlier lessons are going to be repetitive of stuff you've already known but then we're really getting into some cool new stuff in that series of lessons okay guys seriously I really would appreciate it if you guys would give me a thumbs up subscribe to the channel ring the bell so you'll get notifications of my upcoming videos and I really do appreciate you guys that are helping me out on patreon really is an encouragement for me to keep this material coming share these videos with other people okay guys this is Palma quarter with top tech boy comm I will talk to you guys later

Channel: Paul McWhorter

Views: 4,997

Keywords: Machine Learning, Deep Neural Networks, Deep Learning

Id: 1cX0uxd--qo

Length: 24min 22sec (1462 seconds)

Published: Sat Aug 08 2020