AI on the Jetson Nano LESSON 28: Tracking Objects in OpenCV Using Contours

[Music] hello guys this is Palma quarter from top tech boy comm and we are here today with lesson number 28 in our incredible new tutorial series where you are learning artificial intelligence on the Jetson nano I am going to need you to pour yourself a nice enormous mug of iced coffee and today because this is a super exciting and super important in a little bit long video lesson we are gonna supplement our iced coffee with hot coffee yes we will be having iced coffee and hot coffee today so I will need you now to get out your Jetson nano gear and get ready to learn some cool new stuff hey as always I want to give a shout out to you guys who are helping me out over on patreon really appreciate your help and support you keep me in coffee beans and that is very appreciated those of you who are not helping you out yet think about looking down in the description down below there is a link over to my patreon account think about hopping on over there and hooking a brother up but enough of this shameless self-promotion let's just jump right in and talk about what we are going to learn today now where we left you off in lesson number 27 we showed you how you can kind of pick an object of interest coming from your camera and you can pull that object of interest out of your image and just kind of track it based on its color and we set up track bars and everything where we could zero in such that the only thing that we saw was the object of interest and that's the kind of first step in doing object tracking it's to be able to pull it out from the rest of the image and only show that particular object and we were doing that tracking based on on color okay what we're going to do today is we're going to show you how to to the next step where you can actually literally track it by putting a box around it or outlining it and then as you move it around the screen you actually know and the program actually knows where it is and how it's moving in what it's doing so this is kind of like the first real doing of artificial intelligence took 28 lessons but good good news is we are finally there today and in order to show you how to use contours to track an object of interest you gotta understand what contours are so let me go over to this screen and let me see if I can also get out of your way and then let's see if I can explain to you how contours work so I will need to come over here and you know I'm not just really great at drawing with this thing yet but what I'm gonna try to do is I'm going to undo that that was a Fupa okay let's say clear all right and I need to get a square which I have here and now I will need to come here and draw a box that represent your open CV window okay so that is your open CV window and then if you remember we were trying to track an object like this and so let me just try to draw this on the screen as a representation and I will come over here and I will just draw that object on the screen and so we can say that this is kind of a solid color here because we have pulled that object away from everything else so that when we look at our screen we're only seeing that object of interest well there is a command where we can create contours and what that does is it looks at that mask that has only the object of interest and then on that mask it comes in and it kind of creates a set of points that create an outline for that image okay so it's sort of traces around the outside of the image so when you get a car tor what actually is a contour well what the contour actually is is it would be like imagine this is the first point so that's the point x1 comma y1 and this is the second point x2 comma y2 and then this is the third point X 3 comma y3 so what you end up with is you end up with a Python array and it has in the first position the tuple x1 comma y1 and then comma X 2 comma Y 2 and then dot dot dot ok so it creates a list of the points that would be the outline alright but now what happens a lot of times you'll have more than one object so what if I had this and it picked out both of those red objects now it would return two contours all right also if I didn't have it tuned perfectly it would also maybe find some matching spots on my shirt because my shirts kind of red so actually when you give the contour command it returns all the contours so more accurately what you would get is you would get the first contour and so you notice how I have an array of arrays an array of contours so this is the first contour ok and then comma the second contour contour too and then comma contour 3 and then dot dot dot so it might return 30 or 40 contours and that's kind of the problem we're gonna have to figure out how to deal with that because if I have these two objects that I'm trying to track and it's finding 30 or 40 little spots in my shirt that are kind of a matching color I want to be able to deal just with the main contours and so that's what a lot of this lesson is going to be about and so really like let's come over here and show the other case like let's say I had to so I've got this smaller object and I've got this larger object and this smaller object also has you know the points X 1 X 2 X or x1 y1 x2 y2 and so forth okay so we're going to have two contours and what that would look like is again we have the main array which is an array of arrays and we would have X 1 comma Y 1 comma X 2 comma Y 2 and dot dot dot now that would be the first contour and then we would have comma and we would have the second contour X 1 a different X 1 Y 1 comma a different X 2 comma Y 2 dot dot dot ok and then we might have more than two contours so that would go dot dot dot all right so a contour is a set of XY points that define the outline of one object of interest when you do the contour command it returns an array that might have one or more or many of those sets of points in it and the reason I'm kind of kind of going over this a little bit exhaustively here is that when we go in and start coding you've really got to understand that I give the contour command I come back with an array inside that array is the first contour the second contour the third contour and so forth and each contour has a set of points X comma Y comma X comma Y comma the points that outline that given object so if you have three objects you will end up in your contour array with three contours in each one with the suitable number of points have I completely confused you I hope not all right so let's go on back over now let's go over to our Jetson nano and let's take care of a few things I got to kind of get rearranged here a little bit better where I have access to all of my correct keyboard and so forth and so now what we're gonna do is I gotta get right now man I've got different mice and different keyboards and I've got to be mindful of using the right one okay so we're going to come over and we are going to start this remember that we are going to open up visual studio and then we are working inside of the pie Pro folder and we have been doing these lessons inside of you guessed it a folder called open CV so I've selected that now I'm going to create a new program and that new program I believe I am going to call open CV I believe this is about program 18 and then what we are going to call this is contours dot pi okay boom now I've got my stuff my place to start writing my code now what we're gonna do is we're gonna start this lesson off where we left off we're going to start where we left off in lesson number 27 and so if you don't still have your code from last week you can run over and you can open up a browser and you can go to the most excellent ww top tech boy comm go to top tech boy calm and then you can search using this little icon here you can search for the lesson Jetson Nano good thing to search on would be say like Jetson Nano lesson 27 you will come here and then we have the code that we ended up with last week we will click on these two little boxes or these two little page icon and then we're gonna right mouse click and copy right mouse click and copy and then we are going to come over here to our visual studio and we're gonna paste that in okay now that is where we left off last week and now let's jump in and see if we can do some good stuff here first thing I want to do is just run this and make sure that it still runs that we haven't broken anything so I'm going to say run Python file and terminal okay there we go boom alright and this back behind him I'm gonna have to get out of your way a little more here okay so back behind here is our track bars or no let me let me see where our track bars I'll get that out of the way okay here are our track bars and I will kind of move them down and if you remember we started with this full image the Nano cam frame okay and then we created a composite foreground mask and that composite foreground mask was trying to take the foreground away from the background then from that we created the foreground okay we also show the background mask and then we created this final image which was adding the foreground to the background and so looks just kind of maybe it would be easier to show if I picked up this object that we are trying to track and that's red so I'm going to bring hue lower over to about 172 and hue upper to about 179 and then this hue to lower I'll bring to zero and take it to about six remember with red there's red on both sides of zero so we've got to kind of use both of these slider bars and now if you look at our foreground we are doing a pretty darn good job if you look at the final you can see that - if you look at final you can see that we mask out almost everything except this red object that we're interested in now I deliberately chose a red object against a red background because it kind of shows you the problem this this shade of red is different than my shirt so we've really got to try to tune in and we've got to keep as much of this image as we can while getting rid of as much of my shirt as we can so I'm going to come up here to this this top Hugh lower and I'm gonna try to dial it in you see I can take out a little more of it oh that looks pretty good right I'm I've kept I've kept this little nut and bolt and I've gotten rid of most of my shirt but I'm gonna tweak these other ones a little bit more I'm gonna work on Hugh lower a little bit and I can looks like I've taken out quite a bit there and now value low okay look at that that actually I pretty much gotten rid of my shirt there so you can see that you do it and so now if we look at our final window everything has been taken out except our object of interest which is this little bolt that we're trying to track okay so now we have that and you can sort of see it in the final image but now to apply a contour we really don't want we wouldn't apply a contour to final we would apply the contour to our mask in the mask that we're interested in is this foreground mask composite okay because what do I want to track I want to attract that foreground object which is which in the foreground composite mask is what so I want to attract that white thing so I'm going to apply my contour commands to the FG mask composite now if you don't know what I'm talking about here you really need to go back to last week and watch that video so you can kind of understand what this code does but I'm gonna quit out of this and you can see that to move forward I really want my nano cam window and I want my foreground mask window so I'm gonna block out those other ones because I really don't need them so I'm going to come here and quit we are gonna go back to our code and if you remember we kind of set up our track bars up here at the top we're setting up our trap bars and then we come in and start doing our masking okay and we are reading the track bars here inside the while loop all these things are reading the track bars and then we create our foreground mask which is based on that one color range and when we create our foreground mask - which is based on that second set of hue values okay and then we add those two together I think at some point and get the foreground mask composite and that is the summation of the of the two masks okay that's what we're really going to be working with so I think working in in creating the image of the foreground we're gonna not going to want to do that we don't need to create the image of the background we don't need to create the background we don't need to create the final so all of this we are going to take out okay and now let's see I'm trying to see where our while loop is okay so I'm just reading the frame and so I'm reading the frame and I'm showing the frame and now I'm going to actually move the show part down so I'm going to cut that those were I where I am creating the window let me let me put it back there where I'm showing the nano cam window with the frame and then I'm moving it I'm gonna move that after all this other stuff so that I can kind of be working down towards the bottom of this file loop so I'm going to say edit I'm going to cut it and then we're gonna move that down here okay control V make sure that you get your indentation right so what I'm going to do is I'm going to create my foreground composite mask and I'm gonna show it and then I'm gonna show my original frame so let's just run this to make sure that it's still working that I didn't break something there okay there it is so here is the image and here is the mask let's just do a quick tune on that mask to make sure that it's working so we're gonna get rid of everything except the little red object and I'm not gonna fine-tune it I just kind of want to roughly show okay you see we're tracking the red object with our foreground composite mask is that clear hope so okay I'm sorry for the long introduction but because this is a follow-on from last week we kind of had to just remember what we were doing last week okay so let me come back up here and now let's come in and let's start doing some coding so it's going to take me a second to get a good view here okay I will okay so now we have our foreground mask a composite foreground mask and we have our main window so after we create the foreground mask what we are going to want to do is we are going to want to create our contours right so when you have this show up on the screen we want it to give us the points that out one that object of interest and so there is a command called C v2 dot Vaughn contours it returns three parameters and remember I am using open CV three point three point one and they kind of tweak this command so if you're getting strange results make sure that you are using the same version of open CV that I am which is three point three point one okay so it's going to return three parameters we want the second parameter so the first parameter I'll just put it in an underscore meaning it's an unused variable and then I will create the variable contours and then the third parameter we're not going to use and so I will create an underscore there and so we're kind of ignoring the first one in the third one and just grabbing the data set called contours now what is contours contours is an array it's an array of arrays each array inside of contours is a list of points that outlines one of the objects of interest so it is an array of arrays of contours okay with a given contour being sets of x and y coordinates I hope that makes sense I feel like I am repeating myself and that is going to be equal to what CV to dot find contours okay now we've got to find the contours you don't want to do it based on an image right like not based on my foreground image but based on that mask and what was that mask well we created that mask right up here that mask is foreground mask composite so I'm going to come down here I'm going to tell it we want to contour on the mask foreground composite foreground mass FG mask composite alright now we have to tell it how we want to arrange those contours and what you can imagine is like imagine you had a red box with a white box inside of it well it could give you a contour of the outside and then give you a contour of the inside so that it gave you two contours but I don't want that I just want the outside one and not get fancy and so the best way to do that is by saying the parameter C v2 dot R ETR that's retrieve cap upper-caste case r ET r and then you can see that you can do a list a tree if you do a tree you'll get the contours inside of contours inside of contours I just want the outside most contour so you guessed it it will be X turn null okay and now the other thing that we have to decide is do we want every single tiny little point to define an outline on a pixel-by-pixel basis or do we want it to kind of simplify it and just you know kind of get enough points to give you a good shape I want the simpler one because it can run a little faster so I'll say see the two dot and then it will be CH a n cv2 chain and you see you could say chain approximate none well no I don't want chain approximate none that would be a whole lot of point so I want chain approximant simple okay chain underscore approximate simple and so what that should give me is kind of a simplified set of XY coordinates which outline the object of interest in my mask okay so that will give me my contours and now what I want to do is I am just to draw my contours and so I'm going to say CV to dot draw contours so this is gonna draw the contours okay and let's see cv to draw contours where do I want to draw them I want to draw them on frame okay so so now that I have this outline I don't want to put the outline on the mask you use the mass to generate the contour but I want to put the contour on the original image which we called frame okay so I'm gonna put it on frame okay what am I going to put on frame well its contours contours okay right because that's what we named it up here and then and then which one do I want to draw okay well I think that I am going to put minus one and if I put minus one it's going to draw all of them okay if you put minus one it's going to draw all the contours and I misspelled contours all the contours and I can make life a lot easier if I get this out of your way okay there now we got more room sorry didn't do that earlier okay so we're going to draw the array of contours called contours minus one says that we're going to draw all of them if we put a zero there it would draw the first one if we put a one there it would draw the second one the problem is when it creates these contours they're in kind of like random order so it's not like the most important one is first so we're just going to start by putting in minus one and seeing what happened okay now what color do we want it well let's make it blue 255 comma zero comma zero and you notice that's a tuple and then how heavy do we want the line let's make it a three okay so that should create the contours and it should create it on frame and then what am i doing I'm gonna show frame and so I think hopefully this should work hold your breath ah look at that okay so you see it's trying to draw a contour but now we've got to come over and we've got to fine-tune this to find the object that we're interested in and so let me hold it in front of the camera and let's see if we can really dial in on that okay okay look at that now what's the good news and what's the bad news the good news is I am tracking the object do you see how it's tracking the object what's the bad news it's finding a lot of that also inside my shirt so the first thing that we can try to do to address this is just tweak our parameters where it let's see if I can put let's see if I can put this here yeah I think it'll see it now I need to tweak my parameters for its better tracking on that nut and not finding all this noise in my shirt so let's see what would happen there so I'm going to turn my saturation value up a little bit look at that so you see by tweaking my Sat low value I've gotten rid of a lot but not all of the noise in the shirt you still see I'm still getting contours in my shirt and let me also do value low tweak that and if I tweak value low too much I start losing part of that nut and so I'll have that and then let's see if I can tweak this a little bit okay so the bottom line here is is that good news is I'm now tracking this object wherever I put it right wherever I put it I've got a contour around that is pretty darn exciting okay but what's the problem I've got a lot of noise and I kind of did this with this shirt on purpose because in the end no matter how careful you are you are not going to be able to tweak things carefully enough that it's not going to find something that it thinks is the right color somewhere in your image so you see like if I move it gets it can kind of like it looks like it's all gone and perfect but then I move again and then it starts finding it starts finding those problems I hope that that's showing up on your screen okay all right so we found a con we found our contours and we outlined them but you can see that we sort of have a problem and what that problem is the problem is that it finds noise in its draw it's finding those contours and it's drawing those contours so the problem again is is that it's generating those contours in random order and it's not necessarily making the one that you care about the first contour in in that array so what we can do is we can do and this is not an open CV command this is a Python command and what we do is we're going to take contours okay and we are going to sort it because we want the biggest one the biggest contour first okay and that way we could just show the first contour and we would know it would be the right one because it would be the biggest one so we're gonna say contours is equal to sorted you see no CV - this is just a Python command sorted what do I want to sort I want to sort contours so the new contours is going to be a sorted version of the old contour so we're gonna sort contours now we've got to give it a key how are we going to sort it what parameter are we going to sort it on well this is a new thing and that is we're gonna learn about lambda functions and in Python you know we've done functions before we're up top we define something and then we call it well lambda is a way to kind of build a function right into a line of code and so we're gonna use this lambda which is telling it that we want a kind of built-in function right here inside this line of code and what we're gonna say is we're gonna sort on X okay where we take the C v2 dot contour con tour area of X okay so this is gonna step through all the contours and the parameter that is going to look on and sort on is the contour area okay and then if we just did this it would naturally go from smallest to biggest but we want to go opposite of that so we're going to say let me get out of your way a little more contour area comma we are going to say reverse order equal true and what that means is with reverse order equal true it will start with the biggest one down to the smallest one alright so that looks good so now contours is in order and I you know I kind of tried to explain this lambda function that the key is what you're sorting on and we're sorting on this built in this function that we're building in right here which is going to be the the contour area of each one of those elements but at some point you can look this up on your own if you don't understand it or you can just kind of do what I did here sometimes you just got up move on okay now since it's sorted do I want to plot them all no I want to plot the first one so if I want to plot the first one instead of minus one what should I put no not one we are programmers which means the first element is element zero okay so this should draw one it should be the biggest so let's go ahead here and let's try to run this so I'm going to right mouse mouse click run python file in terminal what did it not like true was not defined what do you mean true is not defined ah okay they're right i misspelled true i didn't use the right thing that python wants true uppercase t ru e t ru e and now it lights up blue meaning that it likes it so let's try this again run Python file in terminal unexpected indentation what is that about um my cue I put there by accident alright run Python file in terminal okay so now we've got to get our track bars and try to home in on this little nut again and we're going to kind of come here and we're going to kind of come here okay and this look in this mask I'm gonna deliberately leave a lot of this noise here and I need to bring hue lower to down and I need if you're using a different color you can do this differently okay now I'm gonna bring this up a little up a little okay can you guys see in can you guys see I guess this has been covered up a little bit but can you guys see let me let me turn that off I'm sorry if that was covering you up earlier but okay so you can see right you can see very clearly that it's tracking this object okay but then also look and see that I've got a lot of noise in my shirt let me even let the noise get a little bit worse okay do you see all that noise in my shirt where it's seeing hundreds of contours in my shirt but yet when I track it's only tracking what it's only tracking the biggest one okay now if I put this down you see it's trying to track on my shirt there but if I bring a real object of interest in it simply tracks on that and it's actually doing a pretty good job of outlining that do you see that boom that is pretty cool okay so what we did was the way we did it was we sorted it okay we sorted it and then we kept the biggest one but that's a little bit of a kludge because what if I have two objects I'm trying to track you see it'll track one or the other and it's constantly jumping back and forth as far as which one is bigger like if I move them closer okay it's only tracking the biggest one but sometimes you might have more than one object of interest so imagine you had balloons floating around the room and you were trying to pop them with a laser or something like that you would need to know where all the balloons were not just the biggest one and so we're it's great news that we're tracking okay but we need you know we need to do a little bit better job of that okay so we'll move forward here I'll quit out of that and now what we are going to do is we are going to just kind of comment these out will comment out the getting the contours and compliment out the sorting the contours okay and what we're going to do is we're going to do a for loop okay and what we're going to say is for CN to CNT that's like for contour okay for contour in contours on doors I guess I better leave these up here in fact so I was just kidding you don't want to really comment those out so you get the contours you sort the contours and now we're going to have a for loop where we have this parameter CNT that is going to step through each contour in the main array contours right contours is a list of individual contours contours is an array of contours so it's going to have the zeroth contour the first contour the second so the first time through the loop CNT the first time through the loop CNT will be an individual contour it will be the first individual contour in the list of contours okay and then the next time through C and T is going to be the next contour and the next time through it's going to be in the next contour so this for loop is just going to step through that sorted list of contours okay now what we can say is we can say the area is going to be equal to cv to dot contour area of what of CNT because that's the individual that's the individual contour so we are going to find the area of the individual contour that we're looking at and we're going to step through the contours so each time C and T is going to be a different contour from the list and we're going to calculate the area of it okay now what we're going to say is let's say if area is greater than or equal to 50 and that's kind of like pixels 50 pixels so that would be kind of like if you would have a contour that would roughly be like seven pixels across and seven pixels tall that would be roughly an area of 50 if it's not 50 we don't want to do anything if it's greater than 50 what are we going to want to do if it is greater than 50 what we are going to want to do is let me get down here with a little better okay we are if if the area is greater than 50 what are we going to want to do cv2 dot draw contour draw contours okay and then where do we want to draw them we want to draw them on frame okay now we have to pass it the contour but you can't just say contour because it's expecting a list of contours and this CNT only has one so we've got to put that inside of square brackets so now we're going to have a list and that list is going to have one element but you've got to have those outside brackets or the dimensions aren't going to match with this function okay I hope that makes sense now which one do we want to print or which one do we want to draw the first one which would be zero there's only one but still we got two zero I bet actually here since there's only one you could put zero or minus one because there's only one and then what are we going to do we're going to make it the color 255 comma zero comma zero which will be blue and then how heavy do we want the line we'll make it three all right so now this will print this will draw every contour that has an area greater than 50 so let's go ahead and try to run this okay we need to bring our track bar over and we need to fine tune I'm not going to fine tune too carefully because I want to leave some noise in my shirt so that we have kind of something to work with okay so you see there's okay see see there's a lot of noise in my shirt there let me even let it be a little more noise in my shirt okay let's try that and now look at it you see it is tracking only the object of interest and it's not tracking my shirt but if I take this apart if I take this apart now it's tracking both objects okay it's tracking both objects okay and let's see how far back you see oh look at that you see it's going pretty far back so you see you could tweak that area but you see like if I want to track my finger you see it continues to track it all right that is pretty darn exciting look at that it'll track that let's just try like what if I bring in this box I'd want to tweak the I would want to tweak the parameters this box this red of this box isn't exactly like this red so you could tweak it to track the box or something different let's see this should be pretty close and that's a little bit different so depending on what you're tracking on you got to fine-tune things now look it okay that looks pretty good and it's getting this box in the corners what the problem is all right so that is really working very well okay so that is exciting so now what you can do is you can track an object in OpenCV and now because you know where that contour is you actually know where this is so let me kind of show you a couple of cool things here I'm gonna quit the one thing I didn't like very much I don't like where it tries to draw an outline because it that outline has different levels of perfectness and so as it's kind of changing the outline is kind of jittering around and I don't like that very much so what I would rather do instead of drawing contours what I would like to do is I would like to find like if you think of this what I would like to do is I would like to find the corners of the contour so it had this really complicated shape around it as an outline sort of tracing the outline but what I want is I would want the PERT the the the parameters of the bounding rectangle like imagine a rectangle was around that contour what would be the X Y position of the corners of that bounding rectangle and I would rather do that than trace it and so the way we would do that is is there is a command called bounding rectangle and that command returns four parameters the x position of the upper left corner the Y position of the upper left corner so that's kind of like the X and y position of the upper left corner of the bounding rectangle and then it gives you the width and the height of the rectangle and with that you can draw the rectangle and what is that equal to it is equal to C v2 dot bounding rectangle okay and we want to find the bounding rectangle of what of the individual contour that we're working on which is C NT okay and so now we have the bounding rectangle then down here if that if that area is still we're still calculating the area of the contour if that's greater than 50 what do I want to do C v2 dot rect tank tangle and this is going to draw a rectangle what do I want to draw it on on our original image frame and then what is the upper left corner of the rectangle it is the tuple X comma Y which we just got from the bounding rectangle command and then the draw rectangle command the C v2 dot rectangle it wants the other corner well what would the other corner be it would be X plus width okay and then comma y plus height X plus W comma y plus h and now what color do we want it to be well let's make it the old blue 255 comma zero comma zero and then how heavy do we want the line let's make it a three all right now this should draw a rectangle run pipe ah that was not good that was not good okay let's try it again I hope I didn't break it okay here we go so what we want is we want this and you know what I just really really like having the title of the lesson as I go but I'll try to make sure it's not covering anything up so let's come in and let's see if we can zoom in on this we've got to adjust our track bars okay now look at that I'm getting the rectangle but what did I forget to do oh I still had this all this time down here that is not good okay I'm going to take that command out so I've been read wrong the contours outside of the list which was not good but since I'd put a zero there it was just doing the first one and the first one was a sorted one so that was just redrawing one of the contours so shouldn't have been there but wasn't hurting anything okay now I will run Python file in terminal okay so we are going to adjust our track bars to see this thing we know that we should go to 179 we should go to about let's say 173 and then we should come down here and we should lose okay see see there's a lot of noise still in my shirt but now if you look at the Nano cam if you look at the Nano cam window ah I need to fine-tune it a little bit more because my shirt it is finding some contours of 50 in my shirt that should do it okay look at that you see I have a red object over a red background and it's tracking it I really oh man boom I'll tell you I really like tracking with a rectangle for my object of interest I like doing that a lot better than tracking with the trace contour all right now let's see if we can take it and trace track two objects okay look at that boom tracking two objects two red objects against a red background uh-huh yeah that's pretty darn good double chest bump on that one we should really do that more the professional football players do that to celebrate success I think that we as programmers should adapt the double chest bump when we have had a success like tracking two objects okay tracking two objects look it seems like this one if you move it too fast you can confuse it because you're you're kind of capturing part of the frame there okay but that is working really really well okay so I think that is just super cool so this is like okay now we know where the object is right now we know where the object is and if you think about it if you come over here to the code because you have the corner of the rectangle you the program actually knows physically where the object is and so kind of like I think one of the interesting things I'll give you a heads up of sort of where we're gonna be going here in the future in the not very distant future I think what would be very neat would be to build an object like this where we have two servos one servo will move like this and the other servo will move like this so you can point something so we want to build something like this and what you can see is is that this is two servos and it's a pan tilt and then what I did was I 3d printed a little slot for the Raspberry Pi camera what you could also do if you are using one of these webcams I believe this one is the Logitech webcam you could mount the webcam up there as well and so what we would I think would be neat to do in future lessons is as your object moves around then have the servo track it the the pan-tilt servo track it where no matter where you move the object it points the camera at it so that the object stays in the center of the screen and that starts getting to be some pretty some pretty exciting stuff and so you might not have a 3d printer well if you don't put what I'll do in the next lesson the next lesson what I'm going to do is I'm going to show you the stuff that you need to order and at a minimum what you need to order is you're going to need to order the two servos and then this little pan tilt bracket and then if if you don't have a 3d printer which probably most of you don't that's ok you can just kind of tape you could pop the the Raspberry Pi camera off the Nano and you could just kind of make a little Cluj thing to tape it on here if you don't have a if you don't have a nice 3d printer or you can imagine this is kind of it's hard for me to take off and real time here but let me show you without the bracket on it if you can be patient with me for a second and what I'm gonna do in the next lesson what I'll do is I'll just give you a list of the stuff that you need to order the gear that you need to order kind of like the minimum set of gear and I'll give you links to it so that you can follow along so next lesson will just be the stuff that you will need in order to move forward with these lessons and these servos don't cost much and the little bracket doesn't cost much but let me let me get this 3d printed bracket off of here and man there were many more mucho too many threads in what we really needed okay so this is just the two servos with the bracket and if you have one of these Logitech cameras you can see that it would mount very easily just with tape you could tape it on there or if you have a tripod screw you could put the tripod screw through it and have it mounted on there so I'll just kind of give you a heads up next you know next lesson and for you guys that are watching this on premiere I'll go ahead and premiere it right after this lesson so you can see what what you'll need moving forward but if if you have the logitech camera it's going to work great and that's the easiest way to do it so one simple solution would be to order the the servos in the in the pan tilt bracket and then a logitech camera okay but if you really want to continue to use the Jetson Nano what we're going to need to do is I'm going to need to give you a better deal I mean I don't mean Jetson Nano but if you are going to want to use the Raspberry Pi camera what you can see is you are going to need a longer cable okay so if you're going to continue to use this you can imagine if you have the acrylic case you could pop it out but then you're going to need a longer cable and I've got one that I'll put a link to okay in the next in the next video so next up in the next lesson is basically going to just be going through the equipment that you're going to need to kind of move forward the servos don't cost much the bracket doesn't cost much longer cable doesn't cost much and if you don't have a 3d printer you don't have to make the the nice space and so forth but again the simplest thing would be to get a Logitech camera and man I really I like the Logitech camera and we've had a lot of fun with the Raspberry Pi camera but I do kind of think that if you're going to continue with there's some nice things about the logitech camera ok so you guys that are watching this is a premiere stay tuned we're just going to fire off the next the next premiere right after that and we'll talk more about the equipment that you need but I think this is just really really really cool ok but I do need to give you your homework also so what we're also going to have is we're gonna have a set that we're gonna have a homework and I need to kill this ok and I need to go back over here to this view and then I am going to need to open up my terminal ok this is what your assignment is for next week ok number one you know watch the next video so you'll get your gear ordered but then while you're waiting on your gear this is what I want you to wait work on with your existing gear ok so we are going to start to program and then we are going to tune in on and this is not doing what I thought it was going to do okay let's see here I had this all set up okay I'm sorry let me let me run the correct program I am okay I need to be here okay sorry I was in the wrong terminal window I think this is going to work okay this is your assignment for next week I apologize for running the wrong program okay what I've got to do is we've got a zero in on our target as always okay and we're going to come back here and come back here okay what your assignment is is to track with a crosshair okay to track with a crosshair so that you're putting I need to tune that in a little bit better get a little bit better okay see see what I want is I want you to put a crosshair on your object of interest a crosshair on your object of interest and then you can see that if we have two objects of interest two objects of interest twice the fun okay twice the fun now I would need to tweak these parameters a little bit okay there so you see I could box my face in right do you see all the interesting things or if you think about your aiming something you now have a crosshair that you're aiming at I am very unhappy with all of that nonsense okay there you go all right object of interest put a crosshair on it and track it okay so that is going to be the homework for next week okay guys this has been a long lesson but it's been an important lesson and it's been a fun lesson because this is the first thing that that we now are able to track an object in OpenCV this is starting to get into the real meat and potatoes of artificial intelligence and so that is really exciting to me it's really great and now that we know where something is we could start thinking about pointing something at it like what might you point at it well you might point a camera at it or even more fun would be maybe if we built a little mount for the laser in that if there's an object of interest in the room you could track it with a laser you could put a you could put a dot on it from a laser then you can imagine there's all types of other interesting things to do and so where are we really you're going to want to start going with that is we're gonna want to start working with the GPIO pins on the Jetson Nano or those GPIO pins are starting to control something in the real world and what we're gonna start with is this little servo system okay guys really appreciate you sticking with me leave me a comment down below that if you have made it all the way to the end let me know this you actually watched that you actually watch this video I sometimes wonder if any but he actually makes it to the end of these things all right we will launch the next premiere in just a few minutes so that you can get your gear ordered and then start working on your homework Palma quarter from top tech boy comm I will talk to you guys later

Channel: Paul McWhorter

Views: 11,496

Keywords: Jetson Nano, AI, Artificial Intelligence, OpenCV

Id: lxFPM93Qe8o

Length: 59min 13sec (3553 seconds)

Published: Sat Mar 07 2020