Raspberry Pi LESSON 58: Control System for Pan/Tilt Camera Hat for RPi Camera

hello guys this is Paul McCarter with toptechboy.com and we're here today with episode number 58 and our incredible new tutorial Series where you're teaching your Raspberry Pi who's boss what I will need you to do is for yourself a nice strong glass of ice cold coffee that would be straight up black coffee poured over ice no sugar no sweeteners none needed and as you're pouring your coffee as always I want to give a shout out to our friends over at Sun founder Sun founder is actually sponsoring this most excellent series of video lessons and in the class we are using the sun founder Raphael kit for Raspberry Pi now most of you guys probably already have your gear but if you don't look down in the description there is a link over to Amazon you can hop on over there and pick up your kit and believe me your life and my life are going to be a whole lot easier if we are working on identical Hardware but enough of this Shameless self-promotion let's jump in and talk about what I am going to teach you today and what I'm going to do is I'm going to show you my solution to the homework assignment that I gave you in lesson number 57 and that homework assignment was to take the pan tilt hat that we have have on the Raspberry Pi with the camera let me go ahead and switch over to that view so you can kind of see what I'm talking about and remember what I showed you how to do in lesson number 57 what I showed you how to do in lesson number 57 was to be able to track an object of Interest by panning the camera okay so I showed you how to develop a control system where you could track an object of Interest based on pan and then what your homework assignment was to take what you learned and add to the program where you could also track in tilt and that way you could have arbitrary position of the object of interest and the camera would track it both in pan and tilt that was the first part of the homework the second part of the homework was that what I was Finding is it was kind of hard to train on an object when the servos were on because it was jumping around trying to find an object of interest and then it was hard to actually really dial in on your object of Interest so the second part of the homework assignment was to have two modes of operation in your program the first mode of operation was trained and when you're in train mode you're dialing in on the object of interest but that but the servos aren't moving the servos aren't moving the camera and then once you get it dialed in you can switch over to track mode and then the servos come on and then start actively tracking your object of Interest so I must ask how many of you guys were successful if you were successful leave a comment down below I Am Legend double chest bump and if you were not successful leave a comment down below I folded up like a cheap Walmart lawn chair and just reminder of you guys doing the homework make sure that you post your homework to YouTube leave a link back to this lesson and then down in the comments link over to your homework solution and you guys be be looking at each other's Solutions so you can kind of see how different people are approaching this problem [Music] I think I've talked enough about the setup I think it's time for us to jump in and start coding and so the first thing we want to do is I don't want to start from scratch so let's come over to the most excellent www.toptechboy.com and let's kind of start where we left off last week and so you can use the happy little Search tool and you can search on something like using pan tilt camera Servo to track object of interest in opencv something like that you'll come to this lesson lesson number 57 and you can come down here and you can click on these two little page icons and then you will with a little luck yeah you will be able to copy that code and then once you've got that code copied then we can come back over to the Raspberry Pi view here and then we have Bonnie opened up and then what we can do is we can just come in and we can paste that code and then we're starting off where we we're starting today where we left off last week now we'd like to just run the code to make sure that copy and paste is working and that we're at least starting with a working program so let's come in and run this and remember at this point it doesn't you know it's it starts off with the uh it starts off with the uh [Music] okay there it seems to kind of almost have it so let me see if I can bring this down okay and we bring this up and then we need to just kind of get our track bars that was not good let's get our track bars up here and then let's see if we can kind of dial in on this thing okay that's looking pretty good come down a little let me see if I can get a little more in view okay and then I'm gonna adjust this up a little bit just this up okay so now it looks like I've dialed in on this thing pretty good and you can see the camera there and so let's see what happens okay as I am moving to the left it is tracking okay it is tracking one thing that that happens is it seems like this camera is doing an auto white balance and as it is doing an auto quiet balance it seems like it kind of loses my object of Interest I mean it loses my training but if I scoot back a little bit I think my blue shirt was confusing and my purple shirt was confusing it a little bit but boom I would say that that is working in the pan Direction so I'm going to say we are starting with a working program so that's good let's see if we can come back that's not it that's not where we were where were we okay there we are okay so that is our starting point the working program now what were we supposed to do we were supposed to go in and have a track mode and a train mode and I'm going to go ahead and do that and then I'll come back and I'll kind of draw up our strategy for controlling the servo in the pan Direction okay now what would be the easiest way to switch from track or to switch from train to truck because I want to start in the train mode and then when I'm trained then I want to switch to the track mode what would be the easiest way to get that input from the user what would be the easiest way to get that input from the user we're already doing track bars let's add another track bar so I'm going to say CV2 dot create track bar and this one is going to be I'm just going to say trained is going to be 0 and track is going to be one so that's going to be my label where do I want to put it I want to put it in that same window Which is my tracker okay and now do I want to start in train or track I want to start in train so that is a zero train like teaching not train like you know choo choo okay train like teaching okay and then what do I want to go to one because it's going to be zero or one zero is train one or track did any of you other guy guys think of this kind of pretty simple way of doing this I think you're probably some of you probably got way too complicated on this and probably you're going to see here that this is just going to be a real real simple way okay so now I've created the track bar now I've got to come up and create the function okay and so now this is going to be Define on track seven it's going to get the vowel just like the other ones and then what's going to happen well I'm gonna need uh I'm not going to call this train or track I'm going to call this track okay so uh so I'm going to create a global global variable Global variable is going to be track okay and then track is going to be equal to Val and so if track is one we're going to track if track is zero we're going to train now now uh and then I'll just go ahead and print it so I'll say print value like that I always like to print just because it kind of helps with the debugging and then we'll put track in all right now since I'm using this track and I might run into it before I touch the the track bar I think I should probably come up here and initialize that variable and so just kind of let's see before I get into all of this I'm just going to go ahead and give it track it's going to start out at 0 because why I want to start in train mode so it'll be sure to know that track is starting out at zero and so at this point I should have that I should be able to run this and it should run now it's not going to do anything but I just want to see that it doesn't crash okay so yeah that looks good that looks good didn't crash and let's look at the track bar make sure that I've got that on there and so if we come up here yeah you see I have this new value and it's not going to do anything but you see I can just switch from train to track like that so that's all good so now what are we going to do with it well if track is one I mean if if track is zero I don't want to be doing any I don't want to be doing any of this nonsense of moving the servos so where does that stuff start it starts here where we calculate the error like I still want to find the Contours I still want to box the item of interest but I just don't want to do any of this stuff starting with error so I'm going to say if train equal equal 1 meaning I mean if it's not trained it's track if track is equal equal one then I'm going to do all of this stuff now what's the problem all of this needs to be indented now I could go down and tab each one or I think I can just select them all all the way down to set my pan angle this is all the stuff there for this track business and I can come up to edit and then I can say uh it says indent selected lines but it's not giving me that option okay let's try it again indent the selected lines and boom now you see all of this has been indented over underneath the track so what does this mean this means only if I've switched to track does it go in and find the errors and start moving the servo so let's see if this works let's see how well this works so let's run giddy up no errors you take the little victories in life where you can get them right so sometimes it's just a victory that the thing didn't crash so let's see if I can get this maybe where you can see that a little bit you won't be able to see mask let's get this up here a little bit get this up here a little bit okay now let's get our track bar over here it's hard for me to be able to get this where you can see everything but let me bring this here okay now there you see I should be able to train on that pretty well there and so I'm going to open it up a little bit that looks good I'm going to try to get some of that other tone down a little bit let's see if I go this way yeah you see that looks pretty good so you see how much easier it is to train if the servos aren't running all over the place now I am ready to what I am ready to what I am ready to track now you see I want to just kind of try this easy so I'm going to move this kind of to the edge here and now I'm going to come over if I can find my mouse and let's get this over here now what I'm going to do is I'm going to go from Train to track giddy up look at that okay what if we come back here move it this way move it this way okay do you guys see that that is working so that I think was a big step forward in our software now what do we need to do now we need to go in and we need to get we need to go in and we need to get my sketch pad working there it is it's always a question whether the sketch pad thing is going to come on or not and now we will switch to the most excellent sketch pad View all right and now what are we going to do we're going to kind of map out our strategy for the control system in the Tilt Direction so let's come up and let's see if we can get going here and so what we're going to do is we're going to start with we start with a what I have a frame like that okay and then what I want to do is I come in the what do I want I want the center of the object of interest to be where in the center of the frame so changing that those words into math what do I need I need the coordinates of the center of the object of interest and the coordinates of the center of the frame well I've got the frame here and what do I know in the software go away you aggravating thing I have this is the display wdisp uppercase W and this is disp uppercase H all right and so then what is this Center Point well that would be halfway over which would be the coordinate d i s p w divided by 2 and then where is it this way it is d i s p h divided by 2. and you can remember this because this is the point the upper left point is the origin zero zero and then this point down here which you can't see because it's behind me this point down here is the point point display W comma display height and both of those technically you should subtract one off of because you're starting at zero but we're we're not using those two points so I'm not gonna be that precise okay that's not right we'll come back over here okay so now what do I have I have the coordinates of the center of the frame now what do I need I need the coordinates of the center of the object of Interest so I'm going to come down here and I'm going to get the right I'm going to get the right drawing tool and I'm going to come down here now and I'm going to draw my object of interest I'm going to draw my object of Interest so there it is and now what is it that I want I need to know the coordinates of the center of the object of Interest well what do I know let's come back over here and let's take a quick look at our let's take a quick look at our program we know here at this line of code we find the Contours right we find the object of interest and then after that we take the biggest object of interest and we write we put a bounding rectangle over that contour and then that command returns three it returns four things the x y coordinate of the upper left corner of the bounding rectangle of the object objective interest and then how wide that is and how high that is and so with that it comes it becomes pretty easy what is this point here that is the point X and Y that the program gives us okay and then this is the width and this is the height like that and so what is the coordinates of the center well it is X Plus W over 2 for the x coordinate and then for the y coordinate y plus h divided by 2 like that does that make sense okay I hope that that makes sense give me just a second here okay so now what do we go back and think about we go back and think about our error is this it's called way you aggravating thing I don't know why it keeps doing that uh it's the what do we want we want the center of the object of interest at the center of the frame do we have that no we don't we have a what we have an error okay and now you know that the thing could be over here and you could have a pan error and a tilt error well we've already taken care of the Pan Air now we're going to take care of the Tilt error and so this then what I'm going to call this because I called the pan air error I'll call this tilt error okay how about that I'll call that tilt error and so now let's come up here and let's think what is the tilt error well the Tilt error is this point minus this point the y coordinate of this point minus the y coordinate of that point is going to be the Tilt error and what is the y-coordinate of the center of the object of Interest well that is going to be that is going to be uh I want to make sure I see it right it is y plus h divided by 2 . okay that's this and now we want to minus this which is this which is just display height over 2. now that is the magnitude of the error now what do we want to do we want to drive that error to zero now what are the two possibilities well the error could be greater than zero and in this case you can see the error is greater than zero because this minus this is going to be a positive number well if that is the case if that is the case [Music] appear a little better if the error is greater than 0 then what do we want to do we want to tilt the camera down you want to tilt the camera down and remember down is in the positive direction so what are we going to do we're going to say that the tilt angle the Tilt angle is equal to the Tilt angle plus one and that will make it go down okay the Tilt angle is equal to the Tilt angle plus one well if what if our op man that is starting to annoy me what if our object of Interest was up here well then in that case the error would be less than zero in that case we want to point the camera up and so we're going to say tilt angle is equal to tilt angle minus one okay y minus 1 because remember last week we saw that a negative tilt angle points you up in a positive tilt angle points you down now I think with this information here we have kind of what we need to do to go in and start putting our control system for tilt into place okay so let's come back over to our most excellent ah that was probably an ugly noise for you in the microphone okay so let's come back over here I think that's a nice view and now let's come down and you can see that right here we take care of the error in the pan so we're working with pan error here even though I just called it error it probably would have been good to call it pan error all along but this one we will distinguish by calling it tilt error and tilt error if we look back at our notes is y plus h divided by 2. I always like to use parentheses and then minus what display H divided by two okay is that what we said I believe that's right now once we do that we've got to look at the two cases if error is greater than zero but remember if we say zero we're going to have that oscillation problem so let's go ahead and take advantage of the learning that we had last week and we're not going to try to correct an error unless it's greater than 30 in the positive direction and then what would happen this time it is going to be tilt angle is equal to tilt angle plus one okay tilt angle is equal to tilt angle plus one y because we want to move down some all right now we've got to check we don't want to drive that sensor too far down and in this case it's pretty easy to run this into the hat so I'm going to put a pretty tight limit I'll let you point up but I don't want you to point down very far and so I'm going to say if tilt angle if tilt angle is greater than 40 then what am I going to do I'm just going to pin it I'm going to say tilt angle is going to be equal to 40. so I won't let it go down further than 40 degrees to avoid a crash okay now once I've done that if statement then what can I do I can go ahead and I know I've got a good tilt angle note now so I will say tilt dot uh tilt.set underscore angle like that and what do I want to set the tilt angle like that I think that's good okay I think that's good now what's the other case if error is less than minus 30 pixels then what do I want to do well I want to say tilt angle this time is equal to tilt angle minus one y minus 1 because I want to start pointing it further up I want to start pointing it further up now this time if tilt angle is less than minus 90 because I'll let you go all the way straight up because there's nothing really for the camera to for the uh pan tilt hat to run into going in the up Direction so we'll let you tilt up further than we will let you tilt down so if the Tilt angle is less than or equal to minus 90 then tilt angle equals minus 90. so we'll pin it there and then now we have a good tilt angle so what are we going to do we're going to say tilt dot set underscore angle and then we're going to set it to what tilt angle how many mistakes have I made in here how many mistakes are you guys yelling at me about okay well I wonder I wonder could it really be that easy is there anything else I need to do I don't know we're gonna try to run it I will need you to hold your breath [Music] didn't crash okay what's the good news is now remember we are in TR we are in train mode right we are in train mode and so let's get these windows lined up here I'm hoping you guys are using move Windows commands but it's just because I'm using multiple cameras or I'm using multiple screens it doesn't work very well for me to try to move the windows automatically so let's open this up a little bit that looks good dial it in a little bit closer I'm going to come over here and look that's pretty good I can't really do anything about I can't really do anything about this or this because uh but the thing is this is going to be big enough that should work and so let's look okay yeah you see coming back yeah it seems like it's locking onto that pretty uh pretty well now is the Moment of Truth as we say and what is it that I need to do now I need to go from train to track and let's see what happens boom look at that okay the good news is it's staying stable okay kind of lost me there [Music] okay what's happening it it we need to go back to uh we need to go back to terrain okay we got we got some crazy business going on there so let's see uh let's see if we can get this thing found here so what's happening is there it is okay so now let's see if I can come back and get that thing sort of back up so I don't really know if I have an error or not okay that looks good now I am going to try to dial in now that I've got it in a good position I'm going to see if I need to kind of dial this in a little bit more tightly because it was seeing what got messed up as it was seeing the yellow on the top of the desk and it was starting to lock in on that okay so now let me take this and let's see I'm going up and it's not really working okay do you see how it doesn't want to go up so let's quick what did I do wrong here ah I see my problem did you see my error were you guys screaming at me what do I need to be doing here tilt Terror that one was terrible that one was terrible you guys were probably completely confused and I was panicking man it's not easy to code live guys it is not easy to code I will need to hold your breath this time let's see I think maybe I didn't kill the program completely last time let's make sure the program's killed hold your breath okay look at that so we're gonna bring this over we're gonna bring this over we're gonna get our track bar up we're gonna kind of dial in on that a little bit better okay what are we ready to do we are ready to make sure that you can see this well okay you guys can see that now what we're going to do is we are going to turn on we are going to turn on our tracker okay it kind of stabilized there that's looking kind of good maybe a little bit of up and down but we're not going to quite worry about that yet let's see if we come over okay it is tracking nicely in the pan Direction okay now let's see if it's going to stabilize okay now we're going to come up and it is tilting come down and now let's go across and now let's go up in diagonal you see I'm kind of moving it up in diagonal and so we are getting pan and tilt working at the same time okay so now let's just put it down and see if it'll stabilize [Music] taking a second to stabilize there you know maybe what we would need to do let's do one more adjustment okay let's make this 35 pixels instead of 30 in all these places okay uh let's make it 35 in all of these places so this is going to be 35 [Music] . 35 . okay and that'll just make it a little bit more stable so let's try this again okay we are in train mode so let's get this thing trained on here I'm feeling more confident about getting it trained excuse me okay dial it in a Little Closer okay get this over here get this up all right now we are going to turn on the track mode here so here we go [Music] okay [Music] there it is [Music] okay going up working like a champ let's see if I can turn it over this way moving it over moving it back let's see if I can even track it when it's further away okay guys look at that boom all right that is pretty darn cool Okay so we've completed the homework assignment we're tracking in both directions now you saw we had a little bit of an oscillation issue there what I would do is I would go in and I would tweak that number a little bit more and figure out exactly should be 35 should be 30 25 should it be 40 and I would get a little bit more comfortable with that uh with that parameter of how small of an error do we try to drive out okay now let's talk about your homework for for next week and this is what do we like about where we are now what do we like about where we are now there's a lot of good things to like about this okay a lot of good things to like about this but all right let me uh let me see if I can draw it and say what we like okay so here is our frame okay I don't like that color okay here is our frame [Music] okay like that here is the center of the frame and I think you guys can see that that I'm just going to do it for I'm just going to do it for a uh for a pan but then you can kind of see that that this would also equally uh equally apply for tilt and so let's let's come over here and let's say then my object of interest is way over here and then I have the center of my object of Interest here so what do I have now what do I have now I have a error like this okay that was terrible see if I can draw it better like that this is my what this is my error and it's my error in pan all right now I see I have an error in this case do I have a big error or do I have a small error I have a big error and this object of interest is about to leave the frame and if it leaves the frame what is the problem then I've lost the object of interest in the camera if it continues moving over the camera can't find it because once it is out of the frame then your control system is going to stop all right now think about what we do we see this error and then how do we correct the error we go like this we we just look well I'm not pointed enough to the right so I'm going to click to the right I'm still not I'm going to click to the right I'm still not click click click click click click click click click and as I'm trying to chase this thing this thing might have already left okay and so if I really want to track this thing this is kind of slow because I'm going to click click click click click click all right well what's something uh you know what would be a better way well what if I did big jumps like I made a big jump like that and then I could I could track it a lot better but then if I make big jumps what is the problem with making big jumps well what if it's here okay what if it is here okay what if it is here and then I come in and I make a big jump I jump over it and then I make a big jump back and I run into that problem that we have as we were trying to drive the error to zero so the problem with small jumps is the problem with small jumps is you're moving too slow the problem with big jumps is you'll jump over the target so let's let's kind of look at this and think instead of making always little bitty jumps what would be a better way to do the correction instead of saying pan angle [Music] is equal to pan angle minus 1 right because we're going we're going in this direction so that's going to be a minus one instead of that instead of always making this minus one what would be a different way to do it all right so this is your homework your homework is to go in and create an improved control system where you're not just jumping by one degree you're jumping in a more intelligent way you're jumping in a more intelligent way that's all I'm going to say because you guys some of you guys are real pros and I want this to be a challenge for you okay I want this to be a challenge for you and if you're just completely hung and just you know work on it and work on it but don't worry I'll show you how to do it next week but I really want you guys I want some of you guys to figure out what the better solution is okay I want some of you guys to figure out what the better solution is when you get the better solution post it to YouTube okay leave a link back to this video down in the comments leave the link over to your solution and then you guys start looking at how other people have solved this thing so this is kind of where we're going next week I will show you the solution to a better control algorithm for this and then I think after that I think the next because then we've got our control part of this thing working real well and then after that I think we're going to start doing things other than just tracking objects of Interest based on color maybe could we treat teach opencv to find a face and then track on a face that when it finds a face that it tracks on that so that's kind of the direction that we're going to be going now is after next week our basic control system is going to be working for controlling the position of the camera but now how can we get more and more sophisticated about finding a face finding an eye maybe finding the tip of a pinky finger and then start tracking on a wider variety of things guys I hope you're having as much fun taking this class as I am making them if you enjoyed the lesson be sure to give us a thumbs up if you leave a comment down below that always helps us with the old YouTube juice if you've not already subscribed to the channel when you subscribe make sure you ring that Bell because that will give you notifications when our future classes are dropped and finally share this video with more people most importantly share this video with more people because the world needs more people doing engineering and fewer people sitting around watching silly cat videos Paul mcquorter with toptechboy.com I will talk to you guys later ah

Channel: Paul McWhorter

Views: 4,719

Keywords:

Id: C-ddlvrGPV4

Length: 39min 28sec (2368 seconds)

Published: Wed Apr 19 2023