AI for Everyone LESSON 10: HOMEWORK SOLUTION Create a Region of Interest with the Mouse

hello guys this is paul mcquarter with toptechboy.com and we're here today with lesson number 10 the homework solution edition of our incredible new tutorial series where you're going to learn artificial intelligence or you're going to die trying what i'm going to need you to do is pour yourself a nice strong cup of iced coffee that is straight up black coffee poured over ice no sugar no sweeteners none needed and as you're pouring your coffee as always i want to give a shout out to you guys who are helping me out over at patreon it is your support and your encouragement that keeps this great content coming you guys that are not helping out yet look down in the description there is a link over to my patreon account think about hopping on over there and hooking a brother up but enough of this shameless self-promotion let's jump in and let's talk about what i am going to teach you today what your homework assignment for today was was to take what you learned on processing mouse clicks and mouse events and create a region of interest in opencv based on mouse action alone and what you were supposed to do is the upper left corner of your region of interest was going to be defined when you did the left mouse button down and then you're to drag over to the bottom right corner and then when you let the mouse button up that is to be the other corner of your region of interest and then you were supposed to launch a separate little window showing that region of interest off to the side then you were also supposed to with the right mouse button use that to kill the region of interest if you needed to do that okay so that was the assignment and that's what we are going to do and what i will need you to do now is fire up the most excellent visual studio code we are going to come over here and we are going to create a new program we are going to create a new program i guess uh i guess i better get out of your way and get a view that you can see here okay let's try this again so what i need you to do is launch the most excellent visual studio code and then we're going to come over here and we're going to go to the little page with the plus to create a new program and this is going to be open cv 12 dot p y and the dot p y is kind of important and boom we've got ourselves a fresh new python program just waiting to be written now we don't want to start from scratch we're going to start with that core program that launches our webcam just so that we don't have to do that bookkeeping week after week and you can find that if you go to our website tech boy www.toptechboy.com com and uh with this happy little search bar here you can search on the term you can search on the term faster launch of webcam and that should bring you to this lesson this little snippet of code here you can get by clicking on the two pages come over right mouse click and copy and then you can come over here right mouse click and paste and now just to make sure that we can have the universe in proper order and that we can copy and paste without breaking something we're going to come in and go ahead and run that and it does what we expect we get a webcam window and everything is copacetic just like we would expect okay so now what are we going to do we are going to create a region of interest based on clicking dragging and releasing the mouse all right this should be pretty straightforward if you guys were able to do it leave a comment down below i am legend and if you were not able to do it leave the comment i pulled it up like a cheap walmart lawn chair okay were you legend or were you a cheap walmart lawn chair okay so to do this what we're gonna need to do is we're gonna need to create that little mouse call callback that will be sitting and listening for a mouse click and we do that right above the while loop we are going to do cv2 dot set mouse and you can see it's already trying to guess its set mouse callback make sure you get the capitalization uh the case correct because it is case sensitive and then what are we going to do we're going to put it on the main video window which is my webcam right because we define that down here we define that down here so we're going to be listing for a mouse click over over that window and then if there is a mouse click what do we do well we execute the function mouse click all right now what two problems do we have right now one is we call the function mouse click but we haven't defined it yet so we need to define that and the other is at the point that your program hits this line of code you have not yet defined a window called mywebcam it is defined down here so we need to do that before you get to that set mouse callback or your program will crash and we can do that just real simply with cv2 cv2 dot named named window and that named window is going to be my webcam like that okay and all that does is that just creates the window there's no picture in it it just creates it so that opencv knows that that's going to be your that's going to be your window now we got to take care of this little business here where we have called a function called mouse click but we haven't defined it yet so we're going to do this up here at the top right after our print cb2 version we are going to define def what are we going to define mouse click just because we have not done that yet and now this set mouse callback when it calls mouse click it's going to pass some stuff to it it's going to pass some parameters so we've got to grab those parameters the first parameter is what is the event what is the event what is the mouse event that led to the call then what was the x position of the mouse with the call and what was the y position of the mouse when the mouse event occurred and those are the three things we're going to use but then there's also two other things that are passed that we're not going to use but we've got to grab them nonetheless and so one we'll just call flags flags we're not going to use it and parameters we're not going to use it okay so now we've got this is the function that we go to when the mouse does something when it's over the my webcam window okay so what are we going to do well we want to kind of look for two different things there is if there is if event if event equal equal cv2 dot event and what do we want to look for l button l but on [Music] event l button oh event no wonder i was wondering why it wasn't trying to guess event l button down l button down that's the thing that we're going to look for all right now if the left button is pressed down the main thing is we need to snag the position of it okay and so what we can do is we can go ahead and just print what the event is just because that's always good and now we're going to define a point 1 is going to be equal to a tuple you can think about think of it as a little array of an x value and a y value what is the x value we want well it's x position and y position so when the left mouse button is pressed down when the left mouse button is pressed down the set mouse callback is going to call mouse click it is going to come up here to mouse click and then if the event that led to the call was the left button down then we're going to end up inside this if statement and we are going to snag point 1 which is the upper left corner of our region of interest which is x position comma y position now the problem is this pnt 1 is inside of this function so the rest of the program is not going to know what it is it's a local variable so for it to be accessible to the rest of the program we better make global pnt one okay now the thing is we don't really need to do anything else because we grabbed that point which is all we wanted it's the upper left corner of our box that's all we really want we don't have to do anything else because eventually you're going to have to let that button off and when you let that button off then we'll have the second mouse event and we'll capture the second point but what you can see is is that we're going to need two points and so we're going to need another global pnt 2 all right and p and t 2 we are going to snag when we are going to stag if if event equal equal c v c c v 2 dot event underscore l not l button down but this time l button up all right and i keep putting two points that this is this is one big goat rodeo here isn't it cv2 dot event left button up so what do we do when the left button comes up well let's just go ahead and print the event just in case we need to know that and then what we're going to do is we're going to get point 2 and point 2 is going to be equal to the x position and the y position associated with that second mouse event so the first mouse event is the left button going down and we grab point one and then the second mouse event is left button popping up and then we gra then we grab point two and there's no way that we could let it up before we pressed it down so i think this is a fairly robust little algorithm that we are coming up with here now we better go ahead and make global evt the event and then here we can say ebt is equal to evnt so now the local variable event goes into the global variable evt so the rest of the program will have access to it and we'll do the same thing here evt equal event and that way now when we're in the main loop down there we kind of know what is going on if that makes sense okay now if we remember that left button down that is event four and so we want to start looking for that right we want to start looking for that or is that no i think mouse down is one mouse up is four all right so let's look at that and just make sure that i am right so i'm gonna go left mouse button down and we get an event one if you look over there to the left down there it printed a one and then when i let the mouse button up it prints a 4. and so what we know is when do we draw the rectangle we don't want to draw the rectangle when we press less button down we don't want to draw the rectangle when we press left button down why because we don't know what the other corner is we wait when it goes down we grab p and t one when it comes up we grab p and t two now we're ready to draw the rectangle so we'll start just by drawing a rectangle then we'll come back and we'll actually make a region of interest but let's come down here and what do we do is we say that if e v t equal equal what the mouse up when we release that is evt 4 what do we do well let's just put a rectangle there just to see if we can so we can say cv2 dot rectangle rectangle and then where do we want to put the rectangle we want to put it on our frame and then what is the upper left the upper left point of our rectangle is going to be pnt 1 and then what is the lower right going to be it is going to be pnt 2 okay and now we need to give it a color and so let's make it a red let's make it red so that would be blue is 0 and then green is 0 and red is 255 like that and then how thick do we want the box let's make it a thickness of say 2. and i think that will be pretty good so now what this should be is is that i should press the button down and that's the upper left corner of my region of interest i should drag it let it up and that will be my the lower right corner of the rectangle and then it should just draw a rectangle if i am thinking about this right let's go denied ah okay what is wrong that before you ever press a mouse key you have never been in this function mouse click and because we've never been in this function mouse click we don't know what evt is and so when i'm looking here when the first frame is made it's trying to see what evt is it hasn't seen it yet so i just need to seed that variable evt equals 0 just so it has a value just so the variable is created hope that makes sense so now let's run it and see what happens i need to kill the old program and now i need to run it okay there it is now i'm going to i'm going to press the left mouse button down i'm holding it i'm going to drag it and i'm going to release it and boom region of interest with my head in it my noggin in the region of interest okay so now we can quit that so we kind of got our mouse business working right but now besides just drawing the rectangle we want to create that region of interest now so this is where we have to think a little bit more so we create the rectangle that is all well and good but now i need to create the region of interest and the region of interest is going to be based on the frame but it's not the whole frame right it is from some [Music] remember some starting row to some finishing row starting row colon finishing row and then comma starting column colon ending column all right now you got to think about that so what is the region of interest what is the starting row well that is the first y value okay that's the first y value and then the second y value remember in treating the frame as an array it goes rows first and then columns not x and then y and so it's that little quirky thing that is reversed and so what would this actually be well we want to go to our first point p n t 1 and then n p and t 1 which value of the two values is the row value it's the second one the first one is the zeroth value and then the one value so it would be pnt one and that's actually the second number because the first number is the 0 position you guys should know that by now so we go from 0.11 that's the first y value to pnt 2 1 and that is the second y value so it goes from the y value of 0.1 to the y value of 0.2 those are the rows of interest and now we're going to come up and we are going to go comma and then it's going to be .1 the x value is going to be where in the zeroth position okay and then comma pnt 2 and that zero position as well now guys if you've watched the other lessons this should be crystal clear because we really went in and did this if you're just one of these drive-by shooters that just pops into this lesson and doesn't understand what i'm doing you need to go back and you need to take the class you look at all the lessons and this will make perfect sense because i'm building it piece by piece step by step presup by precept and so if you take the class you will understand all this stuff in the class is free it's free but you got to put a little work in to actually do it so i think that that would then the one mistake i made this should be a colon here this one should be a colon so we go from we go from the y value of the first point to the y value of the second point and then we go from the x value of the first point to the x value of the second point that should define the roi now where should we show the roi should we show it inside this if statement or outside the if statement we need to show it inside the if statement we need to show it inside the if statement because if you try to show it outside and there hasn't been a mouse event then the program is going to crash so you only want to show it if you've gotten down in here so we're going to do a cv2 dot i am show inside the if statement and what are we going to show let's let's call the window roi and then what do we want to show we want to show the image roi like that okay and now we better move that window or it's going to be behind the other one so we're going to say move window and then which window do we want to move we want to move roi and then where do we want to move it well i want to move it over by width okay and i'm going to go watch this i'm going to say integer of width times 1.1 because i don't want it right against the other one i want it just a little bit further over okay and then why an it because any any integer value times 1.1 is probably not going to be an integer and then this command wants an integer so i've got to kind of make it a little bigger and then i have to turn it into an int by forcing it and then i don't need to come down any i'll just scoot it over and so zero should work could it be this easy there's no way it can be this easy let's try and see what happens we're going to come over here and we are going to run it and so far so good so i'm going to come here and i am going to left mouse button down and i see the event one occurred now i'm going to drag over my face and now i'm going to release it and then i have event four i box myself and boom look at it region of interest a second window who's your huckleberry i'm your huckleberry look at that okay so we got that thing working were you guys able to do this on your own i hope so because i think i showed you everything that you needed but in this kind of cool now that you can start interacting with your video by doing things based on what the mouse is doing i kind of showed you how to do that here okay but i do believe let's see i could come up here and let's say i wanted to snag my microphone i could create a different one or like let's go for the creepy eye shot here there's the creepy eye shot okay so you see i can grab whatever anytime i create another one the other one goes right the reason this one didn't go away the the opencv has a minimum width window that it will make and if you make it smaller than that then you're still ending up with the image from the last time around so you know i could go i need to make it big enough that it will erase that old one okay all right so now what do i need to do i need to be able to kill that window if i want and that was going to be the right mouse button up should kill the window and so i'm going to come in and i'm going to quit out of that and now what i'm going to do is i'm going to put another i'm going to say if event is equal to cv2 dot if event is equal to cv2 and then event and what event am i going to look for this time right i keep saying event event and then what am i going to be looking for right button up so i'm going to press the right button when i let the right button up i want it to kill i want it to kill the uh the region of interest all right so i'm going to say right button up like that and then what do i do well in that case e v t is equal to that event which i think is something like 5 probably all right let's just uh let's just try it and see if i just do that as a five if it if that event is a five let's run it oh d9 oh man what is this nonsense it must be like auto filling that for me or something i keep getting that happening okay so we're going to try just to see ooh [Music] okay this is getting serious so why did oh equal equal i'm sorry i hope you guys saw that rookie mistake rookie mistake okay i'm just going to right mouse button click down nothing happens because i didn't have an if statement for that one now i'm going to let that right mouse button up and nothing happened and nothing happened because i did not print i did not print event i could print event or ebt there either one because the function knows both of them we'll kill the old one let's run it okay right mouse button down right mouse button up yeah the right mouse button up is the number five just in case you need to know that all right but if you think about it when that right mouse button comes up at that point evt is going to be what it's going to be 5. and so when you get down here on the next time through this loop evt is not going to be 4 so what's going to happen at this point the image is just going to freeze it's not going to update anymore and let's just see but the window will still be there all right but it should just freeze at this point if i'm thinking of it correctly so left mouse button down drag left mouse button up i got my region of interest it's live now i'm going to come over right mouse button down everything is still copacetic now right mouse button up okay now do you see that my region of interest froze and why did it freeze it freezes because evt when the right mouse button goes up evt is no longer four it no longer gets in this loop so it's no longer able to it's no longer able to update that video because that region of interest because it's not getting in the for loop so what could we do though what we could also do is we could actually explicitly say if e v t equal equal five which is the right mouse button up what could we have happen there well we could say c the two dot dis destroy window and what window do we want to destroy we want to destroy roi like that okay so if it sees the mouse button the right mouse button come up then it's just going to kill that window now just because it's not four anymore it's gonna freeze it and then when i let it up then it's gonna then it's going to kill it i you know i like to live dangerously let's go ahead and make that two different uh now let's just do it like this okay this should kill it i think this should kill it but we will see all right so now right mouse button down drag right mouse button up i got my region of interest okay now or that was left button down left button up now right button down everything's okay right button up ah kill both windows why did it kill both windows what did i do wrong there cv2.destroy window hmm if event is equal to 5cv2 destroy window okay it could be i need to do a try here it could be that that is crashing it the next time around because it's not there so the first time it kills roi the next time through the loop evt is still five now let me let me do it this way okay do you see it's trying to kill a window like on the next frame it's trying to kill the window and the window is no longer there all right so when i let that button up now evt is 5 it goes in it kills the window next time through evt is still 5 but what there's no roi window and so what i need to do is i just need to after this i need to set evt equals 0 and that way it won't come back in this if statement unless the whole thing happens again i hope that makes sense and let me kill this and let me run this okay left mouse button down i drag left mouse button up got my region of interest got my region of interest all right now come over here right mouse button down nothing happens right mouse button up region of interest disappears mainframe still here there you go you see this is really fun this is really easy you just got to play around with a little bit and then actually it turns out to be pretty fun and pretty easy and so i think that is really really cool so what we did this time is we learned how to process mouse clicks and process mouse events and i kind of showed you a little bit more complicated one using the region of interest window and how to interact with it so you've had kind of like two lessons now on how to process mouse clicks i think next week i'm going to show you another way to interact with your graphic interact with your video and that's using slider bars so that you can change numbers by sliding a slider bar and by using these slider bars then you can go in and start tuning the things that you do but you got to know how to do this because for us to do artificial intelligence you've got to be able to have means of interacting with your video window one way you interact with your video window is with mouse clicks and the other way you interact with your video window is using slider bars or tracker bars and so next week i'll show you how to do tracker bars i'll give you an assignment and then after that we will start looking at doing some more interesting stuff after that we will start beginning to approach the edge of actually doing artificial intelligence okay guys i hope you are having as much fun i hope you're having as much fun taking these lessons as i am making the lessons if you enjoy the lesson give me a thumbs up also leave a comment down below i am legend or leave a comment i fold it up like a cheap walmart lawn chair or just leave a comment hey thanks for the video your comments help give me youtube juice and that youtube juice gets this video out in front of more people and we're able to have more people learning how to code and fewer people watching silly cat videos okay guys i will let you go now this is paul mcquarter from toptechboy.com i will talk to you guys later [Music]

Channel: Paul McWhorter

Views: 2,269

Keywords:

Id: QsHFRvRmTAw

Length: 31min 4sec (1864 seconds)

Published: Thu Sep 09 2021