AI for Everyone LESSON 9: HOMEWORK SOLUTION Understanding ROI in OpenCV

hello guys this is paul mcquarter with toptechboy.com and we're here today with episode number nine the homework solution edition in our incredible new tutorial series where you're going to learn artificial intelligence or you're going to die trying what i'm going to need you to do is for yourself a nice strong cup of iced coffee that would be straight up black coffee poured over ice no sugar no sweeteners none needed now as you're getting out your coffee as always i want to give a shout out to you guys who are helping me out over at patreon it is your support and your encouragement that keeps this great content coming you guys that are not helping out yet look down in the description there is a link over to my patreon account think about hopping on over there and hooking a brother up but enough of this shameless self-promotion let's jump in and let's talk about what we are going to do today what i want is before we jump in and start doing the snazzy stuff the really sophisticated artificial intelligence stuff i want you to become very familiar with dealing with an image where when you see a picture you think of it as data you think of it as math you think of of it as rows and columns and the intersection of a row and a column is a pixel and you can address that pixel and you can manipulate it and you can make all types of magic happen but for the advanced stuff to really make sense you've got to get to the point that you're very comfortable manipulating an image on a pixel by pixel basis and that's kind of what your homework was so how many of you guys were successful i know some of you were because some of you have already been posting your solutions to youtube and i really love it when you do that really easy if you do the window key and the g key you will have a little screen recorder that will pop up record the solutions to your homework and post them over on youtube and then in this video in the comment give a link over to the little thing that you posted of your homework solution also when you post that homework solution make sure that you in the description link back to my original video so that if people see it they kind of have context they know that this is the solution for this video set but i really enjoy looking at it anytime you guys post a homework solution i do go over there and look at it okay so what was your homework assignment your homework assignment was to have our standard program that grabs a frame and shows a frame but to take a little snippet to take a little snippet from that image and then convert the whole image to grayscale and then take that color image and put it back in its proper spot take that color image and put it back in its proper spot on the original complete frame and then have that little box bounce around where it sort of bounces around your image and anytime it hits the edge of the image it bounces off so sort of like a bouncing box but that little box is a color image of the frame on a background of a black and white image of the frame okay let me know how many of you guys were successful in doing it if you were post comment down below i am legend or i folded up like a cheap walmart lawn chair okay let me know are you guys doing the homework you actually getting this is it kind of kind of starting to make sense this is not that hard but really to do it to be successful you really need to kind of start out with a strategy and if you just jump in this is the biggest mistake i see with students that i have in real life they jump in and they just start writing code before you write a code you should draw a picture now back in the day we talked about flow charting like where you would you would write flow charts or kind of flow charts for your your uh for your code i'm not a big flowchart guy what i'm a guy is i kind of like to draw a picture of what i'm doing and then i like to label things and if i can have a good picture and i can label it then the coding turns out to be really uh really easy and that's a habit i hope you guys will develop and so i want to help you develop that habit and in this case i will actually go in and i will in fact kind of draw it out the way that i think of it all right so the first thing is the first thing yep i think you can see this okay i think all systems are go here the first thing that we need to do that we have is we have a what we have a frame so i'm going to draw my frame that is not good denied okay we can draw better if we actually get a box okay so i'm going to draw a frame like this okay and we can think of this frame as the whole picture okay we can think of this frame as the whole picture then what we are going to do is we are going to have a little snip okay we are going to have a little snip that we are going to grab like that so you've got two kind of regions that you are interested in you've got the whole frame and then you have the little snip that you take well now what we need to do is we need to go in and kind of start labeling things if i can get my pen to work as i expect okay hopefully now this will go back to drawing so now the first thing that we need to think about is we need to think about the full frame and the full frame is going to have a width a width okay and even when we run that program right when we run the program that grabs a frame and shows a frame one of the parameters is we set the width okay so let's just remind ourselves that this is the number of columns on the picture is the width and then also what do we have here we have the height all right so that frame has a width and a height that as much as i hate being a munchkin i think i'm going to have to get further out of your way to make sure that you can see everything that i'm drawing okay now we have this little box here and this little box you know it has a width and it has a height too but let's call the width of this little box let's call it the snip width okay and then it also has a height and so let's call that the snip height so the little box is the snip so we have the snip width and we have the snip height now we also have the position of that box is the center and remember we're thinking in terms of rows and columns so we're going to kind of do things in terms of row comma column so we're going to put the row and then the column and so what we need to think is that that that box that box has got some center okay it's got some center row and some center column and so this point here we need to think of i think as the box the center row and the box center column okay and so that would give me the position of the center of the box it's some row value and it's some column value and i'm going to call it box call box center row and box center column okay so two coordinates are going to tell me where that point is okay i hope that makes sense now what we know is is that this edge over here this edge over here is going to be at width and this bottom edge is going to be at height so we sort of know we sort of know that our big box goes from row or from yeah from row 0 to row height and it goes from column 0 to column width does that make sense so that's kind of the bounding positions of that box now we got to think of some important points here on our little snip and what we need to see is what do i really care i really care about when this edge hits this edge and so i'm going to call that let's go ahead and call that condition one i need to let's see we're going to call that condition one okay and condition one is where the edge of the snip hits the edge of the frame and then what i'm going to call condition two that i care about is when the bottom of the snip hits the bottom of the frame so we're going to call that condition two that we're we worry about then we're also going to worry about when the top of the snip hits the top of the frame we'll call that condition three and then we're going to worry about when the left of the snip hits the left of the uh the left of the snip hits the left left of the frame and we'll call that condition four well in order to note in order to do this we've kind of got to turn that edge position into math now this edge of the frame is easy these edges of the frame are easy okay the edges of the frame are easy the edges of the frame are just 0 and width on column and 0 and height on row but this moving box we got to kind of put it in terms of a little bit of math and so where is this edge here this edge here is where well that is a column position so it's going to be you start at the box center of the column you start here and then you move over how much plus snip width divided by 2. okay because snip width goes all the way across and we go divide by 2 and add to the center and that's going to take us to this edge well how about this edge here that is going to be box center column what minus half the width the snip so that would be minus snip width over 2. now you're also worried about the top and bottom of the box and so let's do the top of the box that is going to be at box the center row because up and down is row box the center row what minus snip height divided by two and then the bottom down here of the box of the snip is going to be box center row plus snip height divided by 2. okay does that make sense so i have just defined those four edges of my snip i already knew the four edges of my frame well what do i care about i care about let's come in here with a different color i'm going to care about i'm going to care about when this let's start with this one when this one the box center column the box center column plus smit snip width divided by 2 when this hits this and so i'm going to have an if statement right so i need to have like four if statements i can then combine them to two but when those conditions exist i need to change the direction of my box and so this first one would be i need to look when box cc plus snip width divided by 2 when that is greater than or equal to what greater than or equal to i kind of messed that up let me see if i can i get an eraser over here maybe with a little luck there we go okay come back to my pen okay when that is greater than or equal to what width so when that edge of the box hits the edge of the frame i need to change direction so this is one condition where i change the direction the second condition down here the second condition would be when box center row plus snip height divided by 2 is greater than or equal to height then i would change direction okay and that's the second one now the third one that i am worried about is when the top of the snip hits the top of the frame and that is a pretty simple one that would just be box center row minus snip height divided by 2 right that's that edge there when that is less than or equal to zero and then the fourth one that i'm going to worry about would be this one over here and that would be box center column and then the left edge minus snip width divided by 2 when that's less than or equal to 0 okay when any one of these four conditions are met then i change directions now if it hits the top or the bottom i change my row increment i make it the opposite and if it hits the left or right wall i change my column direction by multiplying by negative 1. and so you can see that i'm now going to be looking at all four of these conditions and then based on that i'm going to decide whether to change directions or not and so you can kind of look at this and see that we're going to get four if statements but really i can release i can using an or i can combine those four if statements to just two if statements because because really the two column ones this one and this one i can do with one if statement and then this one and this one i can do with one if statement and so basically i'm going to create this little box and i'm just going to move it around i'm going to move its position around and then anytime it hits a wall i'm going to change direction does that make sense i hope it does and you might even i wish you could take a screen capture of this and print it so then as i begin to do code you can kind of look back and reference back to that but i am going to now need to do a little windows management and we're going to jump in and we're going to start coding i need to move this out of the way and i need to come over here and show you my screen okay all right and let me get this out of the way so i can actually get to my keyboard all right that was not good okay i think we are ready to go now okay so i will come back to my middle size and not be quite so much of a munchkin and we are going to open up our most excellent visual studio code and then what we are going to do is we are going to come over to the explorer and then we are going to come in and we are going to create a new open cv-10 dot py the dot py is kind of important enter boom and you have yourself a fresh new python program just wait to be written we do also want to make sure that we're on the right interpreter so what you need to do what you need to do is you need to just make sure ctrl shift p that you select your python select python interpreter to make sure that you set it to the one that we are doing the artificial intelligence lessons and the one that uh you have opencv installed in and for me that is the p pi ai 3.6 virtual environment so i'll go ahead and go over there and then do a little management here clear that up all right and now i will come over here okay we don't want to write the program from scratch and so let's go out and get that program that grabs a frame and shows a frame and you ought to know this by now you probably already have it but if you don't go to the most excellent www.toptechboy.com and in this happy little search bar you can search on the lesson called faster launch of webcam and smoother video just like faster launch would probably get you there and this is the code we're starting with we're going to click those two little page icons we're gonna right mouse click we're gonna copy and then we're gonna come over here and we're gonna right mouse click and we're going to paste and now let's run this thing just to make sure that we didn't do something incredibly stupid also i'm going to cut this down that is a little bit big i'm going to cut this down to 640 by 360. so i don't have such a gragantuan window so let's run this and see if it works boom we can copy and paste uh-huh anyway i like to run it just to make sure that something unusual hasn't happened and so i know that i'm at least working i'm at least starting with something that is working okay now we need to go in and we need to start coding up that that uh little snippet right first of all let's generate the little snip the little snip in image and then after generating the little stim image then we'll put it back on our original frame and then after we get it back on the original frame then we'll do some coding to actually animate it and make it move around but let's start by kind of setting up some of our variables well we already have width and height and remember that was the blue box that's the width of the whole frame and the height of the whole frame and so we already had that the blue box all right and hey i think i could even maybe come over here and show you oh that got ugly didn't it let's see i do want to be able to switch back and forth here so let me see if i can get this yeah there we go [Music] so we can go back and look at our variables but okay remember the blue box the width and the height that we talked about we already have we already have variables for that so that is good so let's think of some of the other things that we had there besides the width and the height well we also had the small box the green box and remember that was snip width that's the width of our little snip and we had snip height now you can set that to what you want depending on how size your how big your window is but i think i want to make the width about 120 okay and so that's roughly about five times smaller that's like 20 of the whole frame and then on snip height i'm going to make the snip height about 60 pixels so my box my small snip box is going to be about 120 pixels wide and it's going to be about 60 pixels high you can set yours to something else all right now what else we're going to need is we are going to need kind of the initial position of the box that center position of our snip and i'm going to call that box and i'm going to call it it's going to have a center row and then wow that was unexpected so i'm going to have a box and i'm going to have a center row and then i'm going and that is going to be equal to something and then i'm going to have a box and it is going to have a center column is going to be equal to something well i've got to start the box somewhere and write this box cr is the center of the box the center row of the box and then the center column of my small box i got to put it somewhere i'm going to start by putting it in the middle of the frame and where would the middle of the frame be well on row that would be half of the width so that would be about in fact probably the smart way to do it i could just make it an integer of width divided by 2 and that way if i change the width it's still going to start in the center and then int and that row should have been height right row would be height divided by two and the center column of the box would be width divided by two okay so where is my little box going to be where's the center of my box going to be well its row value is going to be half of the height and its column value is going to be half of the width and so that should start it in the middle of my frame okay now i should come in and each time through the loop i want to move my box by some number of pixels well i'm going to move it in the uh i am going to move it in the row dimension and how much am i going to move it well delta rho okay i'm going to move it how many rows i'm going to move it delta rho each time and i'm just going to start by moving it one pixel at a time in the row direction then i'm also going to have a delta column and that is going to be equal to 1. but you see by doing this i could have it moving faster in the column direction and slower in the row direction how many pixels each time through well i'm going to change it in the row direction one pixel and in the column direction one pixel but we could set that to something different if we wanted to okay i believe that that is really kind of what we need we have the snip width we have the snip height and we have the center the the the position of the center of the box and that is kind of what we needed if we go back and we look at our picture over here those are kind of the parameters that we set up in the picture and so now we've at least set them up well what is the first thing that we want to do well the first thing we want to do is let's just go and let's just grab that snip okay let's go and let's grab that snip so after we grab our frame we are going to come in and we're going to create a frame roi so that is going to be our little snip and which do we want that to be well we want that to come off of what we want it to come off of frame all right and then if i think in terms of that center of the snip that center of the snip well we're going to do rows first okay so which rows do i want to be the region of interest well first of all i need to kind of reference things to that box that box center row and then the first row of my little box would be box center row minus what minus the height of the snip no minus half the height of the snip and so that would in fact be snip height divided by two so i take my center position i take my height i take half my height and i subtract half my height from the center and that is going to be the top edge the top row and then the bottom row would be very similar to that but instead of instead of subtracting half the height i would what i would add i would add half the height now when we come up now so that got how many rows is going to be in my snip now i need to have how many columns so i'm going to put a comma and now i'm going to do the column well there i would stop start with the box center column so that is the center here's my here's my here's my little snip the center column okay the center column i'm going to start there and then i am going to subtract what the snip width the snip width divided by 2 and then i'm going to go to box center column plus the snip width divided by 2. if this doesn't make sense go back and look at this picture i am simply defining those four edges of my region of interest or those four edges of my little box or those four edges of my snip okay so now i have defined that now any time we do this indexing we better force it to be an integer right you better force it to be an integer because if you end up with a float in there it is not going to be happy at all and if these were like odd numbers and you divide by two you could end up with a floating point number so pro tip here if you are putting an index on a like on an array where we're manipulating an array with an index better make sure absolutely sure that it is integer the hard thing is you could do this in a way that it might run sometimes because if like the rows and columns were even numbers maybe it would run but then you would do a different box size and then it would crash and so that can be somewhat confusing all right so now i've defined my region of interest i think that we should go ahead and show it so cv2 and then what we are going to do is i am show and then what are we going to show i'm going to show my frame roi i i guess i should give it a name okay i'll call it my r o i like that i'll give it a name and then what do i want to show i want to show frame roi like that now that would put it behind the other window or maybe in in in the way of the other window so let's go ahead and do a cv2 dot move window and then what window do i want to move i want to move my roi and then my roi and then where do i want to move it well i think i want to move it over i want to move it over by width and then i want to move it down by zero okay so that'll just move it over to the side if i'm doing this right okay this is going to be rough i hope this runs this should just generate and show my little region of interest but even in something this simple we could make a mistake so let's run it before we go too much further boom look at that hello you see me in the little box hello okay so now we created our little region of interest okay so now what do we want to do well we want to convert the whole frame to what we want to convert the whole frame to grayscale so what i'm going to do now is after i've taken the snip you wouldn't want to do it before you took the snip because then you would be grabbing a black and white frame so after i've grabbed my snip my frame roi i want to convert frame so frame is going to be equal to cv 2 dot cvt color cv2 dot cvt color all right and then it is going to be what do i want to convert frame and then how do i want to do it well cv2 dot color and what do i want to go i want to go color and i want to go b g b g r to gray bgr to gray like that so now that whole window should be great so let's just make sure that i got the whole window gray now okay yeah you see the whole window's gray hello all right but now if i try to put this image if i try to put my snip back on it the shape of the array is wrong because this this at the intersection of a row and a column on this grayscale image just has one number so i've got to turn it back to bgr just so the array is the right shape so i can move that snip back onto it so we are going to probably the easiest thing would be to just snag that ctrl c ctrl v and this time i want to go what over here i want to go gray to be gr now this will not look any different but it will be a it will be the right array size okay hello there it is okay now let's just make sure that we can take our snip and we can put it back onto we can take our snip and we can put it back onto the original frame so the way we would do that is we would take our frame like that we would take our frame but we only want to put a slice on there and now what would be the range that we would want to go well the same range that we stole so we would want to come here so we're going to take the whole frame and then we are going to start at the row that is the top of the snip and we're going to go to the row that is the bottom of the snip and then we're going to go from the column that is the left of the snip to the column that is the right of the snip okay and that is all defined right here so i'm not going to go through it again but it's basically we are going from the top of the green box to the bottom of the green box okay and that those are in rows and then we're going to go from the left of the green box to the right of the green box and then that green box we are bringing back onto our main image and so we are going to do that by setting that slice of our frame we're going to set it equal to what simply frame roi which should still be color now this isn't going to be moving but simply grabbing it putting it over here doing some manipulation and bringing it back that's like half the battle right there so let's see if we can do this and what should we have my friend we should have a little color square on a gray background okay we have a little color square on a big gray background boom now guys that's half the battle right there now what do we have to do we have to make it move we got to make it move okay and so now how are we going to make that thing move well we are going to say that each time through i'm going to have the box center row is going to be equal to the box center row this is all we got to change is just that center position because our left edge our right edge our top edge and our bottom edge are all defined in terms of the center position so if i move the center position everything is going to move along with it because we were thinking like engineers and we set this up with parameters okay it's a parameterized models so box cr is going to be equal to box cr plus plus what delta rho right because center row box center row we're working with rows so we're going to change the position by delta rho and then we're going to take box we're going to take box center column is going to be equal to box center column plus what delta column like that okay now if we just run this now what's going to happen tell me if i run it now what it's going to happen okay it is going to do this if i run it now i think it'll run but then we'll watch what happens okay there's my box boom but it's going to come down and when it hits the bottom edge it is going to attempt to jailbreak and then it after it gets off the screen the program is going to crash why it's going to crash because i'm going to have an index out of range i'm dealing with a box outside the size of the box and so the whole thing crashes why because i didn't look at those conditionals right i have not put those conditionals in i am afraid i might have lost my let me see if i can wake this thing up i'm sorry but i do want to oh man okay i do want you to see this okay so what i have to do now is i have to come in and i have to look at these conditionals if the top of the snip hits the top of the frame i've got to change directions if the right of the box hits the right of the frame i've got to change the column direction if the bottom of the snip hits the bottom of the frame i've got to change the row direction if the left of the snip hits the left of the frame i've got to change that one so anytime one of those edges hits the edge of the whole frame i've got to change direction now i'd indicated that i can do these four conditions i can do it with two if statements so let's take a look how we are going to do that so let's see what are we going to do first well first of all i'm going to do the conditionals before i actually increment because if i'm already too far to the right i don't want to go further to the right so i want to do my conditionals before i change my directions all right so my first condition let's do the row yeah let's do the rows first so if and the first thing we're going to do is if the top of the snip okay hits the top of the frame so where is the top of the snip the top of the snip is the box the center row the center row and then the top of the box is up by half the height of the snip so this would be if the box center row minus the snip height if that is less than or equal less than or equal to 0 because rho 0 is the top of the overall frame if that if the box cr minus snip height divided by 2 the divided by 2 is quite important or now i want to look at the bottom of the snip when it hits the bottom of the frame well that would be box cr what this time plus snip height divided by 2 and when that's greater than or equal to what if that's greater than or equal to height like that then if that's the case what do i want to do well i want to change that's hitting in this way i want to change the direction of delta rho so delta rho is going to be equal to delta rho times minus 1. and guys i always like to be explicit with my parentheses so that i force it to do what i want it to do and it doesn't do something i didn't expect and so you see like even though with right orders of operation this is meaningful but if you just say delta rho times -1 it's kind of a little like not real clear so if you if you put parentheses you're explicitly saying take delta rho and multiply it by the number negative one and so that is how i like to do that okay now we need to do the other one and what is that if box we're going left and right this time in column so it box the center column minus snip snip width divided by 2 if that is less than or equal to 0 or if box center column plus snip width divided by 2 this would be the right of edge of the box if that's greater than or equal to the width of the frame then i want to change i want to change what i want to change the column direction so now delta column is equal to delta column [Music] times explicitly minus one all right what is the chance that this is going to work the strategy is sound the algorithm is sound the analysis is sound but i could have very easily made a typo in here so let's see what happens let's fire this thing up and see what happens here we go okay good news is it fired up and it's going now but will it bounce will it bounce boom okay now guys always fully test your code don't get excited make sure that it bounces off of all four walls because then it indicates that your algorithm is working okay boom now we still haven't bounced off that left wall and i think we're going to have to bounce off the bottom and then it should come up and then it should bounce off that left wall but then we'll really know that it works okay here it goes who's your huckleberry uh-huh tub techboy.com did you guys get this working i think this is a pretty cool little program is this artificial intelligence no it's not but what is it it is making you comfortable working with an image working with a movie as a simple data set now you're starting to think of your frame and your movie you're starting to think of it as just simple data like any other data and this region of interest really is important because we are going to have to like scan across an image and look for certain things we're going to have to grab parts of an image and do more analysis on it so you've really got to understand and again it's this issue that when you're dealing with a picture you index it by row and then column so row comes first and then column and you got to get used to that because other things want to go x comma y and that's kind of like reverse of row comma column and so you got to always understand is it looking for rows and columns or is it looking for x and y to know which one of the numbers goes first okay guys i really hope that you're having as much fun taking these lessons as i am making them again if you hit the windows key in g it'll pop up a little screen recorder record your solution to this homework post it on your youtube channel then down in the uh comments below give a link over to your solution so other people can look at what you did and also on the video that you post on the video that you post on youtube link back to this video so if someone's watching it they sort of know what you're doing they have some reference of what you're trying to show there i'm really i'm having a lot of fun looking at you guys a solution some of you are taking the 3d python the 3d graphics portion of this course as well and i'm seeing some really really impressive stuff that you guys are doing so anybody that posts a link over to their homework solution i do go over there and i look at it okay guys let's see i guess uh i guess that is going to be the uh i guess that is going to be the lesson for today be sure to tune in later this week because what we are going to start working on now in our next lessons we're going to actually start looking at how to track how to begin to track an object where we're not just you know programming in what the region of interest is but we're actually going to start tracking on other things like we could track on color or we could track on motion we could track on different things and so the next lesson we're actually going to start getting into the fun stuff the artificial intelligence stuff okay if you guys like this be sure to give me a thumbs up leave a comment down below letting me know if you fold it up like a cheap blonde chair or if you are a legend and then share this on social media let's see if we can get other people taking these classes because the world needs more people writing programs and fewer people watching silly cat videos paul mcquarter top tech boy.com i will talk to you guys later [Music]

Channel: Paul McWhorter

Views: 3,124

Keywords:

Id: -bGQNF303QY

Length: 45min 9sec (2709 seconds)

Published: Thu Sep 02 2021