AI on the Jetson Nano LESSON 23: Bitwise and Logical Operations in OpenCV

hello guys this is Palma quarter from top tech boy comm and we are here today with lesson number 23 in our incredible new tutorial series where you are learning artificial intelligence on the Jetson nano I'm gonna need you to pour yourself a nice big mug of iced coffee and I'm going to need you to get out your Jetson nano gear and while you're doing that I want to give a shout out to you guys who are helping me out over at patreon you are keeping me in coffee beans and you are an incredible encouragement to me you guys that are not helping out yet take a look down in the description there's a link over to my patreon account think about hopping over there and hooking a brother up but enough of this shameless self-promotion let's jump right in and talk about what we are going to learn today what we're gonna learn about is we're going to learn about bitwise and logical operations in OpenCV and as we're getting into really starting to do the meat and potatoes of the artificial intelligence and we're doing image processing and image manipulation there's kind of some concepts that we need to begin to become familiar with one is the idea of masking where we apply a mask to an image and we just keep parts of it and we throw other parts away and so masking allows us to do things like separate parts of an image from the background or to block out parts of an image along with the idea of masking there's an idea of threshold inquiry and what we're going to throw away and then along with threshold again masking is the idea of bitwise operators where we kind of have logical functions that all of this gets thrown away and all of this we keep and so those three concepts we're going to be building over the next few lessons bitwise and logical operator operators threshold and masking and we will get into bitwise and logical operations today and so what I need you to do is come on over and let's go ahead and open up visual studio and then we are working in the folder we are working in the folder pipe row and then within pipe row we are working in the folder let me close some of this stuff I got too much stuff okay we're working in the folder OpenCV I am going to select the folder OpenCV come up to our main folder pipe row and click the little page with the plus on it and we're going to create a new file which we are going to call open open sea open CV and I believe we're at 12 and then what we're gonna call this is bitwise dot py the dot py is kind of important apologize you guys I'm getting over a cold and I kind of got over it yesterday but still a little congested but today is the day I have time to make videos so we're just going to power on through and hope you guys understand that if I don't do it today it'd be a few weeks before I could make some more videos so we're going to do the best I can to do this as I am a little stuffy ok we're going to go to our standard program which we get from the most excellent wwp tech boy calm you can search with the little icon here and let me go ahead and get out of your way you can search with the little icon here and search on web camera on the Jetson Nano you should come to this page and then you can click on the little two page over page and then ctrl C you can copy all of this and then we always start with that as the basic program which runs the camera and then since I am using the I am today I'm going to uncomment out this cam set in this cam line if you're using a webcam alternatively you would uncomment out this line okay and then the one thing I need to add is to keep my windows organized I'm going to add CV to dot move window what window do I want to move nano cam where do I want to move it to the upper left corner which is zero comma zero excuse me okay so let's fire this up just to make sure we haven't broken anything okay very good there we go we have our Raspberry Pi camera up and running okay so now what we're gonna do is we're going to start working on these bitwise operators and you might be kind of familiar with this is kind of like the concept of and and the concept of or and the concept of like exclusivo or and kind of like to understand it the and operator like if you have a gate with two inputs and means for the output to be one both of the inputs have to be one so if it's zero zero the output zero zero one the output zero one zero the outputs zero only one one gives an output of one so the first input and the second input have to be one in order for the output to be one similarly for or if either is a one the output is a 1 so a 0 and a 1 would give you a 1 a 1 and a 0 would give you a 1 a 1 and a 1 would give you a one and only zero zero give you a zero and you're probably familiar with that from logic gates we're going to apply that same thing to images only instead of just having one logic gate with two inputs what we're going to have is two images and those two images will be like a matrix so we're combining one matrix with another matrix and then we get an output matrix we're pixel-by-pixel we are doing that comparison so to do that we're going to need to create some images and so let's do that up here I'll just kind of jump in at the top and another thing is we're actually going to be using matrices and right we view images as matrices but we can build that using we can build those matrices using our numpy library so I'm going to need to import numpy as NP and that'll give us help in creating and manipulating these matrices which are really going to be images so let's create our first image IMG one and that's going to be a matrix created by numpy so we're going to call it NP and then zeros is just going to make a matrix that it fills with zeros so all the values put in are going to be zeros and now we've got to give it some parameters okay so we're going to give it a tuple of the matrix dimension it's going to be 480 so that means it's going to have 480 rows it's going to have 640 columns and then at each position we are going to have one number so this is kind of gonna be a matrix where you could think of it as grayscale so where you have the value of the pixel it's just one value between 0 and 255 now if we were creating a color image we would put 3 here because you would have three numbers a blue a green and a red but this is going to be more like a grayscale image and so it's going to be 480 rows by six hundred and forty columns and then it is going to have one color which is going to be gray and now we've got to tell it count of the units or what the format for those numbers is and it's going to be mp4 numpy and then unsigned int you int in 8-bit so that's an 8-bit unsigned in it is the type of number that is going to be there in that image array okay now let's just for fun let's display this and since this is an NP 0 all the value should be 0 and so this should be what kind of window it should be a blank window all black so let's come down here and show it let's see what we called this thing we called it image one so we'll come down here and we will do a CV to dot I am show and then what are we gonna call this we will just call this I am g1 and then what do we want to show an IMG one we want to show IMG one okay and so that should show it it should just be a black window but let's move it so we can keep our windows organized move window what window do we want to move ing one and where do we want to move it well we want to move it below the other one and so I think that we would put it like over an X let's see we would want to move it over an X 0 and then down about 500 because it's 460 leave a little bit for that bar on the top of the window about 500 should put it right below our original image that we're getting from the camera so let's take a look at that I like to run these things every once well okay there it is and that looks pretty good so I've got the main window here that's coming off the camera and then I have this black window that I could just created using the np0 alright let me quit out of this thing and now what I want to do is I want to make that image instead of being all black I want it white on the left and I want it black on the right so we can do a little bit of manipulation here so I'm gonna say IMG one and the pixels that I want to change are between row zero to row 480 so that's all the rows and then from 0 to 320 so I'm going to get all the rows up and down so that's going to be in the range from row zero to row 480 and then going across I want to go from column zero two column three two so I want to go from column zero to two column three 20 and you see this : it's saying from zero to four eighty and then from zero to three 20 so that should be the entire left of that window and then what do I want to set that to I'm going to set it to a value of 255 which would be full-on so it should turn those white so let's go ahead and let's run this python file in terminal hey boom we got it we got it why in black I can't help but just play around with this let's see if we could make a gray let's say we could could make it grave I'm making it 125 let's see if that actually works this should be a grayscale image now yeah look at that we made it great but in bitwise operators really we want it on or off and so we want it really to be the 255 255 okay so that's image number one it's going to be white on the left and it's going to be black on the right now we're going to create image two and it's going to be MP dot Z zeros and this one is again going to be 480 by 540 and again there's gonna it's going to be grayscale so there's just one number there and then it's going to be NP unsigned int eight an 8-bit unsigned int and this one what we're going to do with it is we're going to put a little white box in the middle of the big black box and so right now we have a big black box but now I'm gonna say image two in the range is going to be I want row 192 row 290 so that's gonna be a box a hundred big in the middle so from row 192 to ninety and then from column 272 column 370 so that'll be a hundred tall and 100 wide so this will be 290 comma 370 are you guys kind of getting familiar with this kind of comfortable with what I'm doing now I want to make those white bits so I would set it to what 255 and that probably would be good to be in brackets okay so now let's see if we can show IMG to so I'm gonna come down here and I'm just gonna grab this IMG one these two lines of code here and I'm gonna copy it and then I'm gonna paste those two lines and then what I'm gonna want to show here is instead of IMG one I'm gonna want to show IMG two this should be a black box 640 by 480 640 by 480 and it should have a white box in the middle that's a hundred by a hundred and then this one I want to move over I want to move it over like 700 to the right and it will just be at the top so 700 zero let's see if this works and it doesn't look good what is wrong three said oh did you guys see what horrible mistake I made there this is a range it's not a comma it's caught it's column 292 which is a colon comma column 370 now let's try okay all right but my image too is in a terrible position I certainly didn't resize that right so let's quit out of this and see what I did wrong on my move window I didn't say image to its IMG - did you guys catch that there okay now I think this should work or run Python file in window I still didn't change that right my copy and paste on my I am show I showed IMG one again where it's IMG two right that's the danger of copying and pasting I didn't change everything that I needed to change okay so let's look here I think this time in the Lord we run Python file okay I do not like the position of that though it should have been I didn't do the white box right right it should be in the middle and it is not in the middle so let's see how we get this in the middle I did not do my calculation right so let's quit out of this and let's come up here and look at this okay that would be 192 290 and then it should be 272 370 272 370 because if you think about it I want if it's 640 I want to take a hundred off of that so 640 that would be 540 and then 540 divided by 2 would be 270 so I go from 0 to 270 and then out to 370 and that should do it get a piece of paper and scratch this out you should be able see why this would work for an image that is 480 high and an image that is 640 wide and I want to put a hundred by hundred box square dab in the middle of it so let's try that that is still not in the middle so let me see what I am doing wrong here clear wait okay so let's see 480 by 640 oh here it is that was supposed to be 640 that was one of the errors there okay okay so this image - is 480 tall and it's 640 wide and again when you're dealing with an image is an array you go rows by columns rows are first so I have 480 rose by 640 columns I really hope this works this time ah okay so here I am in the upper left and then I have my first masking type box here as IMG one okay it's white on the left all one's black on the right all zeros and then here I have another masking image which is black everywhere zeros except ones in this Center hundred by hundred box okay now let's start doing a little bit of bitwise operator so let's take that image one and image - okay and now let me create a new image called bit and okay and what bit and is going to be it's equal to C v2 dot bit wise and okay and now what am I going to add together IMG one and I M G - okay now remember white is a one and black is a zero so where is the white going to be in the new image only where both the image 1 and image 2 or white okay so if I'm going to do that I need to show the bit and down here and so I'm gonna live dangerously and try to do better than last time I'm gonna copy those two lines of code and then I'm gonna paste them and then I am going to show I'm gonna need alcohol this the and window and what do I want to show that thing we just created which is bit and and then I want to come down here and I'm going to move the window and and I want to put it below this one so I'm gonna go over 700 and down 500 what do you expect it to be it was white on the left in image 1 it was white just in the center in image 2 where are they both white well that will be what's white in this image oh I am G 2 what is that in line 9 how could I have an error all the way up here in line oh man were you guys screaming at me I tried to do the I tried to do the and before I created image - I hope you guys caught that okay this should work this time okay now you see this this is white all the way on the left and black on the right then over here this is white only in the middle so where are they both white well it's kind of like you split this on the left and only here are they both white so this is in fact the and operator now understand that it's these bitwise functions where you create these black and white regions we're gonna later use this as a mask so that we can mask things off and only let show through the areas that the white are so like I could take this mask and I could apply it to this picture of me or I could take this mask and apply it to this picture of me but in order to be able to do that you've got to be able to do these logical operators so you can create the masks that you need in order to do the masking function that you want okay well let's try another one here if we did the bitwise and can you imagine what the next one would be the next one would be a bit or right bitwise or so I'm going to or those two together and so this will be bitwise or and I still want to bitwise or image one in image 2 so I will come down here and I'm gonna get these two and I'm gonna copy them and then I'm gonna paste them and then I'm gonna adjust them hopefully correctly and so this is going to be my or window and this will be bit or and now this one let's see how far do you do you think I should move this down you look over here and get a see if I can come up with a good number I think that I'm gonna go over thirteen forty this time I'm gonna go over thirteen four D and then I am going to go down by zero and that should work okay and something is very much not happy okay oh I see what okay I didn't move it right let's see I forgot to change the window door alright so this one I really think should work okay so you see this is this one is image one ANDed with image - this one is image one Ord with image two so here this one is white if either is white and I am gonna probably switch over here to this view to get that little thing out of the way okay I know you guys hate that little live in set image of the Nano so I have moved it out of your way down so that is the bitwise or okay now I'll just show you one that's another one that you can do and let's see I'm gonna quit and that would be the bitwise exclusive or an exclusive or means the output is 1 if and only if 1 is a 1 so a 0 1 will give you a 1 a 1 0 will give you a 1 but a 1 1 will give you a 0 and so that is the bitwise exclusive or so let's do that one so I will come down here at copy and I understand I am running out of space here but I will say bitwise exclusive or and I think that is just going to be the exclusive or if I remember it right and now down here I'll just change that or I'll add another one okay so I'm going to come here and then I will copy and then I will paste and I will make this one exclusive or and then what did I call it up there bit exclusive or exclusive or okay in this one I'm gonna go 1340 and then I'm gonna go down 500 so let's look at what that exclusive or does did I do that right I hope so ah what did I do wrong there yeah I just keep making the window exclusive or let me quit that okay this one hopefully will run oh when I tried to quit it put a queue in my code all right okay so you see this is kind of interesting this is the exclusive-or so it is white when only one is white so if the left is white you're fine if this right part is white you're fine but this became black because both of them are one there and the exclusive-or it's 0 1 or 1 0 it's not 1 1 and that's why that became black ok now what the real interesting thing is is now we're going to take our first attempt at creating a mask ok and so what I'm going to do is I am going to come up to frame remember how we grab frame frame is the image that we grab from the camera well now what I am going to do to frame is I'm going to I'm going to cut this out here and move it down to the bottom okay so I will come down here now and I'm gonna on this one on my main image I'm going to apply a mask okay so let's say that I say that frame is now going to be equal to cv 2 and now I'm going to do a bit wise and and ok I'm going to do a bit wise and and I am going to do it on frame and frame so there's a lot of different things that you can do with it why it's like you can do it bit wise with two different images but here we're just working with one image which is frame so we got to list it twice the new frame is frame come a frame and then I'm going to apply a mask ok and what is the mask that I want to apply well let's see what was a good mask up here let's just apply IMG one IMG one okay and then we're gonna show the Nano and so this should just mask the image and only show where our mask is white so let's try that run Python file in terminal ok boom do you see here now in the upper left we have masked with our black we have used this IMG one here we've used this IMG one as our mask and we only show the image where the image is white ok well what other thing could we use as a mask we could use IMG too as a masking now we're only going to have a little bit showing through that white window okay so let's say run Python file in terminal ok so you see now we're masking it with IMG - and you only see through that real little window okay let's do the XOR this time and so we called that bit or did we call it that we called it bit X or alright and now let's see what happens run python file in terminal ok you see so this is kind of strange here ok but we're applying this XOR as a mask on the Nano camp ok now what we're going to be doing so what we've learned today is we've learned these bitwise operators where we can create masks by combining different combinations of black and white masking to create a mask that we then use to mask a live image now in our future lessons we're going to create the masks rather than just using NP zeroes in creating shapes those masks will be created based on some property of the image itself and so imagine that I was masking on something bright blue and I ended up with white a white mask only where the tip of this blue pin is and everything else is black in that way I can mask out everything except for the thing that I'm trying to track or look at okay so that is where we are going in the future and to do that we've got to learn how to do threshold increa and then we with the thresholds then we want to go in and we want to mask the images to pull say pull an object out of the background and to get rid of the background and just to be able to look at the thing that we're interested in and to be able to pull it away from the background that will make more sense in future lessons a little bit let me tell you a little bit about some of the stuff that we have coming up here we're going to be kind of taking images apart in the next lesson where we're going to take an array that is our frame that we're getting from the camera and we're gonna split it into its three different color channels so you remember that that image is just a matrix where we have rows and columns and then groups of three numbers for blue green red we're gonna show how we can pick out and show each one of those color channels individually and then how we can put it back together into an overall image so again we're just learning how to rip apart an image and put it back together and learn how to operate with it not as some mysterious object but just as a good old-fashioned matrix so a 2-dimensional matrix where we have rows and columns and the intersection of rows and columns is a pixel okay so that is what we are going to be doing in future lessons I am wondering did in and make it all the way to the end of this video you can show me that you made it to the end of the video but showing the magic word down in the comments which is eagle10 this is an an image from one of my students recent space flights where we send instrument packages to the edge of space this picture was taken at about a hundred and twenty thousand feet above the surface of the earth and we were able to snap a picture just as the large helium balloon was rupturing on the ground the balloon is about six or seven feet across when it gets up to 120,000 feet it has expanded to where it's 30 or 40 feet across and we were able to snap an image just as that balloon was rupturing and so if you watched all the way to the end let me know by making a comment down below Eagle 10 okay guys I really appreciate you tuning in this week I'm having a lot of fun with this if you're enjoying this give me a thumbs up because that really does help me helps me score better in the algorithms and get more viewers if you guys are giving me thumbs up also if you've not subscribed yet go ahead and subscribe to this channel and think about sharing this video with other people this is polemic order from top tech boy calm I will talk to you guys later

Channel: Paul McWhorter

Views: 6,586

Keywords: openCV, Artificial Intelligence, tutorials, lessons

Id: OF6qYj_Y934

Length: 34min 8sec (2048 seconds)

Published: Sat Feb 01 2020