Rock Paper Scissor using Computer Vision | OpenCV Python

hey everyone welcome to my channel today we are going to build this exciting rock paper scissor game so let me show you a demo how it works so all we have to do is we have to click on the s button it will start the game rock paper scissor and there you go so i had the paper and the ai generated a rock that's why player one got one point so let's do it again rock paper scissor and this time i lost the point because the ai had a rock and i had the scissor so let's try that again rock paper scissor so i have a paper and the ai had a scissor so again i lost so that's not a very good start let me try to win some it's a rock paper scissor and again uh today is not my day it seems so rock paper scissor okay i give up so this is the basic idea for this game it's very exciting so this will be our project for today so we will go step by step write the code from scratch so stay tuned we will also provide you with all the graphics so you can build this by yourself and everything is provided for free on our cv zone platform you can go and download all these files for free link in the description below and let's get started this video is sponsored by clear ml do you want to know how machine learning and computer vision are done in the industry while we all love running experiments and notebooks to learn really fast but when solving problems in the real world things tend to get really complicated but that's just means there's more to learn clear ml and open source toolbox will help you bridge the gap between learning and actually deploying your projects in a way that's scalable to thousands of users if you love machine learning and are looking to make real impact with your skills you can get started for free in 10 minutes check out clearml and let them help you take your projects to the next level if you would like to level up your computer vision skills do check out our premium courses that are available on our cv zone platform links are in the description so without further ado let's get started [Music] so here we are in our python project and as usual we are going to go to our settings and we are going to install the packages now before we start we have named the project rock paper scissor and we are going to go to the interpreter and add and here we are going to add the series on package wait where is the vcv zone yeah there you go so we will install the cv zone package and then we also need to install media pipe which is the google package uh for hand tracking that we will be needing for our hand to be tracked so that is the initial idea so now that the installation is done we have our main.pipe so we are going to start from there we are going to remove all of this and we are going to start typing um our packages so we will import cv2 which is the opencv package and then we will also import cvzone and what else do we need so so far i think that should be enough uh the first thing we have to do is to open up our webcam so we write the code for that we will write cap cap equals cv2 dot video capture video capture and we are going to given the id number 0 which i believe is the camera id that i have attached right now and then we are going to write here while true and we will write image and then we will write success and image equals cb2 dot cap dot read and that should give us the image and the success boolean and what else do we need we need cv to dot weight key and we will give it the value of one which will give it a delay of one millisecond and we also need to write cb2 dot i am show and we will give in the name so let's say it's called image and then we'll just print out the image that we have received in the previous line so let's run that and see if it works so that we can go to the second part which we will discuss later on so there is the image that we are getting i can move my hand and you can see that so let's go ahead and stop it and we can move on to the next part now what exactly is the next part we need to bring in our image which is the background of our game and based on that we can add our webcam into it we can add the score we can add what exactly is the image that we're getting uh as the rock paper or scissor so all these things we need to have in our resources folder so what we will do is we will create a new folder uh it's called directory over here and we will call it resources and within this resources folder we are going to drag all the images that we need so i have these images over here and i was testing a few of them so let's just keep it bg which means background and then we will also add the three images i will show you just now what exactly are we talking about but for now let's just paste them in and there you go so here we have these three images let's check them out one by one so image one is your rock it's the image for rock and it has a transparency at the background so if you want these images all you have to do is go to our website and the series on platform and you can download these images from there all you have to do is you have to sign in if you haven't already register register sign up and then you can download these for free there is no charge for these so we have the first image which is rock the second image which is paper and the third image is our scissor so these are the three images that we need uh which will represent the hand of the ai so and and the other one what we need is the background image and here we have the rock paper scissor and this is the portion for ai so the images we will add here and then the player video we are going to add here so we are going to crop the image we will squeeze it no no we'll not squeeze it we will crop it actually yeah and then we'll push it in here so that this will be the player side and this will be the ai side and in between we'll have the timer so all of this can be designed very easily i used canva for this you can use whatever software you want you can design it your own way so let's go ahead and import this image once we have this image we can add and remove things from it so let's go ahead and do that so we will write here let's call it image bg equals cv2 dot imread and what exactly are we trying to read we are going to read from the resources and within the okay i didn't spell extract resources and within the resources we have vg.png so bg dot png so now normally you would put this outside the while loop but in our case we have to put it inside the while loop and the reason for that is that if we put something on it it will remain for example we need to put one of these images whenever we play a hand we need to put this image on the background image now next time in the next iteration this image will remain on that uh main image on the background image and that is not what we want we want to update the complete image every single frame so that we can decide when to add and when to remove the images so for that reason we need to put it inside the while loop so here we are going to update it it's a little inefficient but it doesn't really make a difference in this case so this will give us our background image and what we can do is so for now we can just show both of them and we will write here bg and we are going to show the image bg so let's run that and we will get two images this time and there you go so this is 1280x720 so it's hd it's not full hd if you want you can make it full hd as well and now what we will do is we will take this and we will put it inside this area so that um it looks looks like it's part of the game um so that is our next step okay so how can we do that first of all we need to know what is the size of our capture device so we will make sure that it is exactly the same whether you are using a laptop or a computer or a webcam whatever you are using we need to have the same pixel size so we are going to write here cap dot set we want to set the property number three which is the width and we will set it to 640 and then we are going to copy that and paste it here and prop id number four which is the height we are going to set it to uh what should we set it 480 okay so now what we need to do is we need to crop it so we know that the width is more so what we will do is we will crop it in the middle we don't want to crop it from the side so we start from zero and we end at some point actually we need to check what is the size of this so um what what exactly is the size of this box so let me check that i had written it somewhere okay so the first thing we need to do is to make this of the same size we need to scale it down so right now both values are bigger than the box so let me actually check what the box is what the size of the boxes so let's open it in explorer so if you are on windows you can simply go to paints or you can write you can go to edit and from edit you can look at the pixel values here so in the corner it's showing 795 and then here it's showing 1193 so 7 so if we open up the calculator so what was the value one one nine three minus seven nine five so it will give us three ninety eight so probably it's four hundred so when i was designing uh i made the width 400 and let's check the height so here if we start the value is 233 and here it is 6 5 3 so six five three minus two three three will give you 420. so we have uh 400 in the width and 420 in the height so that's the size of our image so what we need to do is right now our image is 640 by 480. so it's bigger than both of these so right now if we put our image it will be like this some somewhat like this so first of all we need to scale it down so that it is of the same height so the height is the same like this once the height is same we can remove the sides that that we don't want we can crop these area and then we will get the center image and the end result will be something like this so there are two steps the first one is to scale it down and the second one is to crop the region so the first one let's go ahead and do that so here we are going to write image scaled and we will write cb2 dot scale not scale resize resize and we will give in the image we want to resize which is image um no it's on the image background my bad so we need to put it up here and we'll go back and this so this is the image that we want to resize so we'll write here image and then do we give it specific pixel number uh no we don't give that so we will write here zero and zero and then for the output image we will write none and for the scale so i calculated the the values so it was 480 so let me just show you uh but wait yeah it was 480 the the height should be 480 but right now it is 420 so what we can do is we can divide it so 420 divided by 480 that will give you 0.88 or 0.875 so here we will write 0.875 and then 0.875 so this will squeeze it down uh it will scale it down to what actually we need so instead of just showing the image we will also show the image scaled so let's copy that and we are going to paste it here and we will write scale scaled there you go so let's go ahead and run it so now we should have three images so let's go ahead and check them out okay there you go so this is the scaled one and this is the original one so it's it's not a very big difference but there is a small difference uh that you can see so now what we can do is let's just tally it over here so if i put this you can see that it is almost the same height as what we require and what we need to do now is if we put it here like that we just need to crop it from the sides so that it is of the same size now if you were designing and you don't want to do this part you can actually make it a little bit bigger this region the actual image so that you don't have to crop but in this case we are going to crop it and we will call it uh what should we call it so no we don't need to create a new image what we will do is we will put it on the image background right so image background at what values are we changing so we are starting so let's go down uh first of all we need to scale it my bad so we will write here image scaled sorry we need to crop it so we will write image scale equals image scale and how do we crop because image is just a matrix we just need to tell it use pixels from this value to this value and that's it so if we use our calculator again oh i wrote it here let me just open that calculator and over here if we use so uh our width is 640 multiplied by 0.875 so that's our width it's 560. and what exactly do we have we have 400 so we will subtract 400 from this and we'll get 160. so we will divide it by 2 and then we get the value of 80. so we will start from 0 and we will go till 80. that's the whole idea so let's see so here in the height we are not going to change anything the first parameter is the height and second one is the width so we will start the width at 80 and then we will end it we are going to end it whatever the value was minus 80. so what was the value i believe 560 where is the history yeah it was 560 so it will become 480. so let's see if that worked there you go so if i put that in here now it's exactly what we want so if i am in the center is going to be the same but again uh we don't want to crop it from the edge we want to crop it from the center that's why we are using these values so it's 80 and 480. so that's good now what we need to do is we need to take this image scaled and we need to put it on the background image so let's go ahead and do that so it's very easy again it's a matrix so you will tell that take the background matrix and take the new matrix and take that values and put it in the background image so it might seem difficult but it's very easy so we'll put image background background at what values are we going to replace equals image scaled so we want to put image scaled at these values starting from here till that point so the first value will be the height and the second value will be the width so now uh these values you can check again from paint so if we undo all of this and we check the pixel values here it is seven nine six and two three four so wait no okay seven uh what was it seven nine six and two three four seven nine six was the width so 796 will be the starting point and the heights will be 2 3 4 that will be the starting height and then at the end we will check and it is 1 1 9 4 and 6 5 2. so it will be 1 1 9 4 9 4 and 6 5 2 will be the maximum height so let's run that and see if we are heading in the right direction okay so we are getting an error because you cannot put just random values when you are trying to add one image onto another when you are overlapping it it has to be exactly the same size so here our original image is 420 by 400 and the the pixel values that we have provided are 418 by 398 so it is lacking two pixels on each side so here we are starting at 234 and we are ending at 652. so let's just decrease one from here and we will increase one from here and then we'll do the same from here it's 76 so we'll make it uh sorry 96 it will make it 95 and 94 will make it 95 so that will give us two more pixels in the width and two more pixels in the height so there you go if we push it back and there you go so it is exactly wait is it okay i think it's missing a little bit we can maybe push it down a little bit because there is one white line here which which might be because of the pixels so here um we can push it down let's make it 34 and let's make it 54. so that will push it a little bit down and hopefully that will uh solve our problem of that white line yes it did so now it is exactly in the right position and here we have our hand and it's moving it's real time um even if i push this down i can move this around i can come to the center and play the game and that's all good so we have so far done a few parts where we added the webcam then we added the background then we scaled the image then we overlaid our webcam image onto our background image so that we can we don't have to have a separate webcam image all the time we can see it through our game itself now the next part will be to add the what you call hand tracking so that for that we will require cv zone package so we will write here from cv zone uh dot hand tracking module import hand detector and we will create a detector detector equals hand detector and in the hand detector we can define the maximum number of hands so the maximum number of hands that we want are one we don't want more than one hands and the confidence value by default i believe it's 0.5 and we can keep it that way so that will detect the that will create the detector and once we have the image in fact once we have the scaled image i believe it will be better to use the scaled image or if you want you can use the original image as well but it will be better if you use the scaled image so here we are going to say let's say find find hands and we are going to write here if i forgot let me actually change the hand detector so if we if we press the control button and we right click or left click our mouse then it will take us to the actual code and here i have given some examples so def main will give you the example code and here you can see it says hands image detector dot find hands and there you go so we will copy that we will come to our find hands and we'll paste it so it is hands image detector dot find hands and we will use it on image scale so actually we have to do it before we before we put it on the background otherwise it will not show us the result so let's run that and hopefully we will have it on the image scale okay so this is a new error that has been showing up uh from media pipe so once you install that it tells you to downgrade proto puff so all you have to do is you have to go to file and you have to go to settings and here in the interpreter you will see protobuf and select that and select the version three points uh two point zero there you go and then just install that so hopefully this will not give us that error again so done done and let's run that again and there we have it so this is our image and if i bring in my hand you can see now it's showing me that this is my right hand and it is showing me all the points the joints of my fingers and now what we have to do is we have to check how many fingers are up based on that we can decide whether it's a rock whether it's a scissor or whether it's a paper so that's how simple it is so let's stop that and what we can do is uh we can write here that if hands are detected then get the first hand so we will write hand and equals hands at zero so that will be our main hand and from that we can use to find our fingers so we'll write here detector dot find hands fingers up we want to see how many fingers are up and we will give in the hand so we want to find how many fingers are up of this hand that's what we are saying and we will return it into a variable called fingers so that's the idea and then we can simply print and we can write fingers and let's run that so that that will give us an array of five values so each value represents a finger so here you can see these are the values and if i bring in my hand all five fingers are up if i put my thumb down it goes zero pinky finger ring finger middle finger and index finger all of them up all of them down so this will be rock so we will check when it's zero zero zero zero zero it will be rock when two of them are up index and middle it will be scissor and when all of them are up then it will be paper so that's the basic idea it's very simple the the idea of the the the concept of the game is very simple that's why the logic behind it and the coding part is fairly simple as well there's not a lot of complications so what we can do now is we can remove these parts and we'll just keep the main image which is our game and we will keep the rest aside okay so right now what we need to do is um should we add a counter or should we check the fingers okay let's add a counter so whenever the counter starts we need to check um the timer and based on the timer uh we will say rock paper scissor and then it will be three seconds or two seconds whatever you decide and then it will generate an ai result and then you will it will take your result and then it will compare by the way you can make this game uh to be played as a game that you can never win because the ai you can give it like a 10 millisecond delay and it will always win so you cannot win from this game so you can do that too but we will make it fun so we don't okay so what we'll do is we'll add a timer and we need a timer and we also need a flag that will tell us whether we have the result or not so when we start the timer it will start from zero it will go to zero one two three and at three we are going to turn on a flag that state the result and when it states the result we will stop the timer and we will stop everything else and we will collect the result of the user of the player and we will take the result of the ai and then we will match them so that's how we will see the result so we'll write here timer equals zero so initially it will be zero then we will also write state result results equals false so that is the initial state so once we find the hands we will not check whether there are hands or not we will simply start the timer when we start the game actually we need a flag for the game as well so start game equals false so here start game is false so we will say if start game which means when start game is true then check all of this otherwise don't do that but then the question is how exactly are we starting the game we will start it by pressing a key you can start it by some gesture as well but for simplicity we're going to start it with a key so we will write here key equals cv2 weight key and then we will write if our key equals ord and we use the s key then we are going to set start game equals true so we will start the game as soon as we press the s key so if i run this now initially will it show me yeah it will still show me but it will not show me the output of the console so over here it will give me no results right now it will show me the hand but no result but if i press the s button you will see it will it is showing me the fingers now how many fingers are up so it means the s key is working for us and now we can move on to the next part the next part is the timer so for the timer we are going to simply write um we need to check the state first if state results is false it means that we have not reached the end of the timer so whenever we reach the end of the timer we will we will ask it to state the result so we will make it true so initially it will be false so if it is false then we will update the timer so timer equals initial time and over here we will write time dot time minus initial time we didn't import time i don't think so so we need to import it here imports time and over here it will be time.time and initial time we didn't declare so this time is not when we started uh our game it is when we press the s key and the actual game started not when we opened the game first time so here we will write here we will write initial time equals time dot time so then it will keep giving us this timer and what we can do is we can put it in a text so that we can see so cv2 dot cv2 dot put text and we need to give in image background and we will write string we have to convert it into string because we will first convert it into integer because these values will be really bad so convert it into integer and then what we will do is we will write down the position so it will be 605 this i have checked earlier and 435 so that will be the position and then we have cv two dots font underscore hershey underscore plane and why am i doing that and then we will have the scale and then the color so 2 5 5 0 2 5 5 which means purple and the last thing will be our thickness so thickness will put it at four so this will give us the value of our timer so let's run that and see if it works so it will only work when we press the s button so whenever the game starts it will start the timer so there you go so i'll bring in my hand even if i don't bring in my idea it doesn't matter if i press the s key there you go 0 1 2 3 4 5 but we need to stop it at 3 so we don't want to go any further so what we are going to do is we are going to write here that within this if timer is greater than three don't put three uh don't put equals three maybe equals or greater than three is fine but don't put three equals to because sometimes it it misses because we are converting it uh it's floating value so it might miss it so you have to be careful about that so if the timer is greater than three uh what we can do is we can write state result equals true so that way next time it will not show us the timer it will stop the timer and the the timer we can put it to zero timer equals zero state result is true and timer equals zero and then it will check the hand and all that and this time it will only check once because this will only happen once and then it will stop checking it so let's run that so if i press s one two and three and there you go and it gave us only one value at the end and it is correct because we had all our fingers up and that's why it gave us paper with one one one one one so that is the correct direction we are heading in and the next parts will be to check what hand are we using okay or we can also do the ai first or should we do the hand okay let's do this first so here what we can do is we can write if fingers equals a double equals zero uh the first one we will put it at rock why are we putting the first one rock because here the first image one dot png is rock the second one is paper so we will put paper and the third one is scissor so we'll put scissor so using this convention the first one should be rock so for rock it will be all zeroes and then let's call it player move okay player move equals one so we said that the rock is one then for the second one it was paper so it will be all of them will be one and in that case the value will be two because at the end we are going to match these two values if this image equals two uh with the player move then we will say okay um it's it's die nothing happened and based on that will change value so i will show you later on what that means okay so then we will copy that again and we will paste it by the way a trick to copy if you are on windows you can press window and v and that will give you the history of your clipboard if you have activated it if you are on pycharm you can press ctrl shift and v and that will also give you the history uh within pycharm so you can click on that and it will paste it so sometimes we copy and then copy again and it copies an empty line and it's frustrating and in that case you can press ctrl shift v and then you can paste so here we need a scissor so we'll put 0 1 1 0 0 and then this will be player move number three so these are the three values so now what we can do is we can print the player move so whenever the timer ends we need to print what the player played was it one two three and then we will tell you that with our ai generated image so let's see how that goes let's open that up i'll press the s button and uh this time it should be paper so it's giving us 2 let's check what is 2 if we click on two it is paper so that means we are good to go so this is for the player part now we need to do the ai pawn so ai part is it's very simple it's it's nothing complicated so um we need to import an image so we have these images three of these images so we need to import that and we need to put it on our image the background image so first thing we will write here image ai equals cv2 dot im read and we need to read from the re sources actually let's write an f string because one of the values will be a variable so we write here f and then resources slash one dot png so right now we will just use the first image and later on we will randomize it so this will give us the image ai and then we need to overlay this image on our background image at a specific position so how do we do that where is that specific position we will use the cv zone cv zone dot overlay png function here you have to give in the front image the background image so it will be image background and the front image which will be image ai and then you given the location that's how simple it is so the location will be 149 and 310 and we will put it back in image background so this time around it will always give us a rock because that's the hard-coded value but once we see that it is working fine we are going to put it in a dynamic manner so that it can randomize all these values so let's wait for it to run and see how it works and we will check that out so let's open that up uh let's put in our hands so this is oh there is an error what happened oh yes one very important part i forgot when you are using the cv zone overlay png you have to import it unchanged so you have to write here cv2 otherwise it will remove the alpha channel and you can't use the png part so cv2 i believe it is i am read underscore unchanged so let's run that and hopefully this time it will give us the correct value so let's press the s button one rock paper scissor okay it's a guess there's an error okay no problem uh what does it say prince player move is not defined player move is not defined because um it's only over here but we are giving it a value and the hand was present so why didn't it print okay let's just put it over here and we'll put it as none in the beginning so we'll put in our hands there you go so you saw that one of them showed up but it didn't stay it's uh it just went away so that's not good how can we make it stay the way we can make it stay is by using it outside so how do we do that over here we are going to write if state results then we are going to do this part again keep doing it all the time that and that's how simple it is there's nothing fancy about it because this is with a lot of if conditions this will happen only once so here it will happen again and again so we are putting it outside the loop uh in directly under the while loop so we don't want any what you call loopings within the loop by the way that is called nesting here let's press s and one two three and there you go so if i had the paper and the ai had the png which is rock then it means i1 so that's the idea and then what we can do is we can make it generic or let's call it dynamic so to make it dynamic we need a random number so we will write here that our random number equals rand random dot randint and the range is from one two three so both of them are inclusive so one is also possible three is also possible and in between we have two so it will give us all three values so we will take that random number and we will put it here within the curly brackets and that's how simple it is now it's dynamic and we can use it everywhere so no we cannot use it everywhere actually it will change the values for us so each time we play it will give us a different answer so whenever we press the s key it should start the game it will take the initial time by the way the spellings are wrong initial initial time so let's change that where is it initial time okay so also we need to change the result so state results we need to put false state results equals false so now it will actually reset the whole thing and it will repeat again and again so now we can check with the timer that whether the game is working properly or not so let's go here we'll press s let's put rock this time and it give up it gave us paper we'll press s again and the image went away and now it gave us scissor we'll press s again let's put paper and it gave us rock so each time it is giving us a different value so here let's put that and it's giving us that value so again that is good but now we need to tell who won so we need to check whether the player one or whether the ai one and based on that we need to put the score over here so let's let's put the score first so that we can see whether it's working or not and then we can later on uh put the logic so to put the text we will go down at the very end or let's let's go up first and we are going to declare the score so scores equal zero and zero so the first one is for ai and the second one is for the player so we're just putting it in one variable so that it's easier to manage and then at the end of the day before we show the image background we need to put these so we can simply copy the text part the put text and we can paste it here and we can make a few changes so we need two text we'll put it on the image background we will not show the timer we will show the scores at zero and we will show the scores at one actually it's already integer so we don't need to convert it into integer so we can remove that and then these values will change so the first one will be 4 10 and this one will be 215 and then this one will be 1 1 1 2 and this one will be 215 as well and then uh for the scale it will be four four four and four for the thickness it will be six six again you can change it based on your liking so there you go oh we didn't change the color so let's change the color to whites so we'll put two five five two five five and two five five let's run that okay so there you go so now you can see the score on the right and on the left and uh once we win or lose it will add the score so for losing we are not going to do anything we will just add it to the other person uh for winning we will add one point each time so how do we do that so it's very simple once we have the player move and once we have the random number we need to match them if both of them are same or not same actually we need to check which one is which and based on that we can tell whether the player one or the ai one so let's start with the player so we will write here when player wins so we will write an if statement so we will say that if player move equals one and the random number equals what so if one one is rock you will win with the rock if the finger is scissors right so if the value is three then you will win so you will write this as the first condition we'll go to the next line so it's easier to read what happened there did i copy it wrong okay let's just remove that and we'll push this back a little bit and there you go so player move is when it's two when you have paper then the move should be one so it should be rock and the last one we will remove r and we are going to write when player move let's put it together when player move is 3 which is scissors then the answer should be paper for the player to win if that is the case then we will write scores at one plus equals one so it will add to the score we can remove this part for the print move so it will add to the player score so the same thing we will do for the ai so when ai wins so we'll just put it opposite so we'll put this as 3 and we'll put this as 1 this one will become 1 and this one will become 2 and this one will become 2 and this one will become 3 and then for a i scores at zero plus one there you go so if we run this now it will give us the scores um based on whether we won or not so if i press the s button rock paper scissor so nobody won so there is no change let's play again rock paper scissor so this time i had the scissor and the ai had the paper so that's why player one we have one point then let's play again rock paper scissor so nothing again rock paper scissor so again i had uh paper and the ai had rock so we won let's try to lose again rock so we won let's try to lose and there you go so now the ai got one point and this tells us that our system is working properly so did we miss anything so far i i think that is pretty much it now if you wanted to uh show the result as well okay the player one one point uh the ai one one point or something like that you can do that and based on this you can always make the ai win so you can check what did the player put and then based on that instead of a random number you can decide the opposite the winning number for that and that is kind of a cheating but again if you wanted to do that you can do that as well so this is it uh for our game i hope you have learned something new it is a very uh exciting and playable game you can change your own designs you can add stuff to it some background music some buttons and ui design and you can even export it to a game so if you want to learn all of that we have a course that will teach you how to create all these games on a professional level with buttons ui animations and all that so this is somewhat of a basic idea but over there we will take it to the next step we will even use physics and that is of course a paid course if you want to take your programming skills for gaming to the next level uh that is a perfect way to go because there's some not a lot of tutorials out there that are related to game and computer vision so you might get computer vision uh separate tutorials gaming tutorials separately but uh there's only one course that combines them both and we provide that in our series on platform so this is it for today's video i hope you have learned something new if you liked the video give it a thumbs up if you loved it share it with your friends and comment below what would you like to see next and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 49,818

Keywords:

Id: k2EahPgl0ho

Length: 53min 50sec (3230 seconds)

Published: Sun Jun 26 2022