Interactive Games using Computer Vision

hey everyone welcome to my channel in this video we are going to look at how I created this amazing interactive game using computer vision the idea is to replicate a mouse click and from there the possibilities are endless you can create your own games or play existing games the first time I built this game was back in 2017 but that had a lot of limitations like lighting environment and you could only play on a white background and so on but this latest technique using AI we don't have any of those issues so let's go ahead and have a look at this Amazing Project let's have a look at the brief history of this project I first created this back in 2017 for a bank that wanted to use it for their booth in a maker Fair event back then I used C plus to write the code and it took me about one week to complete the project the idea was to detect the balloons using Color detection and then detect a circle inside the balloon then in 2021 I created this game again this time using python it took me around 12 hours to get the back end working and a few more hours to add some music and animation this project was documented and I shared a video of this as well on my YouTube channel here the game was exactly the same but the balloon detection was made using Contours rather than colors and the circle detection was the same now this worked well but it had three major flaws one was that you cannot change the game or the design so it had to have a white background and the same shape of the balloons so that they would be easy to detect second were the Shadows they were sometimes detected as the ball head so even if the ball was not hitting the balloon if its shadow would hit the balloon then it would be considered as a hit the third was the lighting conditions so if the lighting conditions were altered then the detections were affected therefore you had to fine tune the parameters each time which took a lot of trial and error on the spot but now that AI is maturing quite fast we can utilize it to create this game with better results and more flexibility so let's have a detailed look at how this project works the project consists of three main Hardware components a camera or a webcam a projector and a PC or a laptop the camera is used to grab images of the playing Arena and send those to the computer the computer then processes to find the location of the ball once the location is found it sends an output to the projector to display the game here we can create our own game or we can use an existing game on the internet or even the ones that are offline now let's have a look at the complete steps in the first step I collected the data of the ball for training here I use the same environment in which I would run the game to make it easier for the system to learn and detect the Second Step was to label the data this is a slow process as it takes a while to label all the images and I got quite tired at the end but you can always hire someone to do it for you as well I actually watched the Spider-Man 1 while labeling and by the time I finished it was done the next step is to train the model here I finally took some rest after all the labeling and the training took about two hours to complete and the model was now Ready Step number four was the moment of truth so I tried the model and to my surprise it worked very well the locations were detected with good accuracy but still there was an issue with missing detections so I changed some parameters and retrained the model and after a few attempts saw some significant Improvement in the results now in order to use our ball at the hit point we had to know where the projector is projecting to get the complete area of the project and to remove any angles that might be in the camera feed I created a calibration script this script has to run only one the camera is fixed for the first time now was the fun part creating some games I decided to create a single script that would contain four different modes the first mode is drawing circles where the ball hits this is the most simple one and can be used for testing if the environment changes the second mode is drawing an image where the ball hits the idea is to import a PNG image with transparent background and overlay where the ball hits to demo this I used a crack image to show that the hit of the ball is causing the wall to crack mode 3 is a football game created within the opencv environment here we have a goal and a Target region you can use your hands or your legs to hit the target once the target is hit it moves to a new location and the hit is counted as well this type of games are very easy to make using opencv now advanced games with animations and sounds can also be created using the integration of Pi game and opencv this allows to create much more interactive games with animation physics and much more I have a complete computer vision game development course starting from the very beginning right up to creating exciting games the link will be in the description the final board was where you can simulate a mouse click this opens up a lot of possibilities since you can play and utilize existing games here you can see a pest game being played where you have to hit the insects this game did not require any extra coding and is simply running with the existing Mouse click mode similarly you can use it with other online games such as Fruit Ninja balloon pop and more this whole project took me about three days to complete I have created this as a simple project on our website with all the details on how to create this project including the files code and explanation of the code as well it will take you from the very beginning right after creating this complete project and you don't have to be an expert to follow it is designed for beginners if you like my channel and would like to support it then you can purchase this project from our website this helps us pay our bills and we can continue adding more free tutorials to our 200 plus free tutorial collection when I sold this project in 2017 to the bank the cost was in thousands but don't worry you can get this in just a few dollars this project includes all the details including how many images to take for the training process which model to choose and the parameters to get better results in the training and how to set up the projector and the camera and much more so this is it for today's video I hope you have learned something new if you like the video give it a thumbs up if you loved it share it with your friends and I will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 47,532

Keywords:

Id: Im9M-S_UZsY

Length: 6min 26sec (386 seconds)

Published: Tue Aug 01 2023