Background Removal Like Zoom | OpenCV Python CVZone

hey everyone welcome to my channel in this video we are going to learn background subtraction using the media pipe library they have recently released this model and we are going to use this to do background subtraction so as you can see it looks pretty good on the left hand side you can see we have the original image on the right hand side we have the image with the alternate background and if i press the d button it changes the background in real time and you can see the frame rate is quite good it is around 40ish frames per second it goes even higher and we can keep changing the background as we wish so right now we have three backgrounds as you can see the index at the at the bottom you can see it is changing to up to two but check this out if i close this and if i bring in the folder which has these three images and i simply drag in seven more images now if i run it again it will have a total of 10 backgrounds so now you can see we have a lot more images we have a total of 10 backgrounds and it automatically detected all of these and enhanced our program and now we can click and we can change the background in real time as we like so we are going to learn all of this and all of this is written pretty much let me show you in literally less than 40 lines of code so stay tuned and let's get started so the first thing we will do is to open up pycharm and we are going to create a new project and let's call it background remover you can name it anything you want we will create that so this is our new project what we can do is to remove all of this code it already has a main.pi file so we do not need to create a new file so what we will do first is go to file we will go to settings and then we will go to our project interpreter and then we will add our packages now the first package that we need is the cv zone package so we will click on cv zone and we will install that and this will automatically install opencv for us and numpy as well then the main library that we are using today is the media pipe library so the media pipe package has a new model the selfie segmentation so we will be using that here you can see the result so if we go down they have provided us with the code now if you want to use this again and again this code might be a little bit too much to repeat for every project so what we have done is we have created a module within the cv zone package which allows us to do this within a few lines of code maybe two three lines and we will be able to remove the background and add our own images to it so that's why we will be using the cv zone wrapper package and then we will install the media pipe media pipe and that is pretty much what we need for this project so we can close this and we will wait for it to install so now the libraries are installed and we can go ahead and start writing the code so first of all we are going to import cv2 which is the opencv package and then we will import cv zone and then we are going to import the selfie segmentation module so we will directly import our class so we will say from cv zone cv zone dot selfie segmentation module import selfie segmentation so we are calling it selfie segmentation because this is what they uh name their model and then we will also need os so we will use that later on to get our images so the first thing we will do is we will open up our webcam so right here cap.cv2.videocapture so we are creating the object here and then we will set the frame to 640 so the width will be 640 and the height will be cap dot set the height will be 480 so 3 is for the width and 4 is for the height so once we set that then we are going to display it so we will go down we will write while true we are going to uh get our image so we will write success and image is equals to cap dot read and then we will write cb2 dot i am show and we want to show our image and we want to show the actual image this is the name of the window this is the actual image and then we will write cb2 dot weight key and we will give a delay of one so this is one milliseconds so this should open up our webcam and it should run it so let's try it out before we go any further to see if everything is working there you go so we can see that now the webcam is running so we can go ahead and write the code for our background removal so the first thing we need to do is to create our selfie segmentation object so what we will do is we will write here that our segmenter is equals to selfie segmentation so here we have the option to choose the model number so based on their documentation if you have the model number zero it is a generalized model which will be a little bit slower if you have model number one it will be for landscape which will be a little bit faster so we are using model number one by default if you want to change you can use model number zero as well so then the next thing will be to run our detector or uh not the detector the segmenter so we will write here that our segmenter dot remove background and in that we are just going to give in our image and then we have the option to give in one is the color and the second is the image so if you give a color it will put the background as a color if you give an image it will put it as an image so let's say we will put here 255 0 and two five five so it will look really bad but it will give us a purple background so let's try that and then we will also write we will output an image out and we will print it out so let's copy this and we will write image out and here we are also going to write image out so let's try to run this and see what happens there you go so as you can see now it is very easy to remove the background you just have to write a single line of code and now you can see there is a purple background if you wanted to change it to any other color for example we want it blue so then yeah we will write it 0 and then we will write 0 here as well so if we run that so now it should be blue there you go so now the image is blue now one thing we can do is that on the side if you see that it is not cutting properly you can give a threshold here so there is a option for threshold and the threshold value can be for example if it is one it will cut everything so it will be a complete blue color image now uh they have recommended a value of 0.1 so it will not cut that much but let's try something like 0.8 and then it will cut quite a bit so it will not show the edges that much so let's try that and there you go so now the cutting is much less than before and if we go even further let's say nine 9-5 then it will cut even further and it will give us a little bit better result but if you move your hands a lot then your hands will get cut as well you have to make sure to find that balance so here you can see now the seat is getting cut a lot over here as well so you need to find that balance let's say we will keep it at 0.8 for now and later on we can see if we want to change it so the next thing uh the thing that i don't like is that these two images are separate so what we can do is we can write here that our cv zone dot stack images and we want to stack our image and image out so that's how simple it is we want only two columns and the scale we want it exactly as it is so we will write one so and we can save it in image stacked and then we can simply output the stacked image and we can remove this part so it will display both of them together so this will this way it is easier to play around with it there you go so now you can see both of them are side by side so it is easier to run it and before we go any further let's go ahead and look at the segmenter so if we click on selfie segmentation this is the module that has been created using the code that was provided by media pipe so here you can see that we are initializing in the init function and then we have the remove background method which basically converts the image into rgb and then it sends it to the model selfie segmentation to process and then it we simply use this functionality to see if it is a color then it will change the color of the background if it's not a color then it will change the image so this is how simple it is so you can go ahead and look further into it they have given the documentation of this as well and they have also explained it with a little bit of comments but if you just want to run it then this pretty much is what you require this and this these two lines of code and that is pretty much it so then the next step what we will do is we will write the frame rate so that we can see how much frame rate are we getting so here we are going to write that our fps reader is equals cv zone and we are going to write dot fps so this will help us write the fps on our image we don't need to write or draw anything simply we will say that our fps reader dot updates and we will send in our image on which we want to display so this could be actually let's do it below over here and then we can put it on the stacked image and then it can return us the stacked image and it will also return us the fps so we don't need it but we can store it in a variable or we can just put underscore if we just want to neglect it so then it should display us so let's see how it displays you go so it is displaying this frame rate here it's around 30 frames per second but uh it's not very visible um let's change the color for it so we will write here color and we will change it to 0 0 and 255 so make it red there you go it's a little bit a little bit clearer more clearer but still it's a little bit hard to read maybe if we push it here it will be easier but then we have to add the backgrounds so it might change the color so let's just keep it here one more thing i want to try i haven't tried that out let's try to increase the frame rate so the camera i'm using is goes up to 60fps so let's write here the prop id so let's write it as cap let's write cb2 cv 2 dot cap underscore prop underscore fps and we will make it 60. so let's see if it increases the frame rate so well didn't have much of a big effect i i can see it's more towards 40ish now earlier it was more towards 30ish so maybe a little bit bump we can see okay so then the next thing we will do is to how to add images so i have a lot of images you can download this from uh our website computervision.zone the link will be in the description so what we will do is we will show in explorer and then we are going to create a new folder here and we will call it images and within these images we are going to add a few images at first we will add three later on we will add some more so these are the three images that we added now one thing to note here is that they are exactly the same size 640 by 480 and this is what we are using here so they have to be exactly the same size otherwise they will not work so if you just want to have one image how can you do this it's very simple all you have to do is you have to import that image so what we will say is that image background is equals to cv2 dot im read so it will read the image and we are going to say read it from the images and read lets say one dot jpg so once we read that we will just copy this and we will paste it instead of the color we will paste an image and that's pretty much it so if i run this now it should show me the new image rather than a color and there you go so that's how simple it is now it is showing us this background image and it looks pretty good it looks pretty decent and if you are looking from you know far away you might actually think i am sitting in that house which is pretty neat by the way and so we can try out different images we can change the background but what we want to do is we want to change it while we are running the program so how can we do that it's very simple we will need first of all os and then we need to list down all the images that are in this folder so we will write here that our list for our images is equals to os dot list directory and we want the list directory from the images folder images folder so if we print that out so let's print out what was the name list image so let's run that there you go so it is giving me all these names of these images so we can use these names to import all these images like this and we will store it in a list rather than a single variable so what we will do is we will write here that this is our image list is equals to empty and then we will say that for image path uh in list of our images we are going to write image is equals to cv2 dot im read and we want to read the image within the images folder and then slash the path image path so it will take this part so we can write here image path so one by one it will uh import these images and then we will put them in our list image list dot append and we can write here image so at the end of the day we will have three images here so if i print this and we print the length of our we print the length of our image list it should give us three so let's run that and there you go so it gives us the value of three so next what we can do is instead of writing image background here we can write here image lists at zero image list at 1 image list at 2 and so on so instead of doing that what we will do is we will keep changing this value based on how many times we press the button so we will call this value uh let's say index of the image and let's say it is zero by default so if i run this now it should give me the same image as before and even if i remove this part it should work fine i remove the importing of the single image so even with that it should run fine so there you go it is using that image from the list and then it's displaying this so that's pretty much uh good now what we need to do is we need to keep changing this value if we click a button so how can we do that we can put this into a key so we can write here key is equals to this and down here we are going to check what is this key uh if key is equals to ord which means that we are clicking the a button then we will write that index image minus equals one for a we are going to write minus for d we are going to write plus and then i will copy this same thing and we will paste it two more times and here we are going to write if it is d then we will make it plus one and if it is q we will break the loop we will exit uh but all the rest of them should be else if so we will write here elsif and elsif and what to check if we are heading in the right direction we can simply print the value of the uh index image so that we know uh how are we moving so let's run this okay so now the the image is used as zero and if i press the d button it will go to the next image and if i press d again it will go to the next i can press a to go back and then press a again to go back but what happens if i press a again it will go to minus one then it will go to minus two and then it says out of range so there is no minus three so it will not go there so we need to make sure what happened here uh we need to make sure that it doesn't exceed the limit that we have so for minus we are going to say we need to check first if the image index is greater than 0 then only perform this if it is equal to 0 or if it's less than that then it will not do this because it cannot go below zero and the same thing we will do here we will check if the index image is less than the length of our image list -1 if that is the case then we will keep adding if it's equal to this we will stop we will not add more to it because that's the limit so let's run that and see if it works okay so i will press the d button it will go forward forward and now i'm pressing and it doesn't go any further and if i go back again i'm pressing but it doesn't go any further it doesn't go back so this is good it means we can add images and it will automatically check how many images are there and it will accommodate that figure now to test it further what we can do is we can take this folder and we can add more images to it so i will drag seven more images here and now we have a total of 10 images so we should be able to have 10 different backgrounds now we do not need to change anything in the code we simply run it again and it will automatically detect that there are 10 images and it can go up to 10 and it can go up to 0. so it will automatically do that and we should be able to move around so let's try it so look at the index below so it is at one then two then three then four five six seven eight nine and there is this so it can go up to nine and then we can go backwards and you can see all these uh backgrounds they are changing personally this is my favorite one because it looks natural it's like it's looking like i am sitting in that home well but if you look at the detail it's not that good but still again you can add a little bit of blurring to the edges and you can do a lot of different things to enhance this to make it look more original more natural so you can take it as far as you wish uh this is like a blackboard so again all of these images are available to download from my website again you can use your own images it's not a big deal but if you are a bit lazy you don't want to resize the images you can go ahead and download it from the website and the code will be also available on the website so you can download it from there so this is it for today's video i hope you have learned something new if you like the video give it a thumbs up and don't forget to subscribe and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 24,705

Keywords:

Id: k7cVPGpnels

Length: 23min 36sec (1416 seconds)

Published: Fri Jun 25 2021