Accessing USB Devices and Webcams with OpenCV and Python

can see our window's up all running successfully if i go and hit q closes it again what's happening guys my name is nicholas renate and in this tutorial we're going to be going through how to access your webcam using python and opencv so this opens up a range of possibilities when it comes to performing computer vision using devices connected to your computer now before we take a look at what we're going to be diving into i just wanted to say a huge happy diwali to anyone that is celebrating so without further ado let's take a deep look at what we'll be going through so in this tutorial we're going to be very much focused on how to access your webcam with opencv now this opens up a range of capabilities particularly say you wanted to do something along the lines of pose estimation or face tracking this means that you can literally just plug your webcam into your computer and do some computer vision in real time so it's pretty pretty cool in that sense now there can be a couple of sticking points when it comes to getting this set up so i'm going to walk you through how to actually do that so specifically we're first up going to take a look at what dependencies we need then i'm going to show you how to connect to your webcam and how to find your video capture device number so you might need to iterate through this to find the appropriate capture device and if you don't get this right you're not going to get a feed from your webcam so we're going to set that up and then i'm going to show you how to get a real-time feed so that you can actually do a little bit of computer vision which is going to be super important once we get to some of our future tutorials ready to do it let's get to it alrighty team so we've got three key things that we need to do in order to access our webcam or three key things that we're really going to be focused on the first things first we're going to focus on how to import our dependencies or import the dependencies that we need you probably already know how to import dependencies by now then we are going to hook into our webcam and i'm going to sort of show you how to walk through the sort of debug if you get any errors so effectively you really just need to try out a bunch of different device capture numbers and then we're going to take a look at how we can render in real time but before we do that let's jump over to the whiteboard to see how this is all going to work alrighty guys so the core focus of this particular video is accessing our webcam with opencv now whilst we're accessing it with our webcam this could really be any usb capture device so in this particular case we're going to be doing a webcam but it could just as easily be a capture card so say for example you're streaming a feed from your ps5 or your xbox you could definitely do this in the object detection tutorial or the mega course that we've got on the channel we actually hooked up a microscope to it microscope and this process or the framework that we used was exactly the same so even though these devices were each different the process is the same process is the same you might get different frame sizes you might get slightly different frame rates but in terms of actually hooking up to these different capture devices the process is going to be the same now this opens up a whole range of capabilities right so with the webcam you could obviously do stuff with your face or you could do stuff with pose estimation with a capture card you could might try to automate playing a particular game with your microscope you might do like disease detection or something along those lines the process in itself is going to be the same set of steps so what we're going to be focused on is first up hooking up to our webcam so let's imagine that this is our webcam and our webcam is connected to our computer so this is going to be a pc could just as easily be a mac or it could be another machine and what we're going to do is we're going to hook in or right now it's hooked in using a usb so imagine that's a usb connection what we're going to do is we're effectively going to be able to access the feed from over here to be able to access each one of those frames using opencv now the way that we do that if you cast your minds correctly when we went and used opencv to access videos we use the cv2.video capture class and then to that what we did is we passed through the path to the video now what we're going to do in this particular case is rather than passing through the path to the video we're actually going to pass through a video capture device number so rather than having the path over here we're actually going to pass through a specific id so let's go a completely different color so in this particular case rather than passing through the path we might pass through zero one two three four so on and so forth the core thing is that you just need to identify what this value is for you so what is it or you now this might change as you actually go through and connect to different video capture devices or as you plug in new stuff into your computer so you might need to iterate through and find the appropriate video capture device but that's fine once you've got it all hooked up you should effectively be able to capture frames from that particular device now once we've actually got that set up the process is actually pretty straightforward because what we're going to get back is a set of frames exactly the same as what we did when we were working with videos and we can individually process each one of these frames or access each one of these frames using cap dot read but as per usual we're going to be going through each step of this step by step so you can see how to actually go on ahead and build this up so on that note enough of this let's get back to the code and we're back okay so as i was saying we've got three key things that we need to do so let's kick it off and import our dependencies okay those are our two dependencies imported so pretty straightforward there so really we just imported opencv by typing in import cv2 again pretty straightforward in what we've covered before we've sort of covered that before already as well and then we're importing matplotlib and really the reason that we're importing matplotlib is so that we can visualize inside of our jupyter notebook now again if you didn't want to do this inside of jupyter you don't have to you could do it inside of vs code you could do it inside of pycharm just makes it a little bit easier to interactively go and code particularly when you're doing data sciencey or computer vision type stuff so the first line is import cv2 and then the second line is from matplotlib so remember it's going to import our pi plot package from matplotlib so from matplotlib import a pi plot as plt so this is importing opencv import opencv for computer vision stuff and this is importing matplotlib so we can visualize an image cool now the next thing that we need to do is actually go on ahead and connect to our webcam now this is going to look really familiar to you if you went through the video capture tutorial because it's pretty much the same code there's one key difference and you would have seen in the whiteboard that rather than passing through the path to the webcam we're effectively going to pass through the device capture number now i've actually got two different webcams connected to my computer at the moment so one that i'm using to actually record this video and then the second one that i typically use for computer vision so what we might need to do is iterate through and find the one that we is currently available because the one that i'm currently recording with is going to be unavailable to opencv so let's go ahead and first up write a line to create a connection so i'm going to write cap equals cv2 dot video capture and then i'm going to pass through a device number now i normally start at 0 and hit shift enter and that establishes our captcha then what we can try to do is actually get a feed from it so i'm going to type in ret comma frame equals cap dot read and remember this is no different to what we did with our video remember we used we created a cv2.video capture class except what we did with our video is we actually passed through the full file path to that video here we're actually just passing through a video capture device number as you can see there right so rather than having a path we've just got a number there and then what we're doing is we're actually reading a frame so get a frame from the capture device device so ideally if this runs successfully we should actually get a return value our ret so it looks like that's true and we should get a frame value and it looks like we've got a frame now i don't know if this is the right video capture device so what we can actually do is we can use matplotlib to visualize that so let's do that okay so you can see there that i've actually got some data back right or we've got a valid image now this is actually coming through a virtual video capture device that i've got set up for my sony uh camera in this particular case that's not the one that we actually want to connect to we actually want to connect to my logitech stream cam so i've actually got one of these stream cam so this is currently what i'm using for my all of my con virtually all of my computer vision stuff so i've got two of these hooked up to my computer at the moment now this feed is not the correct one so we know that's not the correct one so what we'll do is we'll release that feed so we'll type in cap dot release and this releases that webcam device so releases capture back into the wild really just releases it so it's not going to be or it's no longer going to be in use so we're going to run cap.release to release our sony cam and we're going to try a different capture device so let's just write some comments there connect to capture device and let's try a different one so if we tried one let's print at our return value all right so we went and ran cap.read let's print our return value all right so this one's returning false so that means that we're unable to get anything from that video capture device this might be because it's currently in use it might there might not actually be anything under that video capture device could be a whole bunch of reasons but in this particular case we have not successfully connected to that video capture device so if we run the next line you can see that we're getting this error here so image data of d type object cannot be converted to float and that's because we don't actually have anything returned back so if we actually take a look at our frame you can see that there's nothing in there right and that's because we do still don't have the correct video capture device so again we'll run our release now what we'll do is we'll try video capture device two and we'll run cap.read looks like we still don't have anything from that one we'll run release we don't need to go through and do the other stuff let's try video capture device 3. and again let's see what we've got all right so that one's returned true so it looks like we've got a frame back and it looks like we've got some valid numbers there if we go and show this one okay this is looking better so you can see that we now successfully have got a frame from our video capture device and the advantage of using these little stream cams is that you've got this little white dot on them so when they have let me zoom in so you can see that a little bit better you can see that you've sort of got this little white dot and this will only turn on once you connect to that webcam right so if you're not 60 or if you haven't successfully connected to it you won't actually see that white dot turned on so i know that mine is successfully connected because my second stream cam has just flicked on now in this particular case we can see that we've got our frame back if i wanted to release that webcam if i run this the little white dot on that webcam so this has just switched off that little white dot has now switched off as soon as i ran cap.release but that effectively shows how you can access your webcam but this only gives us a frame at a time right so if you're happy just taking a photo from your webcam you could actually loop this all together right so we could create a new function um take photo and we could run uh cv2 what is it cap equals cv2 dot video capture and what was our video capture device so it was capture three and what we can do is take a photo so we're going to write red comma frame equals cap dot read and then let's go ahead and write that photo out so we can use cv2 dot i'm right and we're going to call it um webcam photo dot jpg and we're gonna save our frame so if we run take photo now this should we should also release it as well so cap dot release right so if we go and run this function so that should activate our webcam take a photo and then shut back down so i just saw it activate it's run successfully or it doesn't look like we've got any errors so if we go into the same folder that we're currently working in which is opencv you can see that it said we've got webcam photo there if i open that up that is a photo from our webcam right pretty cool so we've now successfully connected to that webcam but this could just about be any video capture device again microphone if you plugged in um like a video capture or like a video gaming device or like a ps5 or an xbox or i don't know whatever else people using switch gamecube whatever you'd actually be able to access that feed through your computer using opencv so let's quickly take a look at what we wrote in that section so remember to connect to our webcam we can use cv2.video capture and that gives us our video capture data or it connects to that video capture device we can then get a frame using cap.read similar to what we did for videos you're going to get two values back so whether or not it's successfully connected and you'll also get the frame we can then print that out using plot.i am show and if we wanted to recolor this we can use cv2.cvt color and we're going to go through these color models a little bit more in an upcoming tutorial cv2 color vgr to rgb so that will give us the correct color we can type plot dot show to get rid of this line over here right so that allows us to get a capture from our webcam and then we can run cap.release to release our video capture device so that we can use it for other stuff and we quickly wrote a function to be able to take a photo from our webcam so remember all we've done inside of here is pretty much we've taken everything that we learned from up here and we've packed it into a function to effectively write out our photo from our webcam now what would happen if we wanted to actually go and render this in real time so we've now successfully gone and completed section one and section two let's go on about and do section three and render in real time so pretty much we're going to replicate all the stuff that we've sort of done up here we're just going to pack it into a loop so let's actually go ahead and write this loop okay so i accidentally went and kicked that off so we don't need it to kick off yet so i went and wrote three different lines of code so first up what we're doing is we're establishing our video capture device so connect to webcam and then what we're doing is we're double checking we're effectively going to loop while our capture is open so while our webcam is currently activated we're going to continuously loop and we're going to continuously run cap.read so the next line is while so this establishes a while loop so establishes or loop through every frame until we close our webcam right so we're going to loop through every single frame so while cap dot is open so this ensures that a capture device or we allows us to check our capture device still open so if i run cap dot is open you can see that right now it's currently false so it's not open so this while loop will close so while it's open so while this is actually true and if let's actually just uh set it to true right so if i write cap equals cv2 dot video capture three right so that's going to connect to our video capture device then if i run cap is open you can see it is now true right so while it is actually opened this is going to read true if i run cap.release and runcap is opened you can see it's now shut off right and that allows us to check whether or not a video capture device is currently connected so what we're doing is we're effectively checking that our video capture device is still open and then we're going to run this line over here so cap.read so we're going to unpack the values so we're unpacking the return value and the frame and we're doing that or returning those values from cap.read what we can then do is we can show our image to our user so show image and we can do that using cv2 dot i am show and again this is similar to what we did in the video feed and remember if you wanted to go and write this video out you could do exactly the same we showed that in how to deal with videos inside of that tutorial but for now we're going to show that video so webcam we're going to call it webcam so this first value is what we want our frame to be called the second value is the frame and then we're going to shut it down successfully so so if cv2 dot wait key zero uh let's set it's one and zero x f f equals equals ord q break and then write two more lines a cap dot release cv2 dot destroy all windows okay so let's take a look at what we wrote there so we use cv2.iamshow and then everything from here on out is pretty much you would have seen in the video tutorial but let me break this down so this line here is effectively checking whether or not we're hitting anything on our keyboard so if cv2.whitekey this gives us a chance to hit a key on our keyboard and this unpacks what's actually being hit so what we're doing is we're checking whether or not a key is being pressed and if the key that's been pressed is q then we're going to break out of our while loop so this gives us a nice way to gracefully quote or gracefully close out of our webcam and assuming we break out of our while loop so remember that is one section there we're then going to run cap.release to release our webcam back into the wild and we're going to run cv2.destroy or windows so this closes the cv2.iamshow frame so this checks whether q has been hit and stops the loop these two lines down here you know this guys so this um releases the webcam or video capture device and then this closes this frame all right let's run this so assuming this runs successfully you should get a pop-up towards the bottom of our screen and we do that is our webcam successfully connected this is a little bit crooked that's a bit better but that effectively shows you what's actually happening so let's take a look at what's happening here so we've connected to our webcam which is what we had from up here we are then looping through while our capture is open so right now our capture is open so we're running this code so we are collecting our frame so we're using cap.read to get a frame at a point in time we then get our return value our frame and then we can show it back and remember how i said the name of the frame is going to be let me zoom in on that so you can see it so you can see it's called webcam up here right and that is because of what we've named it over here now if we wanted to close this right remember we can hit q on our keyboard and this code over here is going to trigger so let me put that side by side so you can see that so if we actually go and hit q this is going to trigger it's then going to release the webcam because it's going to progress out of that loop and it's going to destroy that window so if i go and hit q you can see it's stopped it's released my webcam and it's closed that frame if i go and run it again you can see our window's up or running successfully if i go and hit q closes it again and that in a nutshell is how to work with your video capture device so we went through a bunch of stuff but again we're going to keep this one pretty tight so we imported our dependencies i sort of walked you through how to connect to your webcam and how to find the correct video capture device number a lot of it is just iterating and testing and then we took a look at how we can actually take a photo using our webcam and then last but not least how we can render in real time but on that note that about does wrap it up also happy diwali guys peace thanks so much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a big thumbs up hit subscribe and tick that bell and if you have any questions comments or queries do hit me up in the comments below i'll be happy to help you out thanks again for tuning in peace leds

Channel: Nicholas Renotte

Views: 4,714

Keywords: computer vision, capturing video with python, python capture video, python stream video, python cv2, python laptop camera, computer vision projects, computer vision tutorial

Id: FygLqV15TxQ

Length: 21min 39sec (1299 seconds)

Published: Thu Nov 04 2021