L-4 Use OpenCV with CSI Camera, USB Camera and RTSP streams

hello everyone my name is arohi and welcome to my channel so guys in this video I will show you how to use opencv on Jetson Nano so we will learn how to work with images videos and then different cameras like how to work with CSI camera and then USB cameras like your web cameras and then how to process rtsp streams these kind of things we are going to cover in a today's class so let's begin so uh the first thing is we need python installed on our Jetson Nano so guys there is no need to install this python manually because when you boot your Jetson Nano for the very first time and the jetpack which you have used in that jetpack SDK there is already a python version okay so in my case I'm using a jetpack SDK 4.6 so let's see the python version which we have in Jetpack SDK 4.6 so open your terminal and now we'll see the python version the python version is 2.7 but we don't want to work on 2.7 so let's see the Python 3 version so the Python 3 version is 3.6.9 so this is the default python version on Jetpack SDK 4.6 and guys this Nvidia they recommend to use the default python version so we will not upgrade it okay so otherwise uh suppose if you want to work with some other python versions like you want to work with python 3.8 then you can install Jetpack SDK 5 okay instead of four install Jetpack SDK 5 then you will have it you will have that python 8 3.8 all right so for today's class a python version is 3.6.9 after having this python now the next thing is to install the text editor where we're going to read write a code so the uh text editor which I'm using is G edit you can simply install it by sudo apt get and then install G edit okay so once you install the text editor the next thing is to install pip okay because pip is a package manager and we will install all the other packages using pip so first update your um apt get sudo apt get update and then execute sudo apt-get upgrade by after executing these two commands then you will write a command to install the PIP so we want to install the PIP for Python 3 so we will write sudo APD get install Python 3 hyphen pip okay so now we have Pip installed now guys this is our optional step but I always work in a separate work environment so I will create a separate environment and if you wish to work on a separate environment then you can follow these three steps otherwise uh you can leave the next three steps okay so I am installing the virtually Envy first uh and then using the virtual EnV I'm creating a separate in BIO environment with the name of EnV after executing this command you will get a EnV folder and that will be your separate environment all the packages will be installed in that folder okay so now you create the environment let's activate the environment by writing Source then the environment name which is EnV then bin and then activate now our environment is activated so guys after that we need to install opencv and if if you want to see guys as I've told you in the beginning that there are some default packages which you get with your jetpack which you have installed okay so for example Python 3.6.9 and the python 2.7.17 these two are already there in Jetpack SDK 4.6 in the same way opencv is also present by default in that jetpick SDK but I don't want to use that opencv because the uh that opencv was uh you know giving me some errors while working on object detection models so I upgraded the opencv version okay so if you want to check your current python this opencv version then you can execute this command and you can see that the by default this opencv version is 4.1.1 okay and this was as I've told you that this particular um version was giving me few errors so I have upgraded it now I have python 4.5.1 all right now if you want to learn how to install opencv step by step then you can follow this video and Link is also given in description section you can check that video in that video I have explained step by step how to install opencv which is compatible with good also okay so now now in zooming that you have a python and you have a separate environment and you have opencb installed in your PC now let's execute the programs one by one okay so let's start with the first program which is this this one let's open this program and see what is there so here you can see first we are importing the opencv module and then we are checking the version now in this line you can see that I'm using cv2.im read I am read means to read the image which image you want to read the image which is present in assets folder there is one me image with the name of this okay now I've read the image now I want to show that image display that image so for that we use cv2.im show okay and this is the variable name which we have used above to read the image and then we are using this weight key now this weight key why we use it because guys if we'll not use it then your window will open you the image will open and it will close suddenly okay so if you want to display the image and if you want to want that that window should open for some time then we provide some time over here 0 means infinity and support if you write thousand over here that means one second five thousand means five seconds so this is the time for how much time you want to open that window to display the image okay so let's execute this program now how to execute this Pro program so guys I have told you that I have created a separate environment okay so this is my separate environment I have activated the environment now all my codes are there in one folder let me show you that folder okay so you can see I have created one folder with the name of python underscore examples you can create one folder inside that folder I have my code which I'm going to show you today okay so we are running this first one okay let's run it so let's enter in the folder first python examples now you can see all my codes are here first I'm going to run this one okay so we'll write python then one underscore testing opencv dot py let's execute it so you can see the python version opencv version is this and this is the image so this is the image we have okay so we have successfully executed our first program now let's move to the next program so now the next program is let's go here now let's learn how to work with videos Okay so let's open this second code here now you can see that I have imported the required modules after that I'm using CV2 dot video capture video capture is used whenever you want to work with videos or cameras right then we give provide the source over here so right now we are working with the video so I'm giving the video path inside our asset folder I have this video okay now in the next line we are checking whether uh opencv is able to read that video or not okay so for that we have used this cap is open false false means not able to read okay so if uh opencv is not able to uh read the uh this Source then it will show this error otherwise while why we are using while loop because guys see videos or cameras gives us a series of frames okay video is a series of frames so we want to read all the frames one by one until we reach the last frame okay so that's why we are using a while loop okay and then here you can see the cap dot read so this cap dot read will provide you know a Boolean output like true or false so true means that it is able to read the video frames false means it is not able to read the video frames Okay so after this line if it is true means if if opencv is open um able to read the frames then CV2 dot I am sure using this we will display those frames okay and if in between you want to you know exit from that window then you can press q and you will be exit Okay and otherwise it will work until you reach uh the end the last frame after the last frame this will uh break the loop will break okay so after that we are releasing the video capture object which we have created Above This is the video capture object which we have created right so once you see the whole video displayed the whole video after that you will release that object and then we will destroy that window so now let's run this one okay so where is okay here is it so python to underscore this one so now it will show you a video see it will display you a video it will show you all the frames and let's suppose if in between you want to exit from this video then press q and you will be exit so this is how you work with the videos okay now the next thing is to work with cameras okay so we are working on Jetson Nano so the Jets with Jetson Nano we can use uh the CSI cameras also I'll show you how to use that first only okay so okay let me close this and then this is the code of that yeah so uh now guys uh over here you can see that I am using one module Nano camera so this Nano camera is a another module and what it does is okay so yeah so this is a camera interface and using this camera interface you can easily access CSI camera USB cameras rtsp streams and it is based on opencv so you can access you can use opencv with it okay so how to install it so if you want to use this Nano camera module so first you have to install this module so how you can install just pip install Nano camera and this module will be installed okay so now once you install it you can see here that here this is the line what we are doing is nano DOT camera we are creating the camera instance and inside it these are the parameters we have provided okay so the this is how we uh create the CSI CSI camera object okay so you will write this and after that over here we are printing just printing whether the CSI camera is ready or not so camera is ready we'll provide you that and while camera is ready then what we want we want to read the frames of that camera okay so once camera is open camera is ready what we want we want to capture the stream of it so we'll capture the every frame of that stream okay so that's what we are doing over here using this read method and once you read the frames we want to display the frames and for that we are using cv2.im show and we are displaying all those frames and guys if you want to exit you can press Q you will exit and once you um done with it you can release the camera object and then you delete the camera object all right so let's execute this one Okay so okay let me open the terminal go to the python examples and after that uh our example is three underscore CSI okay so this is the command now you'll see here you can see okay let me set it see you can see the frame is working okay right so guys this is how it works now press Q okay so you can see this is how you can use CSI camera now let's see how to use the USB camera so for that what you need to do is just change this line we will comment this line and now we'll add the code here okay just give me a second so so this is the line which I have added yeah so you can see we are again creating the camera object okay camera instance but this time over here we are providing camera type 1 and device type one so remember whenever you are using Nano camera interface then you have to provide the camera type equals to one in device type equals to one whenever you want to use the USB camera okay and then this width height in fps okay now that's it now we will save it and we will execute it again now when you will execute it you will see the output of USB camera so let's see here okay so this is how you can use oh this USB camera now the next example is rtsp stream okay now I will show you how to use rtsp stream and guys for that I have decided to okay before moving to that part what I want to show you is just a second please all right so over here so the Nano camera module which I'm talking is it is uh freely available at GitHub and you can check it let's go through it okay so these are the feature of that Nano camera and guys this is how you can install it okay so for that guys you can only work with Nano camera if opencv is installed and how to install opencv that I've told you to check that separate video which I've done on that install opencv first then install this Nano camera and there are two ways to install this Nano camera one is this method and other one is this manual process okay and then you can work with these are the different uh things they have explained you can use multiple CSI cameras also this you can explore from here and for the USB camera as I've told you use the camera type 1 and device ID is equal to one if you're working on USB camera now the rtsp stream I'm not showing you the rtsp stream through this because I I think that let's try some other way to process the camera use the camera so I'll show you how to work on rtsp stream using gstreamer okay so but before that just see if you want to work on rtsp stream then you only have to make the camera type equals to 2 and this is how this is the rtsp stream but while providing the location of this rtsp stream and um uh like this in a variable there is no need to write rtsp colon and the double slashes okay just give the path like this IP IP cameras path okay and then you create the object like this and Camera type should be two and the source should be the variable uh variable name where you have your IP of rtsp stream okay so this is how you can use this um rtsp on it okay now let's see the third one okay the how to use uh how to access rtsp stream using gstream Okay so let's see that let me close it first okay okay let's close it and let's close this also because so many windows are here close this as well now what we want to try is this testing rtsp okay let's open it here so guys first let's understand this code then we'll execute so this thing over here these lines okay this is a g streamer pipeline so guys uh I to in this class I am not going to explain you what is gstreamer in detail uh I have a separate lecture on that in that I'll cover the depth things today uh basic idea what is it and then we'll execute the program okay so this is a g streamer pipeline so basically G streamer is a frame a framework multimedia framework with which you can perform various tasks on audios and videos and you can uh perform so many computer vision tasks using this and in gstreamer we create pipelines so pipelines are simply the arrangement of plugins how you want to you know it is just a uh how you arrange all the plugins of gstreamer so that you can data can go through it and and um the plugins all the plugins will do their work so in gstreamer we have different different plugins and all the plugins have their different tasks so over here as you can see this rtsp Source this is one plugin of gstreamer and the task of this plugin is to read the to get the input okay this is a input source so we are basically what we are doing here so uh we are reading the rtsp stream and this location and latency these are the two properties of it locations simply means over here we have provided the location of the rtsp stream and this is the latency Okay so this is the plugin of gstreamer and then we are using the two properties of it and then we have a QQ simply is a it is a it is a basically a memory a buffer where this will this plugin will store its um data and from here the next plugin will pick the data okay this is what queue does and our next plugin is this RTP h.265dp now after that we have this plugin and then we have this plugin so we are linking these three plugins over here so what we are doing is this rtph 264 or 65 what they does is they extract the h265 um video from RTP packets and after that this h.265 Parts what this does this is a video parser G streamer plugin and the task is to parse the h.265 streams okay and then this is our next plugin the task of this plugin this is a um this basically performs the color format conversion okay and then we have this plugin this plugin and this so over here what we are doing is we are providing the data which is in this format to a video convert plugin and now we want to have um data in BGR format because opencv accepts the data in BGR format okay so that's what we are doing so after that finally we will get the data Okay so what this whole plugin does now this this is a pipeline of gstreamer now what it does it reads our tsp stream then it perform all the functionality which is required to perform on that rtsp stream in order to provide it to opencv so that stream should be that rtsp stream should be opencv ready so we approve you know performing all those processes in between and finally our rtsp stream is ready to which opencv will accept Okay so this is what we have written over here and after that over here you can see that CV2 dot video capture as I've told you that video capture will whenever you're working on videos or cameras you use this video capture object right Source what this source is this Source the pipeline which we have created and this Source I have now rtsp stream which opencv will accept and after that here you can see so guys uh I have written the pipeline like this also there are different ways to execute the pipeline you can execute the pipeline like this also and you can execute the pipeline like this also okay so while true me what we are doing we are CV2 dot video capture we are reading the source and if stream is opened then okay if not if stream is not open then Source not available so testing rtsp let's execute it python so this program is what is the name Force underscore testing rtsp okay okay now so here you can see the rtsp Stream okay this is our rtsp stream and now if you want to close it just to control C over here and your rtsp stream will be closed so guys this is how you can use different um ways this is how you can use opencv uh with different different sources I hope this video is helpful thank you for watching

Channel: Code With Aarohi

Views: 14,279

Keywords: JetsonNano, OpenCV, Python, Python3, AI, ArtificialIntelligence, DeepLearning, MachineLearning, NeuralNetworks

Id: o-gOseYQN2I

Length: 26min 44sec (1604 seconds)

Published: Wed Jan 18 2023