C# Image & Video Processing using EmguCV

okay so welcome back um today we're going to start talking about image and video processing and specifically we're going to be talking about c-sharp Visual Studio Windows forms we've done a lot of videos on this channel and we're going to be focusing on a library that we've already started discussing previously called emgu CV and we're going to talk about some of the many many features it has that allow you to do many many things now in the past we talked about using emgu in another Library called a forge and it allowed us to do kind of what you see here on the left which is capturing video coming from a smartphone you can see I've got it here mounted on a window I'm using a smartphone over Wi-Fi and it's capturing that video that's streaming video and it's showing it on the left we showed you how to do that in previous video in the future we're going to start looking at some of the many other capabilities that the same library emgucv has and you can see here we're taking the streaming video and on the right hand side we're processing that and we're using what's called motion detection feature in EMG ucv and we're going to be talking about how to do that c-sharp Visual Studio Windows forms we're also going to show how to do individual image processing using your GPU and here's an example of an application we're going to do in the future it's a very large image of 20 million pixels and we're going to take this image as noisy image we're going to filter it using our GPU and you can see it takes only about two and a half seconds if we used our CPU would take us over a minute to do this so we're going to show you how to use emgucv to do that and we're also going to extend that to show you how to do GPU processing of your video so um in this video we're going to look at some of the details we're going to start to understand emgucv and the libraries that it allows you to access which are opencv and we're going to see what are the basic concepts and how does how does this work it's really going to be helpful to understand that before we get into motion detection and video and image processing so I really encourage you to stick around for this video to see some of the basic concepts behind opencv and emgucv that we can use to develop these applications so now first of all what is emgu CV well I encourage you to look at emgu.com and it's got some information about emgu and what it is and here on the main page it says emgucv is a cross platform.net wrapper for the opencv image processing Library so it allows you to access opencv in a Windows forms.net framework Visual Studio C sharp application so it allows opencv functions to be called from.net compatible languages the wrapper can be compiled by visual studio and unity it can be run in Windows Linux Mac OS IOS and Android and at this emgucv.com it has some fairly recent releases here in April of 2023 October 2022 so you can see it's still being updated on a fairly reasonable basis and here you've got some information there's a tutorial now one of the things you need to know about emgu CV is again it is a wrapper for opencv which means what it does is it allows you to access the opencv libraries which means that emgucv really is probably not going to have a lot of information the the core of this is opencv so one of the challenges is you're going to have to not only be familiar with the mgu CV but also the underlying opencv libraries so emgu.com has some Basics on emgucv I also encourage you to go to docs.opencv.org slash four point x and that shows the latest at the time of this recording the latest 4.7 version of opencv and it's got a little bit more detailed tutorials also to keep in mind that opencv is a C plus plus Library so um the opencv tutorials are going to be kind of based on C plus plus or python so it's a bit of a challenge to understand what opencv is doing and how that's implemented in EMG ucv but you can see main modules image processing image file reading video i o video analysis camera 2D features object detection neural networks machine learning it's got a whole lot of stuff and it's also got some extra modules so this is being developed for many years and it's got a lot of stuff we're going to talk about some of the the core aspects of this so we can understand how to do your application now there is also a very nice educational website called learnopencv.com and here's getting started with opencv and it's got some opencv for beginners it's got some tutorials again these are going to be probably in C plus plus or python so you're going to have to do a little bit of conversion here but it shows some of the core functionality now also keep in mind that emgucv uh has been around for a while and presently in middle of 2023 at version 4.4 4.5 um and this has been around for a while so you're going to see a lot of documentation that might be five years old or older that really doesn't apply completely so um you got to be careful when you see stuff videos on the Internet or you see even documentation on the emgu website some of the stuff can be old and not really appropriate for the the latest version so you got to be careful about that and the same thing goes for opencv so I encourage you to look at this download and installation for emgucv um it's got it for the earlier versions as well as the latest versions now one other thing to know is that opencv and emgucv allows you to use your Cuda based GPU for calculations we're going to show you in future videos how to do that so it's not just CPU based it's also Cuda based which is really nice so um I encourage you to look at the emgu.com download an installation section and it shows you the steps how to do it we'll go through that a little bit later but you need to download some new get packages um emgu.cv.runtime.windows and if we're going to be using Cuda we've got to have emgu.cv.rentime.windows.cuda [Music] so we'll go through the setup process in a bit but first I want to talk about some very basic Core Concepts you need to understand when you're working with emgucv or opencv the most important concept here is what's called an emgu CV map or Matrix and it's the way they describe an image or one image out of a video and you can see that we've got a 640 pixel by 480 pixel Matrix and that is describing each image or each image in a video it's basically an array of pixels however it's not just a two-dimensional array because we have to specify the color of the pixels which is r g and B values so what it becomes is what's called a byte array that in this case is 480 by 640 by 3. so actually it becomes like three two-dimensional arrays one with red values one with the green values one with the blue values for each image so it's a byte array of 480 by 640 by three so that is the emgu CV Matt The Matrix that describes each image and the red green and blue values will be between 0 to 255 since they are each represented by an 8-bit byte which can have 256 total values from 0 to 255. so that is the core component of the emgu CV image and video processing you're working on a mat when you're capturing individual frames or video as we're doing here you see we've got a webcam which is basically a smartphone feeding a camera video over Wi-Fi and we talked about before we're using the erian webcam application on the phone and also on the computer to allow it to grab that video and we did a video on how to do that so emgucv is grabbing from the irian webcam application grabbing that streaming video into these matrices and each time it grabs an image it forms this Matrix of 640 by 40 by 3. and also you specify how many frames per second it's going to grab and that depends on how you set up the webcam and other stuff but basically the video capture is going to be defined by that and then what we can do is we can save that video as a video file which means we have to use a codec or a compress and decompress library that basically takes this uncompressed RGB values and uses some logic to compress it down into something smaller that can be operated a lot easier on your computer now there's some other very important Concepts we need to understand when we are grabbing video or grabbing a frame and there are one two three four methods that you will see in using emgucv that you need to understand what they do and they basically surround whether you're going to do manual grabbing like each frame you're going to manually have your software grab it or if it's going to be automatic grabbing so the first option you have is a method in the capturing where you grab a frame and that's where you basically call the grab method it goes out and grabs a frame all right that's manual grabbing now retrieve assumes there's already been a frame grabbed and if you retrieve it you're saying okay we know there's a frame out there and retrieve will decode and return the grabbed video frame it assumes the frame is already grabbed so grab grabs it and stores it in memory retrieve decodes it and returns it so that you can use it read does both of those read first calls grab and then does a retrieve so you grab it you decode it and you return it why do you have these different options well when you are setting up a a video capture a stream of video capture what you can do is you can set up the software to automatically send out an event after it has grabbed each frame of the streaming video it can send an event to make it a lot easier for you it says Hey I've grabbed a frame if you want to process it then go ahead and use your retrieve method in the event handler and you can process it so for example if I had a timer and I wanted to manually grab a frame I'd use grab but if I had a event handler and a capture event I could use retrieve because the capture event already captures the image and retrieve will just decode it and return it now there's also this query frame which I think is kind of obsoleted because it used instead of a map like we talked about before it uses a blue green red image frame and if you look at the documentation seems like that's being phased out in recent versions instead they're going with a matte so this is another example where you see this documentation but query frame really isn't used in future versions another thing when you do video capture you're going to use a video capture class you're going to Define an object and you can use the start method what that does it starts the grab process as we discussed in a separate thread so it launches a separate thread which is why it's a nice way to grab video because it's often a separate thread it's automatically grabbing every time a frame is available and then it uses this image grabbed event that you can subscribe to and then you can do a retrieve so it use the imagegram event and then you can retrieve to obtain the image that's already been grabbed so that's the basic capturing process now you're also going to need to be familiar with these emgu CV commands such as CV invoke now we mentioned the emgu CV is a wrapper around opencv and the way that you access the opencv methods or functions is you run CV invoke to invoke whatever opencv method or function you want now similarly when we are going to access the opencv Cuda functionality we're going to use Cuda invoke and that goes down into opencv and accesses the Cuda functionality so you're going to be seeing both of these we'll talk about puta in a future video now let's go take a look at Visual Studio C sharp Windows forms and show how we can set up emgu CV okay so we're going to start up visual studio and we're going to show you how to make a very simple application that will allow you to grab a frame or an image from streaming video and then in the future in our future videos we'll be able to do a lot more so we're in Visual Studio 2022 we're going to create a new project and this is going to be Windows forms.net framework next we're going to call it emgu CV base and this is.net framework 4.7.2 create so here's our basic application we're going to hit F7 to get into the code I'm going to right click remove and sort usings and the first thing we have to do is we have to install emgucv so we go to tools new get package manager managing to get packages for look for Solutions go to browse and type in emgu and you can see here the first thing we show is the mgu CD by emgu Corporation 1.56 million downloads make sure it's from emgu Corporation and we're going to click on that go over here and select latest stable 4.7 version we're going to install it's important you leave this package reference and project file leave that checked hit OK and then accept and go to the output window and verify that everything worked okay successfully installed the next thing we have to install is emgu CV runtime.windows so we're going to do the same thing select it 4.7 version install I accept and the output says successfully installed now one other thing we're going to install is [Music] mgu.cv.bitmap it will allow us to convert the matte Matrix representational image into a bitmap we can put in our picture box so we're going to select that we're going to hit install except look at the output make sure it's all installed so it looks like we've got emg.cv runtime windows and bitmap all installed so the next thing we have to do is very important and again if you look through the installation instructions that are recommended they will tell you that you need to instead of using any CPU up here you need to right click on the project go to properties and then go to build and platform Target we're going to select x64 and then up here we're going to have to go to this drop down says any CPU go to configuration manager and right now it says any CPU we're going to select new and instead of arm 64 we're going to select x64 copy settings and hit OK and now we will have x64 available and select that and we should be all set to go now the first thing we need to do is do a using statement using emgu dot CV so now we're all set to go um the first thing we're going to do is we're going to put in a picture box where we can show the grabbed image so we'll scroll down picture box we can resize it and it's important you select the picture box and scroll down to size mode and instead of normal we want it to be Zoom we also want a button to grab the image and we're going to call it we're going to rename it BTN grab and we're going to change the text to grab and we're going to change the size of the font to 12. um then we can also copy paste and make this an exit button call that BTN exit and change the text to exit and then for both of these we want to double click to generate event handlers and with the exit we just say application dot exit so now the first thing we want to do is we want to Define what we described before which is a video capture object so we're going to say video capture and we're going to call it capture so the first thing we do as we said we've got to define a mat and we're going to call it a frame equals new matte and that is the emgu CV Matrix defining the image and then we have to define the new video capture object we're going to say capture equals new and we're going to tell it to use device zero which is camera zero we're going to be feeding this from my smartphone over Wi-Fi and we talked about that in previous videos and then we are going to actually read the image so we're going to say we're going to define a Boolean bull read success equals capture dot read frame now what this does is we just as we said before the read goes through and grabs it and then retrieves it which means it decodes it and makes it available so it does two things and it puts it into this Frame mat or Matrix so if it is successful read success is true then we're going to say picturebox1 dot image and we're going to have to convert the frame to bitmap that's why we installed cv.bitmap and then what we can do is we can dispose of the frame so frame same dot dispose and that cleans things up for the next frame so that should be it we should be able to now capture from our streaming video and display it in the picture box so now let's start this up and I've got my smartphone connected via Wi-Fi so we'll start it up hit the grab button and there we've got one frame captured from a streaming video now we can adjust the size to make it match what's coming out of the camera maybe resize it we'll talk about that in the future but that's basically it we've just grabbed one frame I can do it again and you can see it changes it updates the frame so that's about it now in future videos we're going to take this to the next level we're going to look at detecting motion and then using our GPU so if you like any of these videos I encourage you to hit the like button subscribe hit the Bell notifications but most of all please let others know that we're here so we get some views really appreciate it otherwise take care have a really good day thanks

Channel: EETechStuff

Views: 10,697

Keywords:

Id: vaRC8cG-H3Q

Length: 21min 1sec (1261 seconds)

Published: Tue Jul 18 2023