C# GPU Image Processing

okay so welcome back today we're going to show you how to build this very simple application and what this application does is it allows you to do image processing on an image but it allows you to do it with your GPU so you can hopefully do it a whole lot faster than using your CPU and in this video we're going to talk about doing this image processing using Cuda on your Nvidia gpus and we're going to access it in this application which is c-sharp Windows forms Visual Studio we're going to access it by accessing What's called the very popular opencv library and opencv allows you to do image and video processing and we've talked about that in other Series where we show you how to do motion detection on streaming video coming into your computer and we're going to access the opencv Cuda functionality to allow you to use their gpus with opencv and since we're doing c-sharp Windows forms we're going to have to use a what's called a wrapper and we've talked about this in other videos called emgu CV which is basically a way to access opencv using c-sharp Windows forms and we're going to build this simple application here and it will it will allow us to do image processing on an image that we read in and it will also allow us to compare how fast the GPU does it with how fast the CPU does it using opencv so briefly let's go through what this application does over here I've got a button where I say check GPU and it goes out and finds out what Cuda enabled gpus are there and it responds with does this computer have Cuda connected and it says true I've got two devices device zero and device one one of them is an Nvidia GeForce RTX 2070 super the other is a GeForce GTX 1080 TI and it's going to use device zero which is the RTX 2070 super and what I've done is I've read in an image and we're going to process it so let me um restart this and show you how it works so I've just restarted this and what I'll do is I will click this check GPU and it goes out and finds what Cuda enabled gpus are out there and then what I can do is I can read in an image that I've prepared that I'm going to do image processing on so I hit read and it shows us the image and down here it says the image size is 5472 pixels by 3648 pixels for a total of almost 20 million pixels so there's a lot of pixels here that we're going to have to process and what we're going to do is we're going to process it by doing a kind of a blur filter and we'll talk about that in detail but it's basically going to blur this noisy image and what I'm going to do is I'm first going to do it using my CPU and my CPU is a rising you can see up here AMD ryzen 7 1700 eight core processor with 16 logical processors that you can see here and this is Task Manager you just right click and select task manager and here I've got the CPU showing me the 16 logical processor and you can see right now they're not doing much four percent utilization and it's just under three gigahertz so what I'm going to do is I am going to start the processing using my CPU and you can see how the CPU is going to respond and how long it's going to take to do a blur so I'm going to hit process over here on the right and you can see suddenly all 16 logical processors are going up to 100 utilization you can see the chart has gone up to the maximum and everything is at a hundred percent so it's maxing out my CPU all 16 logical processors and we're going to see how long it takes to compute and over here it's going to give us the elapsed time when this is done and it will update the image to show the new blurred image and you can see there's the updated image and it took 64.3 seconds using all 16 logical processors and you can see the utilization goes back to zero now and it's all done so what I'm going to do is I'm going to redo this I'm going to read the image in the original image in and instead I'm going to select this use Cuda and do the processing instead of monitoring the CPU and I'm going to go down to the GPU that's going to utilize which is the RTX 2070 super and I'm going to go up here to this drop down and I'm going to select Cuda because it's using Nvidia Cuda on the gpus to do the processing and hopefully we'll see what the utilization of the GPU is as it does this and how long it's going to take so I'm using Cuda the last one took 64 seconds I'm going to hit process and you look down here and a blip in two and a half seconds it says it took and it does exactly the same blurred image so we can check that so here's our original image and here is the results of the CPU blurring and I saved the results and here is the image that's the result of the GPU blurring which took only two and a half seconds compared to the 63 seconds of the CPU and you can see these are identical images if you look over here on the right these are matching identically so it did the exact same thing but two and a half seconds versus 63 seconds so what we're going to do is we're going to show you how to write this very simple application using emgucv c-sharp Windows forms Visual Studio so now we first need to understand some basic concepts of how you can program your software to use the GPU and if you want to get into some more of the details I did a series on programming using Cuda and your gpus and using C plus plus and what we did is we we had a series where we talked about how to write a ray tracing application using your GPU and using Cuda and it goes into a lot more detail of how Cuda works and how you can access it here we're using C sharp so we're a little bit more abstracted and a little bit uh easier to implement but you do have to understand some basic concepts so without a GPU if you just have your CPUs you see the CPU here if you're going to process an image what it's going to do is it's going to load the image from your disk into system Ram and the CPU is going to interact with your system Ram as you see here to process the image in our case that took about 63 seconds to process you know 20 million pixels there's not a lot of cores in the CPUs compared to a GPU which has hundreds of thousands of cores and different processors so you can do what's called asynchronous processing and I've done videos on what that means synchronous versus asynchronous so I encourage you to look at that but if you're going to use your GPU you're going to have not just the communication between CPU and system Ram but you're also going to have your GPU and if you look at your motherboard with your GPU you can see that it's plugged in you can see the plug here plugs into the pcie slot and you can see there's a lot of parallel connectors there and that is the pcie bus and what that does is it allows the CPU to take the image data that's in system Ram and it says basically it says to the GPU hey you're a lot better at doing um parallel processing of you know 20 million pixels than I am because you've got a lot more cores and a lot more threads so what I'm going to do is I'm going to take my image that's in system RAM and I'm going to send it over the pcie bus into this big connector with all these parallel bus connectors and we're going to store it in the GPU memory and the GPU is going to go off and take all that data and do the image processing and then it's going to send it back to system Ram so the CPU can do what it wants you know send it to the video display or whatever but it's really important you understand that you now have this pcie bus communication that you're going to have to perform and the other problem is that every time you send data over a pcie bus it takes time you know if you've got like video that you want to send to the GPU um you got to make sure that you do it right or else it's going to take a lot of time you're going to waste a lot of time transferring over pcie bus for every frame and we'll talk in future videos about how that's going to work with video but when we do this with EMG ucv we're going to have to actually write our code so that it sends the data in Ram over to the GPU and when it's done processing it's going to have to bring it back to system Ram so the CPU can interact back with it what that's called in emgucv or opencv is it's called you take the image and you upload it to the GPU and then when it's done we're going to have to also do a download so we have to remember when we write our code when it's all ready to go you upload it to the GPU the GPU processes and then you're going to download it to system Ram so that we can continue processing with the CPU so here is our simple application I'm in Visual Studio this is c-sharp Windows forms application and I've done many many videos on c-sharp and windows forums even for beginners so I encourage you to take a look at the other videos on the channel um what we've got here is some very basic controls you can see I've got one two three four buttons one to check the GPU one to read the image one to process the image one to exit I've got a text box here that's going to print out the results of showing what's gpus are available I've got a picture box here that's going to display the image I've got a check box that allows the user to decide if they want to use the GPU or just the CPU and then I've got two labels one for the elapsed time that's going to display when it finishes processing the image and one for the printout of the image size that we read in so the buttons are BTN check GPU is the name I've got a BTN read I've got a BTN process and a BTN exit and that's about it now the picture box I've done videos on picture boxes before it's important that you set the size mode to zoom to make sure that when you read the image it fits into this picture box so I'll assume you know how to go to the toolbox and just drag and drop these buttons and double click on each button to get the event handlers and the rest is pretty straightforward so here is our application and as I usually do I've got it laid out with regions I've got documentation to-do lists some parameters only a few parameters we're going to need um I've got the methods I've got only three methods want to check the two available Cuda devices one to read the image and want the process the image and then I've got the event handlers for the four buttons check GPU BTN read BTN process and BTN exit so the first thing we're going to need to do is we're going to need to install emgu.cv and there's a few libraries we're going to have to install uh we'll go to tools nuget package manager manage nuget packages for solution and we'll go to browse and we're going to type in emgu dot CV and hit enter and it will show you these emgu.cv by emgu Corporation make sure you get these by emgu Corporation and what we will do is we will download and install emgu.cv emgu.cv Dot bitmap and very important you download emgu.cv.runtime.windows.cuda so there's three that we're going to have to download and install and consult see those are the ones that are installed so I'll assume you know how to download and install those so once we've got those um there's a couple other things we need to do and I've got them listed in the docs uh in order to make this work you need to as we just did download the emgucv runtime.windows.cuda and also the DOT cvn.cv.bitmap um when you're doing that I really encourage you to use this output window when you install them to make sure that there's no errors because they can get some version incompatibilities also we're going to have to set this up to build with not this any CPU but with x64 now to do that you go to the project right click go to properties and go to build and platform Target select x64 or x86 but I'm going to choose x64 but don't select any CPU also under build configuration manager build up here configuration manager you need to make sure you've got one for platform x64. that's very important you may get an error if that doesn't work so once you've got that now we can start to add the using statement so the using statements are emgu.cv [Music] emgu.cv.cuda and then emgu.cv.structure so those are using statements and then system.diagnostics we're going to be using a stopwatch to measure how long this takes system.threading.tests we're going to do some asynchronous I've done videos on asynchronous so I encourage you to look at that and then system Windows dots forms so the first thing is parameters here's the file name I've got it to this location I've got a PNG file that's that noise image but you can select whatever you want and then we've got to Define our two images that we're going to use in our emgu CV and to do that we're defining first the incoming image but we're making it in an emgu CV or opencv format which is this emgu.cv dot image class and we have to define the color which is blue green red and then the depth which is byte so this is the incoming image I'm going to call it image and the processed image is going to also be an image BGR byte called process image and these are going to be accessible to everybody that's why we put it up here and we've got a stopwatch I'm going to call stopwatch and we also have a double called milliseconds elapsed so we can find how many how long it takes to do the processing for the CPU or GPU so I've got public forum one and then processing goes to these event handlers so the first thing is BTN check GPU to do that all we do is we call the method we've got up here called cuda check and we'll look at that in a bit so Cuda check then we're going to read in the image and if you want to do an open file dialog I've got the code here I've got it commented out I'm not going to use it but you can do no open file dialog and come up with a file name which is this parameter up here file name and it will overwrite otherwise we're just going to use that file name we had in the parameters and what we're going to do is we're making this button read I'm adding async because when I do the read I want it to go off and read the image but still allow us to use the user interface so I've got async void button read and again we've got videos on asynchronous and synchronous programming and I'm doing in a weight and I'm I'm firing up a new task and that task is going to run this read image method which again is up here in the methods we'll talk about so it's going to go read the image when it's done it's going to come back and it's going to calculate how many pixels we showed the calculation of how many pixels so I'm going to call it millions of pixels is convert to double and I'm going to take the image that we read in which is this image parameter we had up here and I'm going to take the image dot width times the image dot height convert that to a double divide by 1 million and that tells us how many million pixels in the image and then I'm going to fill in the label image size dot Tech which is down below the image and say image size is image width Plus image height and then we're going to say in parentheses how many millions of pixels and then we're going to put the image.2 bitmap we're using the 2-bit map method in this opencv image class converting that to a bitmap and filling it into the picture box and same thing with the button process we're going to when we hit the button process it's going to zero out this elapsed and then it's going to also I made this async the button process because it's going to go off and it's going to you know it could take 64 seconds to do it via CPU so I'm doing a away test dot run so we can still use the UI and it's going to run process image which will show up here and then when it's done it's going to calculate the elapsed seconds which is the milliseconds elapsed divided by a thousand and it's going to fill out the label elapsed time is elapsed seconds to string and add seconds and then it's going to take the processed image that's coming out of this image processing method convert that to bitmap and fill that into the picture box and then the last button is button exit just application exit the real workers here are these three methods in order to run Cuda methods using emgu.cv.cuda we have to run this Cuda invoke and what this does is it runs some opencv Cuda methods again this is a wrapper this is a c-sharp wrapper so we're using Cuda invoke to invoke some opencv methods and this is going to determine it's going to grab the has Cuda property which says is there a Cuda enabled GPU running if that's true it's going to say Cuda invoke get Cuda devices summary so really nice method in this emgucv.cuda where it will get the list of all of the devices in the summary of all the devices that we showed in the text box and that will um get all of that data on all the available devices and then we're going to append to that what device has been chosen and to do that Cuda invoke get device and that tells us the currently selected Cuda device now you can change that and we'll show that later but this just says what device is selected and it gives us a summary of all the devices if this is false there's no Cuda device available it will just say no Cuda devices found so really simple just going some doing some Cuda invokes and that's it now when we read the image now one of the really nice things about opencv is the image class all you have to do is feed it the file name and it will go out and grab that as a image an opencv image and we're going to call it image and we already defined that up here image BGR byte image so really nice reading the image is just nothing it just you just feed it the path to the file name and now we've got our image and once we've got that then we can process and this is what does most of the work and it's basically an if else statement and the if else is whether or not you want to use Cuda so if the check box use Cuda is checked then we will do this which runs the uh processing with the Cuda GPU or if not it will just run the exact same thing using the CPU so if we have selected to use the GPU first thing we're going to do is do stopwatch restart and that's going to restart the stopwatch to zero so we can start timing um as I said before you can set uh right now we're set to device zero but you can set to whatever device you want by doing a Cuda invoke.set device and if I want the other GPU I've got two gpus I can set it to device one but I'm just going to use it the device zero um the GPU to use the GPU in opencv we have got this class called GPU matte and that is like an opencv matte or Matrix but it's for the GPU so we're going to have two GPU mats one is the original Cuda image that we're going to read in and convert to a Cuda image and that's a new GPU Matt and then there's the result that's the processed mat or processed GPU bat and that's called result mat so we're just defining those up here as I said before now that we've got our image we have to upload that image to the to the GPU and we use this cudaimage.upload and that is basically our Cuda image and it uploads our image and that's a basic image that we loaded in and it then uploads it and converts it to a GPU image a Cuda image and now it's the images on the GPU in Cuda format and then all we have to do is we have to do the blur method and what we're going to use is what's called a bilateral filter in front mode I've been able to determine this bilateral filter is really good at using the speed of the GPU depending on what you process the how you process the image you may or may not get a lot of benefit out of the GPU but I think this is going to give us a whole lot of benefit but keep in mind just because you're using a GPU don't think it's always going to be super excellent um depending on what you're doing as we've described before in our videos not everything is really good on a GPU but we're going to use Cuda invoke and run what's called a bilateral filter and we're going to use the input to that is the Cuda image that we just uploaded the output is the result mat that we defined and I'm going to fill in these properties that seem to work pretty well for the bilateral filter one is a kernels size which is you know how many pixels around each pixel do we use to process and then we're going to use the sigma color and sigma spatial I just picked these numbers you can dig into the bilateral filter to get more information but these seem to work so basically we're going to run a bilateral filter on this Cuda image which is our image converted into a Cuda image and it's going to print out it's going to send out the result map which we then have to download we're now going to Define our processed image which is going to be our image after we download back from the GPU and it's a new opencv image and it's going to be be defined by the original image width and height so there's going to be no change but this is going to be the processed image so the processed GPU image is a result map we're going to convert that to a processed image on the CPU side and call it processed image and then what we do is we take our result mat out of the GPU download it into this processed image on the CPU side that the CPU can now access and then once that's done we do a stopwatch dot stop and everything's ready and now we can access this processed image on the CPU side and stopwatch dot stops in milliseconds elapsed equals stopwatch dot elapsed milliseconds and then what I'm doing is I'm saving that processed image to this location so that basically is very simple it it takes our image uploads it to the GPU does a bilateral filter and then downloads it to the CPU and saves the processed image now if we didn't want to use this the CPU it's a little bit easier we're going to do the same thing stopwatch dot restart processed image we're going to find it as the same size as the input image image.width image.height and instead of doing a Cuda invoke since we're doing it on the CPU side we're using an opencv CV invoke and we're using the bilateral filter which is the same thing but instead we're using our input image our processed image again these are on the CPU side and we're using the same parameters 150 380 380. and once that's done you don't have to do the upload and download we now do the stopwatch dot stop and that will tell us how long it took and we do the same thing milliseconds elapsed same thing and stopwatched out a lot milliseconds and I save that processed image as CPU blur the other one was GPU blur so that's basically it we've just got these three methods we're running them asynchronously and that's all we need to do to run both of these so if you're interested in other types of computer Graphics I've got other videos talking about uh recently we did motion detection in streaming video off a Smartphone coming over Wi-Fi I've got a lot of discussion on Wi-Fi I've also done computer Graphics videos in the future instead of just an individual image processing like we're doing here we're going to look into processing streaming video images and seeing how we can do that using the GPU so if you like any of these videos I encourage you to hit the like button subscribe hit the Bell notifications but most of all please let others know that we're here so we get some views really appreciate it otherwise take care have a really good day thanks

Channel: EETechStuff

Views: 4,417

Keywords:

Id: uHSfGX-W8uU

Length: 28min 52sec (1732 seconds)

Published: Thu Jul 27 2023