DEEP LEARNING in QGIS: Image Segmentation (Aerial and Satellite) with the DEEPNESS Plugin

hi there folks Conrad here with open source options ngos spatial school check out this awesome deep learning segmentation that I was able to do in qgis and I'm going to show you how you can do it so here's the classification I did this all in qgis with a plug-in and pre-trained models and I want to show you how you can take advantage of deep learning in your workflows so go ahead and stay tuned now the first thing you're going to need to do is download this plugin here it's called deepness and so I'll show you how to do that right now let's install the deepness plugin we're going to start in qgis we're going to go to plugins manage and install and we're going to go to all and I'm going to search for deepness deep neural remote sensing and I'm going to install this plugin okay and now we have this error here um because we have to install additional python packages which we will go through now I'm going to close this op it tells us we can install these packages here so if we close that error message we had then we can install the packages and it's going to try to install this itself and you can't see my other screen but it had a command prompt open up and it's running some things there you can see that it's working on some things here and I'll just pause this while this uh finishes and we'll see how it goes okay so those finishes those packages finished installing and we can test and close we can just close we can say text and close and we should be good so now we can close there and now we have our plug-in installed and we'll come back and we'll go through how to use this next now once you have deepness installed and if you do run into issues let me show you where you can get some help here so let's go and just search for deepness qgis and we'll go to the h plugins page for deep neural remote sensing that's the plugin there and we'll go to details and we'll go to the plug-in homepage and there's help here for installation qgis supporter versions all those things um the next thing we want to find is this deepness model Zoo this is super helpful this shows all the models that you can get to run with deepness um so we're going to use this land cover segmentation model here and the way you use this model you're going to click on this and it's going to take you to owncloud to download this model and you're just going to click download here and you're going to download that now I've already downloaded this I'm not going to download it again but it's this deep lab V3 land cover okay 4 C for four class so I've got that one downloaded I might also show you this um land cover segmentation for Sentinel 2 I didn't get as good of results with this one but you can also download this once again just click on it it will bring up the owncloud file and you can download that model and once you have that model downloaded you're good to go now the one thing I will say that was kind of frustrating about these models there's not great documentation on them I mean it gives you a description but I couldn't find out which image bands correspond to which bands in the model things like that the Euros set data set I found didn't have the best documentation maybe I didn't read the paper very well but they didn't even say what landet level they were using or not landet but what quality level it was um Sentinel l1c or L you know or l2a so anyway whatever the case this is still going over to download but that's how you get the models you're going to download them from this model Zoo okay there are regression models so you can have like a place recognition there are object detection so you can um identify planes oil storage cars detection I haven't played around with these a whole lot yet um but there's there's other things here you can your own models so that's where you can get these models in deepness now let's go back to hugis so we've got deepness installed I'm going make a new project here well I don't care I'm going discard this so we've got deepness installed Oops I did the wrong thing there we go we've got deepness installed we have the plugin here we need to add some data so here what I have is I have this Ortho photo this is USA nape imagery the national agricultural imagery project it's not what this high resolution model was trained on we're going to test and see how well it does these data are about 60 cm resolution we can go check it over here so if we go down and check our our pixel size we're about you know 6 M so 60 cmers and we're going to need to know that information for running deepness all right so let's have this here let's go to deepness and just open up our plugin by clicking on deepness if you don't see it here you can go to plugins deepness and click deepness there this will bring up the side panel for the input layer you want to select your imagery I only have one thing there you can do the entire layer you can mask polygons you can do the visible part um we'll just do the entire layer for this example and now we have different model types we have the segmentor the regressor all these other ones we're going to use a segmentor and now I'm going to browse for that model I downloaded I downloaded mine to this temp folder and it is this deep lab V3 so I'm going to open that and now I can reload the model to reload the parameters I can lo the default parameters and you can see the model info there now here we can see the image input requires three bands um our image input is three bands the model input requires three bands or three channels which so we those line up um we're just P we're putting these in in sequence so hopefully it's the right sequence I couldn't find documentation to show otherwise so we're going to go with that here the resolution we're going to put this at 60 cm because our image is 60 cm um we can't change the tile size because that's something for the model that we're just not going to be able to change and we can adjust tiles overlap here if we want to let's try this at 15% um and see what happens and then we can try it again at like 50% or something okay um class probability threshold we probably want to put that at 50% greater and we can remove small segment areas this is usually set at 9 or something like that okay and you can play around parameters to see what gives you the best uh options so once you have that set up now we're ready to run this model and here's where it's just super powerful just inside of qgis we've loaded this deep learning segmentation model now I can just click run oh and it says it's already processing this is strange um anyway there it goes it's going now and you can see that we have this running here and it's processing so this is going to take just a few minutes to complete but it's not going to take as long as some models do because this models already been trained and so we're going to get results within 10 minutes here so I'll just pause this and then we'll come back and see how things look okay so we finished processing and you can see that we have some basic stats here we have this background area which accounts for the majority of our image we have buildings Woodland water and Road um with the other ones so I want to click okay and we'll take a look at this classification segmentation and one thing I want to know here that I forgot before is there's this output format and I selected all classes as separate layers and what that does it gives you model output in this layer group where we have all these different layers um selected here you can change this to separate layers um single class as a vector layer what whatever you like I kind of like this output but let's go ahead and take a look at what we have and see how accurate this is I think we're going to expect some discrepancies just because because of the nature of the data but that's okay so if we look in here you can see I'm going to turn off background so we can get rid of that you can see that a lot of this water here was classified correctly some of it was missed um some of it appears to be classified [Music] as Woodland you can see that some Woodland in the middle um but not too bad overall we have Woodlands um where Woodlands should be it looks like for the most part um you can see that we have some water down here in the shadowy area but overall we did decent I think the thing that looks like it did really well is roads these roads are kind of this green color and if we turn those roads on and off overall it's doing a pretty good job finding those roads and buildings doing pretty good finding buildings um and this is a model I didn't train on these data this is just you know out of the box running on data that it wasn't designed for and qualitatively we're getting some decent results for some classes not great results for other classes there's a lot of background area that I wish we could classify you can see I'm pretty impressed with how those and the buildings do so really cool how you can just pull that deep learning uh algorithm right out of the box okay now I'm going to turn these off let's do this with Sentinal data it's just a little more involved into a little bit of data prep for Sentinel let's go ahead and do it so you can see how that's done so I have this Sentinel 2A scene um this is level 1C data it's in the safe format so in here you go into the granual folder and then you go into this then you go into image data and then we'll slide over and I want bands 1 through 8A so all those bands I add them in say okay let's go back to my layers and now what I want to do is I want to combine these into a virtual raster so let's go to raster miscellus build a virtual raster let's select inputs I'm going to select all to start and I'm going to deselect my Ortho and say okay now I want to place each of these into a separate band I want my resolution to be the highest um of the layers and I want to resample using um let's bilinear cubic I'm going to use bilinear and we'll leave this all the same I'm just going to save this as a temporary file and let's just run this okay and let's close that and now that I have this virtual raster I can remove all these layers to clean things up okay so I have this virtual raster um it's going to have 13 bands that looks kind of cool the way it's displayed like that with that band combination um anyway let's just zoom out here a little um to this extent and and we're going to load a new model and try that new model here so I still want to do a segmentor model but here I want to select virtual I want to change my model so I've downloaded The Sentinel model from the deepness model zoo and I'm going to go into Temp and that model is the eurosat 13 Channel um one here so let's open this and let's load the default parameters the resolution is 10 m so 1,000 CM so that's correct we're going to leave the defaults the same this is where I wasn't able to find any metadata describing which inputs ma to which band so we're just going to leave it as a default we have 13 bands we're going to the 13 channels um we're going to leave it there um tiles overlap 5% um let's give this let's make this like 30% just playing around I don't know if there's a good answer for this I'm just giving it 30% we're leaving these like this all classes separate layers again and let's go ahead and click run and it says it's already in progress but I think that's going to come out in just a minute so I'll pause this again while it runs there it goes it's running now so I'll pause this and I'll get back to you when it's done and there's a couple things I want to tell you about one I made a mistake I meant to do this for just the visible part of the layer but I did it for the entire layer so it's going to take a little while the process it's going about you know 1% per minute probably so we've got a few more minutes maybe we got over an hour till it's done but I'll come back yes since it's got 57 minutes till it's done um I'll come back when it's done and we'll still go through this another thing I just want to point out I opened up this layer styling panel and I restyled the imagery so that it is 432 um so we get true color out of this and it's going to be easier to evaluate that way so just so you know that's what I did and then we'll come back and I'll show you the results here when it's done in about an hour all right that took about an hour we finally got this done you can see with this model we have a lot more classes um and we can go through and just take a quick look at some of these I'm going to use the identify tool for qgis just to identify the features we have going to close that out and let's see what we have let's start with the water you can see that this roughly represents water and let's see what we have here I'm going to click over here and and you can see that this is being classified as Forest which is incorrect this here is being classified as annual corop annual crop it's more like a natural vegetation um but not too far off let's see we have over here in the urban areas it's being classified as a river it's being classified as a highway highway is not too far off river is kind of far off for urban see we have over in this area pasture not horrible that's being classified as pasture industrial buildings industrial buildings and those are more like pastures that looks like we can zoom in and we can turn this model output off yeah you can see that those are more like pastures and are classified as buildings so whereas our other model worked relatively well I feel like the one with the nape and high resolution imagery this model is not working so well so these are just some things to be aware of as you work with these deep learing tools they're out there um these models have been developed there's training data for them and they've been compiled in this deepness plug-in for qgis and it's really powerful you can get started with deep learning really quickly here um but as you do just be careful make sure that you check your data make sure you read the metadata um like I said these models aren't documented really well or at least documentation was not very easy to find which was disappointing um but that doesn't mean you can't use them if you find the model works well or if you want to take that model and train it with other data it might provide a good opportunity for you so I hope you've enjoyed this I hope you've given I hope it's given you an introduction to deep learning with qgis and helped you see what might be possible um I said this is going to be a start for you there's a lot more to get into but you know if you have questions feel free to ask and as always thanks for watching and and good luck with the GIS

Channel: Geospatial School

Views: 5,356

Keywords: open source, open source options, qgis, deep learning, ai, artificial intelligence, geoai, geospatial, neural network, deep neural network, remote sensing, gis

Id: UTiILsy0Mt8

Length: 17min 30sec (1050 seconds)

Published: Wed Feb 21 2024