Geospatial Deep Learning with ArcGIS

hello and welcome to this session on geospatial deep learning with our chairs my name is Rohit Singh I am the lead developer of obvious API for Python and the director of s3's R&D center in New Delhi that's focused on geospatial AI and joining me is Vinay vez la Miran who is the product manager for image analysis and AI in ArcGIS there's going to be a demo rich session so I wanted to start off with a quick session overview so you know what's expected we'll start with an introduction to what is AI what's machine learning and how it's related to deep learning and then we look at the deep learning workflow in ArcGIS going from the imagery to training your own deep learning models and using them within ArcGIS pro as well as using the obvious learn module of the Python API then we look at the different types of deep learning models and how they can be applied to imagery and remote sensing we'll see how you can use image server to scale up the processing and use these models on a large geographic area and finally we'll end with some resources to help you get started so with that let's get started today ai is at at the peak of the hype Chardin you know if you where to go out and look at what's happening in this industry there's a lot of buzz words a lot of terminology being thrown at you and it's easy to get lost and feel intimidated but it really isn't that hard I'd like to parse it down to three important concepts starting with AI or artificial intelligence that describes the ability of machines or computers to exhibit what we typically associate with humans that could be things like understanding speech or or seeing things computer vision machine learning is one way to achieve a I it's based on data-driven techniques where the machine or the computer is looking at a lot of existing data and learning the patterns needed to come up with a prediction when it sees unseen later data that it's not not seen and and deep learning is one type of machine learning that loosely resembles the human brain it's based on deep neural networks with several layers of neurons and where these networks figure out by looking at the data what the outcome should be what the predictions are so that's one way of looking at at machine learning and how it's a superset of deep learning but there's another way and that's how about contrasting traditional machine learning with deep learning how how traditionally machine learning is is different from the newer deep learning techniques and that's in this example on the top you can see how you know on the IMP there's an image on the Left that's of a car and the machine is trying to figure out if it's a car or not and typically there's a human in the loop with traditional machine learning techniques and in the case of computer vision or a really classical computer vision it was human data scientists who were creating things like edge detectors or blob detectors and once those features or edges or blobs or corners were extracted from that imagery we used to apply a machine learning algorithm it could be a support vector machine this diagram shows how a shallow neural network could be applied to come up with a prediction of whether it's a car or not based on those extracted features now wouldn't it be great if the network or the model could do this task by itself just by looking at the raw data if it can figure out what those important features should be so it can come up with the right predictions and that's exactly what what deep learning does in deep learning the deep neural networks figures out or learns what it needs to learn to come up with the right predictions and to me that's the beauty of deep learning so you may be wondering well oh that's great but how does it relate to ArcGIS now in ArcGIS we've had machine learning tools in the broad areas of classification clustering and prediction and now we have a rich set of tools for deep learning going from preparing your data to training a wide variety of deep learning models to using them or you know inferencing detecting objects or classifying features classifying pixels and doing post-processing things like non-math suppression that we you know we name is going to talk about later so why should we be using deep learning for imagery tasks well the short answer is that computer vision is now almost as good if not better than human vision the chart on the right shows the results from a competition the image net visual recognition challenge where a computer was shown an image and it had to come up with a prediction of what's in that image and till 2012 it was machine learning algorithms that were winning that competition and the error rate used to be above 25 percent so one in every four times the machine got it wrong since then since 2012 when the first deep learning algorithm ALX net came it's been deep learning networks that have been winning this competition and they've been getting so good that sometime in the last few years they even started exceeding the human accuracy so because you know computers are now better than humans and tossed like classifying images they got so good that they actually had to discontinue this competition so with that now let me hand it over to Vinnie who's going to tell us how deep learning can be applied to solve a wide variety of problems in GIS what do you Vinay thanks for it now from an industry perspective there are four key tasks of four key categories of deep learning first is pixel classification typical example is land cover classification next we have object detection and as the name suggests designed to detect objects of interest within images cats dogs for instance but from a more practical standpoint it would be cars planes and other structures the third is instant segmentation where users really want to get precise polygonal boundaries of each object in this case building footprints lastly object classification and a typical example is classifying a structure as damaged or undamaged archaea has natively supports each one of these deep learning models now LGS supports end-to-end deep learning workflows we have tools for preparing training samples prepping data to train models model training tools and tools to perform inferencing we support the four key categories of deep learning that I just talked about that's pixel classification object detection instant segmentation and image classification we will be showing you examples of all of these through both RJ's Pro and notebooks and lastly you can run all of these deep learning tools using up J's Pro the map viewer or using notebooks now if you want to run this on desktop the licensing model is you need ArcGIS image analyst extension and if you want to scale out your processing you can run air in a distributed environment you can use enterprise and for this you need of GSM it's over licensed now breaking it down here's the workflow within our chairs step one is you collect training samples step two using the export training data for deep learning tool you generate chips step three training the neural network and last step using out-of-the-box tools we can inferencing now let's go through each one of the steps step one is collecting training samples there are different types of capturing training samples of label data first is you could use ArcGIS probe the editing tools that are natively available with an ArcGIS Pro or the map viewer or you could build your own custom applications using the JavaScript API next is you could use the training sample manager which is historically available with an ArcGIS pro for classification workflows what we've added in ArcGIS pro 2.5 is a new experience the label objects for deep learning experience this is designed specifically for deep learning workflows now there are three information models when it comes to or multiple information models when it comes to capturing training samples if you're a local user if you're working on the desktop you're a single user you could capture training samples using a feature class now if you want more collaborative experience you have multiple users who are writing to the same feature service or simultaneously capturing training samples you can use a feature service and lastly depending on your workflows you could also use a classified thematic rest or as an input for providing training samples the next step step two is essentially exporting the training samples to emit chips these chips will then be used for training the model this can be done using notebooks using ArcGIS API for Python or we can use GP tools from with an ArcGIS pro step 3 is where you train your deep learning model we support a variety of models which we will talk about a little later you can train your model using ArcGIS API for Python and you can also train deep learning models using geoprocessing tools natively with intro and that's something that we've newly added within our JS pro 2.5 the ability to Train deep learning models natively within pro we also support external deep learning frameworks so you can easily tie into tensorflow and CNT k and similar frameworks last step is inferencing we've extended the capabilities in this release to support object classification as well we've previously in the past couple releases we've had object detection and pixel classification and inferencing can be run both on desktop or you could use in a place to run inferencing to recap we've use tools to generate training samples which are used to train a model using tools and api is this produces a deep learning package which can be used for influencing using pro and image server and the intermediate file like I said is a deep learning package so what is a deep learning package a deep learning package or a DLP Kay is essentially just a zip file that's automatically treated by the training tool it created by the training tools it contains the model definition files the Train model and optionally it creates a custom Python raster function for infancy the custom Python rational function is if it's a function that's not natively supported within ArcGIS the DLP K can be shared across the organization enabling users to easily reuse the model for running inferencing on their imagery now here's a tabular representation of the four key pieces that I just talked about and these are the different clients that are available on the left you see RJ's Pro RJ's API for Python map viewer and image server and here you have a representation of all of the tools how they're accessible and available at a higher level here the deep learning workflow in addition to the deep learning pieces RJ's enables efficient management of large volumes of imagery this is done using mosaic datasets and Amit services next ArcGIS enables advanced analysis of the deep learning results and these results can then be used in field mobility and monitoring applications such as out shares workforce and operations dashboard now I'll dive into a demonstration where we'll go through the entire deep learning process using ArcGIS Pro now keep in mind some of the demos we will show you today are scenarios you you've probably seen before but this time they are all being run natively using our chairs Pro or using notebooks with a much more streamlined workflow in place so to provide you with a little context about the scenario here you're looking at the nape imagery it's one meter resolution imagery and what we really want to do is identify pools within this location step one is you need to capture training samples like I said there are multiple ways you could do that one of the options is using the label objects for deep learning tool invoking the tool gives you the option to specify new classes or you could simply just start digitizing features and so on now the export training data tool was available in the past two releases to make the user experience simpler we've embedded it within the label objects tool so you can directly switch to this tab and export the outputs or create chips directly into a specific folder these are default parameters which you could just use by default optionally you could head over to the geoprocessing tool under the image analyst toolbox you have a bunch of tool sets and tools and we can invoke the export training data for deep learning tool in the interest of time I have pre-populated all of the parameters here's my input raster the output folder where the chips will reside he has an input feature class and this is where you define the value you can provide a mask if needed and here are additional parameters once I run my process the output is a bunch of chips in addition to metadata this folder then goes into the training tool so I can call the Train deep learning model tool this is my folder that contains all of the training chips which we created in the previous step this is the output model that is going to be written out you can specify the number of epochs and you really don't need to specify these parameters we use intelligent defaults but in case you want to tweak parameters you can change the backbone model you can change the validation percentage basically the number of chips that you want to use for validation of your results and you can change the model type if needed once you run this now depending on your resources on your machine if you have a higher than GPU the process obviously ends a lot faster you could complete the process a lot faster the output is a trained model once you have that you can call you and we have a variety of inferencing tools in this case I wanted to detect pools so I specify my input raster which is my nape imagery the output detected objects which is a feature class here's my model definition file additional parameters that I can't specify including the non maximum suppression set of parameters which defines how to handle overlapping features once we run the tool before that here's just a quick representation of all the training samples that we've captured once we've run the tool here are all of the pools that have been detected within our area of interest so a matter of clicks we've gone through four key tools to actually go all the way from capturing your training samples exporting these training samples training your model and eventually detecting objects so that was our end-to-end workflow that was - built directly with an ArcGIS pro now I will hand it over to Rohit who's going to be talking a little bit about the Python API that was great when a analyst can now use geoprocessing tools to perform the complete deep learning workflow including training your own deep learning models without leaving our GIS pro but not everyone myself included likes to use tools all the time data scientists prefer to do their work in notebooks and use Python code and libraries and for that reason at RGS Enterprise 10-7 we introduced ArcGIS notebooks that lets them use a rich set of open-source Python libraries such as tensorflow PI tours and Harris for deep learning and combine them with ArcGIS using art pie and arcgis api for python and now with pro 2.5 Python notebooks are seamless integrated right with an obvious pro Python is the lingua franca or the most commonly used language for data science and RTS API for Python allows analysts and data scientists to bring in all the rich capabilities of arcgis into the notebook and we've recently added the archaea sloan module that enables analysts and data scientists to easily train and use deep learning models with a simple intuitive api now if you're a sophisticated data scientist you can use any deep learning frameworks such as tensorflow c and TK or pi torch to train your models provided you know a lot of math linear algebra and calculus you know how backprop works what loss functions are and so on training a model can involve dozens of lines of code and it can be complicated you need to figure out how to install these libraries and believe me it can get pretty complicated with all the downstream dependencies of each library with arcgis learn training a model is super easy it's typically the same three to five lines of code that you need to train a wide variety of models and if you're using obvious pro 2.5 or RGS notebooks all the dependencies can easily be included in that environment and it's not just about training obvious learns suppose the full deep learning workflow starting from exporting the training data to training a wide variety of models ranging from object detection models object classification models to pixel classification and now to models that can be used to extract entities from unstructured text or even processed 3d data such as point clouds it also has tools to deploy these models manage them and use them to derive insight out of imagery OGS learn is closely integrated without yes it's built for ArcGIS it directly assumes the exporter training data from Pro and the models that it creates are directly usable in our GS it has a wide variety of models that can be trained using a consistent API it uses intelligent defaults so these are you know some things that we figured out that makes it suitable to train models on satellite imagery but it also gives you the flexibility to override those defaults when needed and because it's built on top of fast AI and pi torch which are popular deep learning libraries it leverages the latest research and deep learning to train very accurate model using very simple code there are a lot of enhancements and goodies that we've added on top of it to make it sweeter for use and here are some of the things that you can do today with obvious learn this light shows a sprinkling of the different variety of projects that we've accomplished using our GS launch it's best seen through a demo so let's do that now ok so now let's see how we can use our GS learn to perform land cover classification in this example we'll be using data provided by the Chesapeake Conservancy and train a deep learning model that can go from the s3 world imagery to the classified land cover now a Chesapeake Conservancy went through this process and this is actually a very labor-intensive process in their case it took them 10 months where analysts produced a high-resolution land cover data and you know by the time they completed the land cover changes and this was done for 2013 and 14 so how can we use all of the effort that they have put in to label that land cover use that as training data to train a deep learning model so given new imagery it should then be able to come up with in the updated land cover classification so what we've done is that we've taken the from this project for Kent County Delaware and loaded that in arcgis pro so let me switch over to our GS pro here we are looking at the SV world imagery for Delaware and let's bring in the downloaded data the classified land cover from Kent County Delaware and zoom into this area of interest now this land cover has been classified such that houses are showing up in the red color the roads are showing up in the black color the water bodies in blue and so on so what I'm going to do is change a deep learning model to go from the a3 world imagery to the classified land cover and we'll be using this data as training data to train our model and that starts with exporting the training data so and the input raster we are using the s3 world imagery we specified an output folder where image chips will be generated and this is the the Kent County classified land cover operating from the Chesapeake Conservancy we are using exporting the image chips only for a small area of interest because I want to do it live and don't want it to take a whole lot of time so let's just do it for this small area of interest choosing the TIFF file format giving the chip sizes and metadata format the chips should be exported as classified tiles that's the input format that an image classification or a pixel classification task so here I'm kicking off the export training data and let's switch over to Windows Explorer going to that folder we see that the data has been exported and here you know deep learning models cannot just process all of this data at once this huge raster it has to do a lot of computation and that happens on GPUs that have limited capacity and for that reason we chip out the data into these small fragments and use them for training this is the chips for the world imagery and here are the chips for the labels or the classified land cover and now I'm going to switch over to a notebook where we'll be training land cover classification modeling it's right here and this is a code that we'll be using to train our modeling what I've done is that from the obvious learn module I've imported the unit classifier that's a unit model used for pixel classification tasks and the prepared data method and I've pointed the prepare data method at the chips that we just exported and this can you know node in all of that data and applies random transformations such as randomly zooming rotating and flipping those images to create more data more synthetic data from those just those image chips that's a way to create a more generalizable model and here we are visualizing a small batch of that data now you'll notice how we have overlaid the non-covered classification on top of the satellite imagery so if I give a lower transparency you know that the labels are showing up with a lighter color so this is just visualizing a small batch of that data it's already in a format that's ready to Train and here I'm creating my unit classify classifier passing in that data and my model has been instantiated so with a new model let's see if it can come up with the predictions you know given the satellite imagery can it come up with the right predictions and and do the labeling and here we've got the ground truth on the left and the predictions on the right now as you can see the predictions don't match the ground truth at all and that's because the model has not been trained it's like an untrained model and it needs to be trained it needs to be fit to the data and I'll do that using the fit method and I'm training the model making it fit to that data over five epochs or training passes over the entire training data set of those chips that I just exported now I'm also passing a learning rate while fitting and to to pass in that learning rate and the significance of that learning rate I want to use you know while the model is training I want to explain what a learning rate is and how I pass that end so you can imagine that you know a deep neural network when it's first instantiated is like a newly born brain it really doesn't know anything about the world and it needs to look at several examples and learn from them now if it learns at a slow rate it can take forever to learn anything but if you give it a very high learning rate the model can be quick to jump to conclusions if I were to put it loosely in a way you know in in mathematical terms it doesn't find the optimum value where the model converges where the error is the least and to find the optimum learning rate at which the model can can learn we actually provide a handy tool which is based on top of fast years learning rate finder what it does is that it starts training the model at a very tiny learning rate and keeps increasing that learning rate and at some point the model begins to learn and the loss or the error begins to go down modeling is learning and the error is going down and at some point the learning rate becomes so high that the loss starts going up again at that point it's too high for the model to learn so what we do is we can pick a value where the loss is still going down and pick a high value from that in this case it's 1 a negative 3 or 0.001 the loss is still going down it's not high enough that the model will diverge or won't learn anything and it's still fast enough that I can do it live in in front of you while giving this this session so as the model is is learning as we are fitting it to the data we also print out the loss on the training data and the validation data so what does that mean well what we do is we take about 10% of the training data and set it aside and we don't use it for training we only use it to validate or check how good the model is so when we are checking the accuracy of the model we are only doing it on data that the model has not seen and then the third column we are printing the accuracy of the model over each training epoch and we can notice how it's taking about 47 seconds for each epoch and with each training epoch you'll notice how the loss or the error rate is going down both for the training data set and the validation data set and the accuracy is going up from 55% to now 77% and let's see how much it is after the final epoch and hopefully the model would have trained enough that we'll be able to visually see that it's it's beginning to learn the major land cover classes so we are about done and it should complete that last epoch any moment now and we ended up with close to 76 percent accuracy and now we can run the show results method once again and see if the model is doing better than the results we got earlier red you know the prediction that very different from the ground truth and as you can see in just you know less than five minutes and five training epochs on a small amount of training data I have been able to train a model that's able to identify the major land cover classes it's able to now starting to recognize where the buildings are it knows where the forested areas are whether the shrub land is and where the roads are to a little extent now training a more accurate model is all about giving it more training data and training it for for longer over more epochs once you are satisfied with the model you can save it out and that creates a deep learning package that you can use for influencing now in the interest of time I've already trained this data for longer and you can see how the accuracy went up and it's now close to you know giving much better results and here's you know another model that was trained on all of that data from Chesapeake Conservancy and you can see how it's it's so good that the results are almost as good as the ground truth it's almost a perfect match so once we've trained a model we can go back to our chess pro and use that model to to find the the land cover in a new area and I'll do that in this area of interest and switch off the labels for a little while and as you can see you know if you notice closely at this row of houses and this path here in the you know current world imagery you can see these but when this data was labeled by the Chesapeake Conservancy this was missing so most likely these houses under construction and that's why they were not labeled as as homes and in the path now when I used a well-trained model with the classify pixels using deep learning tool I was able to train you know pass in that trained model in in the path here in the model definition in the input raster is the world imagery and we can then pass in the layer the output raster so when I did that I'll just show the results for that I was able to arrive at this inference land-cover using the deep learning model and it's now able to give me the updated land cover based on the current world imagery so that was an example of how obvious learn can be used to train a deep learning model for land cover classification now I'll hand it over to Vinnie who's going to talk about the different types of deep learning models and how they can be applied to GIS so what are you vinay now a few more demos I'll talk about the first model this is the most simplest which is image classification and the model supported here our inception ResNet and more typical scenarios are classifying pools as clean or unclean or classifying structures as damaged or undamaged all right so to provide you with context the scenario here is the Woolsey fires that occurred late 2018 what you're looking at is the fire parameter and what I really want to do here is classify my structures as either damaged or undamaged we have about 10,000 buildings in this Aoi this area of interest step one is capturing training samples and we do the same process we go through the same process we label objects using deep learning we call the label objects for deep learning user experience and these are a bunch of training samples down south that I've captured once we have these tools once we have captured this we can invoke the export training data for deep learning you specify your input raster your output folder similar to what we've done before similar parameters run the process and we have a set of chips these chips then go into your training into your train deep learning model geoprocessing tool you specify your training data which is your chips additional model parameters and you'll notice you'll notice previously it was just object detection but in this case it's object classification because the system automatically recognizes the chips the metadata and the chips indicating that what we're trying to do is object classification and accordingly it populates this drop-down once we run the tool we get our model which we can then use for infants in person purposes and this time it's not object detection it's classifying objects using deep learning we provide an input raster our buildings features is an input the output is the building features classified then we have a model definition file and class labeled field and we can run the process once the process is run here the results 10,000 buildings that have been classified as damaged and undamaged zooming in a little tighter you can see the classifications are accurate now the real value of doing deep learning within ArcGIS is the the geospatial set of things you have access to the complete ecosystem so in this case what I do is I take these results these feature classes I use them as barriers in network analyst and then we identify shelters that are within or locations where we can have shelters there are within 20 minutes work time this is what the results look like right now once we have this you can monitor your project for any given area here's a customized Operations dashboard here you get a dynamic representation or a dynamic view of the average that average damaged buildings the total number of damaged structures undamaged structures and you could populate it with any additional parameters you have need based on how you enrich your data so that was a good example of how we went all the way from raw unstructured imagery it was managed using image services and then we went through the deep learning process used the results in network analyst to provide additional insights and lastly pulled it into an Operations dashboard to get a high-level overview of the project now I'll hand it back over to Rohit who's going to talk a little bit about pixel classification so when I just showed how these homes could be classified as damaged or not and he was able to do that by applying an object classification model and what it did is looked at this layer of existing building footprints and for each building footprint it looked at the imagery around it and classified that images as damaged or not but what if we were missing building footprints you know there can be missing data such as here or if there were new world developments such as in this case in this region could we use the existing building footprints as training data to extract these building footprints for areas where we do not have them well sure we can and let me show you how we did it what we did is we exported training data from the existing building footprints which has been overlaid on top of the imagery here you can see the existing buildings showing up in the pink color here and we've trained once again a unit classifier model this time to extract those building footprints so given the satellite imagery it should find which pixels belong to buildings and which do not so once again we are training a unit classifier model the same training methodology find an optimum learning rate fit the model to the data and after doing that let's look at the results with the ground truth on the left and the prediction on the right and you can already notice even in the first ma chip there's actually building in the top right corner which was missing in our label data that the model has been able to pick out and here's another example where this building was actually missing this one as well but the model is is good enough to find them as well so it's at least as good as the human annotators who labeled that data and this time for performing the inferencing we actually used image server with raster analytics capability all that was needed was getting the imagery layer and to process this imagery we actually did this processing at a larger area of interest all we need to do is call the classifier pixels method and in this case we use a four machine rational analytics cluster consisting of cloud GPUs with 24 GBS of memory each and processing a large area of interest covering over 60,000 buildings only took 20 minutes that's a rate of generating 3,000 building footprints every minute so let's switch back to Pro and look at some examples of the kinds of footprints that were created now because this is a pixel classification model it's only classifying each pixel as belonging to a building or not the results are not perfect and that's where the post-processing tools in arcgis come in handy using which we can go from those masks of buildings to regularize building footprint using tools that we've built just for this purpose so let's look at the results of the regularized footprints and you can see how deep learning has done such an excellent job at extracting these footprints just by looking at the imagery now this is an example of a pixel classification model known as the deep learning world as a semantic segmentation model where we are assigning a label to each pixel of that image for example here the pixels belonging to the cat are in the yellow color the pixels belonging to ground are green and so on and this can be very useful for Jia's for doing things like land cover classification that we saw earlier or in this case for extracting building footprints now for the next type of model let me hand it over back to Vinay so next I'm going to show you a demonstration couple demonstrations on object detection there's one I'd already shown you which was detecting pools and again the types of models that we support a single your detector and retro net applications included acting trees cause airplanes one so forth so in this case our demonstration is going to be detecting well pads okay now the Snyder here is I want to understand visual understand and visualize trends in drilling activity over the Permian Basin so you're looking at the delaware basin and a ten meter resolution sentinel imagery you can clearly see well paths across the landscape this image the sentinel image is great for change detection if you compare two images over two different time periods you can see new well pads have emerged changes in frac ponds new construction sites and here you have new well pads as well now what I really want to do is identify change and detect objects over a much larger area and given the booming rate of development this could be a daunting task and here's where deep learning really comes into play so again going back to our set of processes step one is you capture a bunch of training samples here we have a bunch of training samples that have been captured we have the training samples you can run the export training data tool then you train your deep learning model and lastly you detect objects using deep learning you invoke the tool that will detect the objects zooming out here are results and I actually ran this through a series of tools so what it gives me at this point is going to be set of well pads you can see how it's increased over time so we've got 52 slices of imagery and what I really want to see is how the number of well pads have increased over time and you can see this change gradually so there again we used deep learning and we put in the geospatial aspects we could step through time and see changes now we could visualize emerging patterns over this area also with respect to space and time and here you can see increasing hotspots or increasing cold spots which shows you increasing activity or emergence of new well pads once we have this again you could take this to the next step you could see the total number of well pads for any given day and you can monitor activity near your lease you can see the total number of active well pads frac ponds and you can even filter based on a specific type of asset alright now I'll hand it back over to Roy who's going to talk about instant segmentation okay so when I just showed us object detection models and we typically use object detection models when we want to find the locations of objects in the form of bounding boxes now there's another type of deep learning model architecture this is known as instant segmentation models and I like to think of them as more precise object detection models in that not only can they give us the location of objects but they do that in the form of polygonal you know with their polygonal boundaries so you can see how in the example on the Left not only do we find these cars we also find the precise boundary of each car and in the example on the right it could be used to find you know the boundaries of building footprints now you you would have noticed how earlier I used a unit model a semantic segmentation model for finding building footprints but that would not work in cases where buildings are contiguous so let me switch over to my browser here to show you what I mean by that if I were to use a semantic segmentation model I will only be able to find the pixels that belong to buildings and those that do not and I would not be able to find the each instance of the buildings and that's where instance segmentation models come in we can differentiate each instance of the object and create you know polygonal boundary is for it now I'm also using this opportunity to show us the guide for ArcGIS lon you'll find it in the developer site for our GS APR for Python in the deep learning without GS section and here's an example where we are using a mask our Siena model an instance segmentation model to bring in the the data for building footprints from Berlin where buildings are right next to each other especially in the city center and we want to find their precise boundaries and and create separate building footprints for for these so the same training methodology once again but this time we are creating a mask our skin and model finding a good learning rate fitting it to the data and let's scroll down and look at some results so you can see here the ground truth on the left and how the predictions very closely match the ground truth it's able to delineate each individual building footprint and we took this model you can also make it predict the bounding boxes as well so you can use it in both modes we use this model to extract footprints and here are the results so you can see how when there is a row of contiguous buildings like here the mask our CNN model is able to delineate each individual building and and if I were to you know move here there are these u-shaped buildings and turning off the rest the vectors for those building footprints you can see how the model really had to look at each building see their boundaries and how this would have been like such a labor-intensive task for a human to do and how you know deep learning can can really automate this task so you must be wondering like how can we take such models and and apply them on a large geographic area automating this task and saving a lot of human effort and for that let me hand it over back to Vinay so now I'll talk about scaling deep learning using ArcGIS Enterprise deep learning is an intensive process and depending on the system resources it can take a while to run up J's Enterprise can be used for scaling your deep learning process and the central component or technology enabling distributed processing is the raster analysis server we essentially is a noxious image Silva that's configured in a raster analysis role now once configured a raster analysis server will be able to handle raster processing in parallel by leveraging multiple GPUs leveraging multiple instances of rest analytics servers help speed up what otherwise might be a very long running process here's a sample amp enterprise architecture and this was used for a project internally first of all on the Left we have the arches portal that's your standard enterprise portal it's connected to an ArcGIS server and azure which hosts that portal but also talks to the ArcGIS data store which is what enables you to publish hosted feature layers so this is the data store that contains all your feature outputs from inferencing then on the right you have your as your hosted image server and as your hosted raster analysis server and as I've mentioned the rasa and lyric server is still an image server just assigned to an r8 role and finally you've got your distributed raster data stores as well as as your blob storage that the image servers have direct access to now when it comes to deploying models you can really deploy any type of model you want with any kind of framework provided all the requisite dependencies are installed in the array servers when we perform inference we utilize our chess notebooks or pro-and to run the model as you recall it takes a single function called PR notebooks or a single tool from within our JS pro after running this model we get back a feature class detected objects and these get stored back in the ArcGIS data store when it comes to scaling up inferencing there's actually no hard limit on the number of raster analytics servers you want to use it really depends on the size of data you need to analyze here's an example of scaling processing using rest analytics detecting building footprints this took us about 4.5 hours on a pro instance with a single GP 100 card scaling the process using rest analytic bought down our processing times to 20 minutes that's a significant gain in performance so summarizing and the key takeaways archaea is supports end-to-end deep learning workflows all the way from capturing training samples to running inferencing we support the key deep learning categories within arcgis we support a variety of model types and we show demos for most of them the deep learning tools are accessible through the platform as GP tools and Archie is Pro through the and through ArcGIS learn modules in the ArcGIS API for Python and it's also available as rest analytics servers to be run through him it's over lastly your deep learning processes can be scaled massively using em it's over this is a bunch of resources which will definitely be useful for you it's there's a ton of content that's available for developers and users alike join a GeoNet community and geo a blogs is well on medium so that's all we have time for and thank you so much for attending Thanks

Channel: Esri Events

Views: 16,632

Rating: 4.9521914 out of 5

Keywords: Esri, ArcGIS, GIS, Esri Events, Geographic Information System

Id: adrqRm4q5Ic

Length: 57min 33sec (3453 seconds)

Published: Tue Apr 21 2020