Esri UC 2015 - Analysis and Geoprocessing in ArcGIS Pro

yes we'll be talking about arcgis pro analysis and geoprocessing my name is Drew flater I work on the software development team in Redlands working specifically on an analysis in ArcGIS Pro there's some great capabilities in ArcGIS Pro some things that are new specifically to pro some new capabilities new tools and that's part of what we're going to be sharing with you we're also going to be just sharing kind of a migration story if you've worked in arcmap before run geoprocessing tools used model builder some of this stuff will be very familiar to you we just want to kind of introduce you to working in pro know what my colleague introduced herself morning everyone my name is Jen Langer I'm the product manager for spatial analysis at s3 and Joe and I are doing this presentation for me I'm doing the first time with him and I can see that he's a great presenter and he has really cool demos no pressure okay so first off I just wanted to hear from you guys about if you've downloaded and used pro yet we've had the first release of pros in January ok that's great I have to see a lot of hands so we had the first release in January last week we released a 1.1 version which has more analysis capabilities you're going to see a lot of that today so I hope that you're excited to to kind of get a taste of what's new you maybe haven't had time to look at for a 1.1 or start investigating that but there's a lot of great stuff that just builds on on the work that was done in the first release of Pro just to start off I want to talk about what is analysis there's kind of a lot of things that that could mean to different people kind of in its most basic sense analysis is just transforming raw data into information or knowledge so you start with something that doesn't necessarily mean anything by itself it's the the raw data and it's transforming that it's making it so that it means something and that you can make a decision based on that data and find a pattern or or be able to really just make a decision based on that spatial analysis does exactly what I just described for Geographic or spatial data so that's why we're here that's one of the strengths at at ESRI software is that we have this spatial analysis capability that it's you know it's great to be able to visualize Geographic data but taking it a step further and actually making that data into something that changes lives and is impactful is really important spatial analysis answers a lot of where questions so some of the common examples is where's the best location for a new community center where's an area with statistically high crime rates where has the landscape changed in the last 10 years and you guys know the more of these types of questions they're the questions that you probably ask on a daily basis and that you try to solve so analysis in arcgis pro really provides some interesting and incredible analysis capabilities in 2d 3d and 40 pro is this great native 2d and 3d application but 40 as well you got to remember time and Saudia processing and analysis there's a lot of capabilities that leverage the time aspect of your data that let you find trends and patterns across time and of course there's general I'm aware of it in in pro as well and we'll see a little bit of that through the time slider if you have have heard about that but we'll talk about that in dem in a demo Pro is great for running geoprocessing and spatial analysis because it's scalable it's a 64-bit application so jobs that may have failed in in arcmap because of out of memory errors may now succeed in pro just because it's it has this capability to use more memory on the 64-bit system it has non-blocking threading which allows you to be productive doing other tasks while your analysis is running so say you have something that takes six hours you can kick that off in the morning and still be productive doing other things in the application further for the rest of the day and it has just incredible visualization capabilities you guys saw Pro several times in the plenary yester morning and you'll see during some of our demonstrations that the visualization of of our planet and of geographic data inside Pro is really top-notch so the things that we're going to talk about today are geoprocessing which is really the technology that ties spatial analysis and enables spatial analysis in arcgis desktop and server and really across the platform we're going to talk about spatial statistics we're going to talk about analysis extensions including spatial analyst Network analysts 3d analyst and geostatistical analyst I want to make a note about those an analysis extensions we often get the question is there a new licensing paradigm between our gist Pro and arcmap the answer is that the exact licensing applies so the the license that you've purchased and that you have through your companies that you use in arcmap will be the tool will be the the license that you can use in pro and we'll also talk about ready-to-use services which are some really cool cool functions and capabilities that are available across the platform for running spatial analysis how you accomplish analysis in arcgis pro is really starting through the analysis ribbon tab so pro is this ribbon based application where there's all of these kind of menus up at the top you're all familiar with Microsoft Office applications we went to this ribbon interface in order to to present the rich variety of tasks that you can accomplish in a more simplified way and in the ribbon is something that we feel helps helps with that so it provides access to some of the most powerful analytic tools right kind of on the face of the ribbon tab you can access all the geoprocessing tools for doing data management and spatial analysis you can see the history of the tools that you've run inside your project you can execute Python from a command line can work in model builder there's a network analysis and imagery processing all of this is accessed through the analysis ribbon tab and we'll see that very shortly first again I just want to introduce another topic which is what is geoprocessing do your processing can mean a lot of things to a lot of people so I want to establish it in that geoprocessing is a rich suite of tools for processing Geographic data so there you see the processing and the geo which when switched around geoprocessing so it includes tools for doing spatial analysis as well as GIS data management so really anything that you can think that you've done as a GIS task there's likely a geoprocessing tool that'll help you do it in an automated fashion a typical geoprocessing tool processes input data and produces an output data set one of the most common examples that we have is is doing a buffer so your buffer a map layer and you create areas around that layers features the outputs going to be another data set that contains those buffered areas geoprocessing is also a framework that you can use to model and automate your processes in your work using either Python which is the coding and scripting language of ArcGIS or you can use model builder which is this great diagramming visual scripting language where you can build a process out and we're going to be demoing both of these things quite a bit today so let's start with the geoprocessing demo so I'm going to come in here and open up open up a project so just as a little bit of backstory while the project is opening here for the last month or so the lake fire has burned 30,000 acres of forest here in Southern California are people familiar with that do you remember seeing that in the news yeah it was it was a big deal the fire was happening in the San Bernardino National Forest and in the San Gorgonio wilderness so here we are in San Diego I just want to do a little fly to so you guys can kind of see the relation it's probably about a hundred miles or maybe 150 miles from here in San Diego much closer to our corporate headquarters in Redlands it's just over the mountain from us so this fire burned near several mountain towns and was really close to a number of campgrounds it burned huge areas that had not had wildfires for over a hundred years so you guys might have heard this in the media that was covering it but it was a very impactful fire the fire impacted these towns and campgrounds as well as forced recreation activities like hiking and biking trails so this is a map of the campgrounds and other Forest Service facilities in the area as well as fire data from the National wildfire coordination group and what we're looking at is actually a time series of data so we have 17 days of fire boundaries the fire started in June 17th and lasted lasted about a month but the last data point that we have is on July 4th which was kind of the height of the fire so I mentioned before this notion of a time slider when you have time aware data in ArcGIS pro you have this experience for being able to dynamically set the extent of data that you want to see based on the time so we see here that my first time stamp was June 17th at 10:30 p.m. all the way through July 5th I guess at 6:30 a.m. what's the last time stamp so you can drag this thing around to dynamically set which date you want to set or which date you want to see in the map and you can just do some nice things like plain animation or step through the data to see all of the the time features a nice way to do that is actually by looking at some bookmarks here so I'm going to just zoom in on some bookmarks and we see that when I click that bookmark the bookmark is aware of the time that was set when you were actually looking at at this location so I set this bookmark to be the first day of data we can see where the fire started here on the mountainside I'll just click to see the next day the first day wasn't so bad the second day winds picked up and that fire really spread across the whole mountainside there so again this time slider lets you interactively filter your data to a specified time extent and what's really interesting is when you run analysis and geoprocessing tools it's going to respect that time extent that you have set so if you're only visualizing say two days worth of data when you run your geoprocessing tool it's only going to run again the data that's within that extent let's actually have a look at the last day of data which was on on July 4th when the fire was at its peak so here's my last day of a fire data we can see that it kind of spread over to this region so I want to do some analysis of the characteristics of this fire over time over the 17 days that we have data I want to look at the impacted recreation activities as well as the population and housing units that are in the near vicinity so the first thing that I want to do again I mentioned that when you're running spatial analysis in Pro you go to the analysis tab so here are some of the things that I already discussed with a gallery of some of the most commonly used and powerful analysis tools you can access all the suite of geoprocessing tools model builder etc so the first step that I want to do is to calculate the change in fire area day to day so I'm going to go back and and look at all of my fire data not just that last day so this is really a series of stacked polygons 17 polygons on top of each other the first thing that I want to do is add an attribute for the size of the fire and acres and there's a handy geoprocessing tool that lets me do that it's called add geometry attributes so I can pick my fire boundary and say that I want to add an area attribute in acres and it can run this tool and a new column will be added to my to my layer here that indicates the size of the fire in acres let's just have a little look at that so we can see over here this area geo column so the first day the fire was 86 acres the second day it jumped over 6000 acres and so on so now that I have this acreage field I want to calculate a new field using a percent change so I want to see from day 1 to day 2 what was the percent change of the fire so when you're doing field calculations we have another geoprocessing tool called calculate field I'm going to calculate for the fire boundaries we've got some nice little helpers in here like this percent change helper I scroll down we can see that this is actually a function that is going to calculate directly for me the percent change from the last values in the attribute table so when I run this simple calculation I can come back to the attribute table and see that I have this column area percent change which again is the the percentage change day-to-day so the first day there's there's no change but we look at the second day there when the fire jumped from 86 acres to over 6,000 acres that's nearly a seven thousand percent change so we can see some of the other other patterns in here a couple of the days of note with large percentage change are around the 24th and 25th and it was at this time that a civilian operated drone interfered with some of the fire operation a plane was flying over to make a drop of fire retardant materials and there was a drone in the area so that that plane had to turn around and that actually resulted I think you remember that the fire kind of started over here and was spreading and it was because of that drone interference that it kind of wept over to this other hillside and impacted just a several thousand more acres okay so the next thing that I want to analyze is the recreation impact to this popular hiking and camping area and I'm going to do that by summarizing the length of hiking trails that were burned as well as the number of people on average that use those trails each year so I'm going to use a tool that's new in ArcGIS Pro called summarize within which overlays of overlays a polygon with another layer in order to summarize the number of points length or area as well as attributes that are inside that polygon so I'm going to click the summarize within button up there and it opens this a processing tool that can just really quickly fill out that I'm wanting to summarize my fire boundaries the trails within my my fire boundaries I'm gonna pick this annual usage and say that I want to calculate the sum of of that fields that'll determine how many people were were impacted by this recreational recreation loss and I want to say that I want to calculate the length of the trails and miles give it just a second here I need to drag this layer down so it will become visible so we see here that the number of people that were that were impacted by the loss of these trails was was over 144,000 and there's nearly 15 miles worth of trail that was impacted by that finally let's have a look at some of the human impact to do that I'm going to calculate the population and the number of housing units that are within a near vicinity of the fire so I want to analyze the just the last day of a fire data again this is kind of when the fire was at its peak on July 4th so I just want to look at that kind of farthest extent of the fire to get a rough estimate of the area that was impacted I'm gonna do a buffer for the fire we know that there were impacts because of the fire felt even across state lines smoke from the fire was seen in Nevada and in Colorado so there were some air quality issues there but there wasn't really a a risk to life or to property from that but we're gonna just look at an area that smaller say 5 miles to see if we can can estimate the area that would be impacted that would you know potentially be in harm's way for loss of life or for loss of property so I've got this five mile buffer now the next step that I'm going to do is to add some demographic information to this buffer there's a new geoprocessing tool in ArcGIS pro called enrich layer that adds demographic and landscape attributes to any layer it includes census demographics land use or land cover information marketing tapestry spending habits all kinds of data and and all over the world it's not just specific to the United States you can find population estimates and that for anywhere anywhere in the world in this tool is actually running an ArcGIS online service and the service does consume ArcGIS credits so so please consider that when you're when you're running the enrich layer tool but it's really powerful and allows you to add very useful information to your layers so I'm gonna pick my buffer and the United States then I'm gonna come down and select a couple of variables like the total housing units in 2015 and the total population in 2015 so again I'm gonna run this tool and it's provided me the results which have this enriched information that there were 5,600 housing units that were impacted by the fire or that could have been impacted by the fire that we're in very near vicinity in 3,700 in total population the reason that the population number is lower than the housing units number is because there are a lot of vacation homes and homes up in the mountains that aren't lived in a year-round so those populations aren't aren't counted in the total population as so just in summary the lake fire was a very fast growing fire that had a huge impact on the landscape the surrounding communities and even the recreation opportunities in the San Bernardino National Forest I'm gonna jump back over into some slides again this is just kind of summarizing what we just in the demonstration excuse me that geoprocessing has a very familiar user experience and provides an easy migration for existing arcmap users geoprocessing supports most tools models and python scripts that already work in arcmap so if you have workflows that you've built that you that you rely on you should consider those as safe to be working in in pro the one caveat is that arc objects based custom tools are not supported in Pro pro has a different SDK it's not based on our cobb jex these custom tools were written kind of I guess in a different in a different API than what Pro has if you're interested more about that there's sessions this week about the the Pro API and how you can as developers extend it to build add-ins and just kind of a migration step there's a new tool that's called analyze for pro which checks model scripts models and scripts for unsupported tools data and Python code so that's just an easy thing that you can do both in arcmap and in pro to see if your the models and scripts that you use are ready to be used in pro there's a couple alternatives for creating geoprocessing workflows so we saw the workflow but I just did I just ran doing this field calculation summarized within some buffering in enrich layer those were a few manual steps that I did those could be encapsulated in a geo processing workflow either through Python code or writing it in in model builder using the tools and data the same tools and data that you that you can run manually to make a diagram that represents your workflow and we'll see more about that in just a minute minute I just wanted to make a special note here if there are people who who use Python in ArcGIS desktop are there Python scriptures in here yeah quite a few that's great so in pro the version of python that's used is version 3.4 the version 3 of python is really the future for the Python community arcmap is continuing to support python version 2.7 but pro will be using version 3 of python full python isn't included in the default pro installation there's really just enough to run system script tools and use the Python window if you want to be able to run scripts outside of Pro you need to have this optional Python for arcgis pro installation which enables you to do everything that you're familiar with in terms of python in in arcgis and python name pro includes a few new libraries which are really cool i I won't get into them much right now but if you're curious what they are we can talk a little bit later pandas Syfy and netcdf 4 in terms of what's new in pro 1.1 again 1.1 was released just last week so when you guys get back to the office I really encourage you to upgrade it's going to be more stable there's gonna be more capabilities and fill in more of the workflows that you guys had from from arcmap in transitioning to pro so what's new I already discussed how geoprocessing tools support time extents on your map layers model builder has undo redo and you can create model builder workflows from history so when you run geoprocessing tools it's storing history for your project it's really easy to build these these model builder diagrams from from that history in project packages when you share a project package it's going to include all the geoprocessing history the work that you've done that's really key part of your project so we're including in project packages now ok thanks to and this demo I'm going to show here is all about model builder and how many of you have are using have you been using model builder in arcmap okay great so the experience in RGS Pro related to our model builder is pretty much the same as how you use it in arcmap so in juice demo these show this workflow to do fire analysis and this is the lake fire boundary into 2d what if I want to apply that same workflow to summarize the total usage of affected trails and find out how many people are affected how many housing units are affected what if I want to apply the same workflow in a different area so for example last year along the same time we have a wildfire in San Diego area I know the rain made it almost invincible but that's not the rains not normal wildfires are more normal in this time of the year so what if I want to apply that workflow to the San Diego fire so what I can do is a build model that incorporate all the steps that Jew did so again we access everything related to analysis through the analysis tab I'm going to start with the model builder with a new model and I'm obviously want to give it a name label again drew mention that model builder is a visual diagram visual stream scripting to what I really like is I can simply drag and drop from the history because we run summarize within - before I can just wrap it drag and drop it on my model and I have the first component a little nicer and then because I want to allow people to provide a fire boundary at the runtime I'm going and us as well as the trails so I'm going to make that a model parameter so I ran the summarize women let's check the parameters they look okay to me and the next thing you did was buffered this output from the farmer I summarized within I drag and drop again to my model this tool and I can use the output from my summarized version as the input from my buffer - now let's fill in the parameters which is buffer or five miles and the last thing I want to do is to enrich the output from my buffer so that we can find out how many people are live in our factored in this fire area again I track and drop the enrich layer to and connect the components and I think obviously it's for United States as Jim mentioned this is actually our geo processing service hosted in ArcGIS online as you can see we can use it as just like a regular geoprocessing tool in model builder so I believe what he selected was 2015 total population as as 2015 total housing units okay there I have my model I can I should make this as the model parameter as well so that I can a user can provide the output name and go ahead and save this model and let's give it the test I think I'm really brave to do this on the fly okay so let's go ahead I run this model as you can see at the input the fire boundary now I'm selecting the fuhrerbunker for a sonido area as well as the San Diego recreational trails I want to give it a little bit more meaningful name and go ahead ran it so what it does is it's running the steps I specify in the model first summarized within if you remember it and then buffer five miles I enrich layer applied the affected population and total number of housing units thing is now executing the enriched layer so far so good Wow I was really lucky and now let's change the symbology a little bit to make him not so bright right and just prove that it worked my output has the total number of housing units total number of population as well as summarized you search for the recreational trails and summarize lengths in miles okay so that's the demo of model builder what I really like about it is that it you can really build a model without writing any code so let's switch gear a little bit we like to talk next about some of the and vertical extensions available in RGS Pro I'll start with a special endless extension how many of you have been using specialist for arcmap okay great again the user experience remains very similar in our chest Pro and spatial analyst is one of our most popular analytical extensions is available not only in our chest for tests well it's for desktop including arcmap and our chest Pro it's also available in our chest for server our chests for runtime our chess engine etc spatial analyst extension provides spatial modeling and analysis tools for cell based data which is also called raster data our dimensionless the user experience in our GS Pro for spatial analyst extension is very similar as what you're already familiar with because of the Advanced spatial modeling capabilities for raster data spatial analyst helps us to answer complex questions such as where is the best site to build my new store and this is suitability modeling we can also perform density analysis to find out where point features or line features are most concentrated so for example if you are a business owner you probably want to find out what are you was the distribution of your customers we can use specialists to perform distance analysis to find out the straight-line distance from anywhere on your raster layer to any other cell and we can also create weight weighted costs surface and we can use visual analysts to delineate watersheds trace how water flows not hydrological analysis and finally we have a powerful language called math algebra for raster data analysis just to put in simple words map algebra is like applying mathematical operations on raster data and there's also raster calculator which allows you to write much create math algebra expressions using a dialog or use model builder starting from arches desktop 10.3 release we have some of these spatial analyst functionalities available as ready to use services these are mostly focused on elevation analysis such as profile summarize elevation and create viewsheds as well as some hydrological analysis tools like trace downstream and dealing in delineate watersheds the really big advantage of is ready to use services is not only these are analytical services hosted in our chests online also we host the data that you need to perform these analysis my personal experiences when you perform analysis a lot of work is spent on finding the right data I'm making it fit for my analysis and when you do the profile or create a new shed using these ready to use services you really don't need to worry about the data set because data is part of it it's rain it's like running a direction from A to B you don't have to worry about the network data set behind it so these ready to use services allows you to perform these spatial analyst extension functionalities without having to have the sphere unless extension in RJ's Pro you just need to make a connection to an artist server so we have the detailed connection parameters specified on the slide and all you need is organization subscription to our chess online as well as some service credits so these tools do consume RGS online service credits what's new in RJ's Pro 1.1 regarding two spatial analyst we have some new image classification tools that require spatial analyst license again we are continuing to make more spatial analyst tools available in using parallel processing in order to increase the performance now I would like to show a demonstration using spatial analyst spatial analyst extension to perform suitability analysis our scenario here is to find out the best place to build a wind farm in the North Sea area and when energy is considered up clean energy the energy for the future and because of its natural conditions the North Sea area has one of the most number of wind turbines in the world so I will talk about suitability analysis the first step is always to identify the criterias that are relevant to our decision to build a wind turbine obviously we need area that has adequate wave power and wind speed to lower the cost we also want to build it close to our existing ports and also try to avoid heavy shipping traffic areas finally also cost related we want to build wind farm in relatively shallow water so once we have identified these criterias which generate those using spatial analyst tools and then the next step naturally is to combine them so that we can find the best sites but before we do that we have to make them in the same scale because our wind power is in measured in one unit our distance to seaports measured in the different units we can't really add those together so the next step is making the classes goes in through the reclassification and rescale process to make them into the same scale from zero to one meaning zero is that is not a member in my in this case I'm using the fuzzy logic zero means it's not a member in my fuzzy logic set member set and one means is the most suitable member in my logic okay once we are having them in the same scale now we can overlay them this is really the I would say the wonderful thing about spatial analysis we can combine these layers in order to find get the result the result really shows us where at these areas which are the best areas to build a wind farm so after all that talk and promise that this is really our true demo is not only power slide PowerPoint slides so I'm going to start the project just a no cherien RGS pro you can only have one open project at a time so that's why I had to close my previous demo and open this one I have a lot of data layers in this project okay I think if I do more of these demos life and it's not good for my heart condition okay share on my map I have some boys for the North Sea area which contains wind speed and wave power information I have the ports in this area I have the ship as points and then I also have the PO symmetry that tells me the elevation of the seafloor okay so the first thing I want to do is interpolate my boys to generate a continuous surface for the wind speed and wave power there are many many different interpolations tools and the ones I'm going to use here as example is the ITW tool available in special and the most extension so I'm going to choose my seat boys and the value I want to interpolate is wind speed I'm just going going to go ahead and run this okay there I have the surface for my wind speed I can perform the same thing like say for the Seaport I can do you could Euclidean distance in spatial analysts to generate the distance surface and as well as for my ship I can use kernel density to generate the density surface for my ship traffic so in order to save time I have these previously created already so here I have let me turn off the buoys so you can see better here I have the wind speed surface and here I have the wave power those are created using IDW shipping density a result from kernel density and then distance to ports results from Euclidean distance and also I use the extract because my per symmetry is already a raster surface what I did was I use extract values to select only the ones which are shallow enough it just so I did that so I can combine them so again an idea these are the layers I can combine them remember before I combine them I have to rescale them first you probably can guess by now I really like model builders so I create a model to show that to show the process in this case using model builders not only good for me to automate my workflow it also helps me to explain what I did in my workflow so the first step here is to generating to generate the raster surface and then the next step see this fuzzy membership that's the reclassification rescaling process that turns my each of my surface into the zero to one scale and once I have that I can perform the overlay and get the result I could stop from there but really I wanted to be more correct because in the area we also have some natural protected areas so what I did from the result I performed a condition statement remember the map algebra so that's basic I select only the ones that is not part of the protected areas so let's go ahead run this model I know you trust me that this really works but I still want to show it that in a life just somehow I think make it more convincing I may have said that too early I'm not the part of a development team so anything that goes wrong with the software that's all juice fault thank you okay so I'm just running the model as another nice thing of model builder is you can see where is currently running is highlighted so now let me go to my map voila everything works today so I have the final suitability surface drawing and the ones in light green are the more suitable areas okay that concludes my demo and my part of presentation back to you Joe thanks a lot are people familiar with that kind of workflow do we have people that do site suitability yeah that's like braiding butter stuff right for for a GIS analyst that's that's really great and it's great to see it in pro too so Jen just ran a suitability analysis in Europe's North Sea I want to apply that same methodology to find a good location for a woodwind farm but this time in in the Mediterranean Sea now Jen said several times already that she really loves model builder I love model builder too maybe not as much as as Jin it's great for modeling and automating processes as well as like like Jen said documenting so you can see you know this is how I produced this data set but sometimes I really just want to get back to my true love which is Python this suitability methodology was originally developed in arcmap so i actually have an old script tool that i used in arcmap to run an automated version of this suitability analysis similar to the model builder workflow that was just shown so here I have a Python script and it's doing just just the exact same things so using the arc PI module that's provided in ArcGIS for use in Python I can run tools like IDW Euclidean distance kernel density extract by attributes all of these fuzzy membership tools to transform them to a common measurement scale and finally do the fuzzy overlay to produce the final output so let's try this out in pro first things first though is I want to analyze the tool to determine if there's anything that I need to do to update it for use in Pro like I said it was it was a methodology that was developed in arcmap so there might be some things that that wouldn't work so I'm going to use the tool analyze for pro and I could browse to and select either the py file or the toolbox where where the the script tool is contained I'm going to do that select the toolbox so when I run this tool it's going to print out any any problems that there are in the script so if I look at this it's giving me an a warning message saying found python two to three errors in line 26 line 27 in line 28 so these are are the problems that I'll need to address in order to have this script tool be able to execute successfully in pro so let's have a look at at the script again and see if we can't fix things up so here's the script tool I'm going to right click and say edit and it's going to open it just in my text editor here if you remember it was lines 26 27 and 28 since this is a demonstration I didn't know exactly what the problem is and the problem is that in Python 2 the print function which was how you added a message to to the display was a statement so it was syntax like this print space and then some some variable in Python 3 it's a function so I need to put these prints inside of inside of the parentheses here so if I do that and save if I would run the analyse tool again I would hopefully see that those those error messages were no longer having a problem so now let's try to to run this script tool so here we have my script tool where I've defined a number of parameters again Buie locations the ports ship detection layer both imagery the the depth protected areas layer as well as a study area it'll produce a final output here called Mediterranean suitability so when I click run just like we saw in the model builder example it's going through and running those tools sequentially and will produce a final output which will show up here shortly so here the analysis in the Mediterranean I find to be very interesting because the best place to develop wind suitability is in the French Riviera so I don't know how the the French people would respond to a bunch of of wind turbines offshore but that's really the the best place for them it combines all of the characteristics that we've determined to be important for this suitability so with just that minor change to my Python script that was all that I needed to get this script tool running in pro and the good thing is that this change the print the print statement change that's one of the really most common things that you'll need to change when you're moving from Python 2 to 3 there are other changes and we'll go over some of the tactics that you can can take to avoid these kind of problems in another session it'll be covered either in the Python intro session or if you're interested we have a panel discussion on Thursday morning a pro panel discussion that talks about using arcmap and pro together so if you're interested in in that feel free to come by that session okay a few more slides let's talk about 3d analysis so this is another analysis extension that's provided it has over a hundred geoprocessing tools for a number of different scenarios you can build elevation surfaces and analyze them you can do some of the interpolation that's also in spatial analyst you can use use these tools with vectors raster's and tin based models as well as light our data set that's called last data set so it has some great capabilities for analyzing and visualizing this lidar data by loading it into this last data set we'll see that in the demo in just a minute you can do some other things like measure distances and proximity in 3d as well as spatial relationships so for example is this telephone line intersecting this crown of a tree in three dimensions of course the answer would be different if you're looking at it top-down and doing a 2d analysis versus if you're doing doing things in 3d and it has volumetric and visibility analysis capabilities as well so we've already seen a couple of demos that have a little bit of 3d in them but the important note here is that the 3d mapping and visualization capabilities in Pro are included like I mentioned it's a native 2d and 3d application so you can do all these mapping tasks without needing any extension it's when you want to do the the analytics the analysis capabilities that are provided through this extension that you need that you need to buy that so what's new for Pro 1.1 there's a suite of new tools I won't take the time to go through and describe all of these we'll see a couple of them in a demonstration and I do want to point out that there's a new a new rendering technique again that's not analysis that's something that comes for free with pro that's a point cloud drawing I think that maybe you guys saw the the drone point cloud symbolization yesterday so this is a similar technique in pro the picture here that's in the bottom left that's not an aerial photograph or that's not even an aerial photograph that's been draped on to a lidar point cloud those are the actual lidar point feeds that produce this ultra realistic visualization of an environment and there's my famous quote it's amazing so let's have a look at this are there 3d analysts users in the room today yeah a lot that's great so the demo that I'm going to be looking at actually first off I want to show this this last visualization I think we have time and just gonna do it really quick before I get into the analysis just for a little bit of a WOW factor so this is a scene view inside pro this is the overview of a last data set what a last data set is is simply an Lal ASD file that has references to light our data on disk so it's a really quick and easy process which just wraps on top of raw lidar data and provide some added value added capabilities in performance it has great performance in pro so we've got the overviews which is kind of a typical view when you first add a last data set as we start to zoom in we see the elevation the the point cloud being represented using elevation that can just kind of keep coming in closer here we'll see more and more points continue to draw I forgot to mention that we're looking at three gigabytes of data here so that's not huge but I mean that's pretty pretty big we're looking at millions and millions of points here so this is the elevation that was the return on the the lidar points I want to switch this simply by going to the last dataset appearance picking RGB switch it to that true color representation that was also collected at the same time in the lidar returns so I switched to that let's just fly around to you a little bit and see this virtual environment kind of fill in so here I just have a nice hillside again these are fully 3d data points they're not just a representation or an image draped we're working with actual 3d objects here and I could do this for hours probably just fly around and navigate and check things out it's it's a lot of fun and there's probably there's probably a few work uses that you would do it to I you can talk to the 3d mapping people about that but I mean just for entertainment value flying around these these last data sets is a blast and yeah just the the visualization of it's really fun okay back to the analysis the important stuff right okay so I'm loading up here a 3d scene of downtown Philadelphia the project that I'm gonna be working on is that a developer is designing in a new apartment building and I've been tasked with assessing the view of the cathedral basilica of st. Peter and Paul from this Park apartment building so we see over here the little yellow diamond shape above the cathedral and we're evaluating for this selected new apartment building who can see the cathedral from there from their apartment windows so I'm going to perform a line-of-sight analysis to determine which parts of the building have an obstructed view of the cathedral and this is a somewhat complex but well-documented workflow I'm going to just use again a model to illustrate the steps that are required for this analysis so this is a new a new function in pro 1.1 that you can apply a city engine rule package to create a grid of panels around a multi patch so the input is a selected building here and I provide a city engine rule package and RPK that defines these kinds of rules for how the multi patch should be transformed into this panel this 3d panelized version of the building so it creates a grid around the building and outputs again more multi patches and 3d points and we'll see that in just a little bit the next part of the analysis is to construct 3d lines between target and observer so I mentioned that it's generating the last tool features from city engine rules is generating this grid around the building and so I'm taking each one of those grid points and constructing a sight line between that observer and the target which is the Cathedral so this line of sight analysis is always you construct sight lines which are these 3d lines and then the next step is to run the actual line of sight which determines along that 3d line what parts are impeded by the the 3d environment I have some extra steps down here which are doing some joins to move information from those sight lines to the building grid I won't get into the nitty-gritty details if you're interested in that again we can talk a little bit later for now I just want to to start running some of this analysis so let's start just by running the the City Engine application of the City Engine rule package and we can see this grid that gets generated so there's my grid that's generated around the building and again that's very useful for doing a line-of-sight analysis because it's not just one point that you're measuring the visibility of it's now a facade a grid of all these features so I'm gonna run the full line of sight now which will construct the sight lines and then determine what parts are visible and which parts aren't visible so there we're getting all of these 3d sight lines between the points and in the end the building and if I finish up running this model it's going to take all that information from the line of sight analysis and move it over to the building panels that's not showing up right now that's perfect timing I'm going to just kick it off one more time just so just give it another shot it's worth that I think oh so this is a lesson about arcgis pro it's still it's still in its early release phases if you find crashes or bugs please send the error reports that we can so we can make it better in the future so I'm not going to go back through that all and try to load everything back up but you guys I think at the point that you're constructing sight lines and then doing this visibility analysis it has a really nice payoff at the end so if you're curious about that let's let's look at it a little bit later on down in the the showcase area okay let's talk about a few more a few more things just really quickly network analysis has has a lot of capabilities in arcgis pro and really across the platform there are simple things that you can do for network analysis like routing in directions up to very complex things like a vehicle routing problem which is like a logistics type application where given a suite of delivery vehicles for example you can say what is the the best order that this suite of vehicles should make their stops and deliveries to save gas to save man-hours all of these kinds of things so there's simple things that you can do and really complex and interesting problems that you can solve network analysis in arcgis pro has a number of geoprocessing tools for doing both 2d and 3d network analysis and it's all driven through a contextual ribbon tab workflow so you can find and use the geoprocessing tools but really a nicer user experience is to use the network analysis ribbon so as you're doing network analysis this is all in the duct I mean in the documentation but this is kind of the equivalent of the network analysis toolbar in arcmap it's just been put in the ribbon so it allows you to do these nice workflows where you can make a few changes solve the network problem you know make a few more changes solve it again it's the nice iterative process and again network analysis has ready-to-use services you can use these capabilities without the extension it's exactly as as Jen described for the spatial analyst services it's using curated worldwide Street Network data set so if you want to do a network analysis but you don't have the street network for that area or don't know how to set up and configure one of these networks you can just use use the ready to use services you make an ArcGIS server connection but again you are paying because it is so easy to use with rjs credits okay we don't have a demonstration for network analysis but there's a lot of other demos and especially down in the demo theater area if you're curious about pro network analysis please check that out downstairs in the showcase area i'm going to jump to statistical analysis and that'll be kind of where we finish up in the last 15 minutes so when we talk about statistical analysis there's kinds of two things there's the gist it's geostatistical analyst extension which again is a paid extension and then there's the spatial statistics tool box which is included in the in the core the core installation of ArcGIS it's a little bit about the geostatistical extension it includes modeling tools for creating statistically valid prediction surfaces and a really important part of it and an important concept in geo statistics is that it also predicts uncertainties in its predictions so you'll get this nice surface that says here's what we expect values to be based on my geo statistical model but there is this range of uncertainty that goes that's associated with it because statistics is you know can be a little bit fuzzy sometimes so Jen already introduced this idea of interpolation that you're predicting between known measurements and that's really what geostatistical analyst is great at it has a tool called empirical Bayesian creaking which is an off-the-shelf tool for calculating very highly accurate interpolation surfaces without having to be a statistics person really you can configure it to to be exactly what you need but if you just provide a data set and run it it's going to do a great job I just want to mention that there's no geostatistical wizard in arcgis pro yet that'll be coming in a future release so you're a user that's done creaking modeling and that kind of thing using the geostatistical wizard in arcmap you won't have that capability in Pro 4 for another release or so okay a little bit about spatial statistics it has tools for analyzing distributions patterns processes and relationships again it's doing this analysis in 2d 3d and 4d 4d has really come along in the last couple of releases specifically in ArcGIS pro 1.0 and 1.1 through the inclusion of a new tool box called the space-time pattern mining toolbox did everybody see the becks presentation yesterday where they had their sales data and they were determining crop sales and looking at patterns of sales over the last number of years that was using space-time pattern mining so given just this huge you know cloud of data how can we find patterns and interpret where there's a lot of something or where there's a little of something so space-time pattern mining is really great it aggregates data into this new data structure and then helps you identify hot and cold spot trends and there's new tools in pro version 1.1 that help you visualize that space-time cube and that's what we're going to look at for the last demonstration this morning so again I'm going to start setting up setting up the story as my project opens here and I've got really some some breaking news especially to people that were here this weekend and that's that California is in a drought has anyone heard that yeah of course you have well yes we are in a drought there was record rainfall this weekend but as Jen said that's really out of the out of the norm I actually heard that this weekend had more rain than the last 77 July's combined in San Diego that was on Twitter so don't like fact check me or like say drew said this drew said that it was on Twitter so take it with a grain of salt so for the past several years many areas in California have accumulated less rain and had a lower snowpack than any other year on record so that goes back the records are to about the 1850s and we've been getting on average about one third of expected rain based on trends from the last hundred years so one third of rain results in very dry a very dry environment the US Drought Monitor shows that a hundred percent of the state is under a severe drought that's every area you know it's not just Southern California its Northern California as well we're not getting the rain that's needed and in April California's governor ordered mandatory reduction of residential water usage however there's been no mandatory cutbacks in the use of water that's used for agriculture which actually accounts for 80% of California's water use is there anybody in here that does agriculture GIS application yeah a couple people that's great so this might be interesting to you guys so that brings us to to this map this is California's Central Valley and it's one of the most productive agricultural regions in the whole world agriculture is a forty five billion dollar industry in California and produces nearly half of all US grown fruits nuts and vegetables that's crazy right half half of the cold countries produce central California is also one of the most irrigated places in the world like I just mentioned we don't have rain typically so we have to get water from other sources whether that's from out of state from aquifers from the few natural lakes and rivers that are in the state and from snow snow pack that's that's up in the Sierra Nevada mountains so let's have a look at a specific a specific County here Merced County here in in central California is the fifth largest agriculture producing county in the state it's about three billion dollars of that forty five forty five billion so looking at the map we see that nearly all of the land in the county that's not a town or a mountain Foothill is is agriculture so let's have a look if we get a little closer here we see the agriculture layer drawing in and we can see really just kind of these brown areas which are foothills of the mountain and we can see there's some towns like Merced Atwater some of these areas aren't agriculture but pretty much everywhere else in the county is Merced County has some really beautiful landscapes this is a just a part of the county that I like to show off because it's one of these kind of landscape is art type places if you ignore that layer yeah you'd look at that you would think it was an art installation somewhere but that's the agriculture in Merced County so let's actually look at some data here this is a data layer for Merced I think I gotta change this a little bit there we go so this is a map of agricultural water usage over the last 12 years in Merced County again we have the time slider enabled up here which means that it's time aware data from the data ranges from August 2003 to August 2014 so last summer we're using just water usage data from August of every year since water usage is highest in that month because of the dry weather and heat of the summer we can manipulate the time slider again to reduce the data that we see for example we can look at just the first year worth of data so here we see that now it's been limited to August 2003 to August 2014 and again I got to change my zoom level but what we're working with is actually about 150,000 points which are the farms in in Merced County but they're all kind of stacked on top of each other because again it is 12 years worth of data I'm going to go back to the bookmark that includes the full data set so it's really impossible to to look at this map of water usage data which has 150,000 data points to look at it and try to extract any patterns with your eyes you know you can see some areas that look white some areas that look blue but those areas could have data stacked beneath them which represent higher or lower data usage so being able to analyze any patterns for the last 12 years worth of water usage is really impossible at this point it's important to find these patterns though so that the county agriculture commissioner can monitor for excessive usage and even target irrigation training and site visits so Merced County has a great Agriculture Commissioner who has a lot of GIS data a lot of interesting things they're applying interesting techniques to help solve this exact same problem so using these new space-time pattern mining tools that I just explained can help to analyze this huge amount of data so when you're using the the space-time pattern mining tools it's it's done workflow in which you aggregate all of your data into this into this space-time cube so what I'm doing here it's producing a net netcdf file a dot NC file with the input being my my big water usage point data set it's going to aggregate into a grid that's one mile by one mile so it's not going to be looking at each data point specifically it's going to a great it into this block and calculate kind of averages based on all the points that are inside that one mile by one mile grid now on top of that it's using a one my or a one-year time step interval so it's gonna look at data that's within the same year 2003 to 2004 then 2004 to 2005 it's going to look at both the X&Y as well as the the time dimension to help us determine some hot and cold spot patterns there's a new parameter here on the create space time cube tool that's new in in Pro 1.1 which is called the summarized Fields parameter previously this tool only have the capability to count point but since we're not really interested in measuring the number of farms in each of these kind of blocks were interested in water usage it's really key that we can set this water usage summary field and say that we want to sum the water usage so what this is going to produce is this 3d 3d space time cube that has aggregated all my data and well that'll make a little bit more sense as I as I continue here when I run this it creates a netcdf file I'm not going to run it right now it didn't take that long it's a hundred and fifty thousand points and it took about eight seconds to build this netcdf data cube the next step that I want to do in this workflow is to run the emerging hotspot analysis tool so I provide that netcdf data cube that I just produced and pretty much just give it an output feature class and it produces the layer that you guys may have seen a little bit earlier and to turn this on so running the emerging hotspot analysis tool produces this summary summary layer which includes some trends in the data so we see some points that are persistent hotspots that's these dark red blobs that means that the water usage has been very high in each of the 12 years there's persistent cold spots which means that water usage has been low in each of the last 12 years and then there's some interesting patterns like oscillating hotspots which means water usage has kind of gone up and down and one of the most interesting I think is the intensifying hotspots which means that water usage used to be lower but has been in the last few years really going up so this is an interesting summary layer what is really useful though is being able to visualize the whole cube inside inside of inside a pro so using this tool called the visualized space-time space-time cube in 3d tool you can take that cube and make it into 3d features so I'm going to just fly around a little bit in here this is that kind of same same summary type layer where it's displaying hot and cold spots but as i zoom we can start to see all this data that's stacked on that shows the true patterns in the data so we can see the first the first cube that's kind of down at the bottom represents the first date point so again that started in 2003 and then moving up in in the Z it's getting to more and more recent dates so the top cube on the top of each one of these kind of pole looking things is data from the most recent year so we can see a lot of patterns where specific one mile by one mile areas have been a hot spot in water usage at different points during the last 12 years so I'm going to just go through and look at a couple of the areas these are the persistent hotspots again that's very easy to see now when we're looking at the full space time cube it's red it's a hot hot spot this 99% confidence that it's a statistical hot spot over all over the last 12 years there's these persistent cold spots which again will look kind of like those but they'll be blue all the way up so these have been very low water usage areas of the county over the last 12 years and the last thing that I'll point out is these intensifying hot spots which are again places that may have used to have been hot spots and they're getting hotter and hotter as time goes on so they're using more and more water in summary the California Central Valley is home to world-class agriculture though it requires really substantial irrigation and while water usage is high throughout the state we can use pattern mining tools to find trends within this big data set especially over time if you're interested in more about about the space-time pattern mining you can talk to somebody down in the showcase or again there's more sessions throughout the week dealing specifically with this we just have a couple of slides more to finish up I want to thank you we want to encourage you to please fill out the session survey using the mobile app you can just find the arcgis pro analysis in geoprocessing session and click the workshop survey if you want to learn more about spatial analysis the desktop ArcGIS comm site is the best place to go you can find all kinds of information as well as do case studies that provide the data and tools that are necessary to run some really interesting analysis and solve some great problems there's a lot of analysis sessions this week I'm going to leave this slide up so you guys can take notes if you want the thing I want to point out is you can go to this address ESRI URL comm slash analysis you see 15 to get an agenda which is all of the analysis sessions this week I think that's the last slide so I'm gonna just leave this up gin and I will be up here if you guys want to ask questions we are a little bit over time thank you guys for sticking it out with us we hope that you learned something about analysis in pro thank you [Applause] [Music] [Applause]

Channel: Esri Events

Views: 8,187

Rating: 5 out of 5

Keywords: Esri, ArcGIS, ArcGIS Pro, Analysis and Geoprocessing, Geoprocessing, spatial analysis, geography, GIS, ArcGIS Capabilities, ArcMap to ArcGIS Pro, Esri UC 2015, mapping, maps

Id: 9P5ciTVmGQY

Length: 77min 54sec (4674 seconds)

Published: Tue Feb 20 2018