python Essential in arcgis

hello and welcome to user conference 2010 I hope it's been good a good couple of hours for everybody and hope that you have have some plans to get some good information out of user conference this year today we're going to introduce Python and its role in ArcGIS will show some basic Python concepts and examine what Python scripting can do to improve your geoprocessing workflows just kind of as a for your information these presentation slides are going to be available on the user conference proceedings web page and the DVD that gets sent out in a couple of weeks as well as on the geoprocessing Resource Center so don't feel like you have to go crazy scribbling notes trying to keep up just try to absorb what what we're talking about what we're saying and you can get what we're get the the actual slides later on also we're going to try to take questions as we're going along so if you feel brave enough to have others hear the questions that you have for us please raise your hands and we'll try to spot you and get to an end point then take some questions but we have a lot to cover so we'll have to work quickly with that and of course we'll take some questions at the end just personally if you would rather do it that way so we have an expected audience for this session and I just want to kind of use this as a little bit of a an informal survey so our first point there is that you are new to Python scripting or at kind of a beginner level so if you can just give me a wave if you're at about that level okay that's that's a lot of people and you're in the right spot I don't want to make that sound bad so I guess one prerequisite that we have is that you're comfortable in ArcGIS so maybe you've used geoprocessing before maybe modelbuilder some of that type of functionality but you really want to start using Python so can I get another wave out of those people familiar with ArcGIS geoprocessing okay how about those that are moving to Python maybe from a different scripting language so you're familiar with some programming and just want to start learning Python a good number and maybe you're none of these three or you're all of them and you're just interested in what's new intent and everybody should wave for that one two hands so here's what we'll be covering today we'll try to answer kind of the fundamental question of why should I use Python we'll move to a short section on some general Python basics we'll hope to kind of get everyone to a starting point so that you know some of the terminology and functions that we'll be talking about as we go through the presentation we'll talk about what's new for Python scripting in ArcGIS 10 so for everybody who is waving with two hands that'll be a key part of course we'll cover executing geoprocessing tools in Python that's really the main reason that you're going to be here at this session today is because you want to do geoprocessing in Python so that's kind of our starting point when we're talking about scripting in ArcGIS will deal with messaging and error handling which are important when you're beginning with Python we'll get into some geo processing tasks that you can handle with Python and cover some arc PI functions that you can use to accomplish those tasks will cover batch processing which is again a very common use for Python as well as receiving arguments to make your scripts very flexible and easy to share with with your with your users the users of your scripts so let's get started with some Python scripting basics so this the first question I guess and it might be on some people's minds some others might understand it but what is Python what is you know what is all the fuss that that we're making about Python and Python scripting I guess from our standard view from the ESRI standard view is the scripting language of choice for ArcGIS and we like it for a number of reasons but a couple of those reasons are because it's free for you to you there's no additional cost in in using it like with some scripting languages it works on Windows and Linux platforms so it's cross-platform for those of you who may not use Windows it will work on other platforms and as well it's easy for beginners to use it's a pretty straightforward language once you get some basics down so it's easy to learn but it's also very powerful for our advanced users but that question I guess still remains why should I use Python in ArcGIS there's definitely some overhead to learning Python and how it works in ArcGIS so the question is is it really worth the time the short answer that to that is yes I might be a little bit slanted on that perspective but it's overwhelming yes and the long answer I guess will cover today and in other other Python sessions this week so Python just at its most basic point of reference I guess for arcgis is a way to run geoprocessing tools you can run just a single tool or a string --gel a string of tools so like a geoprocessing workflow but the strength of Python is not just in executing single tools but rather in executing those long and advanced workflows that we know that you make so that ability to develop execute and share custom geoprocessing workflows that make Python scripting powerful is hopefully why why you're here today and why you're interested in learning about Python and from a strictly cost-benefit perspective using Python to automate some of your geoprocessing work can be an enormous time-saver so just I guess from a cost perspective it's it's going to help with that so let's start with just some basic Python concepts what is Python code Python code is can be thought of as simply text text in the text file that can be viewed in any text editor or text viewer but where you actually develop Python script since python code is in an IDE or an integrated development iron environment so that's a program where you can write save run debug Python code some examples of idds are pipe Python Winn Idol as well as winged are some some common ide s starting with arcgis 10 you can also execute Python straight from inside an ArcGIS application in the Python window so we'll we'll show that in some of our demonstrations and talk about it as well so when we're looking at it at a Python file how do we know what different kinds of lines there are essentially how do I know which lines will execute and which won't execute so there's the the lines that will execute and anything that won't execute is usually thought of as a comment in a comment we'll always start with the pound sign you can see it there on the example so that pound sign indicates that it's a comment and that line will not execute so comments are used kind of as placeholders or as a place where you can document the flow of your script and that line won't execute everything else will try to execute now one of the very basic concepts in Python is a variable and that's one of the building blocks that you'll use as your as you're writing scripts so the question is what are these variables we'll use that word a lot today and so we need to start out with a good definition of that essentially a variable is just a name that stores a value it's assigned using the equal sign so you're just assigning some name equal to a raw value and that value can be a string so like a path to a data set it can be a simple number or it can be some other more advanced Python data structures and these variables will just act as substitutes for any raw values you can use these variables many places in your scripts including when you call geoprocessing tools you can stick a variable in the geoprocessing tool and it'll know what to what to do with that variable so and a very important part of python is is the ability to test conditions so you're using branching logic to perform one operation if a condition is met and another operation if that condition is not met the most common form of this is the if-else statement and that just reads something like if this is true execute these indented lines and if it's not true we'll execute some other lines so a colon is put at the end of each condition statement and how the lines like I said are indented beneath that condition statement determines what will execute if the condition is met so there's many different operators for testing conditions here we see mostly the double equal sign and that's used to test equality so that reads is equal to there's also much some other operators like some mathematical operators greater than less than less than or equal to there's many different operators you can use so here in our code sample we see we're just assigning the letter a a string of the letter A to a variable we'll then test that variable if the variable is equal to a will execute what is ever in that indented section beneath it and that if or that else statement in the fourth line I guess down is simply saying if the variable is not equal to a so that's the the false part of the the condition testing and if it's not equal to a we'll do whatever is indented so print variable is not a another important thing that is built into the Python language is the ability to iterate or loop and this keeps us from having to reproduce any code that we want to want to run more than once and we put everything that we want to to run or iterate beneath a loop statement and indent it so there's a couple of different kinds of ways that you can accomplish looping or iteration the first is a while loop and so you see just in this example we have X equal to 1 and while X is less than 5 we just will go into that statement and do anything that is indented so in this case we'll print X and then we'll increase the iteration the x-value by one to continue the iteration and this will execute until X is greater than five another kind of loop is a counted or range loop so you can see in that middle example that we call a range statement and we check for any values in the range starting with one and ending with five and then we'll print the numbers that are inside that range a third kind of loop or iterator is a list loop and that's one that we'll hit quite a bit today and that's just you start with a list and a list is one of those advanced Python data structures that I talked about earlier so it can contain any numbers or data strings in this case it contains numbers and we'll just want to list or we'll want to loop through or iterate through each item in that list and so for each number in X our list we will print the number a few other important things that we want to keep in mind as we start working with Python code is that variable names functions geoprocessing tools that are called in python everything pretty much is case-sensitive and if you start using different cases in your variables than what you originally set you can run into this common name is not defined error so that's one thing to look for another thing is when you're specifying a path in python you should just go ahead and use forward slashes as the the path separators instead of the common backslash there's additional characters you can use as the separators but forward slashes work in both Windows and Linux so it's it's good to just stick stick with that one now two important pieces that you can can think of in Python our functions and modules and a function you can kind of think of just as a tool it's a defined piece of functionality that will perform some specific tasks so again you can just kind of think of it like a tool a module is a Python file where functions live and so often all of the functions in a module are related and these modules have to be imported into your Python scripts in order to be used so the example that I have there on the third bullet is we will use the math module and call the square root function from that math module and provide a value and that function will then return a result value okay so now that we have some some basic Python concepts in mind let's start talking about Python scripting in ArcGIS so in ArcGIS 10 geoprocessing in Python is done through arc PI algea processing tools are accessed or can be accessed through arc PI so this arc PI module is a package that only provides that not only provides access to geoprocessing tools but also several special functions classes and modules that will enhance the scripting experience in arcgis so there are certain functions like the list feature classes function that you can use to list all the feature classes in a workspace the described function to retrieve data properties and the number of cursor functions to get and modify a table or feature classes attribute values there's a number of our PI classes that we can use to create complex data objects like spatial references field maps and geometry objects classes are a topic that will be covered in the next session actually immediately after this session and there are arc PI modules that provide some extended functionality to be used with ArcGIS scripting so there's a mapping module for performing automated mapping tasks and there's a spatial analyst module for performing raster analysis and map algebra now our PI builds on the scripting experience of previous versions of ArcGIS that was known as ArcGIS scripting you can still use the ArcGIS acting module in ArcGIS 10 and almost everything that we talked about today is valid using ArcGIS scripting in previous ArcGIS releases you'll just need to note that if you are on 93 or a previous release of the software before 10 you'll need to refer to your documentation because everything that we talked about today will be ArcGIS 10 specific but the concepts are very similar so so don't be scared off by that also new at 10 is the Python window which I mentioned earlier and this is an embedded interactive Python interface actually built into ArcGIS and this is a great place to explore arc PI and start running geoprocessing tools using Python syntax you can also access any Python functionality that you would be able to in a Python script file right here in the Python window in arcgis so in that window you can execute a single line or multiple lines of codes code as well as load any script files that you've previously worked on or save save your work to a text file or a Python script alright so I'll start our first demonstration now this is my colleague david win he'll be doing the demonstrations Dave any thankful ID is anyone here from Minnesota yeah do you recognize this oh yeah this is uh this is a Hennepin County in Minnesota you can kind of see United Minneapolis and st. Paul down in the corner can the people in the back hear me all right yeah yeah no that's gonna have to talk loud I guess that better all right all right well I think I can do this one-handed so we'll just try that and see what happen all right so one of the things we added in our chest n is the the Python window so if you're here inside arcmap there's this little button up on the main toolbar python window and it's a regular window you can move it around you can dock it anywhere you want so it's divided into two section then the top section is where you put your code in so any we want to run a chi processing tool you want to do any little bit of Python you do it there and the bottom half this is where help shows up for tools functions other things in addition to messages that might return from your tools so if I wanted to start running a tool I can just start by typing in arc PI and then when I hit that period gives me a list of options and as I go through that the list will narrow and I can pick a tool now one thing I know when you're running tools and Giroux kind of alluded to this already is that everything is case sensitive right so you need to make sure that your tool name has correct uppercase and correct lowercase and as well the aliases this management bed for copy features is also required so kind of what the scenario is that I want to try and do here is you'll note that Hennepin County in Minnesota is is bordered by several rivers and if I go in a little bit closer you notice that the the river doesn't really always follow the border of the county all that well I want to be able to take that that river and move it to the county boundary which is more accurate so I'm going to do a series of tools to accomplish this I'm going to use copy features tool and then I'm going to use a couple new editing tools that we have at 10 so write the bracket I get an intelligent drop-down based on choices that are available to me so I'm going to copy this crow river feature class now let's call it river edits the reason I'm doing this the next tool tools which are both editing tools modify the output so you know do I really know what I'm doing maybe not so I'm going to make a copy with copy features tool then I can work directly off of my copy instead of my original so I just hit enter the tool will start running okay you see over here there's going to new I will put add it to my map and that's what I'll be working with so the next tool I'm going to use is the densify tool so again pick the tool like this and I'm going to pick the feature class I just created now densify second parameter is a keyword base parameter so I get a drop-down based on the options that are available to me I'm going to densify based on distance and then I'm going to enter string for the identified distance I want to use which is 25 meters and you'll note as I'm going through these parameters the help and the bottom sort of response to where I am in my list of options so every time I hit a comma basically to move on to another parameter you can see another parameter being highlighted and below there is courses all the tool information more about the parameters essentially a lot of this stuff that you'd see in the help system has been brought directly into the arc pie wrappers for these tools so I've done all the parameters I want here I'm going to close that off and run that again and then I'm going to follow it up with the tool the reason I'm actually densifying that line is to create a more uniform pattern of vertices something that represent and before I do the next parameter I'm just going to open up this tool so you can get a better look at what it actually does I'm just going through the search open a tool dialog so you know this tool has two parameters and the first one is in input features and the second one is this little more detailed parameter she called snap environment not sort of you can see that snap environment is sort of like a like a table almost a series of values that you can enter well in when you have a parameter like that or you have a parameter that's a multi-value where you can select multiple values these can be represented in Python by Python lists so that's represented with a square bracket and then because this is a table there's multiple rows so each rope again can also be represented as a Python list another square bracket so I want to snap to my county's feature class I'm going to snap to the edge and I'm going to set a distance of 25 meters square brackets to close off each list round bracket the close off the tool and this will take probably about eight to nine seconds now you see okay that finish but it isn't really what I was hoping for you can see certainly that you know sure it's snapped in some places but obviously I missed the mark so one of the nice things about the Python window is it's very easy to recall which you've done already and reuse that functionality so I just start clicking my arrow key I can go back up let's start over again right from the beginning and I can run that coffee features tool again and recreate my temporary data and I called densified but I'm going to try make that just a little bit tighter and do that at 10 meters hit enter and then snap I'm going to be very generous and snap in total 100 meters give that a few seconds again all right and now I think we actually got what we were looking for so this is my input I had beginning you can see how far off it is off the line and now with the series of tools and sort of an iterative process and able to get the desired result that I'm looking for okay thanks Dave so like we saw in that demonstration executing a gia processing tool in Python is pretty straightforward only one line of code is really needed to execute a tool so first our PI must be imported and then you'll just enter the tool syntax now the tool syntax is going to be an important thing and it must always be followed with with this convention in its arc PI dot tool name underscore toolbox alias and then the the parameters are specified at the end there so each geoprocessing tool has some input and output parameters that must be specified for the tool to execute so here in my example we see I import arc PI I set an inner environment and then I execute a geoprocessing tool the buffer analysis tool now as you're getting used to using Python to execute geoprocessing tools there are some good places where you can go to see tool syntax and make sure that you're actually using a tool correctly in Python one of those is the image that's on the screen there and that's the results window if you right-click on a tool in the results window you can copy that tool syntax as a Python snippet and so it will have all of the data sets actually right in line there so when you copy it you can just paste that into a script tool or into the Python window for re-executing there's also functionality in model builder to export a model to a Python script so if you have some established workflows that are already models a starting place for you is to export your two scripts you can also drag tools from the arctoolbox window or the search window that dave showed right into the Python window and it'll automatically drop in the correct tool syntax for you you can look at the tool documentation so each of the tools has a help page that has both syntax information that shows how the tool is used in python as well as a more exhaustive script example that uses that tool you can also call the arc by usage function to get help in syntax information about a tool so you see there are pi dot usage and then i just specify the name of the tool that i want help for now while setting environments in arc pi is also very straightforward it has many rewards in making your work easier and adding some additional control when you're running geoprocessing workflows for instance there's the workspace environment and if you're familiar with geoprocessing environment you know that all of the data that you have if if you set a workspace environment you can reference that data by just its name instead of having to specify a full path and doing this in python scripting is very helpful since it makes your script cleaner and easier to manage you don't have all these long paths in your script you can just specify the workspace once and then just use the names of your data instead of the full path there's also other environments like output coordinate system and extent that can be used in different ways and you can look those up if you're interested now when geoprocessing tools are run messages are always raised about tool execution so when you're running just some geoprocessing tools from a tool dialog in arcmap or our catalog messages are displayed in that geoprocessing messages window where they can be followed to see how the tool is executing and there's a little progress bar in that messages can also be seen in the results window and there's three different types of messages that tools will will return that's informative messages so that's just things like that tell you when the tool execution started how long execution took and if the process succeeded there's warning messages that's kind of a step up from informative messages and these warning messages make you aware of any potential problems that might have have occurred during tool execution and there's then of course error messages that indicate that there was a severe problem and that the tool actually failed so if the tool is executed in the ArcGIS Python window and you may have Mitz seen this in Dave's demonstration when you execute the tool there's that special window pane that displays any any messages that you have right there in the Python window it was on the bottom part so you can see any messages that occurred there messages can also be read and displayed in Python but you always need to explicitly call them using the arc PI get messages function so you can specify whether you want to return all messages just the informative messages warning messages or error messages using the syntax that I have there in the slide so here's an example of using get messages to call geoprocessing tool messages so we'll want to execute a geoprocessing tool in this case the buffer tool and then immediately after we execute the tool we'll want to print the execution messages so that's as easy as saying print arcpy get messages and this will return all of the messages that may have occurred since we didn't specify a number there in the get messages call so as you start working with Python you're going to definitely run into some errors and this can be pretty frustrating I had a starting point as a Python scripting users and it was frustrating to me so I'm sure it'll be frustrating to any new users errors can occur because the geoprocessing tool isn't being used right there might be typos in your script or the script has some Python syntax errors now printing geoprocessing messages is a simple way to find errors that occurred when the tool was being executed in Python but Python error handling provides a more intelligent method for dealing when errors when they do occur one of the most common in useful forms of error handling in python is the try accept statement so you can think of it this way you will try to do something like executed geoprocessing tool or perform some computation and if that fails which you can think of as an exception you can do some other steps and just one thing to to make clear is if the operation in the try statement does not fail so if it executes correctly all of the code that's in that accept statement is just skipped over that that accept statement is only for if something wrong happened with the code that was executed in the try statement so try/except statements are really pretty easy to incorporate into your scripts you just you know can copy and try and accept and indent your code correctly so they're easy to incorporate and you should really always use them in your scripts since you might get into the case where you don't even know why the tool isn't failing why the tool is failing or why some Python errors is being raised and you might not even know about it unless you use some kind of error handling technique like a try accept statement so here's just another short example showing using the try accept statement so we will try to run the buffer tool but if an error occurs during tool execution because of perhaps a typo some bad syntax or I'm using the tool incorrectly the script will move then into the accept statement instead of simply failing and ending the script execution so you'll get some information and in this case in my accept statement I print the statement that buffer failed so an obvious message there and then also why the why the buffer tool failed by getting any of the error messages using arc pipe get messages all right I'll pass it back over to day four demonstration of air handling and messaging all right now in the last demo we worked on creating a little workflow inside of arcmap but you know at some point you want to you might want to be able to take the work that you've completed and flushed out and you know got to work and then take it kind of to another level so you can always click inside the Python window and you can save as you can save it out to a Python file and this is that file it's basically just everything that I did in that last session now I know how I do think this is going to work just as is well no of course not that's always something so when you're working inside arcmap you're working inside the Python window art pie is just just there it's imported for you to import any kind of functionality in Python you type in an import statement import art by now of course I went through a couple different renditions so I'm going to delete that out too and if I just try and run this I can go up here and click on this little triangle button called go that'll just execute the script and give me a second and all right well a something didn't work of course well as Drew was sort of explaining there there's some pretty simple mechanisms in in Python to be able to handle errors so I'm going to start making this script a little more detailed by adding a try I can indent that code I type my accept and then I'm going to print out my messages now of course what comes to message is you really only care when it fails it works you know hey good but it failed that's what you're interested in right so that's what the to wait for me any error messages one thing I sort of forget sometimes to do but you can also type in on a trike step you can add it in else so you know the way try/except works and it's going to try anything in that block and if that doesn't work it's going to print anything and or do anything inside the except well if everything in the try works then it'll go down to the else and I can say hey grips some kind of message that hey this didn't work now when you start writing code you know there's going to be several lines and you go to run it and just nothing happens you know well hey down in the bottom there's this little tiny message and says fail to run scripts in tax area well you can also sort of confirm whether or not you have any syntax errors just by clicking on this little check button I click that you notice the cursor jumped down to this except line and simply all I did was forget that : so now that's all fixed up let's run this again okay now I actually got proper error message so what happened here okay it just doesn't recognize that one of my data sets exists at all and well okay why is that well when I'm working inside map I was working most of the layers Here I am in Python when it has no no concept of where arcmap is and what was in there so I need to be able to reference those layers in some other way and one way I can do that is by setting geoprocessing environments so if you need to set an environment it's our apply dot E and V in this environment class and it contains all the environments so I'm going to set my workspace to yeah and for good measure you know once in a while you're going to run a script once and run it again hey then run the second time and that is because you're trying to overwrite something that you did the first time so in Python you want to overwrite data sets and feature classes you set this overwrite output option to true so now if I run this and this time I'm going to click on this Run button and that's going to allow me to step through on a line by line basis so I hit that this little dialog pops up I want to supply any arguments I can I don't I want to step through the debugger though click that option and hit OK and by hitting either this button up here step over button or just hitting f10 I can step through on a line by line basis well somewhere I made another mistake and I can't find it you see it well in any case what should happen and this is maybe another example of the error oh yes true caught it I have a very long complicated folder structure sometimes keeping it simple right makes it easier okay so let's go back that again click on my Run button I'm going to step through in a debugger f10 that my workspace and my overwrite output options run copy features and densify and run snap which if we remember took about eight seconds and then when that eight so that all works it drops down inside the else prints out that last message down in the bottom hit f10 one more time and I'm done three lines well I think I'm pushing my luck so I'm going to turn back the drew thank you Dave and you need to work on that that folder structure all right so at this part of the session we're going to start talking about some actual hands-on kind of real-world geoprocessing tasks that you might want to kind of get started working on in Python so in the arcgis help there's a topic that's pretty much just a flat list of all of the functions that are available through our PI so I'm going to launch to the web help for arcgis and this is the topic that i was saying the alphabetical list of arc pi functions and i just want to kind of introduce some of the the ArcGIS 10 help structure before we get moving on that so everything geoprocessing will now be in this professional library and so that's kind of the first folder that you'll want to get into then under professional library we have a geoprocessing book under this geoprocessing book there's several sections one of them is all about arc PI so that's the arc PI site package here there's also this geoprocessing with python book that contains some additional information for working with scripting working with geoprocessing in python so under this arc PI site packaged book we have another section that's just about all the functions now this is some of the other things that I was talking about that Python or that arc PI is composed of these functions classes modules the bottom three are modules but we want to just get started with some functions so here's the topic and you can reference that if you're ever curious about some of the functionality that's available through our PI so our PI functions these are are very useful in Python scripts and they were actually designed to support and enhance the geoprocessing workflows that you want to do in Python so we've already seen how the get messages function is used to call geoprocessing messages within a script there's another group of functions that is a very powerful group and this can be used in many geo processing workflows related to batch processing and so these are the the listing data functions and they're just used to do things like make a list of all feature classes in a workspace or all rasters in a folder or all fields in a table or feature class there's also another function the describe function that allows us to get the properties of almost any kind of data so we'll talk about that today as well now if you think about it there's a manual way in arcgis to do each of these tasks whether that's getting the properties of some data you can right-click on a future class in our catalog and get you know any properties that you can think of you can do batch processing by right-clicking on a geoprocessing tool in our toolbox window and selecting the batch mode or by using some model builder however these are PI functions that that we'll talk about provide a way to automate all of these tasks that would traditionally have to be done manually and that's really the value of Python scripting is that ability to automate automate your work and make it easier so one of the most common scripting tasks is batch processing and you can think of that as just running a geo processing operation whether that's a single tool or workflow running it multiple times so a simple example of this is clipping every feature class in a workspace to some common study area boundary now instead of running the clip tool from the tool dialog you know five times or ten times once for each feature class you can write a Python script with just a very few lines that performs this batch operation so batch processing tasks often use the art pie list methods that I showed in the previous slide to create a list of data and then perform some operation on all of the data that's in that list and there's several of these list functions like list feature classes list rosters list fields and several others so here's a short code sample showing how the list feature classes method is used to create a list of all feature classes in a feature dataset so we just need to set a workspace environment many of the list functions will require that a workspace be set so we specify a workspace and then we'll create a new variable F c-list which will be equal to the list of all polygons feature classes in that feature dataset and you see that list feature classes has parameters the star in the polygon word and that first parameter that's the star is just used to filter the feature classes that will be included in the list by their names and so with that star there that's just a wildcard character that no matter what the name of the feature class it will be returned in the list you could do something like s and then the star and anything that starts with the letter S would be returned in the list only those that start with the letter s actually in that second parameter you're able to specify a shape type that you want to use as a filter so if we put polygons there only the polygon feature classes will be returned in the list we'll then use a list loop there in the bottom of the sample so for FC in FC list FC list is that list that we created and we'll just want to print the name of that FC and so if we would do this it would turn these three feature classes that are highlighted and those are polygon feature classes in that feature dataset so another one and I'll try to get through this this slide quickly so we can see another demonstration but in this one we'll use the list fields function to create a list of all the fields that are in a table or in this case a feature class a shape file feature class so we'll loop through each of the fields so for filled in fields fields is the list we want to print the field name and the field type and the objects that are returned in list fields are a little bit different than in our previous list example list feature classes list fields actually returns a group of field objects and these field objects have a number of different properties like name type alias name that you can use to get the specific information about that field all right I'll turn it over today for demonstration of batch processing all right well a few weeks ago I was perusing the ArcGIS online and I found this dataset had seven eight hundred nine hundred feature classes of range of trees in North America and personally that's very exciting you know I'm all about trees when I'm not about GIS so any I copied this down and I'm sort of thing starting to work through that ever since one of the things I thought I could do here was sort of take that big mass of feature classes and do something more productive with them so really all I'm going to do with in the first part is just sort of walk through that geo database and list out a series of feature classes in that and then I'm going to see what I can do with them so I'm just going to step through this cone on a line by line basis I'm going to import Eric PI I'm going to impart something called OS and this is a module part of Python there it gives you access to operating system level tools it's very good for working with paths and all tabs in particular now this is a little trick I like to use you saw in the last demo how I totally blew getting the path right well this little thing here will always give you the the location where your Python scripts its so if I just type that in down here so you know you can see how I can use that then I always know where my location is I can build that in other stuff and I don't have to physically type it in and usually mess it up so I'm going to use this because in my folder I have my Python script then I also have a series of G databases so I can use them relatively I'm going to set my override output option to true and I'm going to set my workspace and my workspace I'm going to set to a combination of my home directory and this G database I found an ArcGIS online and the reason I'm doing that is most of the list functions sort of need to be educated well you know what do you want me to list feature classes on so I'm basically telling it to educate use this feature class or this workspace so of course I said there was a lot of future closer than that so I'm going to put in a keyword this ilex a wild-card and that's going to return list of all the teacher classes that are basically holly tree species capture that list a list is a lot of different things you can do with it and one of it is just simply type the length of it so this message prints that out there are 13 feature classes returned from which future classes and then a list of course allows it or lends itself very nicely to iterative type processes so it can walk through that list in a for loop one at a time and we'll the start printing those off now of course you know just walking through something and printing it really you know isn't that exciting so I have this other code down here that I wrote we'll do something a little bit more now the one thing Giroux kind of spent a little time talking about and lists but if you want to just look at a list you can just type it in here like here at the bottom and it'll reveal its contents you want to sort that you know or you want to append something to it lists allow sort of indexing and slicing so I want to get a specific value from a list that can just put in a number I'll give you the first one if I want to get a range out to put a number colon another number and I'll give me a series of feature classes out of that list any case I'm going to go up here and I'm going to set a breakpoint on this line here and I'm just going to click this Run button and that will run through everything I've already done again a second time come down here now usually when you run geoprocessing tools and here I'm creating create file geodatabase you don't really care about what is like the results object every tool when it runs returns a result object when I was learning through that workflow before copy features snap and densify didn't matter that much but I'm going to use I'm going to capture that and then this result has a number of properties and methods and one of the things that has is this get output this is for this case just a shortcut for me so that I can get the full path of this new judez today's on the next few lines I'm going to run to make feature layer tool because I have a feature class of the states of the US but I don't want to clip my tree data based off of the entire United States and when it bays it off of a specific state so I'm writing this query and I'm going to pick out for my layer just the state of South Carolina now you notice to this point most the time when I have had in lines it's all been based on one line the way I've kind of spread it out here is just sort of a nicer way in Python of making a little bit more readable certainly this treats it as if the same line you can imagine if I try to actually put this on one line you know it's very difficult to explain anyone what's actually happening so if I continue to step through now create my lair then I'm going to walk through that list supply each item in that list individually to the clip tool I'm going to clip it with that layer I just created and then I'm creating a path on the fly with the feature class name and that'll be my output fit f10 now when Drew was talking about get messages really that reveals all your messages to me I just want a clumping that kind of gives me a little bit of indication that things are moving along so I'm using a slightly different method called get messages and that basically takes all the messages and just gets a specific one so I'm going to print out the first one you see the first tool succeeded second and then if I just had a five that will continue to run through that list and clip each of them in time until the list has been exhausted see the messages will continue to pump into the bottom half and the prove this is actually working I'm not arcmap now it's done so I can turn these on or off as I want oops try that again and I step through them you can see that each of these layers has been clipped to the boundaries of South Carolina okay thanks Dave all right another function that you'll get a lot of usage out of and that's a very good to include an often necessary in your script is the described function and that's used to get the properties of really any kind of data so when data is described using the described function an object is returned that has a number of dynamic properties that are determined by the type of data that was actually described so among these properties can be things like data type shape type the spatial reference of a feature class the extent of the features or simple things like the path to that data set so again I will launch the the help system and so again we're under this functions book in the arc PI site package and this is just a description of what the describe function is and how it can be used here's a list of all of the different data types that can be described so there's things like feature classes tables any other kind of data really that you can think of to use in ArcGIS and at the most basic when you describe a piece of data you'll get the describe object properties and so these will be those basic things like the path to the data set the name of the data set any children the data set might have the data type and the extension of the data so those aren't very interesting they can be useful but those properties are a little bit dry so if we would wanted to describe something else like a feature class we can just go to the feature class properties here and here's a listing of all of the properties that you get when you describe a feature class so you get feature type information about the M and Z values spatial index the the field that's the shape the geometry field as well as the shape type so whether the feature classes polygons line point what type of geometry now up here in the summary section we also see that when you describe a feature class you have access to the table properties and data set properties so if we look at that we can see that when you describe a feature class you also get these table properties as well as some data set properties so here's an example of actually using the described function in a script we'll start by creating a new variable desc that will equal the describe object of the feature class specified we'll then use some of the conditional logic that we introduced earlier to print some different statements based on what the inputs shape type property is found to be so depending on what the describe object desc shape type property is found to be we'll print an appropriate message stating the feature class shape type so after we create that describe object we'll look at it that shape type attribute and say if the shape type is equal to a polyline we'll just do something simple like print the FC is a line feature class else if the describe object's shape type property is polygons will print the FC is a polygon feature class else neither of the above conditions are true so we'll print the feature class is not a line or polygon now as you start to create script something that you may want to do is to make your script capable of receiving arguments and when I say arguments the thing that I'm talking about you can just think of as an input and input to a script in all of our previous examples we were using hard-coded input values so hard-coded paths or hard-coded numbers that we wanted to use if we wanted to reuse that code that we've already written but specify some different inputs or change some other settings we would have to go in and actually manually edit to change the paths or those values and that's kind of where arguments come in in making a script more flexible and easy to share with others because they allow for values to be entered by the user and pass to the tools and functions in the script instead of actually hard-coded write in the script now in your arcgis scripts you can use the get parameter as text function to read arguments into your script arguments that are read using get parameters texts have a zero-based index so that's just saying that the first argument has an index of zero the second argument has an index of one and we'll see that on the next slide now an important reason to use arguments is because it allows you to connect a Python script to an ArcGIS script tool so you'll be able to specify any inputs and outputs for your script actually in ArcGIS in a script tool dialog then pass those values to the Python script for execution so another short example showing how to receive arguments we see that I just create two new variables input FC in output FC equal to arc PI get parameter as text and we see the first one has an index of 0 and the second has an index of 1 now we can use these variables that are coming from script arguments in a geoprocessing tool the same way we would if they were a variable set to a hard-coded string or value so we have arc PI clip the clip tool and we use one of the argument variables input FC as the input feature class we then use a hard-coded value for the clip features and then back to another argument for the output feature class so there's no rules about being on being able to only use arguments and no more hard-coded values you can use either or in your in your script and in the functions and tools inside of them alright let's go back for another demonstration of using describe and implementing arguments in your scripts yeah alright so I'm going to go back to the script I had in the second demo and we're going to try and make it more generic so in each of this each of the demos I've had so far I've mostly been working with scripts that are almost kind of like a macro right there the paths have been hard-coded there's been lots of assumptions that I problem we've been making that not even aware of so I'm going to take some of those that script and sort of evolve it to a point where it's something that could be adaptable and I could use with other data sets so as Drew showed one of the things you can do pass in parameter is used to get parameter as text method based with an index value so I'm going to define three parameters one for my input feature class I'm going to assign another one to a snap feature class I'll update value my index and then a note feature class I'm going to remove this workspace environment because really you know I'm going to be passing in full full feature classes with with PABs I'm not necessarily going to want them to be in the same location anyways I'm going to strip that out and I'm going to take these variables that I created and substitute them instead of these hard-coded strings so I'm going to copy my input to my output I'm going to densify that output I'm going to snap that output and then the last one was the feature class like actually was snapping to County now drew kind of got into the describe a little bit to describe allows you to make decisions based on what your data is so does the script is right now it's still préval to somebody putting in something that I didn't intend to so really this workflow only makes sense if I'm trying to snap mating lines or polygons to another line of polygons you know if I put in an annotation feature class or points maybe that's just something I want to stop right up front so you can use code like this and all I'm really doing here then creating a describe object of my input feature class whatever that might be and then I'm going to evaluate the feature type the logic here sets basically if it isn't a polygon or polyline feature class then that isn't what I want and then I'm going to come here and I'm going to throw an error that's really not the direction I want to go I want to try something just slightly different so one of the nice things about creating Python scripts that are adaptable and take input arguments is that you can then take those that's why I wasn't at the UC last year if anyone was looking for me you can create Python script tools now anytime you have a toolbox and actually back up one step then you're in a workspace you can click on here the new toolbox from toolbox you can click in there you can add a script so I'm going to create a stool with that script that I've been using in mine and I'm going to call it snap the polygons something generic and the name is what you would you say if you were calling tool from Python the label called same thing that's what use would use if say you know you on the top of the tool dialog or if you added this tool the model builder that would be on top I'm going to locate that file and hit next and then on this screen this is where I define my parameters so I had three parameters I'm going to define the first one call input features and I'm going to use feature layer so Y layer not feature class well if I specify this feature class that means I can only use full tabs and feature classes to the data say I had added that feature class of the map previously I couldn't just drag and drop that layer in so feature layer allows me to use feature classes and layers instead of snap features I'm going to also with that a feature layer and I'm going to define an output feature class and that will be a feature class the number of properties down here below for this scenario don't have to change to me my last one this is definitely not an input this is an output so I will identify that and then when I went through that exercise in the script of using the scribe what I was trying to do was basically filter out feature types that weren't acceptable well you can also do that when you create a script tool in the wizard so for my input features I click down here the filter is currently nine I can use a feature class filter and I can check off types that are not acceptable to them it will apply finish now I have this fully functional tool this will behave in the same manner than any G processing tool will behave so you know before I was working with this river called crow River well Hennepin County is also bounded by the upper reaches of the Mississippi so in this case I'm going to snap a Mississippi layer I'm going to snap it to the County sort of like before you can see how it's off a little bit right here if I run that tool based on the code that I've written hopefully this will come back and 5-10 seconds and that is my outputted achieve what I wanted to do but with different data great thanks Dave all right we just have a couple more slides here's a resources slide that we wanted to just call some special attention to the ArcGIS Resource Center is a central location for finding all of our online documentation help forums and that these were just they're brand new sites that were just put live a few weeks ago and so that's a good place to check if you have any questions about our GIS we have a specific geoprocessing Resource Center then that contains a script gallery some blogs as well as presentations related specifically to Python so that's a good place to check there are a few Python reference books that we recommend above some other ones so if you're interested in learning more just about Python as a language these are a couple good books to check out there's a lot of information as well on the Python organization website so you can go there and and at dive into Python org there is a free tutorial a very very long tutorial actually that you can download and work through okay thank you for attending today if you're interested in filling out an evaluation form and having gotten one there's a student assistant over by the door so you can stop and see her yep oh thank you

Channel: Sherry S

Views: 18,959

Rating: 4.9633026 out of 5

Keywords:

Id: VNfr3R8J3vM

Length: 71min 31sec (4291 seconds)

Published: Wed Nov 06 2013