Accuracy Assessment of an Image Classification in ArcMap

howdy in this tutorial we will be creating a confusion matrix in arcmap to assess the accuracy of an image classification for instructions on the classification process please see one of our image classification notorious thematic maps created from imagery will have some classification errors accuracy assessments provide the user with more information on where the errors are occurring depending on the acceptable level of error the user will be able to determine if their classification is usable or if they need to reclassify the image the class accuracies are determined by comparing test pixels with the corresponding location in the classified image in a perfect world we would be able to use field verified ground reference locations or that has pixels this is not always possible in which case the user may also select references that they have visually identified from the imagery the test pixel should be evenly distributed across the image they should also be distinct pixels from the training areas used for supervised classifications confusion matrixes are widely accepted method of determining the accuracy of a classification but it is important for the user to remember that the biases that are present in their test pixels will also bias the accuracy of their confusion matrix the rule of thumb that I learned was to have ten times the number of pixels for each class as there are classes so if there are three land cover classes then there should be thirty test pixels for each land cover so a total of 90 test pixels it may not always be possible to have an equal number of pixels for each each class in a classification if you know that there is not very much forest in your classification and there is a large amount of water it would make more sense to have twenty test pixels for the forest and forty for the water the test pixels need to be as near to evenly distribute it as possible if all of your test pixels are from one section of the image the result will be biased to only the accuracy of the section of the that you had test pixels for in arcmap open your pre classification image on the top of the map open the arc catalog interface navigate to the file that you wish to work in or create one I have created a file for my accuracy assessment now we will create a new shape file I'm going to name mine reference points make sure that the type of shape file is set to point now I'm going to drag my new shape file into the map reference points is currently empty before I add points to the shape file I'm going to add two fields to the attribute table open the attribute table by right clicking on the shape file and selecting attribute table we will left click on the button on the upper left and then select add field the first field I am adding will be the reference for me and any other user of this data set the data type will be 'text and i will name it land cover I'm going to repeat the process of adding a column but this time I'm going to name it class and leave it as a short integer in order to add a point I am going to use the edit function to use this open the editor toolbar by right-clicking at the top of the map and selecting the editor toolbar left-click on the arrow next to the word editor and select start editing next we will select reference points shape file to open an editor window select the reference points file in the editor box which will open options in the construction tool select points you will now be able to add points when adding points be sure that you are using the image that you use to create the classified image be zoomed in sufficiently that you are able to see where you are placing the point before I start placing points I need to determine how many land covers I can identify in the image I am noticing four trees pasture water in urban areas based on the four classes and the rule of thumb to have ten times the number of test points for each class as there are classes I will select forty points for each class and we'll end up with a total of 160 points for the moment I am just going to select a few points for the water class once you have selected your points for one class you will need to add information to the attribute table so that you know what the points represent and so that the computer knows this is why we added both a text and a number column you may not know what the class number is now because you should not have your classified image open as it leads to the temptation to select points that match your classified image when adding the information to the attribute table it is easiest to use the select by attribute tool and choose land cover from the dialog box now it will equal the unique value of the quotation marked space the attribute table for the reference points will now have all of the blank fields highlighted and I can input the value of water by using the field calculator because water is text I need to you put quotes around the word water now you can start adding points to the next class these will be blank in the attribute table and you can fill in their values the same way that water's value was infilled I'm going to skip ahead here and open up a point file that I have already completed adding all of my reference points too after completing all of your points add the classified image to your map document check what the number code is for your classes and then assign them to the point file so that class 1 is the same in both in my case class one is pasture so I will go to selection select by attributes and enter land cover equal and pasture and hit OK the attribute table for the reference points will now have all of the pasture highlighted and I can input the values the value one by using the field calculator I have already input the numbers into my file there's one major problem with my data that is now parent based on the classes that I observed in the classes that the computer observed there's no urban class in my classified image at this point I have two options I can stop and perform the classification over again or I may go ahead and run the accuracy assessment so that I know what the urban areas are being classified as so that I may pay more attention to them when selecting my training data if I am using a supervised classification method to continue with our accuracy assessment we need to set the system to align the pixels we will create from the reference points with the pixels of the classification to do this go to the reprocessing tab environments processing extent fill in the extent as your classified raster and set the snap raster to the classified raster as well now hit OK if you do not have the spatial analyst extension on let's do that at this time go to customize extensions and select spatial analyst now we will convert the reference points into reference pixels open the toolbox and go to conversion tools then to raster and finally point to raster the input feature is the reference points the value field will be class and then browse to the location you would like to save the output raster file and name it you can leave the cell assignment and priority fields in their default position but make sure that the cell size is correct mine should be 30 because it is based on a Landsat image if you do not know what size it should be you may also drag the classified raster to the cell size eyes and the computer will match it hit OK zoom in to one of your points to see if it appears to be aligned correctly I'm going to zoom into a river because this is a small object and it is easier to tell if the pixel is in the same is the same size when it is next to multiple classes mine looks good if yours has issues revisit the environments and cell size settings we will combine the reference points and classified image go to the tool box spatial analyst local and combine looking at the results of the combine is interesting if it were not for the urban class I would say this was a very successful classification class 3 has 39 out of 40 points correctly classed and is missing the missing point isn't showing up in any of the other classes which means it was an unclassified pixel class one and two were 100% placed my urban class was nearly equally placed in pasture and forests although this has told us a lot of information it is still not a confusion matrix to create a confusion matrix we will need to use the pivot tool but this cannot be done with the attribute table so we will have to export the table click the button on the upper left corner and select export and then browse to the location that you wish the table to be saved in save the table as ad based table the PivotTable tool may be found under the data management tables and then the pivot table tools should be there the input table should be the table you just exported select the classified raster field as your input and the reference points as the pivot field the value field will be count now navigate to the location you wish to save your table and name it then hit OK my matrix is in an order that makes it more difficult to read but that's okay for now I'm going to use Excel from this point forward this may not be the best way but I like to export my table as a text file then in Excel I will open the text file and select the delineated hit next and then the file is comma delineate it so check that box and then next and next I'm going to get rid of the object ID field next I'm going to rearrange my columns so that they are in 1 2 3 4 order I'm going to add a class for wrote for my urban class and input zeros for all four columns to make it easier to interpret I am going to change my classes from numbers into land cover names in my case that is pasture forest water and urban so now the matrix is looking pretty and it is time to add the formulas so that we can get meaningful numbers out of the data the measurements that we will be finding are the Kappa coefficient the overall accuracy the class accuracies the Commission and the omission I'm going to insert several rows above my matrix I'm going to find the total number of pixels for each class I can do this easily by using the formula equal sum then select the cells that I want and then drag the formula into the neighboring cells I am going to find the ground truth totals in the same manner I know that I had 40 points in water so I'm going to manually correct the total for the water class next we will find the ground truth percent these are the class accuracies these will have the same headers as the matrix so i'm going to copy my matrix and then delete the pixel data from my copy in my first cell i will input the formula equal my pasture cell divided by the pasture reference total multiplied by 100 add a dollar sign in between the letter and the number of the reference total cell now put the formula down and then let's pull it across next we'll find the Commission Commission is how many test pixels were incorrectly classified as a class so it is the incorrectly classified pixels in the row divided by the total number of pixels in the row there's a little more this is a little more time-consuming to fill in the cells because you cannot just put them down the way you could for the percent we can think of this as the rate at which the class has been over classified I'm going to skip ahead here omission is the opposite it is the incorrectly classified pixels in the column divided by the total number of the column let's take a moment to fill these in I'm going to put one in for water because I had added that one back in earlier but I know it was not classified in mine I have 0% omission of pasture and forest but I have 100% omission of urban this makes sense because there were no pixels classified as urban in my classified image next we will add the producers accuracy these are the correctly classified cells for pasture forests water and urban / the reference point total for the class which are all 40 and then we will find the percentage for each land cover the users accuracy is like the producers accuracy in that it is the correctly classified cells for pasture forests water in urban but this time it is divided by the total points appearing in a given class so 57 60 339 and 0 now let's go back to the very top and find the overall accuracy and the cap of coefficient the overall accuracy is the sum of the correctly classified cells divided by the total number of cells this one is pretty easy to find the Kappa coefficient is a little less straightforward so let's take a moment to talk about it the Kappa coefficient may be used as a measure of agreement between the model predictions so that would be our classified image and reality or to determine if the values contained in an error matrix represent a result significantly better than random a1 would indicate perfect agreement between reality and our classified image and a0 is representative of complete randomness in here's here's the Kappa coefficient equation n is the total number of sites in the matrix R is the number of rows in the matrix XII is the number in row I and column I then X plus I is the total for row I and X I plus is the total for column L so what is this actually asking us to do we need to multiply the total number of reference points by the sum of the correctly classified pixels from this number we will subtract the sum of all the class row total by the class column total next we will divide all of this by the square of the total number of reference points and subtracting the sum of the class row total by the last column total it this is more sophisticated than the overall accuracy as it takes the misclassifications into account as well as the correctly classified and the ideal classification the harder part is now to put this all into Excel but I'm going to leave that to you your CAPA results should be similar to those of the overall accuracy this concludes our tutorial on accuracy assessments and arcmap thanks for listening and for more information or resources from Texas A&M University map and GIS library please visit our website at library.csun.edu

Channel: Map & GIS Library

Views: 96,770

Rating: 4.9108472 out of 5

Keywords: Microsoft Excel (Software)

Id: FaZGAUS_Nlo

Length: 19min 34sec (1174 seconds)

Published: Mon Sep 30 2013