Estimating Classification Accuracy Using ArcGIS

welcome back in this session I'm going to teach you how to assess the accuracy of classified images in arcmap so from our last session you saw it's pretty easy to come up with a raster of cover but the question is well how accurate is it so here's an example of what's called an error matrix which is typically used to assess the accuracy of classes so we have a classified raster where one represents water and two represents sea ice and we have ground truth points so these ground truth points are actually GPS points were on the ground we determine whether it was water or sea ice so then basically we have columns representing ground truth and rows representing our classified raster or the predictions for each of those locations on the ground so along the diagonal of the error matrix are the correct predictions so for example there are 48 ground truth locations that in real life they were water and on our classified roster they were predicted to be water and there were forty ground truth points that were sea ice and those were all correctly predicted in our classified raster to be sea ice so the total number of predictions that were correct were 48 plus 40 or 110 so our overall accuracy is the total number of correct predictions divided by the total number of predictions or the total number of ground truth points so in this case will be 48 correct for water plus 40 correct for sea ice divided by a hundred and ten was a total number of predictions made and then we could also ask ok for any class what was the accuracy so what was the accuracy for sea ice well you can look at it from two different perspectives so one would be we've got 50 ground truth points that on the ground where sea ice and of those 50 ground truth points forty were correctly predicted so our accuracy in that example would be forty divided by 50 or we could say for sea-ice there were 52 pixels that were predicted to be sea-ice and of those 52 pixels that were predicted to BCI's there were 40 correctly predicted so in that case our accuracy would be 40 divided by 52 so whenever we're talking about class accuracy there'll be two estimates so one will be of the ground truth points how many were correctly predicted or of the predictions how many were correctly predicted okay so how do we do this in arcmap okay so here's an example of a classified raster and we've got four classes so all the pixels that have fours in them are predicted to be water all the pixels that have three in them are predicted to be Spruce the ones are aspen birch predictions and twos are mixed Aspen birch spruce forests and then we have our ground truth location so these are randomly located points and they have values 1 2 3 & 4 so if we turn this on you can see that most of the water graduate points were correctly predicted and most of the spruce ground truth points were correctly predicted and here's an aspen birch ground truth point that was correctly predicted so what we want to do is for each point grab the value from the classified roster so we can compare for every point we know the truth from the ground what was its prediction so to do that we'll use a tool called extract values 2 points ok in this case I've got a hundred points in each class so the most efficient way to do this would be to use this tool extract values 2 points so for every point from these random location layer go to the pixel from this input raster and add that pixel value and then we'll output that to a point in this case a point shapefile and okay so the output is a another point feature class where we have the original ground truth values and for every ground truth location what the raster value was so typically what I like to do is add a field called predict and then we'll calculate predict to be equal to this field okay and then I'll delete the original field called raster value because that's not a very descriptive field name okay and in this case there were some pixels that had no data values that were flagged as negative 999 so what I'll do is a definition query to eliminate any of those points where there was no predicted pixel for that point location not equal negative 99 and just okay okay so we've got 370 ground truth points that actually had pixels that had predicted values so now what we want to do is create a summary table of every different class that on the ground was true how many predictions were there for that class so we'll use a frequency tool to create that table okay so for every different value for truth and every different value for predict tell us what the frequency is and we'll output to a table in this case I'm going to get out to a folder so in DB F table so the result is a table so for example there were 90 ground truth locations that on the ground were class 1 aspen birch and they were correctly predicted to be on the ground aspen birch and there were seven locations at on the ground where class 1 aspen birch they were incorrectly classified in our raster as being mixed force pixels so the last step is to take this information and rearrange it into an our matrix format so to do that we'll use a tool called pivot table okay so if you click on show help it'll tell you what each of these dialogues is for so for example this dialogue is the field that defines the records so the records are the rows and what we want in our rows is the classes in terms of predictions so predictions will be in rows and our ground truth points will be in columns so prediction is in rows and then if we click on pivot field that will be the values used to generate field names our field names are our columns so that's going to be truth and then our value field will simply be frequency so how many pixels are there in each class and then we'll output to a new table so in this case I'm going to name my table error matrix D be F and then just okay so the resulting table is in an error matrix format so the cells along the diagonal are the correct prediction so for example there are ninety ground truth points that on the ground they were class 1 which is Aspen birch and they were all predicted to be class 1 which is Aspen birch so our overall classification accuracy would be 90 plus 35 Plus 70 plus 86 divided by the total number in this table and the other thing we could do is we'll add a label so we don't have to remember what is class 1 versus class 2 versus class 3 we'll actually put that in the table so we'll add a field it'll be a text field and I'll make it let's make it 32 characters and we'll just call it a class name so we have our text field class name and then we'll simply edit and fill in our class names for these four rows and editor save edits editor stop editing and then we could add another field a double precision field so we'll calculate the percent correct for each class prediction so for example for aspen birch there are 90 ground truth points that were correctly predicted so we would take 90 plus the 41 or 90 correct divided by the total number of predictions which is 90 plus 41 plus 7 plus 2 so we'll use a fill calculator so truth one divided by the total number of predictions times a hundred so our accuracy for class one was 64 percent accurate and then we would repeat the process for each class okay so water was the most accurately classified class at about 96 percent accuracy and Aspen birch was the least accurate prediction so 90 out of 90 plus 41 plus seven plus two and then we could format this as a percent so if we go to properties and then numeric we'll say format that as a percent and then we'll give it a format of one to the right of the decimal okay and if you desire what you can do is use Excel to finish creating your error matrix so what we'll do is that Windows Explorer will make a copy of this so just control-c and control-v to copy it then we'll change the name of that error matrix and we'll rename it to error matrix whatever you want to rename it to and then we'll open it with Excel okay so in Excel what we can do is we'll save this as a Excel spreadsheet so file save as and I'm going to save it as an XLS ex extension so I just formatted it so we've got in bold the number of correct predictions so we had 90 from aspen birch ground truth that were correctly predicted 35 for mix forest 70 ground two points from spruce and 86 from water so if we take 90 + 35 + 70 + 86 divided by the total number of ground truth points that gives us our overall classification accuracy of about 76 percent and then if you're interested in in the class so for example Aspen birch there were 90 ground truth points that were asked members that were correctly predicted out of a total of 100 ground truth points that were Aspen birch so that gives us 90 percent from that perspective however there were 140 pixels that were predicted to be asked member janda but those 140 pixels only 90 were aspen birch so that gives us from that perspective an accuracy of about 64% so for classes there's two perspectives in terms of class accuracy okay so that's what we're going to do this week in lab after we do our supervised classification we're going to assess the accuracy using ground truth points and create an error matrix

Channel: David Verbyla

Views: 47,937

Rating: 4.8448277 out of 5

Keywords: Classification Accuracy, Frequency, Pivot Table, Accuracy Assessment, ArcMap, ArcGIS, Raster Analysis

Id: 9dGjuEQie7Y

Length: 12min 28sec (748 seconds)

Published: Mon Nov 11 2013