ArcGIS Pro - Fishnet & KDE

all right hello everyone hopefully you've had a couple nice weeks i haven't uploaded a video to youtube in a couple weeks mostly based on the assignments for the class it kind of combined two different areas so i gave them a longer assignment but with that today i want to jump into and we're in kind of the raster phase and one of the big parts when it comes to raster analysis in arcgis and when we're looking at crime are hot spots and that is truly a density map of crime itself tons of papers out there tons of research using uh kde kernel density estimations for hotspot identification with that there's a lot of parameters that you can choose and set within kernel density i typically have done it in crime set free spatial stat software through nij uh arc pro does give you the ability to use and change up things with python and even the parameters in here there's some articles out there and i'll link to one by heart and zane bergen on the kde parameter selection i've done it in the past and one of my papers ended up being just an appendix and a footnote but it does shift some of what you find depending on what you're trying to do with it more so on the smoothie and cringing part of it but with that before i dive into specifically the hot spot part and the kernel density parts we'll deal with raster data at that point i want to get into and show you another tool another tip that you can use within gis so when you look at little rock here i still have our crime data for 2019 i also have our block groups and streets have used these in other videos nothing new there but what i want to show you what you can create and depending on the type of analysis you want to do this can be beneficial in a number of capacities but with that i want to create a fishnet when i say a fishnet imagine a grid going across little rock with equal cell sizes across so it's truly putting a fishnet over our boundary file and that way it comes back as polygon so we can attach data to it we can split it up into smaller units of analysis so here even though we have block groups and there's what about 150 block groups in little rock we can divide these up into much much smaller cell sizes and with that we can attach data to it be business data health data social data crime data there's really cool micro level when i say micro level i believe it's about 90 meter grid cells for ambient populations that's different than residential population which is different from the census and connecting it to the block group or census tract level so you can weight it based on an ambient population side not going to get into that for this video that's a bit more in depth but for now let's just go over how to make a fishnet and i'll join some crime to that so if we go into fishnet and i typically forget where the actual tool is so i typically search for it and it's in data management tools but if we just go ahead and open it i want to specify where i want to save this and i'm going to do it it's already in my week 12 folder so we're good to go there i'm going to put little rock fishnet and i know the grid cell size i want to use so i'm going to do 500 feet i typically put a description at the end just so when i'm looking at it i know what i did without going into the properties of it with that this is where you have to specify your coordinate like where you want to actually make the grid this becomes a bit difficult if you don't have a template keep in mind when i say a template it has a default of nothing here but if you change this and just click on the down arrow down arrow sorry about that you can pull up your boundary file so this is going to bring up the x and y extents and everything with that so you don't have to go in and find those with that all i'm interested now and this will give you a couple clues as you hover over it the size i want to be 500 and the height 500 and if i put zeros in the number of rows and columns that will maximize it for me but since i put those in there it's fine right now i don't want points with these so imagine for each grid cell you can create label points for those i'm not interested in that polyline if i did lines it would truly create a grid but each one would not be individual polygon each individual grid cell would be unique so that would have a unique id it's separate from one another you can display data in a different fashion that way so i would recommend using polygon here and with that can hit run keep in mind little rocks about 120 square miles we're now dividing that into 500 by 500 foot cells give your computer a little bit of time you can tell mine even though i just restarted my computer hoping that it wouldn't have too many issues and it didn't do too bad there so here you can tell if i change my drawing order it went to the boundaries or the extent of my boundary file of little rock itself that's the block group but it still shows it now you might look at this and be like well it also included probably 20 to 30 000 probably more than that outside of little rock itself so if we change these up real quick and just open this and let's see how many cells this created it's about 32 000 cells at 500 by 500 feet oh so actually fewer than i thought i've usually made smaller ones i guess so with that though we want to select our cells that are intersecting or within little rock itself so keep in mind at this point we can do a select by location we want to select cells that intersect or within at this point we can do within so we might lose some on the edges but it's okay for what we're doing today we want to say that they're within our boundary layer itself so if i hit apply you can now see the ones that are within little rock highlighted and it drops down to thirteen thousand with that a key part is once you select it you wanna export it out as a separate layer now so this one's gonna be and save it to the folder go back click on it and there we are and it's going to be little rock fishnet 500 feet clipped i'm just essentially saying what i'm doing i'm not technically clipping it but in my mind and kind of my mental notes i know what that means that i've selected just little rock itself and once that pops up can close out can take off our original grid fishnet itself and i deleted the wrong one of course but i can hop back in this is for a terrorism project uh and let me go back into the courses and re-add that layer i probably could have just hit undo yes control z to the rescue so there we go remove the full one and now you can see when i zoom in the cells themselves when i say it's a smaller unit of analysis if i bring back up the block groups you can see now that there's many many smaller grid cells within each block group this is giving you the ability to pick up on variation so smaller units you can also see here what i meant on the border itself it's fully within you could also have done intersex contains you can also get the centroid and make the centroid the determining factor for us at the center point the xy center for each cell can be the determining factor but now we actually have a grid for little rock when it comes to cells the cool part is we also have a ton of crime data with that a couple just notes for this video i'm not going to select cells that only intersect streets with that say i'm going to turn this off and turn on the streets say you did end up geocoding all of your crime to street center line with no offset it would be important then to only select your grid cells up from the fishnet that intersects streets that's because you're only having an outcome or event that's geocoded to a street by including more cells that don't intersect streets you're not going to have data that's attached to them so depending on the type of analysis or metric you want to calculate run you're going to have a lot of empty values that just can't actually have an event occur so you can have false positives and a lot of other issues there it's not fully representing the analytical ability to actually predict or understand that issue in reality crime could occur technically anywhere in little rock but in reality when we're address matching it can't so with that we have to keep that in mind with how we're representing the data and then what we're trying to do on the next step all right now that was a bit tangential but let's go forward all right so let's open up our crime file and let's create a subset from this so attribute table let's go over to our descriptions or category well technically i'm having a lot of my students this week do something with from trace the gun violence archive so to keep in line with them let's go ahead and do something gun related so you can see here that there are quite a few different weapon types with that let's go ahead and choose i'll do fire arm handgun there was rifle so let's do a select by attribute and it's going to have a couple or statements in it so we go to select by attribute i want to select from our crime layer develop a new expression i want what was that weapon type weapon type is equal to and i'll give you the drop down options here so i'm going to do firearm i want to add a clause i want it to be an or statement rather than and because if i use and it would have to be if a firearm not stated and in this case if i'm using the same variable of weapon type it can't be anything else there's only one label by doing an or it could be this or that so with that weapon type is equal to at this one let's hop down to handgun add another one similar process until we grab all of the identifiable other firearms firearms in general so let's do another ore weapon type and we have rifle and the last one will be a shotgun if you're not familiar with little rock even though i've talked about it in many many videos little rock is a fairly violent city united crime type and not weapon type so the use of firearms is not surprising little rock also has received gun crime intelligence support from the federal government uh has recently in the past i think four or five years implemented shot spotter i have not evaluated that or spent much time looking at their data there but they do have shot spotter i won't get into that tangential argument of should people use it or not but for now we do have 1431 crimes that included some type of firearm so again since these are highlighted we want to hop over right click data export our features and it's going to go back to the geodatabase where i'm pulling all this from this is really my base one that i use a lot of little rock data in i have everything saved to the geodata it makes life easier for projects with that i'm still in my prior folder so i'm good there so i'm going to put little rock 2019 firearm related oops crimes cool we're good there and now i can turn off the selection here can turn off the streets i want to turn on the grid cells with that similar to before you can change the symbology of these so if you just wanted to do an outline so you can still see through obviously it does look checkered but if you zoom in you can see where they fall with it there is the chance if there were any along the border edge that had itself cut off looking pretty good there might be [Music] so you can see here of why this would matter of why you'd want to do an intersect because you do have a couple crimes that occur right on the edge in these cells based on how i did the select your selection part was within you want to do an intersection one there and you would miss these or they wouldn't be joined to it you can see that's where i say it's always good to be familiar and play around with the data to make sure you're capturing all an easy way to double check this is when we do the join we'll see how many and i'll do a joint count we'll see the sum of it and see how many were not fully joined to it it looks like there's probably going to be close to a dozen or so without counting each one all right so we have little rock we have our cells and we have our firearm related data now we might be interested we just want to display these based on counts it's it's kind of and i'll show you not drastically different from creating a hotspot map this one just counts alone but it doesn't look at density it's just truly counts itself so if we do a join again we want to do a spatial join and remember we want to join to our cells so the grid cells themselves so since we want to join data to that we're going to right click we're going to do a join and we're going to do a spatial join here and with that we want to join our little rock 2019 firearm give it a second to load there we are so i want to join the firearm incidents and i'm going to rename this one being lr efficient counts firearm 2019. again find a name instructor structure that works for you join operation this is what it's meaning of i only want each point to go to one polygon each grid cell is an individual one and i want to do it for this since i'm joining points to it we're going to do a contains within it so our target feature contains our points and we should be good to go from there so if we go ahead and okay we could run it but you'll see that i might as well show you a couple other tricks since we have done in a while i'm deleting out a couple of these because since we're joining it's not as important and since we're joining multiple ones it's not as big a deal for it but with that i might be interested and say even though it gives me a join count you can also create a field and add it in if i just want to do a count and put that down there obviously it's in red because it wants you to specify what's going on with it and i'm just going to hop over there and hit count and there we go count of there we go and here i'm just choosing any one so it can be fid i just want a count of how many of those there are we have that i specified it there so we're okay it's going to run for us remember we had i believe it was 1431 firearm crimes i'm going to hit pause i'm going to let this run because just how many cells there are of course when i say that it already runs for us so you can see now that it has generated i'm going to turn off our firearm crimes again another way if you want to zoom out you can zoom to layer this just brings you back out to the extent but now since it's joined we can right click open the attribute table we have a drawing count so if we do a descending part it will show you the highest one so the highest value is 22 in a given cell and that's similar to what we see over here with our count feature itself that i created and just add it in if i if you didn't know that the join count would be there you also have the option that you can other run other statistics with it as well with that if we remember we had the about thirteen hundred fourteen hundred thirty one got them backwards so we had 1431 original firearm crimes if we click on this and hit statistics it's going to give us our summary statistics our descriptives of it and we'll see how many we are missing from this hopefully doesn't freeze on me actually had arc pro crash on me for the first time last week and there you can see that we had 1431 originally so we're missing 17 max is 22 plenty of zeros we barely have a mean of over one so you can even see here that we have a highly kurdish and skewed obviously this should be appropriate to do kind of a count based model something with that but we also have a lot of cells in here that honestly even with 13 000 we'd probably still take out another 4 000 that can't actually have crime occur in them but for now based on what we're doing we don't need to do much with it but cool so we have this let's get into the symbology part so you can actually and we'll get into and i'll keep this layer and we'll see how this looks oops wrong one so we want to do it based on a graduated color if you ever see this this is where arc pro has a default set to ten thousand we have over ten thousand with it and when i say that we have grid cells over ten thousand so to get around that oh where is it advanced symbology sample size and i just add on another zero make it a hundred thousand now we can display everything not a big deal that's just understand if you see that warning it's your sample size aspect to it let's hop back over to our symbology part it has joint count up there i'm okay with that for now with it though natural breaks given our data set i'm not a huge fan of if we did equal interval it's not a great way to display the data either we can do quantile and for this we're going to have a ton of zeros so this is breaking up into 20 percentiles with that i'm really only interested in let's say our high ones here so we could go in and i didn't really mean to do that go back which i guess yeah i did mean to do that i was going to change just the color scheme but since we're in here let's just go ahead and do for our lower ones and no color and no outline part to it so you can start to see now where some of our higher regions are so these took out all the zeros so you can see we have far fewer cells that actually have values within them an easy way to double check that and kind of just see is if we went back and just did a quick select by attribute we want to choose from our firearm a new expression we want to do join count is equal to or graded to one hit apply we're going to see all those highlight open the attribute table real quick and we see that 560 cells actually had more than one one or more sorry so a small subset of that and you can see there are some clusters formed and if you're familiar with our prior videos in little rock we've done some other types of videos based on tracks and block groups and this is where i say variation matters greatly depending on the unit of analysis that you're looking at so you can have a high block group a high sensors track but when you delve down into it a bit more and you can do this at the street segment level so i've joined some of this crime data before to street you can see certain streets are high this is just another way to visualize the data itself but you can see within any i'm just going to make the boundaries so a bit larger just so we can tell in some of these areas there we go thicker wider there we go so when we zoom in we can see that even though and i'm familiar with little rock in these areas in particular then down here as well often come back as our higher rates and counts across crime types not just firearm itself but with a you can see that a lot of these areas even within specific neighborhoods of our block groups don't have any and it's really coming down to a few blocks street segments that are contributing to the higher counts as a whole which depending on the neighborhood population can increase the rates to a certain extent as well so keep in mind variation exists at many units from a hot point or address and if you haven't heard of that read up on from radcliffs hotspot matrix i think it was 0.407 one of those years good article that discusses the different types of temporal spatial aspects that people should pay attention to but you can see this variation across a lot of our different neighborhoods some have none relate or no firearm related crimes from 2019 some of these have a lot and that's where we say crime's not random it's not uniform it's not equally spatially distributed if so the 1431 firearm crimes would have occurred in 1431 different cells we saw that they didn't there was a reduction of it we missed out on the 17 that were outside of the boundary but if we assume spatial equality they would have been at least one per cell to some extent or randomness to some of that we haven't assessed statistically if this is different from random but if we also did the average nearest neighbor based on our point data and we've already done that before in another video we would likely find that this is clustered based on what's observed versus expected so no shocks there so at face value this is a quick way to do a count join to a grid and how to make the grid often times nowadays people make a grid cell size based on the average block length you can do it that way you can go to even smaller units a very common thing to do now is attaching everything to a street segment based on how much data is joined to that in some capacities especially with our cells you can get into an issue of intersections and how cells fall can integrate and fall across so street segments or blocks are becoming the norm but cells still have a lot of value with it especially when you want to do map algebra super easy and when i say map algebra and this is going to date myself probably a little bit if you think about overhead projectors and these sheets that you put on you can layer and layer and layer and layer them on one another and still see the image as it as you put more on imagine doing that with grids so when you have data attached to the grids you can keep layering on and do different mathematics and statistics with it based on the grid layers itself we're not going to get into that here but tobler gets into that as well so if that's of interest and that tobler's law i'm blanking on oh something else but again really just comes into map algebra but now let's turn these off and we'll do a simple comparison here in a bit let's just do a kernel density estimate of the same data all right so let's hop over to our geoprocessing we just have to go back so it's still in the fishnet creating part go back again now we're in our toolboxes we scroll down to the bottom if we get into our space spatial analyst tools we have density which is what we want to essentially do we have a couple different ones and i'll link to this kernel density one you have the options on the on the website to click on the other ones if you want to learn a bit more but for now let's get into kernel density so with that we want to input our point you could also use lines but we have our points little rock firearm crime population field we don't have one right now and that's where i say you could use a population such as the ambient one which i'll make a video at a certain point i just want to become more familiar with the newer ambient population data myself so i'm going to do kde little rock 2019 firearm hit save and yep so if you ever see this it probably has too many characters yep and that's the issue so go back in and i'm just going to do kde firearm there we go cell size and this is where it's giving you a default one now i'm gonna make it the 500 to make it similar to the fishnet that we already created search radius so this is where it's going to do the density part how far out it's going to search for it so typically you if you're doing it based on your block size you'd want it to search three or four blocks out from a routine activities perspective so with that i'm going to keep my cell size of 500 and do a search radius of 1500 for now so it's going to be relatively small and i'll say this because if you read and look at the article that i'll link to for heart it gets into how parameters matter for this so for now i'm not going to get into it too much but i'm going to put 1500 here so three times around with that you can choose your output unit square miles square feet for us these are going to be small all around if i do our part here our map units are and feet are ready so we're fine there but i'm going to choose square feet we're interested in and this is where it will give you a couple different options expect accounts and i'm not going to get into the whole difference now of expected counts and densities it gives you those definitions in the walk-through in the website alone but essentially they expect it counts the density times the area kind of thing but for now we're okay and we just want to do our density our method is defaulted to planar which is fine since it's much more of a local projected part to it so we're good there barrier features i've honestly this is different than i've used before so let's hover over it ah bears within our data set but for now we're okay you can always and this is where i always set the extent when it comes to kernel density and arc gis just out of habit of if we turn on our crime real quick you see our highest and lowest points and it still might do it to where it would only run kind of a box around this but i actually want the box around the entire part of little rock so the processing extent i hop in and just go and set the boundary this is what i've kind of just got in the habit of doing over the years but with that let's hit run and there we go and we can pull this up and you can see here where it's starting to show i'm going to turn off the values here and you can see the darker it gets this was in square feet so that's why they look so small the dark it is the more densely populated that is so if i open the attribute table uh so key part with raster data you have to actually convert it back to git to it so raster data are different from uh polygon polyline obviously that's talked about in the book but with that if you wanted to change this so you have an output we know our hotspot so you can see they're here with purple we can still change the symbology and there everything with it if you wanted to do a different spectrum i was trying to find a red one where'd it go there it is i'm just going to flip it if it lets me reverse the order it's actually not letting me do that but for our bottom one here i'm just going to hit no color you can see we have our hot spot map i'm going to show you here in a second how to export out of raster into a polygon file but now if we turn these off and on you can kind of see now what it's doing differently of even though there's crimes occurring elsewhere they're not densely populated in certain areas so you have standalone ones what kernel density is is using the search radius to check how many essentially grab those crimes around and understand how densely populated that area is to that one cell so all the cells surrounding it how many are there within 1500 feet and from that how densely populated based on the area is it so with that you can see that this lines up to what we could guess if i were to ask all right where would you think high density clusters would be you would probably guess a lot of these so we'd be in good shape here so with that kernel density and this is a quick way to display the data is what you often see and this one i say a hot spot map this is what i tell people if i change this to have say a green part to or is that one at what people think of almost as a weather map there we go red's bad storm's coming yellow is still bad but not as bad as it goes down to green yes there's potential but not as much no color you're in the clear for the most part rarity does exist but this is how if you look at it weather map is what comes to mind a lot of people look at it red bad green blues you cooler colors not as bad with it and you can see here it's set to equal intervals which makes it better for this for it so 10. now with that what i mention with this is we run into the issue with raster data the ability to manipulate it so we have our density values here which are great but how do we manipulate those and how do we use those so if we come back up into here and go to our data management we want to oh why am i blanking on it ah because it's not data management it's conversion from raster so when i do from raster to point you can do polygons but i typically do point then to polygon because it will give you the centroid and then you can join your centroid to another fishnet layer but this is where you would go and transfer if you wanted to do raster to polygon you just input your raster our firearm one our value field i'm not sure why it's giving me that error give me one second all right let's try this and i want to troubleshoot kind of on the fly so what i want to do i'm going to make this similar kde but i want to change up one part to it so population's fine i don't have anything that i want to use as my population base it i don't want it to save uh that might be the issue of where i'm saving it so let's do that i'll keep it this is going to my geodatabase instead of a folder so 500 foot 1500 density i'm going to keep i'm just going to keep it a square mile now keep in mind i'm going to go in and change my environment to be the boundary hop back over so keep in mind that this is saving to a geodatabase rather than my folder let's see here it's going to generate that cool we have the same map higher value so you can do more with it now but now let's take a look if i go back here scroll up i have a raster and i want to do raster to polygon and it's still giving me the same error codes ah so maybe that's my fault of not understand why i won't go to polygon but it will do a point for us so you could take the points and since we already have if i turn this off our grid here you could do a grid afterwards so if i we have our kde raster output it's it's gonna save there which i'm fine with if i just hit run we now have all of our points again similar to the same issue we had when we first did the grid now we have points everywhere if you zoom in though they are dead center as centroids for our individual cells because the extent layers for both of them were set to the boundary of little rock and use the same cell sizes so at this point all you would have to do is do another spatial join and you would have the grid cells with your density values with them and if you go to your attribute table your value itself so let me sort descending keep in mind this is the one that i did originally in square feet so you're seeing these values here that are the red to green spectrum versus the purple ones so the higher it is the more densely populated it is you can see and i'll just highlight a couple and then zoom out you can see where those were originally located and those are the areas that come back with it so with that i apologize about the end part going trying to do polygon but it's point and you can take your points and go to and attach to your good cells and you're good to go there with it we went over how to make efficient how to join crime data to that how to do a select by attribute based on multiple weapon types with or statements also went over a very quick way to run a kernel density hotspot tool how densely populated is a point item or you can use it with lines within your study area you can specify you can change the parameters from cell size to search radius like i said there's articles out there that spend time going into that and testing it depending on what you want to do what matters more so than others with it if there's any questions that come up interest for it let me know in the future and i say that probably over the summer i will get into the ambient population data and how to use that because population does matter and when we think about risk and victimization and everything with that you have to have two parties an offender and a potential target a victim so how those join in space and time matter so you need to understand how people move in ambient populations one way i won't go on my tangent about twitter data but we'll get into that another day if there's any questions please reach out to me but like i always say experiment with your data especially when it comes to kernel density maps you can do it a number of different different ways display it different ways a good way to typically break down your values is based on the mean and standard deviation so what areas are plus minus one standard deviation from the mean you have your high and lows but since we're really only interested in our high crime areas so what's the top 10 percent what's the top 20 percent that's probably going to give you more than the standard deviations because our mean value is typically going to be super low but that's where it's play around with your data understand it look at what it's displaying what story are you trying to tell with it and if there's any questions please reach out take care

Channel: Crime & Security Data Analytics Lab - CASDAL

Views: 360

Rating: 5 out of 5

Keywords: ArcGIS Pro, KDE, Hot spot mapping, Crime Analysis, Crime Hotspot, Crime Hot Spot

Id: qlsZsuzP6G4

Length: 38min 31sec (2311 seconds)

Published: Thu Apr 01 2021