Processing LiDAR and Drone Point Clouds in GRASS GIS 7.8.3 - Part 2 of 2

okay sorry had a little glitch with the old video recorder here so we're back basically I hope where we left off we had imported the both the drone and the lidar point clouds we're looking at the ortho mosaic and so now we have our base data in and remember we set up our location grass location we just have the permanent map set and the sort of best practices with grass is to put your base data in permanent and then create a new map set to do any potentially destructive work or additional work with that so you're not working in permanent you're just importing your base data then we should go create a new map set so there's a couple of ways we could do this we could potentially do it from one of these modules over here by creating a new map set we could also just close grass start it again and create a new map set but since we're here right now let's just do it right here and I'm gonna call this a point cloud processing that's gonna be the name of my map set click OK and it gives you this little message and we've technically changed map sets so it's a little bit you know if you're not really following along with what you're doing you might get confused which is what I still like the ritual of closing grass and starting it again and creating my map sets that way but now I just want to let you know we are definitely in that new map set if you want you can even just close the display and then start a new display by new map display window like so and so now we can see that we actually are in the point cloud processing map set just make this a little bigger for y'all okay so new map set we have no Maps in it all our maps are impermanent at this point what we want to do I'm just gonna quickly read in our mosaic just because I have photo mosaic just because it gives us a nice something to look at while we're talking about the next little bit here okay so that's essentially what we were looking at before now we could process we're actually at the point right now where we need to process the point clouds both of them to remove the tree so if I was to lay in the point points here the lidar would be kind of and we were to zoom in we would see kind of like trance X where the laser was shining and the drone point clouds more densely packed right in and around their eye if I was to do the whole point cloud especially for the lidar point cloud on my laptop we'd be here for several hours so I'm gonna suggest for those of you students following along at home doing this on your laptops yeah we're gonna zoom in and just work in a little area and this little upper area around these houses is kind of nice there's a slope here and then there's some trees so it gives us a nice little area to focus on and it's not a bad idea when you're working with any kind of line or data to zoom in at first and work in a small area that's representative of the topography and vegetation cover until you find the parameters that seem to filter the point cloud best and then you can zoom back out to the whole point cloud set it to those parameters so through routine to those parameters and then you can run it and walk away and trust that it will finish it and create a good filtered point cloud afterwards so what we can do it's pretty straightforward to clip our what we're gonna do is clip our point clouds out to just this little area over here so I'm just gonna zoom in just gonna use my own zoom tool here yeah that looks pretty good I got a little bit of the wash I got some of the houses I got some of the trees I feel pretty good about this I might just push it over so I get the entire corner of the house let's get all of those houses yeah that looks pretty good to me like that okay it's not super important and our display is set here and what we want to do is to set the computational region from the extent display so there we go we're now set our region to match the area that we're zooming just the boundaries yeah we can clip by going to vector and then [Music] basically down here where it says the overlay vector maps clip vector maps V dot clip and we should be able to clip by the region now usually what I do is I make a just a little polygon based on the region and if this doesn't work this way I'm going to I'm gonna do it that way so let's just see what happens if I select that twice and clipped hopefully I believe like by doing clipped by a region it should should actually clip it yeah 9,000 that's pretty good just FYI if you do this V clip and it gives you an arrant says the topology is not built then you're gonna need to rebuild the topology for your original drone or lidar point clouds are both of them and you do that by the vector menu topology maintenance create a rebuild topology V dot build and then you would pick like your you'd have to be back in permanent directory to do this so you'd have to go back into the permanent director II select drum point cloud can't run build the topology don't display it and it will just run through it and I take a couple of minutes because there's what 2 million in in the lidar point cloud and almost a million in the drone point cloud and then we'll just map out the topology and then it will let you click that and so if it doesn't give you if it gives you an error that's what you have to do I actually rebuilt the topology on my points while the video was glitched out because I knew this was gonna be a problem so sorry for doing that off camera I should have done that I should have waited to the video was running so we've just clipped the drones we can see the drone points here and there's only 9000 of those points let's now clip the lidar points and just do this name this right our point cloud clipped and we'll hit run and look there's only ten thousand of those so we've clipped him out let's take a look at these two differences between them now that we're zoomed in you see the drone point cloud was sort of bunched the trees are kind of missing a little bit so that drone has already tried to you know web OD a mice tried to process off the trees a little way which is pretty good but there's still some on the rooves and in the trees around here so let's uncheck the drone point cloud and here's the lighter one you can see yeah it looks more like trance X and it definitely goes across some of those trees so we're certainly going to want to process both of these guys out so there's a variety ways of actually processing or filtering out vegetation if you have the return data which is the actual reflectance is it the first return second third return you know you can use that to help filter out the trees if that is probably the most advanced way of doing it you would need to have compiled grass with lidar support lis support and you would do that while importing and import raster I believe yeah RN lidar or I think there's also a V in lidar and you would filter based on the returns at that particular with that particular tool but my version of grass wasn't compiled with that mmm I think most of the binaries you download from the grass website don't have that you have to know how to compile it on your own from source because of the LA s source tools have to be compiled as well which is a bit of an annoyance but we can work around that there is if you wanted to go the easiest way to create at the with point clouds is under import raster data are in X Y Z and then you essentially just pick a resolution and then let's say average the elevation within that but it's not particularly robust in terms of removing things like vegetation so that's why we did V in X Y Z then we're filtering it this way there are some tools under vector lidar analysis these three detect edges detect interiors and then correct and reclassify you run them in sequence with your input point cloud here and then the output of this goes into here the output of that goes into here and you might have to run that two or three times to fully filter it out and then you could filter out the vegetation that way but luckily for us there is an easier way and that is an extension called V dot lidar not MC sea so to install an extension you go to settings add-ons extensions and then install extensions from add-ons Qi dot extension and then you can just scroll down through in this case to the vector area of vector add ons and keep scrolling down until you see it whoops I passed it there was Feli Tower MCC and click install now I already previously installed this so I can find it under my modules tab under the add-ons it won't appear here for you until you restart grass one more time so if you want it to be here you can do it and you know restart grass and then they'll be here then you could double click and we'll start the other way to do it even if it doesn't show up here if you're after you've installed it you can go into the console and type V dot lidar dot MCC and then hit enter might have to enter twice just to make it read it and it pops up V lidar MCC so before we run V dot V light our MCC what we need to do is to set our region resolution essentially to match our desired output resolution for our final digital terrain model that we want to make and for these data that we have here you can see the drone point cloud has reasonably dense points but it's patchy it's missing some big holes whereas the lidar data is much more evenly covered but the points aren't quite as close together and if we're to go up here and draw just measure the distance between one row and the next it's telling me it's like 1.3 meters you know from between rows of the lidar data and if we were looking at the sort of big patches that are missing in the drone point clouds waiting for it to load back up oops we're missing have to mess around with this stuff okay display the points for me please okay it's something like some of these things are like twelve or fifteen meters okay tool to clear it and so we have to pick a resolution that's reasonable even though we have some really dense points we're gonna filter out the vegetation so our points are only gonna get less dense and our holes are only gonna get sort of bigger in these in these point clouds and so we need to pick a reasonable resolution to do this at probably with these data you know and to make our lives easier well just try and do about a meter one meter pixel size is going to be our output desired map potentially we could make a higher resolution one but again we're doing this you can do this three interpolation and we don't want to start interpolating to the point where we're sort of making up data so a meter maybe two meters depending is a reasonable resolution let's go to our settings menu computational region set region and just so we can see where we're at right now if we hit go to the print tab and hit print current region the ortho mosaic that we set our region to match is like three centimeter resolution which is super-high resolution because the ortho photo can resolve that but if we tried to run the light our MCC and then to interpolate a map even in our zoomed in area at 3 centimeter resolution it's going to take a long time on my laptop so we definitely need to increase the resolution to 1 meter so go to the resolution tab 2d grid resolution 1 pretty straightforward hit run we haven't changed the boundaries of our box so our boundaries are the same and our resolution is now 1 at this point we can close G region we can bring back up our V light our MCC and get going so let's start with the actual lidar point cloud clipped that one right there and what this is gonna do is it's gonna look at the point cloud in 3d and it's gonna have a canopy and the ground down below and then it's going to create a it's going to interpolate a really stiff surface 3d surface somewhere in the middle of that like the average of that and then it's going to run through at a couple of different resolution scales using this V outlier tool V light our outlier tool to pick out any points that are above this medium surface and say those are definitely trees let's delete them and separate out two sets of points points that are probably the ground and points that are probably the trees and so that's what we're gonna do here we're just gonna put lidar grnd pts ground points and then lidar TRS pts trees points right that's just shorthand I like to do it you could name them whenever you want this helps me keep it organized over here we have a few parameters that that could be tweaked the tension parameters would tweak how stiff the this theoretical mid-surface that it's going to use to determine what our trees and what aren't would be so you can change that same thing with the spline steps we'll make it smoother and flatter if you increase the spline steps and then you could change some of these things but these two are the ones apparently that are that are the most sensitive Alain's are the number of different different resolutions it's going to run by default with three it'll run at once at a slightly finer resolution once at the resolution you set one meter in this case and one set a slightly coarser resolution and I believe it would in this case do something like point five meters one meter and one point five meters well if you increase this you can potentially filter out at you know a little bit more in a more detailed way but three is usually fine and the rest of these are a little less important you might have to get into them and I would certainly suggest you to read the manual for this for this guy it's actually pretty pretty straightforward when you read it and it's got some nice displays in here but at the beginning at this point we're just gonna leave all the defaults the defaults are reasonable and we'll just see what it looks like so we might have to run this one this one module a couple of times but we're gonna have to first then look at our point cloud or at least look at the you know render up you know surface and look at that in 3d see if it looks good and then we'll might have to repeat this so I'm just gonna hit run at this point and because we zoomed way in and we set our region a resolution pretty you know at one meter this guy went fast we were doing the whole area it would be a little bit longer looks like i zoomed in maybe more than I had to but that's okay because again this is iterative and we want to find the right settings to create the best map and it's much more useful to do this in a fast little area and then expanding in if we need to so we did this let's take a look at what our results would be I'm gonna just clear that out because it's frenetic over there okay so V dot V light our MCC and then now we need to interpolate our ground points into a raster map so we've set our res into one meter all we have to do at this point let's go to our raster menu go down to its as interpolate surfaces and we're gonna use our V sort b-spline from way back when we talked about interpolation and I like to just sort of have these two guys up side-by-side because again you're not might have to go back and forth between them so I don't close I'm not going to close me to let light our mcc until I'm certain that I got everything correct right so over here the name of the input vector map it's gonna be lidar ground points my phone is over here it's it's got the things I'm asking it Google to do something super anyway okay so light our ground points yeah then our settings over here the spline step we're gonna want to figure out the spline step and then the smoothing so the simplest thing to do is to go to the optional tab and check this box that says estimate point density and distance you hit it runs oops okay I have to put in some required names here the output tabs so we're gonna put lighter DTM and whoops put in okay so then now we have this checked and we hit run it says mean distance between points is about a meter and point nine five centimeters right so what we want what we want to do is we can uncheck that at this point go back to settings and the spline step minimum should be about one point five times the that average distance so at a minimum we should be putting one point five here if we increase this if we make this number bigger than that it's gonna smooth out or kind of stiffen up the surface a little bit so we might need to do that but at the beginning we're just going to start with that we're going to choose a bicubic style interpolation and then we're gonna leave the smoothing parameter pretty low if we wanted if it looks a little bumpy we can make this larger we would make it to be just point one but let's take a look at what this looks like when we do this so I should render pretty quickly because we're a pretty small area okay so this is our map here and let's make sure it's the only one checked because we're gonna pop into the 3d view have a look at it okay so let's go to the data tab make sure our fine mode is set to one so that we're seeing everything you know know the View tab let's pull it around so we can get a nice view of the area that we're interested in and let's increase the Z exaggeration too - yeah that's really showing me that the trees aren't fully gone in this so we can try we can try to work just inside V surf b-spline by smoothing it a little bit with these settings so we could take this up to three and we can change this we'll just go all the way down to one and then on the optional tab will kind of overwrite those files and hit run and this may work you know it may be enough just use these tricks of interpolation but we may need to be filtering out our trees more so now that we've done that we're gonna just refresh our display and pop into the 3d View mode and let's take a look so it looks like we have smoothed it out but you see you know I think we may have been a little heavy-handed on our interpolation I think we can try one more time with interpolations back to 2d mode and this time I'm gonna leave my smoothing to the original point on one and we'll just let that do its thing I like to refresh the display and then go into 3d view yeah so I think we're being a little heavy-handed with the smoothing so the trees are still there you can see them popping out there a little bit so that tells me we need to go back to V light our MCC and tweak our parameters a little bit so let's go back to our 2d view mode we'll bring up our V light our MCC will go here and probably let me just open the manual and see what it tells us to do in this case it's always good to double check that you're doing things the right way okay so optimal settings are it says trial and error usually required spline steps have the and threshold T curve prince turki yeah spline steps and curvature tolerance T parameters have the most influence on the results large Ursulines footsteps and lower curvature tolerance threshold lead to more points classified as non ground points so that's the one that we wanted tweak so let's go up to 15 and T it says what should we do with T lower curvature tolerance so let's just take this down by 0.1 make it point two and let's overwrite and hit run okay so at this point we got to go back to our be serve b-spline and we can leave our settings the same and just hit run again because it's got the new input we just remember we over wrote this ground points so we're gonna render a brand new map this time and we're back over here and we can pop into 3d view and it looks like we're doing a little bit better here but we still have some of these trees so let's go again this is an iterative process so where did my then I closed it silly what a silly thing to do okay so I believe I closed there's b-spline to many windows yeah it looks like I know here it is haha there it is it's just large I was looking for something small okay so let's go back to our optional tab and let's try making these way bigger and this tolerance a lot smaller and it's already got the override tab checked so let's see it run as soon as that's done we can just hit run over here because everything's set up and let's take a look so we're doing better I think it's still a little lumpy we could continue doing this I'm not going to take up your time until I get it perfectly right instead what I'm going to do is go back into here and I am gonna increase my smoothing by a little bit and well that's happening and go back and forth and by that time it should yeah so that's reasonable you can continue playing with this until you get it exactly the way you want so we've done that now for our this is our light our points let's do it for our drone point-cloud points and for V lidar MCC I am going to I'm just gonna set everything back to the way it was like so under the required tab I'm going to find my drone point cloud clipped and then I'm gonna change this to drone ground points and this to drone trees points I'm gonna run so it's gonna take a little longer because there's actually more points in the drone point cloud in this little area and here's our V sort of be spot line I'm just gonna pick now find drone ground points I'm actually gonna leave everything else exactly the same it seems reasonable to me and speed things up just a little bit and we're gonna hit run and then now we see I should be seeing our new drone DTM oh look at that I accidentally forgot to change the output let's quickly go back here to light our ground points because this is the danger right here the optional override tab was checked and I forgot to change the output file name so I accidentally over wrote it so lidar ground points we just restored that file now let's pick our drone ground points go to our output change that to drone DTM lower case now I'm gonna hit run over here okay and here we go drone point cloud was added okay so making sure just drone cloud is checked we can go into the 3d view and just take a look at our work here and again we have to go to the data tab first and make sure that the fine mode resolution is set to 1 and under the view we can see we have a lot of trees and that drum point-cloud data still in there so we're gonna need to really filter this guy a little heavily in light our mcc more so than the actual lidar data so bring our two modules back up our B spline and our V light our MCC go to our optional tab let's really increase our supply steps to like 30 and our curvature tolerance let's bring it way down to like 0.05 and see what that does for us so let's see definitely by doing that we're taking a little longer right okay so now we're just gonna hit run again without tweaking our interpolation settings cuz we wanna check it out oh we may have gone way too far we may have gone way too far we're gonna get just like that we filtered out all the points yeah okay let's go back to our whoops our v-lite our mcc we'll take that back down to a more reasonable step just the same ones we used for our actual light our data - so let's hit run over here well that's to get my B spline up again as soon as it's done okay and we can run hopefully it won't look all yellow ah maybe we we have been way too aggressive over here so let's take this down well let's keep this high and we'll put this curvature back up to three and we'll see what happens when we do that again this is trial and error there are no obviously clearly there are no single set of parameters that work in every situation so this is why it's important to do it in this sort of back and forth way checking your work working in a little area until you get some numbers at work okay so it looks like we were we got a better result here in the background let's pop again into the 3d view and yeah it's looking looking like the trees are still kind of influencing yeah the trees are definitely still influencing us over here which is a little unfortunate so let's see let's see what we can do keep going and try one more time before I give up okay so let's just let's make a really stiff one maybe I'll try two more times I'll try this with 30 like we started with leaving clearly the threshold value was very sensitive in this particular case so I'm gonna leave that alone for now and then pop back and forth between 2d and 3d so that we can reload the new treaty topography and yeah those trees are still in there and with this drone point cloud trees are a killer so let's try reducing this to two leaving that at 30 and see what happens it may not it may not work but we're gonna give it a try all right you can see if you were doing this on the entire area it would take a long time okay it may not it may not have worked yeah we went too far it's that it's this guy right here clearly is a minimum number that you can use so I'm just gonna try it one more time with a real stiff plate and that I'm gonna try with the smoothing parameter and visa-free spline that's going to be the last time I mess with this and it may just be drawn point-cloud in this particular case because of the orchard it's really that regular spaced trees is really not going to do it for us so let's see where that looks like and if I have to all increase the smoothing and then we'll move on with the rest of the tutorial because there's still stuff to do okay one more time with the smoothing just b-spline optional tab settings tab sorry and we're going up this to pretty high smoothing I can see that it's still gonna be lumpy down there let's try let's just keep going let's push this let's push this really far we're gonna have to mess with our interpolation settings and may have pushed it way too far yeah it's looking like nothing up there it's really smoothing it out in the upper regions but okay well I'll just go with this as our you know as our basic result so what we wanted to do now is we want to compare our 2d TMS and you can kind of look at the colors but it's useful let's just create some hill shades and we can look at them side by side so terrain analysis under the raster menu and then we want the compute shaded relief our relief and where I start I guess we have the drone one selected so drone GTM shade I usually leave these alone but let's set the zigzag for two just so we can get more shadow going on I'll just hit run does it and then we'll do the same thing for the lidar DTM run okay so with these two maps selected what we can do is use a little tool called the map swipe under the file menu go down to the low almost the bottom map swipe G GUI map swipe and if you have those two selected it automatically selects them but you can use the ad maps to select them in the you know a little menu over here if you'd like it as well and what this does I'll just expand it is essentially lets you swipe across so both maps are in there both are registered and you have the drone one in this case on the right and the lidar one on the left and so you can do this to quickly see the differences between them in this particular case the shaded relief but any kind of map you can do this as well so that's really cool we can definitely see the lidar one is looking better a lot better than the drone one for a variety of reasons but mostly because of its eat nice and even coverage and the fact that the lasers can penetrate through the foliage better than the drone well drone can't penetrate through the foliage ok cool so that gives us our first little area we can interpolate or interpret visually what these shadows mean could lay the colors on we could do slope but let's just quickly classify this based on landform type and we'll go back to terrain analysis we could use our params gal but we can use our geo more fun why not it's a nice new one let's pick our lidar DTM and we'll leave everything basically by default we the only thing we want is our pattern output so we'll put right our geo morph and I think we can basically leave that alone for now this is a very sophisticated tool but we don't have time to get into it right now so we're just going to run that this is now classified Ridge peak valley that kind of stuff let's while we do this while we have it open let's do it for our drone one as well throw them DTM the patterns drone geo morph and we can see they're gonna be totally different patterns you can throw up the raster legend doesn't matter which one you choose it's just gonna give you a little label so you know what you're looking at for the colors and what you can do is go back and forth between them this old way or why not let's pull up the map swipe again oops and the two wrong one selected okay we'll just do it this way drone geo morph and hide our geo more okay so there we go and now we can well we could swipe it back and forth between them I'm not entirely sure why doesn't want to zoom it all the way but you know is what it is if we do this I might just have to use the zoom tools to do that just zooms in on my little area anyway fun fun fun with the map swipe it's kind of a cool tool to sort of compare maps quickly back and forth but clearly we can see there's a difference between the two renderings so the last thing that I'm going to have you all do is a real basic difference map in the map calculator and so here we can go the raster map calculator and we just insert let's start with our light our DTM and we're gonna put a minus sign and we'll put our own DTM like so and we'll call this diffe lidar drone so we know that it was light our - drone and just hit run so over here in our layers let's just unselect all the rest of them we have diff light our drone let's go right click on it set color table define the color table and find the one that says differences and so 0 will be white negative will be blue and positive will be red I believe yeah so it looks like basically they're not even the same the probably what's happening is the actual elevation values are different between them so we can figure that out if we select the actual DTM's both of them like so and get our query tool and click around put this up here and so you can see the value for the drone point cloud it's 56 above sea level whereas for the lidar it's 137 and it looks like it's a pretty standard offset Oh interesting so we can correct for that so let's take a look at our differences map over here the one that we just created and let's take a look at its histogram so blow this up it looks like there's a smattering of them between yeah between about 80 and 84 meters difference but the vast majority of them are it looks like 80 1.75 more or less so that's the value we can just add that let's see what happens let's go back into our we closed our map calculator didn't we so let's get our map calculator up let's take our lidar not our light our drone DTM and we will add what did I say 87 80 80 1.75 is what I said and what we're gonna do is call this drone DTM corrected so that's just that and while we're in here let's delete all this stuff out our let's do our difference again lidar DTM minus our drone DTM corrected and diff light our drone corrected okay so we have here our diff light our drone collected and by default it's just various color scheme so let's change it back to our difficulties so we can see what's going on that color table define differences color scheme hit run and there we go and swear to have one of these guys you know raster legend already displayed I'm just going to double click on it and change it to match our diff one line our drone corrected and so now we can see the difference is related to our our interpolation and it looks about three meters difference between the lidar and the drone and these areas and versus two meters down in those areas so why don't we take a look at this a slightly different way let's let's add in a shaded relief map layer will pick our lidar DTM shade and then we'll drape this diff map over it just so we can kind of get a sense for the topography and it looks like it's just sort of about the density at the points if we show our drone point cloud gonna take a little bit of time here underneath it looks like over here there's just too many big gaps and there's a big gap down here in the drone point cloud and that is causing that's causing a lot of error in our final map it has to do with the way the drone point cloud is generated and the fact that it's spottier and so in this particular area the drone photogrammetry technique if we're trying to get rid of the trees is not going to produce as accurate of a DTM surface of the ground surface as lidar and that will be the case if you have trees or thick vegetation anywhere if you're out in the desert or something and the drone can the drone camera can see the ground surface then potentially the drone photogrammetry technique can produce results that rival wide are so at this point what you would do is you choose your data set you'd figure out which values work and V lidar MCC and V surf b-spline you have zoom back out to the whole area that you want to interpolate and then you'd hit go and then you'd go have coffee tea lunch maybe also lunch and dinner and breakfast the next day and then come back and finish the routine with the interpolation and go have more lunch and more dinner the next day and then it would be rendered 1 meter resolution de M now I'm being a little facetious about how long that might take but it could take quite a while so you know get a better computer or have a lot of patience so that's basically it for this I know this is kind of a monster tutorial I'm gonna break it apart into two sections and hopefully hopefully it will it will make some sense when you run through it yourself alrighty and this is the last one for the semester so bad yourself on the back yay ok that'll do I will see you guys in class one more time next Tuesday

Channel: Isaac Ullah

Views: 630

Keywords: GIS, Tutorials, GRASS, QGIS

Id: hnBXaGu0GOU

Length: 47min 1sec (2821 seconds)

Published: Thu Apr 30 2020