3D point clouds from drone imagery -- WebODM, Meshlab, GRASS 7.4

hello everybody this is our very last tutorial video tutorial practicum of the semester yay and what we're gonna do today is to do drone processing of drone imagery to make them some 3d models alright so this is not part of Project 4 but it is an extension of it so you could kind of look at this as a bonus lesson because it's pretty cool so what we're going to do is to look at some of those drone images that we took that I took last summer and that you you had you know one or two of in project four for the qualitative analysis in now in this particular project what we're going to do is to use the idea that as you as you see as I scroll through these you could match potentially specific points between these things and to measure various offsets and the parallax and compute a 3d model of this particular landscape and that analysis is called structure for motion and the dense point cloud extraction is multi-view stereo and the idea is that from the very small differences in offsets as we move from one image to the other we can compute where the camera was and then we can reconstruct the entire scene in 3d based on the overlaps and offsets between all the matched features so this is a small subset of drone imagery just so that the model will render quickly in the software but I've rendered this whole valley in 3b using these same images plus some more ok so these are the images you'll download this thing called drone images and you'll extract it somewhere let's talk a little bit about software ok we'll get to this in a second what we really need is open drone map all right that's the software that is gonna render this for us there's multiple ways to install it the way I use it is from the command line but there is actually this sort of graphical interface that runs through your web browser there are two editions there is the Community Free Edition and then they have a small Edition where they charge you a small fee 47 dollars just to make it easier to install it to give you some support now we're going to use the free edition but if this is something that interests you and you just want to have support from them $47 is not a lot compared to some of the other commercial software options that can do the same thing so just a little plug for these guys this is how they get support to do all the software development so not a bad project at all man you install we go to their github site github is a repository where open-source projects can make their code available if you want to install this it's not too bad it's a little technical right so you have to follow along follow along follow along do it the lab computers have this installed already so to run it it's a little complex you have to find the folder on your computer that web odm is installed into and it should just be called web odm and you started terminal in there and you have to start it with this little bit of code here dot slash web o dmsh start that's a little startup script it does a few things here and then says it's ok and at that point you actually open your web browser and you put in localhost 8080 basically to this interface right here and the very first thing that you have to do is to make a new project I have a couple other ones made over here but I'll make one and the name of this particular area Wow okay I'll just do collab RIA Calabria drone right for now and we'll just hit create project and at this point all we have to do is hit upload images and GCP but we're just going to upload images open our own images I'm holding my shift key down to select all of them and then I just hit open and it loads them all in now this is not uploading to the web or anything like this this is just running in your web browser this is all happening on your local machine so all this is local the reason why it's web is because technically you could install this on a server and you could do it remotely but this is how they built it so this is how we're gonna run it so now all my 17 images have uploaded correctly the first thing that we're going to want to do we can pick some default options or high quality options but we need to do a little bit of custom right so custom and then you click this little widget here and you get this options over here there's a whole bunch of them we need to edit just a couple of things force focal actually starting out force focal we need force CCD which is somewhere down here yeah force CCD the CCD is the the sensor size and for most images the software can read that from little tags in there but for my particular girl it doesn't read it so you have to enter 6.17 that's the width and millimeters of the actual sensor inside the camera on my drone the other thing we're going to change is ortho photo resolution this is the number of pixels per meter by default it's 20 so that's actually a five centimeter resolution which is fine if you're really really high in air but let's increase that to let's increase that to about a three centimeter resolution so we'll do 33 all right they can go even higher it's just going to take longer to render in the file will be bigger but 33 will be fine that'll be basically a three centimeter ground resolution not bad at this point we can hit save and we can hit start processing and you're gonna see the little thing cog running over here if you want information about what's happening you can hit this plus sign and this is actually the output of a terminal and then I'll just give you information about what's going on and you know as it goes along it'll say things like running this thing for this thing matching these points for these things finding you know this 3d solution etc etc etc and it will take some time you know depending on your computer it's one processor it's not paralyzed this is why I got on the command line because I can run in in parallel and it goes by faster okay so what I'm gonna do now is I'm going to pause the video I'm gonna let this thing happen it shouldn't take too long 10-15 minutes at the very most for this number of photos and then I will come back and show you the final rendering alright we're back and the routine ran through all this this stuff and at the end they gave us this little symbol right here the little guy going yes we're done it only took five minutes because this particular image that's pretty pretty minimal alright and says completed in green and there's a little checkbox and now you have a bunch of other things that are available so the first thing to do is to view your 3d mall and over here you get a really cool sort of rendering and as you can see I've got big gaps in mind that's because I didn't give it all the the imagery that I actually had for this area but hey it looks pretty cool you can zoom in I'm using the scroll wheel of my mouse and I'm just dragging and moving around each one of these little dots is one of our 3d points right and as you can see the model that's pretty cool actually right we have a couple of tools over here we could actually do some measurement on here because the GPS in my drone information for each photo is actually used to calibrate this model so this is actually a geo-referenced textured point cloud right I can draw boxes and polygons if I wanted to and calculate areas of you know different things and I can delete that stuff so for example one thing I could do potentially is rotate this around until I see looks sometimes it's kind of funny like this pretty cool right each one of these dots so let me zoom out so I can see this house that we actually dealt with in project for this particular aerial view and now what I could potentially do is to find how big this house is what's the square footage of this particular house right and it's telling me now so it's seventeen point nine two or forty seven point nine square feet hey that's pretty cool all right and I can delete it here I can do linear measurements height measurements if I wanted to which is kind of neat you know here two here two here that kind of stuff right in fact I can create a profile across the landscape if I wanted to and let's put it there don't move me up to here and you can actually see the profile kind of coming on over here and you can save this out which is pretty neat right so this gives you a very many number of tools that you can use to navigate and and analyze your D you can even make like you can even make little fly through videos and this kind of stuff and change the 3d rendering and get these specific viewpoints and that kind of stuff which is pretty pretty cool so you actually have a lot of control over how to view and render your final image and you can do it fullscreen like this you can look at tools you can render a texture mesh it takes a little bit of time here so we're sort of looking at just the point cloud you can actually connect all the points and then drape the imagery over so it looks like a solid object right here which is pretty cool and I'm going to change my navigation style and so that's pretty cool like that and you could potentially look at it in 2d so you can actually look at your ortho photo as well so this this is Google Earth imagery behind but this is the rendered orthophoto from the drone imagery right so you can actually see we can zoom it and remember we made this only three centimeter resolution we probably could have gone down to maybe one centimeter so although when I plan this mission I think I planned it for about three to four centimeters resolution according to the height of the flight of the drone and the focal length of the particular camera that was attached to this particular job so that's pretty cool in fact we can see it's better resolution than Google Earth right and that's kind of what we want which is pretty neat okay so if we go back to our our particular project and our particular task what we can do now is to download the assets individually and there's a bunch of that so we can include downloading the ortho photo so what I'm going to do is to download that into my same working directory project for and I'll just stick that here and then we can download that's the geo tippers and that's the geo rectifying one that you can load into grass or QGIS and then I'm gonna load the textured model I'm gonna download hello textured model okay there we go save that there you know come down as a zip file and then I'm gonna download this thing that says point cloud in LA s format that's a lidar format do a reference model and well I'll just briefly show you is how you can I guess it's taking its time to download this kind of stuff how you can actually view this stuff and in some other software so or the photo is totally rectified so we can actually open it up in QGIS and and when QGIS opens up here in just a minute you will see that we can do all the stuff that we can do with our other kinds of GIS data we can measure stuff just like we were doing in the you know in the open drone map view view window we can do it here in QGIS just like this and what's really cool is you can add this now to your standard QGIS projects and you can it's got its a standard CRS format and you can digitize on top of this just like we were doing back in project one so that's really really cool and very very very very useful if you give me just a second I'm gonna pause the video until it finishes downloading some of these other things and I'll show you what else you can do with it and software called mesh lab and then in good old grass okay that's because I wasn't paying attention and downloaded them into drone images but I brought them both out so you have your lis model and then you have your textured model dot zip so the zip one has to be extracted so we'll just extract it and now we have a bunch of files inside here and particularly what we want to do is to open up this obj in mesh lab all right so let me just open mesh lab up might take just a second or two to open up like this and what we can do is to go to file and then import mesh right here and gotta find mine working directory for project for textured model and should find this odn textured model dot obj click open and then down here there's a progress bar at the bottom of the screen it'll take a little bit of time to load the mesh in but it'll progress and then basically what we have is that same textured mesh that we were looking at now we could have downloaded just the point clouds pmy or something like that but here what's really cool in mesh lab is that that gives you a whole bunch of tools for example for filling gaps and all this kind of stuff mesh lab is incredibly powerful not just for viewing this information but really actually for for for editing it and moving nodes and and meshing and texturing and all this kind of stuff and it is very very powerful and potentially a little complex so I'm just gonna show you a couple of different things so you can do you can turn lighting on and off you can turn the texture on it off you can look at just the actual meshes themselves without the texture or any color whatsoever you can look at the points that underlie all of this kind of stuff right here and and then you can't you have some editing tools here going to delete you know parts of it that you don't want and then you have filters to clean and repair to close gaps and you know all of this kind of stuff tons and tons and tons and tons and tons and tons of tools this is too much to cover in one video tutorial but you know if you want to use mesh lab there are some really great tutorials available on YouTube but what I highly recommend is mr. P's mesh lab tutorial and they go through in very very good detail how to use mesh lab at a at a professional level and one thing that's really cool that you can do in mesh lab is to load in meshes of the same landscape could take in it let's say different time intervals and you can compare them and figure out what's changed you can measure very precisely for example erosion or the construction of buildings or hay excavation of archaeological units you could actually calculate the volume of sediment that you've removed after the fact as long as you capture imagery at each stage of the excavation you could go back and much like them back compute all of that it's very very very cool very very powerful one thing that you can do is to save a snapshot so you get a view that you like use hit save snapshot and you can save it output as a you know a typical graphics file so if you do anything in mesh lab that might be it you just get your get your view that you like and you save it out and use it in a presentation or something like that so this is a real quick blast overview of mesh lab you can they see how cool that is and how you can mess around with it there's a few other pieces of software cloud compare is one of them something called blender which is for 3d animation you can use to view these kinds of things but I like meshlab quite a bit for that now the last thing that we're going to try and do is to bring out all of our points into grass and see if we can make a nice de M out of that so what we are going to do is to start grass as we normally do and what's really cool is that once we get grass started up here we're going to make a new location and we're going to call this Calabria drone just for now we'll go next and then when it says how to make it pick the one that says read projections and data in terms from a geo reference data file then hit next and then here we hit browse and we go to where we downloaded all of our stuff now we're going to pick up orthophoto right this is gonna be our basic our base - for grass to automatically make a new location and get all the projection and all this kind of stuff and by default it will figure out exactly where in the world and to the right region so gets us off right on the right foot we just sit finish says do you want to import that sure why not let's import it and then it did I mean we have Calabria drone permanent and then we hit start grass session after it finishes importing a little little pop-up will come let's say okay now we hit grass session and load up normal grass and you'll see that we have the ortho photo and of course it breaks it apart into red green blue and alpha just like we did in our project for right so we have our nice black and white individual bands what we can do is go to raster manage raster create RGB and then we can meld our green red blue oops green and blue back together and we can put grown RGB I can't run will process it along and we'll have our color imagery English not technically a necessary step for any 3d rendering but hey let's do it because that's in project for it's always good to reinforce our our skills okay so here we go our final re put back together RGB drone image this is going to be our little base footprint okay now what we want to do is to import this la s file this is a point cloud file that's typical actually la s is a lidar format but don't worry about that just closing QGIS so that we have a little bit more memory to deal with we're going to be fine using it for this particular point cloud the point clouds that are generated by lidar and that are generated by this sfm are pretty similar so don't worry about that in fact what we're going to do is to go to import let's see raster data even though these are points right it's kind of strange and what it says la has light our points import import our en dot my dog okay and we get our dot in that liner up here and we have la as input file we browse and we go back to where we exported that my project for zero Erin's model dot Elias and we hit open and then in the output tab we can do wow there's a couple of different things that we can actually do in fact what we can do is to do the bidding technique altogether here so what we should do is to put in the output tab name for alpha around we'll just put count this is going to count our points and we'll count them in a 50 centimeter cell size all right and the statistic that we choose for this will be n all right run and now we get a bit so you see how fast that was right and what we can do now is render a histogram of this particular raster map and what we can see is that most of them don't have very many points in a 50 by 50 centimeter but all of them have between 1 to 15 or so points in there this is probably the middle part so all this purple part over here this is where we don't have any information but inside here 50 centimeter seems to be a pretty good cell size what we can do is to increase that up to 25 and will click optional where it's Ohau over right and we'll hit run again and oh what did I do I've put 25 that's 25 meters what I wanted to do is point 25 right we'll hit run a slightly finer resolution and well re render this and now we'll see we have a slightly different rendering area but 25 7 is actually pretty good I'm at this point so let's say we'll stick with 25 centimeter resolution now what we can do is go to our output and instead of count we'll put max and the statistic we'll choose instead of n is max so what we'll find is the highest elevations in our 25 centimeter pixels and we'll hit run like this and what we'll do is we'll take it another look at the histogram this guy and what we see is we have maximum values somewhere in between 50 and 60 and then a smaller number have them going up I don't see any huge outliers like 3000 or something like that so that tells me my model is pretty cool in terms of maximum values I'm what we're doing here looking for outliers so these are like the tree tops and all that kind of stuff at the 27 at 25 centimeter resolution so let's look at the min valued here men value here and in value here and we'll hit run and I'll leave this guy up over here and we'll render another histogram of this one and we'll put them together well actually see there's not a lot of difference between these two histograms so the 25 centimeter resolution the max and the min values are basically the same in all the cells at this particular point we can do is render our 25 centimeter DSM by in fact we'll call this DSM 25 cm right and we'll go to our statistic over here and we'll actually take the mean actually we'll take the median in this particular case so we'll get a nice averaged value DSM and we'll hit run all right and so now if we look at this in 3d we'll just uncheck all of these guys and we're here what we can do is to change this to our 3d view over here and as soon as it starts loading in there we go so we can actually see now our 25 centimeter DSM and of course it's got a bunch of holes in it because there are some places where there's just not data at 25 centimeter resolution all right so that's the sort of last night is to check the quality here of course we want to increase the resolution I mean to put it to one so we get it looking exactly the way it ought to and I'm freezing my computer trying to do this there we go all right okay I think probably 25 centimeter was too much right so let's go back to our 2d view over here and what I will do is to go back over here input and I will try again my max but I'm gonna change this - let's actually do one meter one meter just for fun all right so let's hit run over here and we'll take a look at max at 1 meter resolution and we'll reload this guy and we will see all right so similar pattern over here let's try doing min now at 1 meter resolution statistics run over here and let's take a look at this we'll reload and we'll see okay so there might be a little bit of a difference now between Max and min in fact min maybe what we want to run in fact probably what I needed to do is to change the statistic to max for max I think I still had it on median so wants to do that real quick hit run and it will reload our histogram so both of these things and we'll sort of put them side-by-side we can actually see some differences now between these two histograms which tells us that the 1 meter resolution we might actually be able to filter out some of the trees right when people filter out some of the trees so as I flash back and forth between them maybe we can do that all right so I don't believe both min and Max checked and now I'm going to go back into the 3d view and wait for it to render a little bit again I've probably got a lot running on my particular computer but that's cool so let's this is we got min let's bring it down to 1 wait for everything to catch up and then once it does that we click the button here and we can set max down to 1 hopefully we will get our our model to look a little nicer and then when we scroll down from here once it's caught up again the 3d rendering with this super-high resolution is you know a little taxing for the computer over here one thing that we can do is to go to the analysis tab and to make a cutting plane again it's going to take a little while for my computer to render this but once we get the thing rendered we can potentially move it back and forth across and we can see potentially the difference between Max and min and to speed this up I may actually set my resolution back a little higher so let's just wait a second here okay so let's set my resolution to like three for Max and three from min when it lets me do that in just a minute here hopefully we'll do that in just a second and then things will render a little faster okay that's fine so now when I go to my analysis plane you can see how I can cut back and forth across that and at some point you'll start to see the difference between Max and mins show up as the objects as I go back and forth through the cutting plane which is pretty cool okay so let's go back to our 2d view and what I can do one last thing that we can do is we can do an interpolation and what we'd want to do is to zoom in pretty close so that it in this particular case goes a little faster I'll actually zoom in even closer than that I will set my computational region from display and then what I will do is I'll set my current region settings let's see set region and I will set it to match my I guess I'll just use min right here a run like so in fact I think that's what I had the resolution up too high that's why I was taking too long to render things in the 3d view yeah there we go no now we're talking see I make mistakes too doing that nice to know that professor hula makes mistakes too so don't worry about that right now we can actually render render our cutting planes in pretty fast but I was wondering why I was taking so long I think I had it set to like one centimeter resolution for this one meter data which is funny right alright so now we can actually see what the cutting planes are offering us on the other thing that we can do is to so offset our max up just a little bit and when we go back to our cutting plane we can blend the colors together and we can actually see for example by view from this angle we can actually see the difference between Max and min that's what I wanted to show you guys okay so now what we can do is to zoom in to our map and what we can do is very quickly interpolate or fill in the gaps so what I'm going to do is I'm gonna resize my window here so I match the sort of general shape and that I'm going to zoom in just a Titch to cut off some of the edges like so and then I'm gonna set my computational region from extent from display I'm gonna go to raster develop map and then I'm gonna do resample using b-spline like this and then I'm gonna sit here and I put in oh I'm gonna do let's say I think I'll just call this in Terp de m right here under our settings we're going to do 1.5 for this line steps so that's the length and map units will do by cubic and then I think we will just hit run at that particular point and it will chug along take a minute or so at this particular resolution and what we're doing is rendering a smooth 1 meter resolution de M I interpolated de m and now what we can do is look at this interpolated de M in the 3d view and see the difference between our original and our interpolated de M and actually what you can see here right now already even so interpolated the e/m vs. men right here okay in fact if we view this from a slightly higher angle what we can see if we go back over here and we do our Z offset like so is that our interpolated de em is quite a bit smoother in fact and we filled in some of those some of those holes that we should have seen go like this just trying to get the view a little easier for you to see maybe I should move my face out of the way like so all right and there we go turn my cutting plane off there we go so now what we can see is exactly how we filled in some of those holes in our original in our original 3d model filled in all those holes through interpolation now remember this is not adding data this is just estimating data based on the surrounding points but we've done it all of these little lumps right here the orchard trees if we wanted to process this further we could get rid of those orchard trees if we wanted to by a combination of increasing the cell size and trying to find the minimum value which would make a much smoother interpolated ëem so they go back to do that you go all the way back to your are in lidar and let's change this to like five meters or so and we'll call this min 5m and a statistic will be minimum we'll hit run and we definitely are more pixelated right for sure definitely more pixelated I'll set our region to match that computational region and what we'll do is we'll take a look now with this coarser resolution rendering in 3d and we'll see we actually got rid of all the trees because we took the minimum values which was the ground surface not the top of the trees right in that and you can see here how we have rendered a very smooth I am at that level and what's really cool is we can then go back to our our resample b-spline we can pick our min at where are we at I think it was called min 5m right min 5m and what we can do is to change this to 7.5 so this last step has to be 1.5 times the cell size okay and let's see over here what we can do is now reset our raster resolution setting computational region set region resolution to be let's say 1 meter in both bounds and then we hit run over here and what we are doing is interpolating a de M from our 5-minute output at 1 meter resolution and it will be very very smooth look at that so this is a nicely smooth surface based on our own data obviously this whatever this is a house or something like that is wrong or maybe it's a big tree that was bigger than five minutes but what we've done is got rid of the orchard trees and in fact what we've done is made a nicely smooth rendering of the valley floor right so this is sort of the power of point cloud and point cloud rendering and analysis in grass and I think that's where I'm gonna leave it there's a lot more that you could do you could actually pick out objects by analyzing the difference between the ground and the and the foliage top you could pick out the individual trees if you wanted to you could pick out the individual houses but all of that is sort of very technical extra stuff that you can do here is a basic workflow to render a nicely smooth 2d and getting rid of objects that you don't want to have in there while at the same time having the ability to to include those objects if you wanted to just by changing some of these resolutions and cell size and with our n light R and then resampling with b-spline and i think that's what i'm going to leave it for this one again this is all icing on the cake so feel free to look at this stuff and if this intrigues you we can talk further I do a lot of this kind of processing in my lab so if any of you are interested in this kind of stuff there's opportunities in the lab to do this on a more intense basis all right yeah that'll do

Channel: Isaac Ullah

Views: 26,074

Keywords: 3D, Drones, photogrammetry, SfM, MVS, ODM, WebODM, Meshlab, GRASS, GIS, DSM, DEM, Tutorial, Lesson, Screencast

Id: 0zVtZxWyBsw

Length: 39min 26sec (2366 seconds)

Published: Thu Apr 26 2018