Lidar Mapping Accuracy 101

hello again so this is the second of the two follow-up videos based on the questions and requests i've received after i put out the first video sorry it took this long we just have been very busy over here yeah you can see it's over 6 30 p.m now here in berkeley and myself and my colleague our senior customer success manager liam meyer are still still stuck here in downtown berkeley uh too bad because of government 19 he can't really get too close but anyways uh this one is on accuracy and i hope uh this is good enough so enjoy it accuracy is a very important topic we get asked about it all the time but the thing is too often the conversation quickly zoomed into only certain aspects of this issue and forgot all about the other factors that could be problematic and eventually could lead to disappointment and dissatisfaction in addition there might be just a bit too much overhype in the marketplace these days you sometimes hear some vendors telling you that any system can get just about any job done that is simply not true and cannot be true so i felt it was helpful to put out a basic introduction to explain as clearly as i can about this topic to show you what you are actually looking at and what you should be looking at and take into consideration especially now as we are seeing a growing number of new customers taking on lida for the first time so here we go to start we need to clarify a few terms or concepts some of these can be a bit confusing and from time to time almost used interchangeably first range accuracy is not overall system accuracy range accuracy refers to the robustness of the measurement by the laser sensor it is one important factor but not at all fully representative of the performance you should expect from integrated laser mapping system as a customer you should ask more about the overall system accuracy it is a systems overall performance that really matters to you and the overall system accuracy is determined by a lot of factors as you will see when we continue next relative accuracy versus absolute accuracy okay these two concepts do have different meanings in other industries in our context of geospatial science relative accuracy is the assessment of error measured between two points or features with respect to each other while absolute accuracy means the accuracy of the measured location or position of an object with respect to a reference space in other words relative accuracy tells you how close the measurements of an object are compared to its true dimension or size but absolute accuracy tells you how close the measured location or position of that object is compared to its true position in the defund spatial reference system let's say you mapped a house with the uav ladder system so if the data has high relative accuracy that means the house will look right in other words the sizes of the doors windows and things like that will be close to the two sizes of these features but that doesn't tell you anything about where that house is high absolute accuracy on the other hand means that the recorded coordinates of the house in the captured data will be close to its true location and based on those coordinates you should be able to find exactly where that house is again in that selected spatial reference system i mean basically that's that's how navigation works right vertical accuracy versus horizontal accuracy the names really tell you what they are and there's not much confusion there what's tricky here however goes to how accuracy is assessed for 3d point cloud from laser mapping unlike photogrammetry where you are working with a mathematically reconstructed author image and can relatively easily identify a point on the surface to compare against the control point with point cloud it's not so easy to select the point which is supposed to match a given ground truth if you think about it every single point in the real world will produce literally a blob of points in the 3d point cloud due to the inevitable uncertainty in each of the hundreds if not thousands of measurements it gets during any survey mission so selecting one single point to represent the true point can be subjective it's often necessary to generate a surface first and then calculate the distance between that surface and your control points to estimate the error values this is why for uav lidar mapping you would do a ground points classification first and then use the derived ground surface to evaluate against your survey control points which will give you the vertical error or elevation accuracy this can be done relatively easily with automated tools in many software however it may not be so easy in the horizontal dimension and almost always requires manually selecting the sample points you could make it simpler and less subjective by using specially designed survey target objects that said you should generally expect the vertical accuracy and the horizontal accuracy to be in the same range with somewhat more pronounced error in the horizontal direction lastly design accuracy versus mission accuracy well i kind of made up these two terms but the idea is simple design accuracy is a spect published by the manufacturer but it's important to recognize that this is a result you can expect if the system is used properly and under good operating environment it is not a guarantee that you will get that level of accuracy from every single mission mission accuracy refers to the data quality from a specific project you can also call it the project data accuracy again if all goes well and done right emissions data accuracy should be close to the published system design accuracy and possibly better but it may also be outside the range we will talk more about that later so what are the factors that will determine lidar systems design accuracy and the actual data accuracy you may get from a mapping mission here is overview i can think of well i can't possibly list everything and i'm sure not everything is covered but still just from this overview you can see that there are quite a lot in my opinion at high level at least four layers of factors in general together will determine the data quality when deploying a laser mapping solution these are hardware components product integration software algorithms and operation proficiency within each layer there are multiple factors to be considered and let's break it down and go through them one by one first hardware okay this is surely the topic everybody likes to talk about and sometimes it's the only thing that gets asked about to some degree that's a bit unfortunate that said when it comes to hardware it's obviously the big three sensors laser imu and gnss not much debate here about the importance of these components but what's worth mentioning here is that to those who like to drill deep into the technical specs it's important to understand that all the manufacturer published performance specs are the results achieved under certain testing conditions in other words you don't necessarily get that level of performance every time and all the time now please do look into the fine prints in their spec sheets footnotes if you are really interested and again this is why if somebody tells you that particular system can get the highest level of performance every time and all the time you probably should be worried there are other components in a complete hardware system besides the big three how good vr does make a difference so this is the first layer with hardware components setting a floor the next layer of impact comes naturally from how these components are put together like for any hardware product product design and product build are typically the two aspects to look at this is pretty straightforward and this is where the system integrators first begin to differentiate themselves from each other moving on mark and jason the co-founder of netscape and andreessen horowitz one of the biggest names in silicon valley vcs once famously said that software is eating the world yes having shining hardware is very cool and very important but with many technologies software often really is in their driver's seat software will significantly impact the overall performance both at the product level and the operation level you will need good firmware onboard the hardware to make sure all the pieces are working properly not only individually but also together collectively as a unified system that makes sure the system collects high quality raw data but then you will also need good software to process the raw data and produce the actual final products that you will deliver to your clients such as the contour tree count dbh or whatever believe it or not for the products we are talking about here it takes as much if not even more resources to develop the full software stack i'm not saying that software is the dominating factor in lidar mapping solutions but it is absolutely a very important factor that you should not overlook when making your purchase decisions now you have hardware the hardwares will build will design that you got the software ready to go but then you still need to get the job done here finally is operation this is where rubber really meets the road even the best hardware and software will not get the job done for you at least not yet you need to understand what a project requires and the environment where it takes place select the right equipment for the job device the proper mission plan and have the hands-on skills needed to execute that plan and then you'll also need the right tools and the skills to process the data and to take you across the finish line the environment will play a role to give you an example temperature will affect the performance of the laser and the gps signal strength will fluctuate planning will play a role is the base station placed at a safe and open location does the flight plan give you sufficient coverage but at the same time not forcing you to have to use the outer laser channels or larger scanning angles where the data quality is going to be lower execution is critical is the operator following the proper best practice protocols and using the equipment even correctly and lastly are the best possible parameters and configurations used in data processing i mean i tell you having spent almost 15 years in the field myself i know how critical operation is so never overlook the human factor and value your experienced operators because this will significantly affect what you will get in the end and bring it all back together as you can see many things will affect the actual data quality you will need not just a good product but actually a right solution that includes both hardware and software and it will also take the right skills to accomplish the mission successfully all things must be considered all these factors plus others that i didn't think of will together determine the actual data accuracy you will be achieving from any light on mapping mission so i want to quickly illustrate some of these things we just went through and let's take a look at this simple example so you are conducting a uv lighter topo mapping project assume at a given time t0 the system is in the air at point p0 it shoots a positive laser which hits the ground at point p2 note that laser mapping system reconstructs the 3d environment basically based on measured distance and recorded angle and then throw it on top of your gps trajectory so there is already a slight inaccuracy in the measured distance p0 to p2 due to the uncertainties in the laser measurement then let's say there's a also a slight offset angle of beta in the recorded imu data i'm simplifying a few steps here but anyways now the system will think that the scanning angle for this particular pulse is alpha instead of alpha plus beta therefore it thinks that the pulse must have come back from ground point p1 now guess what when it applies the distance p0 to p2 in the direction of p0 to p1 we got a problem you can see that right there you now have a vertical error of delta h and the horizontal error of delta d and this happens to every single pulse measurement but should be pretty small in addition from this diagram you should also be able to see that as the scanning angle and flight altitude increase the errors will also increase and that's why to achieve the best possible data quality you should consider to limit the mission agl and not to use all collected laser data from all channels and angles and then you put the raw data through processing i mean is gps data even good enough during the mission is the software used giving you the best trajectory ppk and pp pose result what about parameters for the ground points classification what resolution used for producing a dem is the data even transformed properly into the correct coordinate system using the correct datum both horizontal and vertical i mean all this will affect the final data quality hardware integration software and operation it's far more than just one thing and what you should really pay attention to are what overall performance the system can deliver and how you can get the best possible results well that's it i hope this was helpful and you find the information useful to you again this is only meant to be a basic introduction and not in-depth technical discussions so if you are interested in learning more or if you have any questions at all please feel free to drop me a line or just comment here um and we are always here to help so until next time stay safe and ciao

Channel: Leo Liu

Views: 1,651

Rating: 5 out of 5

Keywords: lidar, lidar mapping, lidar accuracy, point cloud accuracy, lidar mapping accuracy, uav lidar, drone lidar, mobile lidar mapping, liAir, LiAir V, LiAir 250, GreenValley International, GVI, Livox, Riegl, Velodyne, Hesai

Id: sDq0VqowsHw

Length: 20min 9sec (1209 seconds)

Published: Thu Aug 27 2020