ASM/REM 404 - Lab8: Multispectral Image Processing, Part 1

[Music] all right so in the previous labs we've been working with imagery that came from a dji mavic camera which is just rgb imagery and in this lab and the next one we're going to switch it up and look at multi-spectral data and how we can process multi-spectral data into orthomosaics and point clouds and elevation models with metashape as well so i have uh some imagery that we collected out of parker farm on the 5th of october using the mica sense red edge camera which is a 5-band multi-spectral camera and i've got it open here in windows explorer and so it's in this uh folder here called you know zero zero zero three set and if i open it up there's a couple of folders within that the way the micasense camera works is it will collect a whole bunch of photos into uh this first directory until it reaches a certain number and then it'll create another directory and then add photos into that right so you could have multiple photos here just depending on how many uh you know total exposures that you had in your in your mission but let's go in here and uh just open this up and take a look at what's going on here so notice that all these photos are in black and white and that's because the mica sense camera actually consists of five separate uh sensors five separate lenses on that camera and each one samples a different portion of the electromagnetic electromagnetic spectrum and so each one is black and white and then we'll use metashape to kind of assemble those together and so if i look at this photo here notice the this is image uh four underscore one so this is the first image uh here and then all the way up to the fifth image so all five of these together are one click of the shutter button in the micasense camera and so let me open this up here just so we can look at it in the uh in the photo viewer and uh you know you can see some nice patterns here so so image one corresponds to the blue band and as i click the next arrow here that's going to take me to the next one which is going to be green and then red and then red edge and then finally near infrared okay so um that's how these things are are sort of ordered but notice how if i go back and forth these images are moving around okay and that's because each one of these has a different position on the camera and so the principal point of the photos is slightly different for each one that can pose a real challenge if you were just trying to smash these things together and merge them but using meta shape we can actually use that in the bundle block adjustment phase it's going to sort of line all these photos up for us which is also really a handy thing to to do so let me close out of here we'll minimize that and so i've got a new project in meta shape first thing we want to do is add our photos here so we're going to use add folder that's going to be the easiest way to do it and then let's see i want the one from the 5th of october and i can just select that folder here and it's going to read all the images that are in that folder okay so now this is a new option for us so um because each of those uh bands from a single photo is a separate file then we need to tell it that it's a multi-camera system all right so so there are 294 total exposures but the total number of images that we have is five times that so we have about 1500 images that represent 294 clicks of the of the shutter button on the camera okay so i'm going to go ahead and hit ok here it's going to load those things in which might take it a second to do all right now i got this thing that looks like a warning message but this is actually a good thing so um the mica sense camera the micasense sensor itself is a calibrated multi-spectral sensor and we have these uh images of the calibration panel that will allow us to convert the values that we get in the images to surface reflectance and micasense is just telling us hey i found some photos here that correspond to a calibration plate and i disabled those so they don't mess with your uh photo alignment and i move them into a separate folder so that's actually a good thing that's what we want okay now let's uh kind of take a look here at what we got so we've got a whole bunch of photos here and this is actually data from a couple of different uh things that we were doing during that flight so there's some images here that we don't need and so let's kind of spin this up and look at it from the side here and you can see that i've got a whole set of images at one altitude i've got a smaller set from a higher altitude and then i've got some images that it looks like it took kind of on its way up or way down and then these guys here they're light blue those are our reference panel images that it's already turned off okay so what i want to do now at this point is uh i want to select and disable sort of all of these upper elevation images and then all of these guys in here and i can just do that with the with the select tool here okay and then once i've got them selected i can come down here and find them and then just disable these cameras okay so now you can see they're all light blue same thing in here i just want to turn these guys off because we're not going to end up using these images we just want this set in the middle here okay all right so now at this point we're getting close to being able to do our image alignment but we want to look at a couple of things first so up here in the tools menu i'm going to choose camera calibration first and i want to kind of note a couple of things so first of all there are five separate sensors that make up this camera this is different than what you'd see if you had like dji imagery which would just be one right this is a frame camera so it does not have a rolling shutter so we we want to make sure that that is not uh checked where there's no rolling shutter compensation to use okay and we have parameters for all of these uh attributes of the lens model and that's because the micasense camera is a calibrated camera so they've actually measured what these things are for the lens that's in the camera and if i click on each one of these notice that the parameters change because each one of them is in a different position on the on the physical camera itself okay so that's sort of first thing to note right second thing is if i click over here to the bands then that gives me some sort of information about like like what are the different band values that we have and then there's this normalized band sensitivity option we're going to leave that alone for now but we'll come back to that and what that's going to do is uh allow mica sense or not mice and allow meta shape to adjust for any color differences between these images at least to the best that it that it can all right so we're not going to really change anything in here or mess with this too much i just didn't want to point this uh this out all right so go ahead and click out of that and then the one thing we do need to do though to start with is bring in this reflectance panel information so that we can do the conversion to reflectance values okay before i do that i did want to point out and just remember this that within each of these images okay it's only showing us the first image of the set of five okay so this is the blue band values or the blue band images for uh for for each one of these okay and it's doing that just so you don't have to look at like five duplicate uh ones okay uh you can change which one that you want it to display or use by coming up here and in the tools menu and selecting set primary channel for most of the time it's it's fine to just leave it uh like it is okay at least i find it is all right so now let's go up to tools and we're gonna go calibrate reflectance and this is going to open up a a box here where um it listed our images that it found reference panels in and then it's going to give us a chance to sort of load the reference data for that for that panel so this is a little bit confusing there is a set of instructions that metashape that adjasoff publishes on how to sort of work with micasense imagery and metashape and this part is not very clear in the uh in the instructions but so metashape found that these two images have reflectance panels in them but it has not actually like located the panel in the image and so we need to click this button here and it's going to take a second and actually like scan through these images and find that the panel in there and then each the panel has a qr code and and that qr code is the unique identity of that specific panel okay so every panel that that they produce that microsense produces gets a unique id number and a unique qr code and then they do a wavelength calibration on that panel and so we will we have a file here that we'll upload in a second that will give it the uh or or tell mica sense tell mike sense tell meta shape the uh actual reflectance values to use for uh for calibrating the the bands okay okay so we can see here that it actually found the the the sort of panel the unique panel identifier uh here over here on the side then we need to uh tell it sort of or give it the official calibration file for that panel and i've already sort of loaded it in that's why it's showing up here but if you haven't loaded it in before then you just sort of click the open button there and then it's this csv file that i'll make available to everybody for the for the lab but that's the one that you want you just click that and open it and you'll be able to select it here and hit ok alright and now it's going to supply the actual reflectance values right the other thing that the micasense camera has is a downwelling light sensor that measures the incoming radiation to help it with that reflectance calibration so we need to make sure that that option is turned on and then we can go ahead and hit ok and it's going to calibrate all of these images for us and you'll notice that it just turned all of them black here down in the photo tray they're not actually black it just sort of changed the brightness values of them okay so we can we can fix that here by coming to tools and set brightness and i think we're going to want sort of a brightness value of something around 600 let's see what that does yeah we could probably go a little bit brighter even let's see 700. this is just really like for for visual purposes right and and you know kind of working with these images this doesn't actually affect any of the products that you'll get in the output so we'll go ahead and hit ok there that's good enough for now so at this point we're ready to to work the standard process for aligning the photos and creating the products throughout the structure from motion workflow i'll point out a couple of things here so standard options here for our general alignment uh we can turn the adaptive camera model fitting off because we actually already have really good uh values for those uh for the lens model okay and then we'll hit okay here and uh let that go i'll pause while this runs uh it should go pretty fast but uh we'll pick up when it when it's done here and uh and then look at the optimization steps okay so we're done with our photo alignment and looks like everything aligned pretty well we've got a nice uh looking model here to start with and that we're ready to start our optimization of this the one thing that i wanted to point out that i've noticed is that uh with the mica synth camera it's easy to uh over optimize this model and so if you get too crazy with the optimization steps then when you go to build your dense point cloud then it you end up with some kind of weird results it'll it'll not reconstruct certain areas or it'll it'll uh have kind of strange artifacts so when you optimize you know it's a it's a fairly kind of uh uh gentle optimization if we want to call it that so uh but but standard process here um for the for the optimization um just like we've done with the other image sets so i will just kind of put instructions for that those optimization steps in the instructions for the lab and and we can go from there i won't sort of do all of that uh in the in the video here and once you're done with the optimization and building the dense cloud building the elevation model uh all those things are you know just like with the other image sets that we've worked with and so i will pause here and we'll pick it back up when we create the ortho mosaic all right i've gone as far as creating my dem uh from the the multi-spectral imagery here and we're ready to do our ortho but there's one couple of things that we want to do first i'm going to first come up to tools and camera calibration and uh just show you or point out that if i look at the band sort of tab for each of the different sensors then the normalized band sensitivity option is now turned on the reason that's turned on is that we've calibrated the images and so metashape is now expecting to use that calibration and adjust for the uh for the differences in exposure of each of the images okay which will just give us a better looking ortho mosaic at the end all right so when we come to uh create our ortho mosaic a couple of things to note um you know we're doing it from the dem we've got about a four centimeter pixel size here for the blending mode i found the best results by using an average uh rather than the uh than the mosaic um but you know you can play around with that and see which one you like the best otherwise it's pretty straightforward uh to to do this and so once we get this ortho mosaic here i'll pause and come back and show it to you all right so here's our ortho mosaic for the multi-spectral imagery it looks pretty good i mean it's not the best time of year we got a lot of shadows and stuff but but in general it uh it stitched together pretty nice and it's going to be a good product for us to work with in the next lab coming up so that's it for this lab what i would like you to do is go ahead and you know go to export and generate a report for this we'll have you submit that report for the lab write-up and then next time for the part two of this we're gonna pick up at this point and uh look at how we can do some raster transformations to calculate some indices from this multi-spectral data and then we'll we'll actually export it from uh meta shape and look at it in in qgis or arcgis and what we can do with the imagery there so that's it for this time and we'll see you uh with the next lab [Music]

Channel: Jason Karl

Views: 636

Keywords: drone, UAS, drone remote sensing, Agisoft, Metashape, Micasense, RedEdge-M, photogrammetry, multispectral, structure from motion, point cloud, DEM, orthomosaic

Id: i3WB5TrkRy8

Length: 18min 5sec (1085 seconds)

Published: Fri Oct 23 2020