Tutorial : Masking for Photogrammetry | CLICK 3D EP 12 | 3D Forensics CSI

everybody it's eugene lisho here welcome  to click 3d this is the program where we   talk about photogrammetry and how you can use  a digital camera and some software to make some   really compelling 3d models today i thought what  we would try is a model of a trolley here and   this is a little bit of a complex shape  only because there's some really fine   details in here but what i really want to do  today is not so much focus on the trolley as i   do want to show you a technique called masking and  masking is really useful especially when you have   objects that you want to ignore in your photos so  there's a few examples of this one would be if i'm   going to be taking pictures of this trolley on a  turntable i don't want to capture the background i   don't want to capture the turntable or things that  are off way off in the background and so masking   will help us do that another reason would be let's  say for example you're using a drone and you're   flying on a roadway and maybe there's people and  vehicles which are going to be moving around well   in that case you want to get rid of the people and  the cars so masking will help you to do that and   there's different techniques when using masking  some are manual so you can do it externally in   another program and but a lot of photogrammetry  software will have a masking feature included   in their software and so there's manual techniques  there's semi-automated and automated techniques   so i'll show you some of those today not all but  let's first talk about uh the subject that's the   first thing i do whenever i begin a photogrammetry  project and so this is small it's in my control   meaning that i can hold it i can pick it up i  can easily move it around now today i want to   be able to capture the whole thing so i want to  create a 3d model of the top and of the bottom   here okay so it's not just i can put this on a  turntable and take photos going all the way around   i actually are i'm gonna need to flip this somehow  either upside down or sideways a couple of ways   and then capture the bottom and when i do that  i'll be able to reconstruct the entire model now   uh the other thing is that as i said before this  has got some really fine details on it so i may   need to take some additional photographs i'm just  going to try it with you know sort of my first set   of photographs and see what i get and if i need  to i may go back and then take some additional   photographs i'm not really interested so much  in the model as i am in the masking and that's   something that i will be showing you otherwise  it's a pretty good texture it's not too shiny   it is you know i can see that there are some shiny  things here in the lights that are showing up   but you know this particular model is not too bad  it's fairly flat um it's made out of some kind of   metal and some plastic as well but you know quite  quite a decent model to work with an easy one   to work with so that's what we're going to start  with and i've got a bunch of lights that are   around me and i'm just trying to sort of cut some  of the the shadows and make sure that it's well   lit so i've got one two three i've got four lights  that are just kind of spread out these are just   led lights you can pick up at the local um you  know hardware store or whatever nothing too too   expensive actually two of the lights are uh leds  that i just picked up on amazon they're like 50   watt outdoor lights but they have a nice bright  white light and because i can move them around and   i can do some different things there you know i  can help to avoid any strong shadows on this thing   i also have some overhead lights and some track  lighting here so i can you know point lights if   i need to but i'm not going to get too too fussy  with the lighting today like i said the purpose is   going to be masking so let's talk about my setup  and the turntable and then taking photographs   so when thinking about what i'm going to do  with this part there's one thing about it it's   longer than it is wide and so what that means  is i'm planning to use a turntable now it's not   absolutely necessary because i could just put this  on a table here and just kind of move it by hand   but a turntable will make it a little bit easier  in some cases so i'll try the turntable with this   particular model now one thing that i always try  to do whenever i photograph an object is i try   to get close and i've seen a lot of people  when they'll take photographs of something   it's like they're really far back so the object  makes up a very very small piece of the overall   image and there's no reason to do that in fact  i'll often get so close i'll try to cut off parts   of the image it it doesn't cause a problem so long  as that you have enough overlap between the images   so i may leave a little bit more background  only because when i start to rotate this   part of this is going to you know be outside  the camera view because the camera view is going   to be wider than it is tall let's say so i can  even move the camera too so if i'm on the side   i can take photographs this way or i can just  rotate until this point and the other thing that a   lot of people don't do or they forget is that you  can rotate the camera 90 degrees so if the object   fits better in portrait versus landscape mode go  ahead and just rotate the camera photogrammetry   software doesn't care it just cares that you have  a good photograph of that particular image so   when i'm rotating around if i'm going to have  the camera kind of pointed this way then i'll   use sort of like a landscape kind of this way  and then otherwise when i get to this particular   direction or that perspective then what i'm  going to do is i may rotate the camera up   so it's going to take quite a bit of work to  manipulate the camera and that sort of thing   but that's okay i have a bit of time and it's not  a big deal now in terms of a turntable i have this   one right here and this is one i just picked up  on amazon and you'll notice that one side is black   and the other side is white now i actually painted  this so that i could have the option so if i   have enough i don't know a part or whatever  that's dark and i want a white background   then i can just turn it on this side otherwise  i'll just flip it over and i can use this side   now i haven't really decided which side would  be better so i have to think about that if i put   this on this side on the white side that might  work because there isn't a lot of really really   white uh paint or white texture on this particular  model so that could potentially work but i'm   thinking black could be okay too it's just maybe  where these fine little parts are up on the top   that could potentially cause a small issue i don't  know so i think i'll start with the white and i'll   see how that goes and then from there you know  we'll see how that progresses i can always come   back and just take more photos the other way so  let's start taking photographs and set this all up okay so we're going to be taking photographs  now and so a couple of things i need to consider   one is the camera one is the object  and the setup and everything else   so first thing you know i talked about  some of the complexities with doing this   i've thought about it and i think all i'm going  to do because i'm just really going to be focusing   on the masking part today is i'll take a round  of shots from the top here on the turntable and   then i'm going to flip it upside down and then  i'll take a round of shots from the bottom and   that should pretty much do it i shouldn't need to  actually clip it on its side or that sort of thing   and so that'll just simplify things and keep  it most efficient with respect to the camera   i've talked about some of the camera settings uh  iso i'm going to keep down very low but iso 100   and the f stop number i'm going to raise high so  i'm going to have f 22 or f 25 and by doing that   i'm going to basically let in very little light  okay so the orifice or the opening is going to   be small so as a result i'm going to need a  longer shutter speed so right now my shutter   speed is about two seconds or so i'm going to play  with it a bit just to make sure that it's okay   i don't want it too bright or too dark kind of in  between where i can pick up all the details quite   nicely and like i said i'll start from here and  i'll just rotate this in about 10 or 15 degree   increments all the way around making sure that  i get good overlap and once i do that i'll take   the model i'll flip it and then run it again so  i'm gonna get started taking photos here let's go   all right let's do a quick test here just going to  make sure that this is in view as it goes around   it's not too bad but i need to get uh either a  little bit closer a little bit higher like that   and that does most of what i need around the top  okay good and then i can just flip it now i will   be using a remote and this is really helpful  so i avoid any shaking or anything like that   if i was to use this by hand i'm going to  have to change my settings so i'm going to   need a shutter speed of at least 1 80th of a  second to ensure that they're nice and crisp   and so that's one of the limitations of um you  know holding things by hand you can't have an   extended shutter now just looking at this so my  settings right now it looks like i'm at about uh   well a third of a second maybe and f 25 it looks  like it's right on in terms of exposure maybe a   little bit underexposed so what i'm going to do is  i'm just going to take a practice shot but first   i'm going to enable my my remote here so give me  a second make sure that's on so i can do this and   almost there remote control there we go and i'm  going to have a quick mode here so let's see what   this looks like yeah so this doesn't look too  bad uh in the viewport there i can see around   the lcd it looks pretty good so i'm going to take  that first one and move it move it a few degrees   take another picture and like i said i'm not  going to trouble myself too much with this model   in terms of all the fine details i  really want to focus in on the masking and you know by doing just the top and the bottom  i'm really doing this in the most efficient manner   possible can you add more photos later absolutely  can you take more photos now absolutely if you   really really want to you know beat this thing  up into the ground you can just take all kinds   of photos getting close to all the little details  i just want to create a nice overall model um   just so that i can show you how we remove  the background okay that's done there so now   what i'm gonna do is i'm just gonna flip this  upside down like that let's see if it'll stay   yeah something might lean up yeah it'll lean  over a little bit let's see what that looks   like and let me rotate it around yeah i pretty  much get everything there it should be okay so   i might have to take a few other overalls after  but let's go from here and see what we get okay uh probably what i'll do is just take  a few extra little photos just in case but   that's pretty much it um in fact i'll probably  just process these and see what they look like   and so let's go up to the computer  uh we're to just do a quick check   on the images make sure they all  sort of come together pretty well   or at least i have enough images and then  what i'm going to do is i'll show you how the   masking works and then i'll just continue on  and create a final model so let's head over okay so we're at the computer now and  what i want to do before i do anything   is just check the images to make sure that  they are somewhat okay so i'm looking here   you know not too bad and they look you know you  can see the turntable you can see i cut off part   of it part of it will come back when we flip and  here we go so we flip this so you'll see that   other stuff will come into view and so we should  be able to reconstruct this so not too bad i am   okay with the images there the other thing that  i'm gonna do just to try is i'm gonna bring these   in and i'm just gonna do the sparse reconstruction  just to make sure that you know they're gonna   connect it even like this now there's going  to be some issues with these particular   types of photos when the back there's a lot of  background showing and that sort of thing so what   i'm going to do in this case is i'm going to make  sure that i choose close range but i'm going to   also choose deep for the preset normally i would  just use default if i wasn't using a turntable and   i didn't have a lot of background i would just  choose the default i'm just going to check and   i'm going to go next and i'm going to go run and  what this is going to do is just see if i can get   most of the photos oriented because if there's a  problem i don't want to go through all the trouble   of masking and then go back and have to retake  all the photos and mask again so this will tell me   something about the images that i have and since  i flipped the model on the turntable what i'm   going to expect to see is part of the turntable  being reconstructed so there'll be like a flat   ground or flat bottom set of points underneath  the trolley and then when i flipped it upside   down there was the turntable which is really on  the top now so i should see two little kind of   sets of points or planes which are on the top and  on the bottom of the trolley if i get that and if   most of the photos reconstruct then i'm in good  shape and i'll jump into the masking and then   kind of go from there okay so right off the bat  i can see i got a bunch of photos that uh were   correctly oriented that's great and if i zoom  out you'll see i've got these rings that i did   and look at the model this is interesting you'll  see that i have you can just make out the wheels   here i've got the base of the uh the turntable  but i also have it on the top here and because the   you know again i go back upside down right when  it was upside down it was kind of sitting on an   angle so you can see how the turntable is being  reconstructed here as well so that's a good sign   that's a good start that means that i've got  something that i can use now i'm not going to   save this because i know that it worked here so  what i'm going to do is just restart a new project   and then start the masking and i'll show  you some different things there so let's   switch over so when it comes to masking there  are three basic ways that you can do this and   they're um basically down at the bottom here but  one is just manually okay so you can go through   the whole thing um and uh just you know on this  image here there's a couple of brushes the first   one is i use red to tell me or tell the software  what i want to keep and then i choose blue to tell   it what i want to get rid of so i want to get rid  of the background in this case and what it'll do   is it'll automatically compute and if i look at  this black and white mask you'll see white is   what it keeps black is what it's going to get rid  of and there's little things here that i need to   keep oop that's the wrong one i want to go back to  red i want to keep that i want to keep this i want   to keep the step back here i want to keep this  headlight at the front so little by little you   can start to tweak this and you know keep certain  things and get rid of other things so you can see   the little space in here i can get rid of now i'm  just holding down the shift key so when the red   brush is enabled then if i hit shift it actually  turns to blue so it's a shortcut key all these   little gaps i could clean up of course this is  time consuming right this is going to take quite   a bit of time so i'd have to go through each image  little by little and then kind of go from there   so that's not something that i probably want to  do now there is this turntable mode so if i tell   it hey this is gonna this is on a turntable and  i'm slowly turning this around what i can do is   have it automatically compute and i can do this  with or without the turntable enabled so but if   i hit this automatic compute i'm going to do that  now what it's going to do is use the first mask   that i did and you'll see that as it's running  through here it's creating all the masks for the   subsequent images so that's really helpful  because it does the bulk of the work for me   now it's not perfect it's not going to isolate  every little single feature and in some cases   i may have to tweak it but if depending on  your setup this may work really really well   now you can also create masks externally so  you can do this in photoshop create a black   and white image and basically use that as your  mask if you want i'm not going to be showing that   there's even another feature here at the bottom  that is a color picker basically so you can just   select this button choose your background and  then you know once it does that you can just tell   it to go ahead and automatically get rid of all  the white or lighter portions of the background   so there's a few options here manual automated or  semi-automated tools and then even external like   with photoshop or other image editing programs  so let's just process this and when i'm done i'm   just going to close this i had all the masks  i refresh and you'll see that it loads all of   the masks here no problem i'm going to go next and  next again close range deep i'm going to keep that   on i'm going to go next and i'm going to go run  and what i'm going to do is pause it here once   it's done i'm going to come back and i'll show  you what it's got for the sparse reconstruction   all right so we're back and if i scroll through  all the images it looks like they've all been   reconstructed now so i'll go finish and you look  at my model i have much much less of the um you   know the background and such it's just gotten rid  of it all so that's not too bad so what i'm going   to do is i'm just going to continue on and i'm  going to create a deep or a dense point cloud   from this particular model i'm just going to  go close range in default now that it's all   reconstructed and when that's complete  again i'll come back and then i'll let you   see what that looks like um but i think after that  what i'm going to do is i'm going to go back down   shoot another round of images with the black part  of the turntable and see how that might help us   all right so the dense point cloud doesn't  look too bad if you look at it here it's fairly   complete if you look on the bottom okay we've got  the bottom pretty much set up and let me come back   here what happened there we go but overall you  know we got a half decent model there so not too   bad yeah there's a lot of things we could probably  improve but i'm just going to go ahead and just   make a very quick mesh just to see what the mesh  is going to look like if it's going to be anything   decent i'll go to default go next and run and  when that's done we'll have a quick look and   like i said after this point i think i'm going  to go back and shoot another set with the black   base and see if the contrast is going to help us  create a really nice uh or a better automated mask all right so the trolley got reconstructed here  you can see that it's it's not too bad there's   some things that i could do just right off  the bat to improve here but i'm okay with this   in terms of the masking that was really what  i was after anyway so you can see the model i   haven't textured this yet so there are there's  some things i can do to improve the colors and   even the mesh there's probably some things i  can do with more photos and that sort of thing   but i think you get the idea here and let's go  back i'm going to shoot those other photos with   the black part of the turntable and then i want  to come back and reprocess and see what i've got   okay so we're back again and i'm going to try  this one more time with a black the black side   of the turntable and a black background i  didn't have anything really black like a big   black sheet or whatever so i just had this shirt  and i just tucked it up close to the turntable   and i just took the camera bag behind it to kind  of raise it up a bit so now when i take my photos   i basically see just all black in the  background and that's going to help   now i did notice right off the bat before i went  ahead everything which was white um you know my   my exposure was different and so i'm going  to have to adjust that because right now um   in order to get this properly exposed uh i've  got about uh oh boy looks like about a second   maybe a little bit less so i'm just going to  make some adjustments here and i'll check this   out i am going to use the remote again and  let me just make sure that i can turn this   and it's going to get me what i want looks like  it so let's try another set of photos like this   yeah it looks pretty good i might even just  back up a tiny bit and i'm going to go up a   tiny bit higher by pointing this up like that  and this way i'll definitely get a little bit   of background and i should be able to separate  this quite well so let's see how this goes okay plenty of photos there  so let's flip this over   and this was leaning a little bit so just  stabilize that let's see what that looks like   yeah i'm just gonna spin it around see what i  get i think that looks okay so let's give it a go okay well that should about do it  so i did two rounds one with it   upright one with it upside down  let's go see what it looks like okay so these are the new images that i have and  i'm going to be masking them and trying this again   so i'm going to go next and i'm going to go  next again and i've got to go to masquerade so   it doesn't look too bad so that could potentially  work for me so i think i'm going to try turntable   mode i'm going to tell it what i want to keep  by doing oh i went out there so i'm going to   undo that just by hitting ctrl z and i'm going to  hit all this like that and maybe what i will do   is hit some of this and then i'm gonna do the  background which is more or less black like that   let's see what i get here so let me check the  mask that's not too bad so that to me is a big   improvement on what we had before because a lot  of the bottom here is uh has been you know sort   of gotten rid of now there is some stuff up here  i'm going to want to keep and there's some stuff   i'm gonna want to get rid of now it may not have  the intelligence to do all of these little spots   but that's okay uh that's not a problem for me  because it may not reconstruct all the little in   between spaces here the majority of it it's picked  up let me see if it's gotten this piece down here   yeah it looks like it's okay so i'm going to keep  this i have turntable mode enabled and i'm just   going to go ahead and say automatic computation  and it's going to do the rest of them for me and   as it's doing this i'll be able to get just  very briefly a snapshot of uh you know where   it's masking and where it isn't masking and when  it flashes i can kind of see what it's doing it   doesn't look too bad actually looks like it's  um behaving kind of properly but once this is   all done i'm gonna come back and review this all  and see if it did what i wanted okay so it's gone   through all the images and what i'm gonna do is  i'm just gonna have a peek at some of these oops   and see how it looks oops let me see here okay so  this one here if i look at that okay so not too   bad it got rid of most of the background which  is fine if i go back here yeah i'll save that   and let's have a look at this not too bad it  didn't get the step but most of the background is   gone it did get this back step so what i can do is  i'll go through some of these and just have a look   and make sure that you know i'm not missing out on  some of the finer details but most of the work is   done for me and then i'm going to go ahead and i'm  going to proceed to turn this into a nice 3d model   all right so all my masks are done and i'm just  going to go ahead and start processing this   normally i may do deep mode here  just to make sure i get everything   i don't mind taking the extra time  just to make sure that everything gets   reconstructed properly so let's let this cook a  little bit and then we'll come back and we'll see   how the model looks like or at least the sparse  point cloud looks like okay so this just finished   processing and all of the image is here all the  yeah all the images look like they reconstructed   and so it really helps to remove the background  and i found that if i just let it go and with the   background and without the background removing the  background seems to help a bit so you can see all   the images on the top and on the bottom very  very well organized you know small increments   and equidistant from the model just a few straight  points that are here not too bad what i'm going to   do is just go ahead and reconstruct the dense  model and see what that looks like so let's go   ahead and let's do that i'm going to just do close  range i'm just going to take the default settings   and see what that looks like and we'll come back  in a little bit again and see what the dense   point cloud looks like all right so the dense  point cloud is done and it doesn't look too bad   just looking around here yeah the top looks a  little uh different there i have to look at that   a bit more closely but the trolley the base and  that sort of thing doesn't appear too too bad and   okay that's all right it's not as bad as i  thought what i'm going to do is i'll create   a mesh and i'll just continue from there just a  rough mesh to see what it looks like and i'll just   stop there i'm not really interested in going more  the real purpose of this was to be able to use the   masking to get a full 360 model and to show  you that you know it's easy to do now in   the other thing i should mention in this case was  i masked all of the images but you don't have to   so if some of the images are just fine on their  own and you're finding that there's issues with   you know you're doing an outdoor scene and people  are walking in front and they're caught in the   image or something like that you can easily just  mask them out one two three four images and those   portions of the images that you've masked will  not get reconstructed so that's a useful tip in   case uh or for example if you're texturing in  some of the you know people walking or whatever   end up in the textures you can obviously remove  them there as well so yeah there's some good good   things you can do with masking and once this is  done i'll give you a peek at what it looks like   so here's the mesh model and again more that i  could do here for cleanup for accuracy and that   sort of thing but i think you get the idea you  know the cameras nice and organized using the   turntable using the masking and finally getting a  full 3d model top and bottom all complete that's   really what's important here alright folks thank  you very much and see you next time on click 3d

Channel: 3D Forensics

Views: 18,483

Keywords: photogrammetry, 3d scanning, 3d scanner, reality capture, laser scanning, 3d point cloud data, 3d scan point cloud, land surveying, forensic evidence, 3d model, 3d modelling, Vr, virtual reality, forensic scientist, crime investigation, accident reconstruction, zephyr, photomodeler, 3dsmax, cloud compare, archaeology, Autodesk ReCap Pro, Agisoft Metashape, Meshroom, Pix4D, 3DF Zephyr, Regard3D, Drone, aerial data, RealityCapture, point clouds, textured models, OpenDroneMap, 3DFlow

Id: v-1vuFUDE0o

Length: 29min 24sec (1764 seconds)

Published: Tue Mar 02 2021