How to 3D Scan All Sides of Objects - RealityCapture

so if you've ever tried photogrammetry or 3d scanning at some point and you were frustrated at the inability to capture the bottom side of your objects or get a closed mesh you're not alone there are lots of clever ways to approach this by placing your object on something thin or suspending the object by wire but this isn't always possible depending on the shape and weight of the object you're trying to scan doing it this way is risky it can make your object move which will negatively impact the quality of your scan and it requires a bit of mesh and texture cleanup in post it's not very convenient now fortunately there is a simpler way to do this that does not involve using sketchy rigs to hold your object upright now full disclaimer this video is sponsored by capturing reality and we'll be using reality capture software to generate our 3d model so let me show you how to get a clean closed mesh and how to scan the bottom side of any object you want now first off as always the basics of photogrammetry apply you can watch my intro to photogrammetry video right here i'll put the video link down below the like button so let's grab the object that we want to scan and set it down on a rock solid surface i do want to take a moment to talk about what kind of objects you can scan using the method i'm about to show you it should go without saying that you have to pick an object that does not deform when you move it it needs to be an object you can grab rotate flip onto its side that won't bend or move or change shape a solid object if you try to scan something like this or anything soft that does deform this method i'm about to show you will not work the alignment process is going to fail you're going to end up with a complete mess with that said ensure flat even lighting overcast days are perfect or shoot in the shade always shoot raw photos and if you're using a phone do consider an app that will allow you to manually choose as many settings as possible with these settings in mind let's start taking photos of our subject and we want to get photos from every possible angle of our model here at various levels of elevation so take photos around your object like this while following the rough shape of it the more photos you get of any single point the better your alignment process will be and as a result the cleaner the model as always make sure you have enough overlap between photos to ensure proper alignment later 50 to 60 overlap is a good starting point if you're in doubt it's better to err on the side of shooting too many photos than you think you need without enough overlap you're going to have trouble aligning the top and bottom half of your object later as you're going to see toward the end of the video so stick around to see that with your first loop done increase your elevation just a little bit while maintaining the same distance from the object and perform yet another loop around it repeat this process until you're almost shooting straight down you might even think you're taking too many photos but again it's way better to have too many than not enough you can always discard some later but it's not always possible to go reshoot not to mention it is very inconvenient so with the top half of our object here scanned let's flip it upside down and repeat the same process as earlier again be sure not to deform your object at all when flipping it because bad things will happen once you've taken photos from both sides we're now going to have two sets of images one for the top half and one for the bottom half usually i like to take one dark frame between image sets so i can easily find which set is which later with all of our photos taken here the hard part is done seriously the rest is easy next let's bring our photos into adobe lightroom i'm using a lightroom but you can use any other photo editing package that you want lightroom is just really great for batch editing photos and we're going to be using lightroom to delight them a bit so this is going to help the texturing process again you'll see me go through this in depth in my intro to reality capture video right here link below start by editing one of the photos i'm simply going to lift the shadows lower the highlights and create a flatter image adjust the exposure a tiny little bit and make sure your white balance is okay and because we didn't change our camera settings during the shoot we can sync this one photo correction to all of our photos and everything should look the same by clicking on the sync settings here click ok and now you'll see our edits have been applied to all of our images now here's the important part we're going to export them in two batches in separate folders i'm going to select the images for the top half of my object here and export that into a folder named top by clicking on the export button right here and selecting my destination folder we can export these to jpegs for now the reason we wanted to shoot in raw was to give us the latitude to edit and turn down those highlights and lift those shadows later so exporting as jpeg is fine then i'm going to do the same for the bottom half by selecting the photos of the bottom part of this log and exporting those into the bottom folder now let's open up reality capture and we can drag and drop the entire top folder that we exported from lightroom directly into our viewport of reality capture let's hit the f6 key to align our images and after waiting a bit you're going to get your point cloud right here and it looks great now let's just set our reconstruction region by clicking right up here and click on set reconstruction region the reason we do this is the reality capture will only process the area that we want a handy tip here is to use the numpad keys to switch to orthographic views which makes lining this up a lot easier you can press 0 on the numpad to revert back to perspective view and now we are ready to generate the mesh by clicking on the mesh model tab we can simply start by creating a preview mesh as this meshing process is not really important for now and you're going to see why real soon normal and high mode are going to take way longer so preview is fine for now once the mesh is generated you'll see we only have the top part of this log so now we need to generate what we call object mask to combine the top and bottom part of our object here so let's start off by cutting out any part that is not part of the object we're trying to scan and in this case is the support beam that we have the log resting on right here so by going to the tools tab cut by box and then clicking on set reconstruction region i'm going to go to the orthographic side view and i'm going to chop out the bottom part here clipping out any of the support beams and for the record this does not have to be perfect don't worry then we can click on cut outer to remove everything that is outside our reconstruction region from our scan perfect with all the unwanted triangles gone now we can generate our masks again in the tools tab click on depth and mask and set where you want your mask to be exported to we need to navigate to the exact same top folder where these images are found in the dialog box that shows up here the only thing to change is disable the export camera depth set this to no and that's it now just a heads up for full transparency if you're using reality captures ppi setup you'll need to use some ppi credits when exporting the mask just like when you use credits to pay for exporting a model out of reality capture however once these images that you're using are licensed so to speak you will not need to pay for them again later when you export your model it's a bit of a bummer because i would love to have the option to at least see my final model combined and all before having to pay but that's the way it is for now and i believe the devs are working on a solution and a compromise here so with that done let's go open the top folder here and you're gonna see we have perfect mask generated for every single image this is absolutely awesome because manually painting masks for you know two or three hundred images is not fun so this mask is going to tell rc to only use the parts of the image that are white it eliminates the background and allows us to merge the top and bottom halves together with the top half done now we can process the bottom half of this log in reality capture let's select all of our images by pressing ctrl a and you'll see all the cameras are selected now we can press ctrl r to disable them from further processing you'll see they change color here it's a handy visual indication that they are disabled now let's import the images from the bottom half of the log by drag and dropping the bottom folder directly into our viewport and you'll see here we now have 295 images up from 167. press f6 again to align these and now that these new images are aligned properly you'll see we now have two components which isn't quite what we want to keep it tidy i'm going to rename this component to top and rename the new component here to bottom now we know which one is which now again we can set our reconstruction region and then generate our mesh just like before using the preview mode to quickly generate a mesh with that mesh completed let's use the cut by box mode again in the tools tab and cut outer again this is going to remove the support beams which we don't want same as before next again let's generate the mask for this component by going to tool depth and mask again again you need to export these masks to the bottom folder respectively and rc will remember the setting that we used earlier so we don't need to do anything else now we have two batches of masks generated previously in older versions of reality capture we needed to have two separate projects to combine the two halves but not anymore now we can simply drag and drop both the top and bottom folders of our images here directly into the viewport but you'll see the total amount of images did not change it's still 295 but if we click on any of the images here and go to the image 2d view tab and then click on mask you'll see every single image has its mask loaded in automatically for us it's magical so by changing my layout back to the oneplus one mode i'm going to select all of my images with ctrl a again and press ctrl r again to enable everything and now with that done we can cross our fingers press f6 to align everything again and if everything turned out great you should get one perfect model with all of the cameras aligned perfectly you get a fully closed mesh everyone is happy with both sides matching at one whole complete closed object but sometimes a lot of times you'll get two or more new components the two sets didn't align perfectly and this is usually caused by a few different things either you didn't take enough photos the space you moved between photos was too large some photos were blurry noisy or just too dark getting this to align perfectly takes practice but don't worry we can still fix that first obviously you can go reshoot and take more or better photos but reshooting is not always an option so we need to use what we call control points to help reality capture align those two components so i was initially going to narrate this part like i did the rest of the video but this part here can be a little bit tricky so i figured i would hold your hand through the process to make sure you don't miss anything so now what we need to do is we need to assign control points to both of our components here in order to merge the top and bottom half of our object so what i'm going to do here is i'm going to go ahead and change the layout right here up top here to give us a 2d image preview next we're going to click on the scene 3d tools tab and click on control point right here and you'll see what's pretty cool is as i move the mouse around all these lines you're seeing are all of the cameras being aligned to that specific point on the point cloud so we can use this to manually place control points in 3d space now what's really important is we need to assign a control point to something that is very easily recognizable very easily distinguishable across both components something that really stands out a little spot that has high contrast that you can easily say yep that is indeed the correct point very important so right here i'm going to zoom in and you'll see i have this little piece of wood sticking out here this is perfect this is exactly what i want this is very obvious it stands out it's high contrast easy to recognize i should be able to find this pretty easily on my bottom component so i'm going to go ahead and left click right here and don't worry that spot doesn't need to be perfect right now i'm going to right click now to get out of control point mode click on my first image right here on the left hand side over here and in the 2d view mode i'm going to zoom in here and i'm going to move this control point by clicking and dragging on it right over here because i know for a fact all my other images this spot stands out a little bit more now again i can click on hold on it and press the down arrow key to go to my next image again i'm still holding the left click here i'm going to move the control point to that same spot right here let go and now you'll see that control point has moved and as a result we now have way more cameras being aligned to that specific spot it was indeed a better control point spot so now again i'm going to click on the top one here click again in my 2d view press the f key to zoom to that control point spot now we need to go ahead and visually confirm all of these images you see all these images have a nice little green plush next to it by clicking on the image here we can go confirm that yep that's indeed the right control point we can confirm it there are two ways to do this there's the simple way to do it by clicking on the little green confirm button right here on the left hand side or an easier way to do that is to click the next image in our 2d view press the f key to zoom to that spot zoom in click and hold on the control point and again press the down arrow key to confirm it it's automatically going to bring you to the next image and we can just press the down arrow key over and over like this and as you'll see it is cycling through the images to confirm that yes indeed this is the correct control point do this for all images that were aligned and now you'll see on the left hand side all of these images have been aligned correctly we've now assigned our first control point on our top component now we need to go ahead and assign the same control point to our bottom component so in order to do that we're going to go on the left-hand side scroll down and we're going to select the component 1 because this is our new bottom component click on this and now we need to find that same little piece of wood sticking out on the bottom component we need to find that same spot and assign a control point to that same one we are now in the process of telling reality capture which point is which across both components and then as a result it will be able to merge the two together so what we're going to do again is we're going to go to the scene 3d tools tab up here again click on control points but this time we don't want to just click here because if we do that it's going to create a new control point and we don't want that we want to assign the same control point to this component so we need to go ahead and click on point zero right here and then we can go find our point in common now you'll see right here it's a little bit hard to see but that is indeed our little piece of wood our little stick that we saw in our previous component so i'm going to go ahead and left click on this right here and now you'll see we have a whole bunch of images that were aligned correctly now we need to go confirm those by clicking on them right here and zooming in i'm going to confirm that it is indeed yep at the right place i'm going to click and hold the control point press the down arrow key again and as a result you'll see it confirmed for you on the left hand side now i do want to take a moment to talk about why realitycatcher had a hard time aligning these images to begin with and that's because you'll see right here where i place my control point this little twig here is completely out of focus my depth of field was too shallow as you can see if i zoom out i focused on the front of the log here but because i was so close even though i was shooting with an aperture of f10 i didn't have enough of the shot in focus here and so that's why reality capture may have had a hard time aligning this initially that's why it is very important to get a clean data set to begin with to avoid alignment issues like this now realitycatcher does know that it should focus its attention on what's on the center of the object and less so on the edges but still having more of your object in sharp focus is absolutely very helpful so now as you can see we have our first control point assigned to both of our components now we need to repeat this process across two or more control points we need to find yet another point in common across both components in order for reality capture to merge those components together two is the absolute minimum sometimes three or more is necessary usually in my experience two or three has worked fine but again depending on your data set you may need more control points so what we're going to do now is we're going to find yet another point in common and assign a control point there so again i'm going to go to the tools tab up here click on control points and try to find something else in common across our images here i'm liking this area right here this seems like a pretty good spot easily recognizable it will be very easy to find on my other component zooming in i like this little point right here it should be very easy to find so i'm going to click on that right here and as a result you'll see on the left hand side we have a whole bunch of images that have been aligned so i'm going to right click to get out of control point mode click on an image here on the left hand side zoom in click and hold on the control point and press the down arrow key to confirm it and proceed to the next image cycle through these images until you've confirmed all of them now with the control point added to our bottom component we need to go here on the left hand side select our top component which is component 0 and assign our second control point to that same spot on the top component so again going to the scene 3d tool tab up here click on control points and of course again make sure i click on point 1 right here otherwise again it's going to create a new control point and we don't want that find that same spot and in this case this little thing right here and left click to add a control point and you'll see as a result we have a whole bunch more images that have been aligned successfully right click to get out of control point mode click on the first image on the left here and in the 2d view you'll see that my 2d view is selected because i have a blue border here press the f key to zoom in 2.1 and click and hold and press the down arrow key to confirm it do the same for all of these images as well and just like that we now have two control points assigned to both of our components the top and the bottom one you may need to repeat this process for a third or more control points but that totally depends on your scan now i do want to take the time to show you another way that we can assign control points because sometimes when we're using the control points tool it can be a little bit hard to really accurately add this control point like this another simple way of doing this we're going to manually create a new control point right here on the left hand side so now we have control point two and now using the 2d view here i'm going to zoom in and let's say i want to add my control point to this thing right here i can click and drag control point 2 onto that point right here using the 2d view drag and drop like this and now using the arrow keys to toggle between these images i can drag and drop point 2 on that same spot on another image and you'll see just like that we have a whole bunch of these cameras aligned to this point as i toggle through the images you'll see these cameras were able to find this point correctly so i thought it would show you just another way to assign control points now we can attempt to merge these two components by going to the alignment tab up top here and clicking on merge components if everything worked out correctly we can see all of our cameras in a dome around our log here and we have the top and bottom half of our log combined as one seamless closed object if however the merge components didn't work you'll see you have new components created here you can safely delete those and try again with our previous components find more common areas between the components and assign more control points that way as you can see this can be a very slow and tedious process and it's why making sure your images aren't too blurry or making sure you have enough depth of field to work with your images are well exposed not too noisy taking the time to shoot correctly may seem boring and dull but i can assure you that taking the time to get a clean data set will save you a lot of time in the long run it takes practice but eventually you're going to start scanning objects and getting the alignment perfect on the first try without the need for more control points now we can finally go to the mesh model tab and generate our final model if you're going for the absolute best quality go with high high does take way way longer to make though so keep that in mind often normal reconstruction it's good enough but i thought you might want to know that there is a pretty big difference with that done now we have a completed fully closed model that we can see and use from all angles guys i know this with a lot to take in but it's a very powerful tool to have under your belt because close 3d meshes as you well know are infinitely more useful to us 3d artists so just a bit of an announcement i wanted to let you know that i've just started my patreon page so if ever you're interested in supporting this channel go check out my patreon link down below as a patron you'll get plenty of 3d models that i create using photogrammetry moving forward so guys i hope you found this video helpful if you did do consider subscribing and giving it the old thumbs up and as happy rendering

Channel: William Faucher

Views: 70,597

Keywords: Unreal Engine 4, Unreal Engine, Cinematics, UE4 4.26, UE5, Realtime, realtime rendering, rendering, CGI, 3D, 3D Artist, UE4, Unreal Engine 5, photogrammetry, 3d scanning, under side, underside, bottom of object, 360 scan

Id: 3mLBL7T-zgs

Length: 24min 33sec (1473 seconds)

Published: Thu Jun 02 2022