LiDAR Scanning in Blender - Environment and Asset Creation in-Depth Tutorial

hi my name's zac and today we'll be exploring lidar scanning in blender for environment and asset creation i'll cover all the ins and outs of lidar scanning from which apps and devices to use right through to a step-by-step breakdown on importing your scans into blender and setting up your first scene i've been using lidar scanning as part of my daily workflow for the last few months now utilizing this amazing technology in personal projects commercials and short films with that said let's get on with the video so what is lidar scanning well the lighthouse scanner is a light detection and ranging sensor over the last year apple has begun introducing lidar technology into their consumer mobile devices starting with the ipad pro 2020 iphone 12 pro and iphone 12 pro max lidar however isn't a new technology and has been around for a long time now mostly used in science autonomous vehicles and surveying essentially the lighter scanner works by projecting an array of lasers onto the environment you are in the rays of light bounce off objects and back to the lidar scanner on the device and with that is able to create a 3d map of the environment you are in if you'd like to learn more about lidar and how it works check out this video by the unlocker for more details two devices that scan as i mentioned before the only devices with lidar scanning capabilities at the time of filming are the iphone 12 pro iphone 12 pro max and the ipad pro 2020 for all of my scanning recently i've just been using my iphone 12 pro max mainly because of how portable it is compared to my ipad pro having something that fits into your pocket and is there whenever you find something to scan while you're out is super helpful the ipad however is king when it comes to processing power screen size and battery life if you plan on using scanning as part of your daily workflow then i recommend getting the ipad pro one downside to the ipad pro over the iphone is the camera specs and crop being the same camera as the iphone 11 pro from 2019 it has worse low light performance and the ipad camera crop is especially frustrating when scanning interiors without a lot of space the iphone on the other hand suffers from overheating after about 15 minutes of continuous scanning i've found that the scanning app has more glitches and crashes on ios than ipad os but these are still exceptional devices and these issues are rather picky and also the technology will only continue to get better over time 3. how to scan there are plenty of highly rated lidar scanning apps available on the app store today some with more features than others but the app i'll be recommending and using is the 3d scanner app this app has performed the most consistently and accurately across all sizes of scans and from my testing produces the best texture mapping which in our case is what we want when you open the app you'll be in live view this is where we'll do all of our scanning to start let's look at the interface down in the bottom right we have the scanning accuracy low and high resolution basically low res is just very primitive shapes and the things that you don't want any high details in i use hi-res so that's what i'll be showing in this video above we have some settings now first is confidence basically i leave it at high and it works really well so i'm going to leave it next is range range clips the distance that you want the lidar scan to capture if i set the range to 3 meters it won't scan anything over 3 meters away from the camera this helps keep unwanted objects from being captured in the skin next is masking the masking feature predicts what you're scanning and tries to disregard everything else from the scan basically i don't use masking but sometimes when scanning people the person mask works quite nicely and lastly resolution this just lets you dial in how much geometry you'd like to be generated and also determines the accuracy of the skin i usually leave it at five millimeters but some objects simply don't need that much detail and will only make the file bigger and harder to work with for most of my scanning i leave the resolution at five millimeters but when you're scanning large items like environments or big rooms i would usually put the resolution up to around 10 to 15 millimeters once your settings are dialed in hit the red record button to start your scan slowly move your device around your object making sure to get all angles and not moving too fast as you move around the device will be taking photographs at intervals usually about 30 centimeters in movement so make sure you're not moving the device too quickly otherwise those photos will be blurred and will not get a good result once you finish hit the red record button again and it will automatically bring us into the preview mode here we can edit our mesh and we can also texture it i'll chuck on a quick texture to show you how this looks and this is our result so that's the basics of scanning and now we'll move on to environments number four actually scanning our environment so once you've found the environment room building that you want to scan it's important to plan how you will scan it and scanning will require some test scans to find the best method one rule though that applies to every scan is the lighting when you scan all the lighting is baked into the texture so if it's a sunny day you will have harsh and usually unwanted shadows in the texture always try to plan your scanning on overcast days where as much of the light as possible is ambient essentially no harsh shadows you can always add lighting later in blender and it's recommended to do it in post now for area specifics for areas with lots of foliage grass trees sticks etc you want to try and keep as much superfine foliage and leaf out of the skins as possible the lidar scanner isn't accurate enough to capture individual leaves or thin twigs and will only mess up the skin resulting in a mess or even errors for my forest skin i tried to only scan rocks and areas without lots of foliage and heavily used a short range clipping to keep all the unwanted plants out of the skin for buildings and large surfaces like walls you need to keep in mind that the lighter scanner has a max range of 5 metres so for this building i was unable to get the entire height of the wall from the ground where i was standing also observe if there's enough detail in the surface for the device to track if there isn't enough contrast in the camera frame the scan could slip and basically you need to start again sometimes you can add a poster or an object into the scene if it continues to slip around for more contrast for the device to track for the best scan and textures try not to rescan areas too much and work in a sweeping manner just slightly overlapping if you find your object or environment is too large to scan it once or you just want to take a break there's a neat feature to extend your scan after stopping the recording just hit extend and point the camera at an area you've already scanned and it will try its best to match them up i do recommend duplicating a scan first in case there is a glitch and your original scan gets ruined i've found the bigger the scan the more likely it will crash or fail when texturing or even scanning a good rule of thumb is to stop scanning when a prompt appears at the top allowing you that you're at fifty percent of the max scan size this is usually about one to two gigabytes now that you have your scan just texture and check that everything is fine before exporting then hit share i prefer obj as it is universal and well adopted make sure you check textured and zip files to save space i'm going to upload to my personal server but you can email airdrop or chuck on a usb drive by saving to files having the ipad is especially useful in this case as it has a usb port and lets you save your scans to any usb c drive or ssd once you've got your scans onto your computer let's jump into blender alright before we can jump into blender though we'll have to unzip the zip file that we exported before and save it somewhere we can find it later i'm going to use winrar to extract my files and once they're extracted you'll find three files here one will be the textured output which is what it's called by default and this is all the textures put together in like a uv map next is your mtl file which will be imported with your obj file when we're in blender now that they're in let's open up blender today i'll be using blender 2.9 1.2 but the version doesn't really matter for what we're going to be doing today so with that let's open up a new general file and we don't need any of this stuff here so we can hit a to select everything x delete just like that i'm going to turn on screencast keys this just is a handy little add-on that lets you see the different buttons that i'm pressing so with an empty scene we can now import our scan doing that is easy file import wavefront.obj which is what we exported it as so clicking on that we get the file browser and we can navigate to where we saved that scan here's mine here and just by selecting on the dot obj it will also import the dot mtl which is our texture file once that's there hit import obj and this could take anywhere from 30 seconds to 2 minutes i know with this particular scan it takes about a minute 30 to import our scan has successfully imported and everything looks good from here now don't worry that there's no textures we'll turn them on ourselves later before we get started though need a save because there's nothing like losing all of your progress just before you hit render to save this as whatever you want and hit save blender file now that we've saved our file we can move on to cleaning up our scan and getting our scenes set up before we begin though i personally like to keep things nice and clean and we can see that our scan has been imported on a bit of an angle so i'm just going to tie that up rotate it a bit so it's in line with this red x-axis so to do that we're going to select our mesh so it turns yellow outline that means it's selected we're going to go into top view which is 7 on the numpad or you can go up to view viewport select top numpad 7 tells you what it is here once we're in top view we can then rotate our mesh so that's r on the keyboard which is r for rotate and we can just align it here with the x just eyeballing it so there we go that looks good and now we can move on we're in object mode at the moment and that allows us to grab objects and move them and position them around but that's about it so if we want to get in and actually delete some of the geometry or fix holes and stuff we need to be in edit mode so to do that make sure your mesh is selected make sure it's yellow come up to object mode drop down and just change it to edit mode you can also switch by hitting tab on your keyboard now this might take a few seconds because this mesh is super dense so keep that in mind when you're recording just to make sure your computer doesn't die when you're trying to edit these scans zooming in we can actually see the density of our skin all these are the individual vertices of our skin and because i recorded this at five millimeters there's a lot of geometry but we're here to do some cleanup cleanup is essentially identifying areas of the mesh that need to be removed holes that need fixing and sometimes even re-topology if the geometry is too dense today we'll just be covering removing pieces and fixing problems when i approach cleanup i first identify geometry i don't want in the scene i've got a piece down here i can see i can also see these pieces jutting out and because i know what i'm doing with my scene i'm going to take out the entire ceiling up here i just don't need it and it didn't scan all that well anyway to kick things off i'm going to remove the ceiling to do that we're going to jump into front view so that's one on the numpad or view viewport front numpad one just to give you a sense of orientation this is now the ceiling and this down here is the floor so we're going to be removing this part up here now to do a box selection in blender you just click and drag that easy just like that and that will select our ceiling portion but before you hit delete if we pivot around we'll see that we haven't actually selected the whole ceiling and that's because from this view we can't actually see that portion of the ceiling so to select everything what we need to do is come up to the top right and toggle on x-ray mode that allows us to see through the walls and select everything from this one view so we'll just go ahead and select that again and now when we pivot we'll see we've actually got the whole ceiling and now we can hit x and delete all those vertices all right now that the ceiling's gone we can go and identify other pieces that we'd like to remove and this is one i've already spotted so zooming in here we can just make our selection again just like this or we can check out some of the other selection features blender has by clicking we can get rid of that selection and we're going to use a tool called circle select that's c on your keyboard for circle or you can come up to select circle select right here circle select just allows you to select a circle instead of a box we can change the size of the circle by using our scroll wheel once we have the right size just select your selection and to exit the circle tool it's enter on the keyboard or the right mouse button now we've made the selection it's x delete vertices and basically just keep moving around your mesh finding different things that you can remove so i can see a piece in here this would have been my ceiling lights that just got in the skin and we'll check out another way to select things in blender and that is the lasso tool so hitting t on your keyboard will bring up this little menu or you can hit this little tiny arrow on the left here holding down on the select menu we can select select lasso that allows us to draw a selection around what we want now it's selected x delete vertices and because i don't use lasso much i'm going to switch it back to select box t to get rid of that taking a quick look looks like that's all the geometry that i want to be removing today and that means i can move on to the second phase of cleanup and that's identifying issues like holes and problems with the texture but to do that we need to be in object mode so i'm going to switch over to object mode or you can hit tab on your keyboard and we're presented with this glitching looking effect on our scan but that's just because we left toggle x-rayed mode on if we hit that off we're back to normal and that's all the cleanup i'll be doing at the moment so back here in object mode we can get a better look i'm pivoting it around i can't see any more loose objects that i'm wanting to remove so that part is finished what we're going to do next is start setting up our scene so pulling in a camera start positioning it so that we get a good idea of what we're going to get once we render it out now a common problem when working with interiors is not being able to see through the wall so it can get kind of finicky once you're in here trying to move around and you keep popping through the wall here when you're trying to move around and edit things so to fix this issue a little tip here top right let's turn on back face culling what this does is it removes any faces that we're looking at the back side of so if we pivot around here this wall comes back and we lose these walls here super handy working in interiors this wall isn't gone it's just not visible when we're looking at the back of it let's bring in a camera so what we can do is come up to the add menu find camera bring in a camera just like that now moving a camera works just like moving any other objects in blender and if you don't know how to do that we can hit t on the keyboard or this little arrow up here and bring in our move rotate and scale gizmos so cycling through them we can do the standard operations by moving this around rotating it and scale doesn't work on a camera but it's there for other objects or you could use the shortcuts which is g on the keyboard r and s and you can also constrain to the axes by hitting well so g and then y will lock it onto the y g x lock it onto the x rotate on the z just like that so that's how we can move our camera around the scene but i don't usually move it around this way it gets kind of finicky and annoying because you don't know where the camera is and you don't know where you are an easier way to place the camera in blender is just to line it up with your view so i'm going to just move around checking different angles and moving my view somewhere where i like and as this is the seventh time i've recorded this video i know that i'm going to put the camera in this location once i'm happy with this position here all we have to do is hit ctrl alt and 0 on the numpad and that will actually snap our camera to our view it does however punch it in a little bit so to pull that back out zoom it out a bit it's g and double tap z just pull it back a little bit just like this so i'm pretty happy with that and if you didn't know hitting zero on your number will bring us in and out of the camera view just in case now that we're here we can go on to the second phase of our cleanup and that's identifying holes or issues with our mesh and or our textures we can't actually see our textures at the moment we'll do that in a second but first let's have a look at some holes it's pretty obvious but we can see a hole right here this is exactly why i find it important to position your camera before you do too much cleanup because here we can see that this hole here is in the frame but per se this hole here on the side of the couch it doesn't matter because it's not inside the frame so it would be a waste of time if we went and fixed that so make sure you position your camera get an idea of the scene that you're going for so you can only fix the things that actually matter and because this hole is right in the middle of my frame this is what we're going to be fixing now we could come in here and try and manually patch this up but that takes a lot of time and effort and to be honest i'm kind of lazy in 3d so we're going to use a quick tip to replace this wall completely in as little effort as possible what i mean is we're just going to delete this wall and replace it with a photo i took in the actual location now when i was scanning this room i actually found this issue in the app and i saw it and i thought i'm just going to take some pictures just in case i need them so it's a good idea just to take some pictures just in case you need to fill in a hole or fix something that glitched out that you weren't expecting and i could have gone through and just rescanned this room but it's a good little tip to throw in the video so what we're going to do is fix this wall but to fix the wall we have to get rid of the wall so just like before removing things we're going to tab back into edit mode or you can go up to object mode switch into edit front view which is one on the keyboard or using the menu toggle on x-ray mode and we're just going to select this wall and delete it just like that nothing fancy tabbing back into object mode and disabling x-ray we can now worry about bringing in our image or our photo that we took bringing in images super easy so file import import images as planes now if this doesn't show up for you don't worry we just need to enable it enabling add-ons we can go up to edit preferences in the add-ons tab we can just type in images oops images and just make sure this is checked now when you come to file import we can import images as planes clicking on that just navigate to where your photo is mine is lounge wall and in the settings tab just make sure you're unprincipled then hit import and there is our image it's on a plane so we can move it around nothing different like that but we can't see anything so to bring textures into our scene top right just click on texture i'm just going to move this into place using the same tools as i showed you before so just roughly bringing this in scaling it down oops let's rotate bringing it up just trying to massage it into place so that it fits into our scene don't have to be too nitpicky about this but just try and get it roughly in place depending on the project you're working on you might want to spend a bit more time but for me this is looking pretty good once your wall is in place let's check out what the render is going to look like over here on the right in your render properties we need to check that we're in cycles so just check cycles most of you will be in eevee so just switch over to cycles for this tutorial and once we're in there we can come up to the top right and click on viewport shading and this will enter rendered view so rendered view might get a bit fuzzy on the screen but it should clear up pretty quickly um that's just how cycles renders so that's not a bug that's how it looks but you'll notice there's no light so to turn the lights on we'll come over to our world properties click on color just boost it up to white now usually you'd come into your ad menu which is shift a or up here and you'd add some lights and that sort of fancy stuff but because our lighting is baked into our texture we can get away with just white ambient light and just like that we've got our skin ready to use for whatever we want inside a blender from here it's only limited by your creativity and you can literally do anything with this now vfx animation you name it so this basically wraps up our tutorial today that's the background behind lidar how to light our scan different tips and tricks how to import and how to set up a scene and clean up your scan thank you for watching if you'd like to stay till the end you can watch time lapse of me going through this scene and adding a bit of creative flair and having a bit of fun with some rtx cards so thank you for watching and enjoy the clip [Music] [Music] [Music] [Music] [Music]

Channel: Default Cube VFX

Views: 136,034

Rating: 4.9375849 out of 5

Keywords:

Id: IN1nyU_CL7A

Length: 28min 8sec (1688 seconds)

Published: Mon Mar 15 2021