Meshroom Blender photogrammetry workflow

hi my name is Jack Lynch and I'm a visual effects artist based in Manchester in this video I'm going to show you my workflow on how I created this 3d model of a junction box I used photogrammetry techniques and mostly free software so let's get started when recreating a real-life object using photogrammetry it's important to take lots of photos of your object so as you can see here I took lots of photos going round in concentric circles around this junction box it's important to capture everything you want to recreate so if you don't have photos of it you are not going to have the geometry at the end especially where detail is occluded by the object itself I took 219 photos for this object and I went close-up detail as well to get the detail on the lock and as well the detail on the floor where it meets the ground this object was slightly blocked by this other junction box so that's going to generate some issues on this side here ideally you want plenty of room all the way around your object so you can get plenty of photos on each side so I got more photos on the clear side here rather than the side days blocked by this other box so once you've taken your photos you're going to need to download mesh room it's a free and open source piece of software specifically made for photogrammetry it's node-based so each operation is a specific node with different parameters to change the default options are very very good however however I've changed my texture size as well as to use.you dims to get the maximum detail out of my raw scan the first thing we need to do to get our images and drag them into yeah so it's going to analyze all the images and look at the metadata to find out what lens what image sensor and some other details about the photos so once you've loaded in your photos it's important that you save your project just need to click start and you get this big loading bar at the top and it will process all your images align them generate a dense point cloud then create a mesh out of that and then project textures onto the mesh mesh room has finished calculating now and it's found 318 out of 319 photos and the photo but it didn't find was this photo here which I'm fine with it didn't have much detail anyway and there's not really much key features for mesh room to match with the other photographs so your mesh will probably come in a strange angle as well mine came in totally off and what you'll have on by default is icons of all the cameras so this is where each photo was taken in 3d space so you can see I went around in circles and got blocked off and then went around here and you see this far far more photos on this side as opposed to the side where I was blocked by the other box so if you double click one of these so meshing for example you get this icon here which you can click and turn off the other one and you can see the model has been reconstructed and we have some errors over here and if you want to view the filtering which basically clears out lots of the artifacts and bigger mesh errors that's here also smooths it as well if you want to change any of the smoothing iterations it's just here and you can also keep the large triangles that are generated so these ones here and then we can apply our texturing because I clicked use udimm it's gonna generate as many younam's required at 8 K to fulfill maximum quality so in this case it generated 17 maps we can view those maps by right-clicking and opening the folder that will open up a file explorer window with the mesh and the 17 image maps so you can see the UV unwrapping that mesh room does is not very efficient at all a lot of black space and doesn't use doesn't use most of most of the UV map which is why I wanted new dims and maximum quality at 8k so let's bring this into blender and start cleaning it up got my fresh blender scene here I'm just going import an obj paste in the location and because it has a material but file as well it should come linked with all the textures this might take a while for blender to open up and find all the textures because it does have quite a few you dims okay so my mesh has been imported into blender now and you can see it is at this weird angle upside down and rotated weirdly okay so if we align that and rotate it round so these faces up okay so once it's lined up you can put the floor in the origin roundabout there and now let's have a look at our wireframe it is so dense so if we wanted to put this into a game engine like unity with three million almost 4 million faces it would not work very well you see that our main subject has the most detailing and then towards the background you see it's kind of the details fading away and now let's preview it with our textures ok so it all looks good so something can do to preview it is put into rendered mode then change the sky to white and that will basically put a uniform light everywhere so an issue we're gonna have with the textures is this side is somewhat cluded by this junction box here and if there was a tree area here so this side is in shadowed basically and I think yeah the back is in shadow compared to where it's very well lit on the top here it's quite reflective and here so we're gonna have to clean that up so first thing we need to do is only select the geometry that we want we don't want all this grass back here and we don't even want the road here or the pavement so let's do that so I'm gonna jump into the Select masu into edit mode and it might take a while because there is almost four million polygons and I'm gonna go to x-ray mode and back in till I see you select faces and another select try that again I'm gonna select just the junction box and it's how you be around it okay so I'm gonna click ctrl I to invert the selection then X event delete faces okay so we've got some floating geometry out here that we selected in our last selection so what I'm going to do is go back to the tweak grab one of these faces then click ctrl L yeah and that is gonna select all linked I'm gonna link it by the seam so that's going to get rid of anything floating control eye X delete faces okay so we've got it down to two and a half million polygons now but that's still way too much we need to fix this issue here so I'm gonna rename this to raw scan check our texture still work they should do don't see why not we haven't messed with ru V's no so first of all let's clean up the huge gap here okay so jump into edit mode I'm also gonna delete these faces here because we're not gonna need any of that grass and it's slowing us down and this as well it's just gonna speed up the viewport performance quite a lot so I'm going to actually duplicate this to kind of make a backup and I'm gonna rename it to scan scan cleanup one now we can do a few things to fix this we can have a look at sculpting let's see if we can fix it in sculpting so then you use the fill tool can you drag this out that you can actually see the names and use the fill tool turn off symmetry and just by using the mouse going to fill this hole trying not to affect any of the other geometry we can go to advanced and auto mask Auto topology auto masking and we're gonna find it if we can find a cumulate while me triking right in here this might help to start off with by bringing out our then let's try to fill again working slightly better now far too much so I'm gonna shifts to smooth that just so we can get it kind of flat okay I've got this side cleaned up now a little bit better than before I just used the fill tool and this smooth tool in the clay tour as well the fill tool isn't so good around the edges but you see if we do it around the edges it starts filling everything basically so I found it works quite well on the center where it's not faced by any edges and I'm just gonna use the clay strips to actually fill in detail and this is where we have accumulate on so if we add in some clay around here then brush a bit bigger and smooth it out and now we can go in and fill it with a bit of a bigger brush tried using flatten but don't really think that works very well in this case yeah it does flatten out detail by makes this bit bumpy found smooth actually flattens things out quite well so we can go go around and sculpt in some more detail if we want which i think is what we're going to do we're going to go ahead clean up some of this noise because this is not what it truly is like so I'm just going to go ahead flatten out some of this noise smooth it out just in there just so we get tiny bit less noisy using this flatten tool and ok so now we've got our mesh smoothed out a little bit we can begin to remodel this luckily it's quite a simple shape so we will be able to remodel this very easily so then I go back into layout mode and then I had a cube resize it and just like with modeling anything from scratch just go ahead and line up all the verts you I'm gonna shrink wrap it to our cleanup see we're getting some issues here that is simply because of the limit so if we make it zero we're getting something see some weird things this down sample that okay I'm gonna make backup of this so I'm gonna shift D duplicate that not in edit mode though now we need to UV unwrap our object so first thing I'm going to do is delete these bottom faces because we're not going to see them and now we can simply unwrap our object so I've placed some cuts in the obvious areas now if we go to UV unwrapping unwrap it we still have some issues here so it's simply cut piss off now we have unwrapped our object we can begin to project our textures and onto it so I'm going to shrink wrap it to our original scan and then add some subdivisions before we actually bake we need to set up a shader for our new object so in the shading tab click on our new box new and then we need to create a map for baking so I'm going to create a new a k-map now change the color to black with 100% alpha create that I'm gonna call it J box color there we go from our scan hold down control click on the clean box under under the rendering tab is baked change it from diffuse color only selected to active change this to 0.01 and then bake we have our textures projected now and there have been some errors that we would need to fix so we've got lots of black spots here which I think we may be able to fix with our projection limit or raid limit let's have a look now with the object with smooth shading on now yes that looks a lot better still got lots of issues here though with the black clipping almost it looks like another reason why it might be looking strange is because we unwrapped our object before we applied the shrink wrap so I'm going to unwrap it again I'm actually going to duplicate model and unwrap it with the modifiers already applied I like everything unwrapped and it is a lot different and it is a lot different now it's not a square because I've got a lot of different shapes going on now okay - I'm going to bake again and I'm gonna up the up the Ray count to zero point is there a radius and this is zero point one and bake so if we wanted to that's it we could leave this put this in our game engine and it'll look great but I'd like to go a little bit further fix these errors here which will be simple enough so I'm going to export this into substance painter where we can do a little bit of cleaning up and hopefully convert this to a PBR shaders model I've imported my model into substance painter now now I need to import original scan so that we can bake some normal data and ambient occlusion to help us do liked so I'm gonna bake that 8k not going to worry about the thickness or ID everything else will be helpful I need to do some basic D lighting around the lip of the lid of the junction box so I'm going to do that by filling it with ambient occlusion I just generated I'm going to generate this to duplicate this add a levels and then add a mask with a fill and use that same ambient occlusion as a fill for our masks we might need to invert this and there we go we're just affecting any although dark patches here so we can slide this to see if we can get some uniform out of this and if we just preview that it definitely does have a do lighting effect and in our mask here we could generators we can do a number of different things but in this case I'm going to fix the different lighting on the lid now and we're going to do that just the same way so I call this base I'm going to waste do light main do light duplicate that and call it lid delights now we can simply change the mask in this case I'm going to paint the quick mask remove that add a paint we remove there that's fine paint a mask on to that you and with that we can change the levels so instead of brightening brightening it we can darken it and that already looks a lot more even I've added a roughness channel now so everything that is black is shiny and everything that is white is rough however we still don't have a metal and there are some things on this I'd like to be metal like this shiny metal lock here I'm just going to be fairly simple going to add a new paint layer or fill with just metal selected and one what white is metal and black is not metal I'm going to set this to black add a new paint layer set that metal and just paint some paint some white here you use a fairly hard brush maybe maybe a brush with some scratches or metal fibers about about this dirt brushed here very small just to get this bit and go directly into the metal channel make sure on white might need actually a paint layer there we go just see if that works um here yeah okay you can just delete this on there perfect and to roughen this up I'm going to multiply that with a grunge map and we're gonna use metal scratches possibly or rust and put in try planer have a look at it metallic multiply see if we can up the contrast a little bit there we go stir a little bit shiny now and I don't think we should add anything else metallic because it's painted let's have a look actually see what happens if we if we add just a base metallic fill to everything of just a very small amount just by using the clone stamp tool I've created a cleaned-up version without the graffiti so we can put our own refugee on let's have a look at how we can change that now for my original fairly simple we just need to drag in our new one and call it a box as a texture into our project we then just need to replace all the instances where we have it filled so if we turn off everything so far got the D light we need to replace that got the lid d-light we should really replace that even though it won't make much of a difference because we run the lid do anyway we have this cleanup on the paint so let's don't need that sits on the pass-through don't need this anymore that was our original attempt at the D light our rough base now that was just a color corrected version of our original color so replace that this is our rough mix of our AO we don't need to worry about that we need to replace the rough with grips even more drips that we added with a paintbrush don't need to change the metallic because that is just a rust map now we have a cleaned up version we can now put in our own graffiti if we want to do substance painter has a really good preset for adding graffiti with spray tag a spray paint which we can add as a decal on to our object somewhere need to rotate it into the right position so 90 degrees and make it a bit bigger change the color to let's change the color to white for example so we can bring this below everything else or just yeah below everything else so that our roughness still has an effect on it so I'm gonna add and add it here rotators it's not perfectly centered scale it up a little bit bigger and now we can add an image that we want to be graffiti so I'm going to add the river logo scale it up a little bit I'm going to play with some of the attributes so we can add some drips and leaks so we need to adjust this and it's perfectly where gravity sit so the leaks go down then we can just change the image rotation here so something like that will work don't like this leak here to change the density and here it is we've got a finished model we can put into a game engine or into any scene if you have any questions about the technique or the workflow make sure to let me know and thank you for watching

Channel: Jack Lynch

Views: 18,451

Keywords:

Id: HZPAqL0JJiw

Length: 39min 18sec (2358 seconds)

Published: Tue Jul 14 2020