Environment Art - Turning a Cliff into a vista prop using photogrammetry

[Music] hello everyone jakers para here in version 1.6 aqui soft has introduced a new photogrammetry reconstruction mode in this video I decided to give it a try and share the results with you so this time I'm going to present full photogrammetry workflow where I captured a cliff rock formation and turn it into a Vista prop in details I'm going to show cliff rock formation capture with the drone an image was processing with different way for ambient shadow removal in photo op free full 3d reconstruction in meta shape using newer construction mode I will build a low poly model in ZBrush which I'm going to your team up with there is a movie I will bake textures using substance designer next I'm going to fix missing areas in substance painter and finally I'm going to compose a quick scene in marmoset toolbag free to present the result I hope someone finds this video interesting and fun to watch and let's begin the newer construction mode was introduced in version 1.6 of meta shape this mode is way faster and way less resource demanding for a construction when compared with other modes as well as previous meta shape versions bear in mind that new mode works in arbitrary reconstruction mode with depth maps selected as a source only the high field mode but also the arbitrary but based on a dense cloud as a source of data work as before have no out of car implementation and no GPU support this new method can benefit from additional amount of from a variable and also fast communication with the project folder as quite a lot of intermediate data is stored in the project folder and being read and rewritten so using fast SSD drive to store the project files should give faster reconstruction and likely outperform slow hard disk drive we should also get them significantly faster when we compare a construction times before the 1.6 version and last but not least bear in mind that the current depth map vice reconstruction mode gives a bit lower than City mesh to the one based on a - cloud fortunately these are things adhesive is currently working on so should be fixed soon let's move to the capture part done for this capture I picked pretty cloudy but windy day the Sun was shining through clouds from time to time but it was fine seems material capture wasn't my top priority this time I wanted to find and capture a larger but consistent rock formation which can be used as a quite generic and Universal Vista prop the loved ID gave better access to interesting rock formations and exposed bottom parts of the rocks but I had to be careful since the high tide was coming and without paying attention I could end up in a water it took me a while to get to the nice spot but finally I have found a really nice cliff wall with interesting shape since I was planning to capture it and turn it into a Vista prop my aim was to get a full image coverage from every possible angle the only fast and easy way to capture this type of probe is to use a drone without the drone I wouldn't have access to big parts of this cliff since the ground wasn't flat I decided for her hands tied off for this capture I set the drones camera f-stop too far but I left I saw in outer mount hopefully light was good enough and all he murders were captured with the ISO 100 this is the setting I have found the most optimal for my big two pro and close surface scanning it gives short exposure time while this value is small enough to compensate the distance to some degree and give sharp images also on image edges before I started the scan I took a few images of colorchecker for future white balance correction and color calibration unfortunately since this cliff is quite huge and has limited access I didn't set up any rulers as a sky reference scanning takes some time especially for larger props it is quite easy to get lost and leave some areas uncovered and the same time over shooting others this is why it is worth to keep scanning paths organized and this is the path I have found most efficient and useful when I scan larger vertical drops with the drone I simply fly in vertical lines trying to keep the same distance to the subject so I fly up taking series of pictures until there is nothing more to capture and when done I move the drone to the next viewer to our column and fly down taking another series of images when done I repeat this steps over and over until everything is covered and this is exactly what I have done in here to make sure I didn't miss anything I also took a few shots from larger distance as even lower quality data is better to no data there were a few birds flying around and a few crazy seagulls but I have learned that seagulls need more space to attack they are predators but they usually dive from the top to hit the target to pull up when it is done it works with fishes in the open sea but not in this case since the cliff wall limits the space so as long as I kept that wrong quite close to the cliff wall it was safe and seagulls were just flying around in big circles I'm not saying that these seagulls wanted to attack the drone just that if would be quite hard for them if they was trying finally I managed to capture 274 key images plus a few additional four called hydration all captured images were fine when the capture was over I hand landed the drone and packed it back to my backpack the next step is the image photo editing to get the best from all captured images [Music] for photo editing I use a photo lab free next let's jump into customize section and select the image which is going to be used to set up the white balance we have this image selected let's press ctrl + a to select all images next let's zoom in to see the colorchecker clips and select neutral clip with the white balance color picker it sets the white balance based on this value to all selected images [Music] next let's remove some ambient occlusion shadows using selective down section scenes we work on row image we work on very dense color data and it is totally fine to shift colors and push values because while working on row we deal with way more additional data that we really need so even if we limit the histogram area we still have a lot of real capture based data to pick from just to remind you one of my previous videos this is the histogram based chart showing how many colors depending on their color that RAW files can store it means that as long as we operate on dense 14 bit zero data we can consider any color shift within a dynamic range lossless and this is the stage where we can easily remove any ambient occlusion shadows or column down highlights by simply playing with options in selective tone section it would affect their construction quality if we would use 8 bits for reconstruction but as long we export the result into 16 bits and use that 16 bits results for reconstruction photogrammetry software gets more data to what it even needs it is good to understand that the photogrammetry reconstruction isn't based on shadows but positions of points as long FATA gramma tree software has enough information to see the difference between neighbor pixels and while we work on 16 bits it definitely has we should be totally fine we should also turn the crop correction off as we don't want photolab to remove anything from our images also we should turn any geometry distortion correction of as the photogrammetry software will do it way better and I think we should be ready to export the data for reconstruction we just need to make sure we export in format which supports 16 bits like TIFF or PNG I usually use Steve since we convert 275 images this way is going to take a while well done we can jump into meta shape and upload those images for a construction [Music] the next step is the reconstruction first to reconstruct the subject we need to load and align all the images in 3d space I did it by selecting workload and images I skipped the image with myself holding the colorchecker and selected the first image with actual rock formation next I moved to the last one and with shift pressed I selected it with all between next we need to align those images in 3d space I did it by jumping into workflow and select align images let's try it with the default setting since we work on 16-bits images it should be enough but in case not all images are aligned we can play with numbers in advance setting and increase values depends on alignment setting this process can take from a couple of minutes to even a few hours this alignment took about two hours so let's keep this video to the moment when it was over when the alignment is done all images should have a thick this marking means that the image was aligned and it's going to be used for reconstruction with all images aligned we are sure that we are going to use all collected data we have for construction after image alignment we can see the camera positions and type points these points are an avid navigational points shared between cameras to estimate the positioning in 3d space for the reconstruction blue planes are representing the camera position from the moment when the image was taken looks like this rock formation is quite well covered from close and medium distance and it should be enough information for a construction the box around the tie points limits their construction area let's extend it a bit to cover skipped part to the right side I think everything looks okay so let's move with the next step which is the reconstruction as you can see we get more options after the images were aligned since we want to proceed with a full freezer construction let's use the reconstruction mode I mentioned at the beginning of this video to do this we need to select workflow from the top tab and pick option to generate mesh as I mentioned the new mode works only with the depth maps used as a source of data and only in the arbitrary freedom mode to get mesh as dense as possible let's keep the quality as ultra high and maximum face count since I want color to be stored in vertexes let's make sure that option to calculate vertex color is active next let's hit the ok button and start direct construction next flat speed time a bit since full freezer construction took almost 22 hours in previous version of meta shape it would probably take even more and highly likely crash at the end giving totally nothing back as you can see mesh we generated has 56 million faces and is quite accurate unfortunately I didn't manage to cover everything and there are a few gaps left these gaps can be easily covered with the low poly model and filled with texture data using clone tool in substance painter in next stage but before that let's save the model and export it as a high poly source for baking exporting 60 million polygons FBX file might take even an hour this one took 40 minutes since it is very hard to navigate in ZBrush with such a heavy mesh we need to decimate it in meta shape to create a lighter version we can do it easily by running a decimate tool from tools and mesh tab I think that 5 million polygons should be enough to keep all the shape details we might need in the next step so let's set the summation to 5 millions and press ok or now let's pick 10 millions as ZBrush also should handle 10 millions without any problems just be careful and don't save any changes after the summation is done so we will keep the heavy model in a project to be able to resume tit to whatever value in it as you can see 10 millions is enough to store all these shape details so is definitely enough as a reference to build a low poly model in ZBrush the summation itself is quite fast and when finished we can export the summated mesh as another FBX file exporting 10 millions poly took me just 3 minutes went down we can close Mehta shape just without saving to do not overwrite the high poly wave it's decimated version now since we have a high poly model for baking and medium poly model as reference we need to build a fully functional opera version of it and make all high poly information into its texture so let's jump into ZBrush and load our 10 million faces dance model as a reference to build a low poly one [Music] [Music] next let's bring the topology and create a low poly model using the detailed model as our reference the topology brush is a very handy and efficient tool for this job next let's set the proper spline accuracy using draw distance slider and let's start building the model with the topology brush we simply draw lines when they cross to each other they create a connection point so we don't need to be very accurate drawing these lines what really matters are just yet the intersections [Music] [Music] areas defined by free or for collection points are being filled with a face because we are going to project this low-poly model on our heavy reference we don't need to be very accurate we just need to focus on line topology to follow the main shape this way we will get enough geometry to wrap around details after we subdivide the mesh [Music] [Music] when done we need to separate low poly mesh from the reference and project it under its surface to make sure it's aligned properly to get a bit more information from geometry itself all adds divided a bit and project it on 10 million polygons mesh and let's repeat that until we get the density we want I think on fifth subdivision damage is done enough to carry all silhouette details for the Vista prop seen from long distance first or the second subdivision level will be totally enough but since this is a Vista prop for a still shot in marmoset toolbag scene I can go a bit crazy and not worry about performance too much just bear in mind that in workflow totally nothing changes and no matter what mesh density I pick next steps are exactly the same if it would be the game asset I would go with way lower density now the mesh is ready to be exported to make the final scene a bit more interesting let's create something for the ground surface and export it out now it's time to UV map it UV mapping can be done directly in ZBrush with UV master tool but there is way better tool design just for this and this is the tool I'm going to use so for UV mapping let's jump in tourism UV to start UV mapping in recent movie we need to drag and drop the mesh we have made we can UV map it manually health manually or totally trust there is a movie and let it do the job for us since we are going to use a dedicated texture without an overlapping with information back directly from the high poly model let's save some time and let resume UV to do the full job let's move into outer seams and full outer UV stop next select out of select music mode and next let's select free for a number of cuts this one means less distortion but more cuts next let's activate overlaps and stretching detection as it's important for us to do not have anything back in the same space twice and let's proceed with UV mapping how to select edges button selects the UVs for cutting following the rows we set we can change the rules or calculate them or hit the second button if we are okay with what we see the next button cuts and flatten the UV shells when done it's time to hit the last button which will rescale and pack all these UV Islands within a single udimm and will use as much space as possible now it's time to hit ctrl + S to save our UV mapping to the FBX file and consider UV mapping process as done now it's time to bake all color and high information from the high poly model to the low poly one we made to do this let's jump into substance designer and use its baker when opened let's create any substance it doesn't matter which since we need just to run the baker next let's bring the low poly model we want to use for baking and activate the baker on it let's configure the baker window and start the baking process [Music] [Music] [Music] looks like I missed the vertex color as a source for color map so let's fix it and repack just the color map now looks like we have full set of textures to deal with so it's time to jump into the painter and fix what's left to be fixed since we didn't break a roughness map let's generate one in substance designer using our pegs I quickly made one by processing albedo with the high pass filter [Music] [Music] next I multiply the value with ambient occlusion map since we don't want nothing what is hidden in a shadow to be reflective and at the end since is a roughness map I inverted all values now it's time to jump into painter for final tweaks let's bring our UV map low poly rock formation next let's add a fill layer and fill the channels with baked textures first we need to import them to the project and add to the channels to be able to use clone tool we need to apply paint to the lair and set it to pastro for every channel with paint active we can activate the clone tool and paint on the mesh by pressing V we select the area where we want to clone from now we need to find areas to fix and over paint them with correct data it is worth to switch between channels to get better visibility it can be done by pressing C button with M we come back to material view [Music] [Music] [Music] looks like I missed the ambient occlusion map so let's bring it in [Music] [Music] [Music] okay looks like everything is done now we can export all the textures and apply them in the next step to our render scene in marmoset toolbag let's open the marmoset toolbag van and put all elements together at the beginning let's bring our main hero the rock formation to the scene [Music] next let's apply all the textures we might [Music] [Music] [Music] since this rock formation look scrapped without the context let's at the context then and bring about ground meshes we made before to the steam this one will be used as a solid ground [Music] next let's bring another one and try to set it as a background water plane since this is not the subject of this video to cover the ground planes I will just bring to environment materials I already made for this beach I'm gonna pick the one I captured while ago for the water I will use procedural water material I made HS ago - this one isn't the best but should do the job next let's drag both materials two meshes and set up the tiling to the value which makes sense [Music] next let's copy the cliff and set it in a longer distance and we arrange the scene to build nice and interesting composition [Music] [Music] to make lighting a bit more interesting let's apply one of my sky maps I captured while ago let's add additional light which will cast a sharp shadow by tapping on HDR I'm preview map and adjust the angle and brightness next let's add some fog and set up the render [Music] [Music] and finally let's bring some life to the steam [Music] let's do a bit more tweaking until we are happy with what we see and I guess we can consider the sting done I really hope you found something useful in this video if you found it interesting and want me to create more content like this one please drop the comment leave the thumbs up and if you haven't already please subscribe to my channel big thanks to those who did it already as it really motivates and helps me to move forward and create more content I can share with you I guess that's it and hopefully see you in the next video bye [Music] [Music] [Music]

Channel: Grzegorz Baran

Views: 6,532

Keywords: drone photogrammetry, substance, photogrammetry, scanning, tutorial, mavic 2 pro, dji mavic, zbrush, rizomuv, environment art, vista, how to, game development, workflow, game artist, rock formation, cliff, retopology, substance designer, substance painter, pbr, texture baking, 3dart, 3d modeling, Metashape, Agisoft, Marmoset Toolbag, material, game developer, digital art, graphics, game arts, photogrammetry software, Unreal5, photogrammetry tutorial, reality capture, environment artist

Id: _WJLjOrzNsU

Length: 40min 26sec (2426 seconds)

Published: Thu Feb 06 2020

Reddit Comments

[u/brad3378]

Keep up the great work Grzegorz!

I'm still trying to take what I learned from your polarized light video and try it with my own 3D captures. This latest video also makes me want to explore color corrections.

I just ordered a light tent from Amazon a few weeks ago that features integrated adjustable Bi-color lighting so I can adapt the color temperature as needed. It's pretty cool but I don't think I'll be able to use it with polarized lights because of the reflective internal surfaces. Now I wish I would have tried something else with custom polarized lighting and polarized camera filters.

It's really cool to see our community is innovating so quickly. There's always something new to learn.

Right now, my best camera lens is non-motorized, so my old Nikon D3200 camera body can't adjust the focus automatically because it doesn't have a focus motor either. As soon as I upgrade my camera body to one with a focus motor I want to start experimenting with focus stacking because I have a hunch that it will greatly improve my capture quality.

[u/[deleted]]

Really great video! Thanks for doing the work and sharing with us!