Hey everyone, Grzegorz Baran here. In this video I am going to present a 'single image to PBR material' workflow with the Substance Alchemist and compare it to the photogrammetry based one. As a subject for this comparison I picked a brick wall surface. This type of surface should be a good example for a comparison as it is quite obvious when something isn't correct and I believe its a good case to rate overall reconstruction quality. I captured this wall a while ago to make a video about photogrammetry reconstruction with the 3DF Zephyr. For those interested in photogrammetry workflow used for this capture I put the link to the video in the description. Basically I took 164 perpendicular to the surface images. 2 of them were captured from a long distance so they covered larger chunk of the wall to be useful for the Image2Material based reconstruction I calibrated the color for each and processed delighting pass using photo editing software
Next I processed them all with the 3DF Zephyr to get the high poly model to bake PBR data from. I built a low poly model in ZBrush to use it as a canvas for PBR baking
and baked 4 textures: an albedo, normal, height and an ambient occlusion maps with the Substance Designer baker. When done I used Substance Painter to manually remove seams and tile the texture I processed baked maps with the Substance Designer to generate roughness to finally receive a high quality fully functional PBR material of the brick wall.
And this is the result which I am going to compare with the one I am going to get from the single image based reconstruction with the Alchemist and its AI reconstruction algorithm. This entire photogrammetry process took me about 3.5 hours. I spent 10 minutes to capture the surface with the camera 10 minutes to tweak and process all RAW images with the photo editing software 150 mins to photogrammetry reconstruction with the 3DF Zephyr, 5 minutes to build a lowpoly model using ZBrush 10 minutes to bake 4 maps with the Substance Designer, 30 minutes to manually tile the texture in Substance Painter
and about 10 minutes to generate roughness map in Substance Designer. All maps generated this way turned out to be really good, but the complexity of the process and the overall time needed to obtain the material this way might seem to be a little over the top. So what if I can simplify the process and use quicker and simplier technique? And instead of taking hundreds of photos I would capture just a single one? Is the single image based reconstruction technique based on a Artificial Intelligence good enough to replace photogrammetry approach?
What is the actual difference between two techniques? I hope this video is going to answer all this There are many tools which we can use to process a single image and turn it into a PBR material. To name a few these are KNALD, CRAZY BUMP, B2M or ArtEngine. So far each of them delivers pretty similar, quite crappy in my opinion and fake when compared to the photogrammetry based materials results.
But there is one tool which stands out from the crowd and this is the Substance Alchemist.
The difference between Alchemist and its competition is that the Substance Alchemist offers a bit different and modern approach as it introduced an Artificial Intelligence based algorithm to deal with reconstruction.
I tested it while ago and even if the results were promissing and better to any competitive solution, they were still off and not very reliable. But recently the algorithm received another buff and turned Alchemist into probably the best available option to proceed with the 'Single Image to PBR Material' reconstruction.
So lets give it a try and test the improved Artificial Intelligence in action. When done I will compare the result from the Alchemist with the one generated using photogrammetry workflow to see the difference. As I mentioned, to capture the brick wall I took 164 images. Most of them were taken from a close range which is useful for photogrammetry reconstruction, but wont work for the Image to PBR reconstruction as we need at least 2 meters to be covered in a single image. This is why I captured 2 additional photos from a longer distance Lets open the Alchemist and bring one of these
Next lets switch the preview for 2D map into Scan1. As long as we dont generate a PBR material we wont see anything in 3D preview view. For now lets switch the view to 2D only as this way we are gonna get better image visibility.
and apply a crop layer to crop the area we want to be turned into a material
To apply any layer we can use an ADD A LAYER button, or hit the SPACE BAR. Both open list of all available layers we can apply. Bare in mind that these layers aren't destructive
and can be reorganised or even removed at anytime. Unfortunately in the current version of Alchemist we need to set resolution of the image we use manually.
This way we can preview it in proper aspect ratio, which is extremely important when we want the cropping to be perfectly square. In this case we need to set it to 6001 by 4000 pixels And we can crop the area we are going to turn into material. To make sure we get the same coverage as the photogrammetry based material lets open it and count the bricks
and use that count to define similar area for cropping on our single image. Its 22 so we should limit the crop coverage to cover vertically exactly 22 bricks in Alchemist To make seam removal process easier, lets adjust the crop rotation and make sure that the top and the bottom edges end between bricks. To make sure the CROP is perfectly square lets hit the MAKE SQUARE button and continue with adjustments until we are happy Next lets switch the 2D view to the material outputs. We should have 22 bricks from the bottom to the top edge. Next lets hit the T to preview tiling and adjust the rotation to get even better horizontal alignment for future tiling Lets adjust distance between bricks on top and the bottom edge so it is also easy to tile. I think it is good enough. Next lets bring the 3D preview window and apply the image to material layer. There are two types of Image to Material algorithms we can use here. The old one which is based on a B2M algorithm. I would say that results coming from this one are quite decent sometimes but usually they are really off when compared with the quality and accuracy of the photogrammetry based materials. So lets delete this one and use the second one an AI powered Image to Material layer
When applied the Alchemist use the AI algorithm to interpret the surface data and return complete set of PBR materials. Tthis one takes slightly more time to process the input image when compared to the B2M but he overall result as you can see is way better.
Lets adjust setting a bit to get even more from it Lets tweak generated roughness so it isn't as shiny The height seems to be really good but feels like some areas were misinterpreted and need to be calm down a bit. but lets leave it for now and fix it later. As I mentioned this is totally non destructive workflow so we can go back anytime to any of layers and readjust any setting we want, and when done the final result gets auto updated. As you can see the brick we have doesn't tile and have very noticeable edge seams. Alchemist offers an auto-tile tool called 'Make it tile advanced' but there is no chance it can handle this type of manmade surface with such strong pattern.
This auto-tile layer is based on system of masks, supported by the content aware algorithm to get better transition between each.
It works quite well for generic surfaces like solid sand, concrete or rock, but totally don't for human made surfaces with strongly defined pattern. So lets drop this one for this and try to tile it manually using CLONE PATCH filter.
We can change the brush size using brackets on a keyboard Next we need to find the place to copy from and clone the values. This DOT marks the source area we are going to clone from. We can change its position by holding an ALT key and hitting a different place with the LMB. Next we can paint around the seams until we are happy with the result. This tool also isn't destructive so we can modify the paint and change painting setting anytime we want. If we want to clone from a different place, we need to apply another CLONE PATCH layer. Lets do this also to to remove a seam from the bottom and top edge Any Clone Patch layer can be also readjusted using the layers own setting Ok, I think we did pretty decent job in here.
Feels like the height map and maybe the color map need a bit more cross surface consistency. Lets bring the Equaliser, activate the equalisation for the height map and tweak radius values to balance the height consistency. We can do the same for any other channel of our choice to decrease overall material repetition Finally, we can bring the delighter layer which removes shadows from the albedo map.
Unfortunately we don't have any control for this one so we can ether accept the result if we like it or remove it completely if we don't.
Since we equalised the color a bit, I cant see any noticeable shadows to remove and since the delighter seems to rise overall saturation a bit, lets drop it for this material and go with the result we already have. Its a shame that delighter still lacks any control as I can see how useful it can be if it does.
Lets take a final look how it tiles and export the result for comparison I can see that AI struggles a bit with dark spots like this one, so lets quickly fix it with the clone patch. I think its way better. So since I believe we are done in here, lets export the final results so we can compare them with the photogrammetry based material based on the same brick wall. This is a texture-set generated from a single image with the Alchemist. An albedo map, a normal map a height map, a roughness map and an ambient occlusion map. And this is the result when we apply them all to the preview sphere and simulate PBR lighting. I have to admit its actually pretty damn good and the Alchemist AI did a really good job here. Its not perfect and definitely has some weird surface misinterpretations, feels a bit blobby and could be sharper, but come on, it was generated in just a few seconds from a single photograph. To compare there are maps generated using the photogrammetry based workflow An albedo, a normal map, a height map, a roughness map and an ambient occlusion map. And the final result applied to the same preview sphere with exactly the same PBR lighting this one seems to be sharper and has way better defined shape. Honesty I think both renders look really good.
But if we compare them with the real wall we can clearly see that the photogrammetry based one is definitely closer to the original while the Image to PBR based one is a kind of fuzzy interpretation of it. The photogrammetry-based textures are cleaner and have better shape definition.
The Image to PBR based textures are more fuzzy, a bit blurry, and lack of some medium and small details. So even if both materials look good, the photogrammetry based one, definitely better and more accurate reflects the reality while the one based on a single image just pretends it. Regarding the time, it took me about 15 minutes to get a fully functional PBR material with the Alchemist while I needed about 3.5 hours to get the material using photogrammetry based approach. From the other side, over 2.5 hours of this time was pure processing time without any actual work. So are materials based on photogrammetry better to those generated from a single image? Of course they are. But I can see that the
quality difference between them isn't so obvious anymore and I can see areas where one technique might be a better choice and areas where the other technique makes more sense.
So there is no simple answer. If I would need to deliver the texture-set in high production quality with the Alchemist I would probably add a few more tools to improve the workflow and get a higher quality output. For example I would probably introduce a photo editing tool and tweak the input image before it is processed because everything depends on its quality.
I would also pick a different tool for seam removal and tiling, especially for this type of surfaces. These are things which would affects the final production time and when introduced the time difference between both techniques wouldn't be so obvious anymore. Also the downside of the image to PBR material reconstruction when compared to the photogrammetry approach is the output size limit as we are simply limited by the resolution of the image used to generate the material.
We don't have this problem with the photogrammetry approach. So, the Alchemist
seems to be a very interesting tool with a lot potential. Its Image to PBR algorithm is great, but some of its features could be really better. I still wouldn't use Alchemist to tile my photogrammetry based material for fear of losing its quality. The manual seam removal in Painter still seems to be the better option, even at the price of additional effort and time. Delighter is very promising and does the job pretty well but lacks of control. It definitely can be useful for delighting photogrammetry based albedos but I would probably generate the map and blend it in the Substance Designer to control its power instead of using direct Alchemists result.
I think Image to Material approach with the Alchemist is a great optional tool to consider in material creation workflows. It can be very useful if we don't need to reflect the reality in 100% or when we simply don't have enough data or time for photogrammetry reconstruction. But if I would have to choose between the library of photogrammetry based materials and a library of Image to PBR Material based materials I would still pick the first one without any hesitation. But I believe that there is no and never was 'the best' or 'the only one' tool to do the job, and we have to know them all to have a choice. Alchemist is definitely an option to consider. I hope this comparison is going to help to get better understanding of pros and cones of each approach and in result let us to pick the better tool for the job. I really hope someone has found this video useful. If you enjoyed it and want me to create even more content like this one, please leave the thumbs up, drop the comment and subscribe to my channel. Big thanks to all of you who did it already. Take care and stay safe And hopefully till the next one! Bye!! :)
