Crafting Photorealistic Environments with Houdini & USD

foreign you some versatile USD workflows using Maya and Houdini I'll be using kit bash to generate a big environment scene using Houdini's native scatter tools I'll be also using Vellum I will be using effects volumes and all sorts of little other tricks I'll be creating some hdas as well so it will be a quite action-packed tutorial class today so make sure to support me and follow me on patreon alright as you can see we are in Maya and I open up the kit bash scene I got and it is an Arnold scene so every geometry here has a Shader assigned already you can see a couple of shaders connected they are pointing to some texture on disk but before we want to convert this to USD what I always like to do I want to clean up my scene so I don't have any weird naming conventions or my file paths are corrupt or something like that I want to first of all rename everything that has this prefix for the kit bar scene in it and I use chat GPT to help me quickly create me a full loop to clean things up and I just need to run it and you can see all of the prefix and suffixes are gone from all the nodes you can see in this scene so texture notes Shader nodes everything the other step what I need to change is my texture paths are pointing somewhere they also have that name which I also want to remove so I have a different path here which essentially does the same adjust hard hard codes um the the path it finds from disk into this section here so now this is filled in properly as well um and that's pretty much all I do and now I'll just do a quick test so I'm opening uh or creating just a simple Sky Dome and I also save my scene and once that is done we can hit render and see if everything is there all right and as you can see all the textures are connected and we get shaded assets from the smallest to the biggest one is all working quite nicely so in the Arnold menu if you have that installed and you're on a pretty recent version you can go scene export and Export everything to USD like a little bit of a caveat is you want to disable a couple of things because by default it will literally ex export everything so I'll just I won't overwrite this because this is the one I was setting things up but I just created a new one called training and just make sure you disable a couple of things you don't need color managers you don't need operators no filters no drivers essentially you need shapes and shaders and that's pretty much it and then full paths export and you export this and then you get a UST file all right so all I did now in Houdini I switched my view to Solaris I have the stage on the right and my scene view on the left and all I created was a sub layer like that and I'm just pointing that to the USD file we just exported so it's The Outpost the USD info and it's the same thing as we had in Maya it's the same layouts everything is the same it's just essentially converted to a USD scene also all our materials live now here as well converted for USD it's Arnold materials so that is all in order to create an environment I want to be able to pick and place certain buildings certain foliages Flags antennas and I want to be able to place them manually and Maya potentially you would just select them and transform them where you need them to go it's a pretty cumbersome way or there is obviously you can use measure other things in Maya to do it but I like the Houdini way it's very procedural and you do it obviously through USD what you right now cannot do in Maya my script here is split in two sections the top one is the ingest where I'm actually doing lots of work to separate each element and make sure I've got proxies and shaders and textures and then the second part is actually where we do the environment stuff and I'll walk through everything in detail so you will not be missing out on anything we are starting out with creating a selection the selection a collection name is a subset and it's a similar like a Maya set and I'm just putting everything what is a geometry subset in here so all subsets essentially live under this collection name and they are being stored under collections let's see if we find that in this long list of things I've got collections here on the left and in the scene graph details view you can always see what is included and excluded in this and right now this one includes all of my subsets and a subset is essentially a geometry selection or face selection you can see all these subsets and they have different material assignments that's how this specific kit by scene is set up I'm just doing a attribute on the subsets named material bind this is used later on to actually assign shaders back onto each individual subset and then another collection of every top group of every asset is essentially in here all right and now we you can see now out top in is analysis it's essentially a pointer say okay this is now a next step in our graph you can see at the bottom of the first section there's like this top net and the top mat is actually pretty cool you can see I'm pointing it to a lop path and generate that file and you can see I'm looping over every asset in my collection that I just created so everything which has geop in the name is part of that asset collection right so I'm looping over everything and you can see now if I hover over this this Prim name is antenna a then we've got building MD or whatever so this is essentially my Loop over everything I want to go and then I'm creating a natural because this print have sometimes changes and I just want to store this attribute into a new attribute I'm creating so if you look at this guy here you can see now I've got a current print path which is the same as a prim name or the prime path here it's just a copy which I reuse in the notes above and this rub USD will actually write the prim name to disk in a separate single USD file of that asset name and this is where I'm generating a USD file per asset now if we go back to the Top Gear you can see that the variables I created in the top net will be used in here for instance in this prune here I'm literally removing everything except the current print path you can see it's it's set to deactivate the other things and put unselected it's essentially an isolate mode and this red node here it's a python script which took me quite a while to create and think about how I can do this and by default the Arnold shaders are not represented in the opengl viewer and in order for this to work you would need to create a material Network create a USD preview surface and connect everything I can show you an example in a little bit the problem though is with this ingested scene I do not have access to the original material networks so all I'm stuck with is the USD materials in here so I had to find a clever way using using python to actually generate these USD preview materials on the Fly and create USD textures and roughness Maps hook them all up into a USD preview surface and connect them to this material node and you can see once I'm visualizing this red node you can see at the bottom here in this in the details that there is now a output surface connection with a USD preview material and this took me a little bit of trial and error I was using again chat GPT to help me out a little bit quicker because otherwise I had to go through the API and find the correct calls here but with chatgpt it's quite easy to find apis and now this is literally displaying my base color texture in the viewport which Super handy then I'm just doing some cleanup because not every geometry here has the same Shader assigned because I've got all the shaders always for every asset and here's what I was saying before I'm looking at the current geometry print path and I'm looping over the subsets and I'm extracting the material names which which are on this asset and then I'm looping over every material in this slower section here and I'm deleting the ones or deactivating the ones which are not um in this on this asset and then they just get tagged off and then I'm just left over with whatever is needed and that was the material assignment stage now we always want to create proxies to work very efficiently and I was creating a sub modifying Network which is essentially a sub context which allows me to do some sub operations and all I'm doing here I'm putting in or I'm bringing in the geometry of that asset and deleting all the unneeded nodes all the deleting all the groups and then I'm scanning scattering a lot of points on this geometry and then I'm creating VDB from those points reshaping them converting them back and then poly reducing them that I just have this really ugly looking proxy which is exactly what I'm after um next up is I'm just cleaning up The Primitives and Primitives are everything which are in this in this stage essentially and I'm just defining what is a proxy for instance um once I bring that in I've got this proxy cell option here and I'm just saying okay this primitive path is its purpose is proxy that just means that whenever I change my preview to final render that they will be toggling between peroxy and render and there's a couple of things just changing the the scope types and setting the purpose to render stuff um all of that and then at the end here I'm specifying my safe path for this current USD file you can see if I expand this this goes to antenna a group a USD file if I now change to a different task um you can see that this process be re-cooked like it's restructuring it's redoing everything and now a different asset is here and you can see that this process runs over every print path and now this maybe works better yet now I can easily switch between proxy and final and this is essentially what is being done and this is the first step of my ingest so I did that obviously it took some time to set everything up and again if you want this whole setup which is quite complex especially a couple of the Python nodes in here um it's all accessible through my patreon if you want to support me over there you just will not get the individual assets itself because they were purchased and you would need to do your own purchase or create your own materials or geometries let's just quickly jump into nuke so I can show you what we will be creating right I guess you know because of the thumbnail anyways this is the final result you can see there are lots of things going on we've got obviously the buildings called props which are not in the kit bash scene it's a mega scan props got a telephone poles we've got wires everywhere we've got volume smoke um all of that so I'll be showing you step by step what happened I have a ground plane it's just very basic a plane with a mountain I believe and then I have a camera view which is just very low to the ground and building stall is essentially already my building placement you can see these are my simple buildings and more buildings being added on top of the buildings like that and it's pretty nice pretty easy to do and I show you quickly using the um the love layout node if I hook that in here and then in my layout asset Gallery what I can do I can say Okay I want to have these buildings as my Essence objects essentially I want to do a couple of things before I don't want a scale I just want to keep this very simple scale from original size aligned to normal that's fine asset can be cycled I want to lock the scale and I don't want to save rotations go to the manipulator and you can see now um we can we already see the first asset popping through I just want to go over input instead of all this just means that Collision objects are a little bit more simple and now it's just about creating these buildings so I can click and then I can scroll to pick a different one click and you see this is essentially how I created a basic layout you can also use the line which helps as well for placement and you can essentially click and drag and it just populates the line with buildings and that's also what I did as well like that and then you can pick smaller items you can switch them to final render to actually get some textured views you can remove your your objects from here and maybe if you want to create smaller props now you can drag these guys in um activate them and maybe do a second row of smaller pieces I don't know I'm just doing it very quick now right I was I think in most cases using the place node or you can click and and rotate things I disabled the scale but we can just enable this back and see now we've got a huge Tower there um but yeah this is essentially how I created the layout and then I was going down low of the camera and then creating the camera from view but yeah this process takes some time to populate the scene but yeah you can see this is how it is technically you just see maybe four buildings but the scene itself it's actually a lot bigger there's a lot more going on but yeah I have a big fov so I think 105 is my lens yeah so it's really long lens but yeah now let's talk about the rest of the things I did so this is a volume in Center here and this is instancing volumes which I save to disk and you can see it's called Smoke chimney and we also have um another smoke chimney B I believe and they were created in the object context under chimneys it's essentially an axiom simulation with a very simple very simple Source shape Axiom solver and this is one of those smoke blooms and they look like this right and I've got two of these and I just rotate them and scale them and these are showing the older Fields but that's one of those plumes if I go back to the lops context you can see I have both in here and um you might wonder now how did I scatter them in the instancer node I brought in all the buildings and I pruned all the antennas and then I found a way to just um I obviously I deleted a couple of packed points so I just have a little bit less antennas and then using group Expressions I have picked the top points of each antenna and packed that so in the end I was just left with the top points and then I applied a random P scale so I have bigger and smaller smoke plumes and then they are just being instanced on these antennas essentially very simple way to scatter smoke they it might not be placed perfectly but I think it suits the shot or whatever that's yeah it just seems to work quite well and then the next one is dead shrubs instancing so these dead shrubs are coming from my acid Library from Mega scans and I have I believe under where is it 3D plants that trap so I'm using OD asset Library you can find this third-party plugin in the description if you feel like it and it just lists my mega scan assets on disk and I can drag them in and it will create me this little sub context here and within is pulling in the fbx data and my shrubs it's loading in the LOD it's creating the Arnold material and this is what I was saying before you have to create this USD Network in order for this to actually preview and opengl and all it's doing it's creating a preview um USD surface and then I have color roughness and opacity I've got a control which I just need to change the name to the current Shader and then hook both up to this output thing and then once that is connected I need to make sure that the output is assigned to my Geo I'm output being this node here and then on top level now you can see if I go smooth shaded this is now my opengl view of a single shrub and back if we jump back into the point instance so I'll just show you how I did scatter them I create obviously brought in the geometry I just kept the ground subdivided a little bit more and created some density noise and I also brought in all the buildings rate them flat onto the ground and then created a density attribute on the flat buildings and the reason for that is I didn't actually transfer and whatever is a building I don't want to grow shrubs around the build underneath the building so that's my method of doing that and then a custom build of custom curves to clean up some Pathways did some blurring of the attributes nothing crazy brought in the shot camera to remove everything which is outside the camera so I do not scatter points which are out behind behind essentially the viewer which we don't need and then I'm using scatter align to create all these points I have my orientation on them so I have random tilts essentially and a random y-axis rotations random p-scale adjustments as well and that's my points and now if we look at the end sensor but now everything is back to scale and if I look through my shot camera you can now see we've got these little shrubs here in the foreground and it's more apparent if I bring back the wireframe then we've got small little rocks and very similar setup I won't go into this so it's essentially the copy of the previous one just the different assets coming in and then same way with the grass I just did a different seed value and now we've got also Pebbles being instant on the ground and everywhere around you can see they're very subtle but in your face enough to to be to see them through oh yeah this is I I did a different method to bring in my wires um and they were created in here you can see it's a little bit more complicated but they all start out with the same way I'm pulling in the buildings and then on the buildings I am finding the top points again and you can see now on the antennas I've got my top points here I did a connected Json points uh pieces to points not to create me these lines and then I'm just re-sampling them so they're all nice and smooth pinning the points which are where the antennas are applying a Vellum cloth Sim on it and see if I go to First frame if I hit play Loop that's all that's doing so now they're full down I have a Time shift to always just jump to this Frame so it's a bit of cleanup work poly wire to give them some thickness and material pretty much that's it and then I create a component this time so it's actually written by writing out to disk as well I've got this USD file for the wires and I'm just grafting it in and placing them in a certain location in my hierarchy View bone masked pretty similar thing actually don't know where I created this guy I think it stops and then the object context actually yeah so in here I did a very basic um model of a thorn mask you can see it's super basic but essentially did the same thing with vellum just let them hang down like that some random strength so they're all a little bit hanging longer or shorter like that and then it's all like the same story write them to disk and graph them back in and now we've got the wires I'm transforming them twice so I get two different ones small Geo stuff is a little bit more interesting because it creates all these little props but it's the same way as before I'm you pulling in my mega scan assets same setup as before I just did that several times and merge them into a single output node which then allows the instancer to randomly pick one piece from the subnet Hero Rock is my foreground Rock which is here nothing crazy I just copied and place it um I created a volume box just to give me some depth it's hardly visible in the render in the end but this is essentially it and I'm assigning vdbs to the Smoke blooms volume Shader to the Smoke blooms and the volume and then it's about a single hdri which lights the whole scene and you can now see the depth of the volume in here I can disable that and it's a little bit more noticeable towards the end it's just about creating Arnold render vars so I'm creating spec direct the fuse direct all the fun stuff you usually do I had to do a little bit manual stuff because there's I think there's some bugs with the current Arnold render settings node um not respecting custom renewal so I just recreated the render product myself um set everything up as I would and have my render settings Untitled and then my sampling pretty simple actually I forgot to enable adaptive for my final render but that's not a big issue notable adaptive and then I'm just storing the raw the final scene as the entry USD file so this scene this USD file could be rendered in any other application that supports USD and Arnold essentially and now we can quickly jump into nuke so I can show you my work that I did in here very simple setup I just have my render coming in that's how the raw render looks not too much to do honestly did a couple of few shifts just made remove the warmth slightly color correction more contrast and what did I do here just Shuffle the Z into depth then I create the depth of field it's maybe a bit much but I just wanted to direct the focus to the mid ground here to that front building and then light wrap to and make the light rep will make more sense in a sec this is my background image did some color corrections to it Hugh corrected it scale it and then merge them on top and the light wrap Pro is essentially wrapping the background light the illumination onto my foreground geometry might be again a bit too much maybe a bit too hazy but I kind of like the look in the end so I just went with that um here I created a depth pass from camera distance just to help me fog up the background a little bit so it's a little bit more tied in Distortion I like to do that because it just bends the bends the image a little bit on the edges this is what a lens would do as well this is not what a lens is doing that's pretty much what you see with your eyes probably but the lens is always a little bit distorted on the edges I created some glow plastic on top it's a little bit the same as a light rib essentially then I created highlights from glow from the highlights this always adds a nice touch I feel to metal pieces metal surfaces get this nice glow here a top grade to get a little bit more warmth back and then I lifted the black slightly just to get this more like a old film camera look essentially lifting the blacks a little chromatic aberration helps a little too it's probably if I zoom in like that it's just too strong but what it does it kind of enables or shows like a lens fringing which happens on the lens on the on the edges of a lens here's the way if things are getting uh worn apart essentially and a vignette towards the end and some re-graining and that's that's that's it so we can go back from front to back let's go uh here go maybe um full screen yeah and that is how I did this quick thing and this took me roughly let's say two days and again if you want to get the new scene files the Houdini scene files they are my patreon source files tier so if you want to support me there you get access to this and all my other previously uploaded scene files um yeah that is it thank you so much for watching and I will see you in the next one [Music]

Channel: Arvid Schneider

Views: 17,470

Keywords: vfx, houdini fx, sidefx houdini, autodesk maya, maya 2020, arnold, arnold renderer, 3d, 3d modelling, 3d texturing, lookdev, rendering, 3d rendering, lighting, 3d lighting, substance tutorial, substance painter tutorial, #ArvidSchneider, #Houdini, #USD, #UniversalSceneDescription, #Photorealistic, #CGI, #3DModeling, #EnvironmentDesign, #VisualEffects, #VFX, #DesertScene, #Rendering, #Shading, #ProceduralModeling, #DigitalArt, #AssetInterchange, #FilmMaking, #GameDesign, #CGTutorial, #3DEnvironments

Id: UVW59srZgo0

Length: 26min 27sec (1587 seconds)

Published: Thu Jul 06 2023