A New Way to Approach Photorealism in 3D!

water realism like most 3D artists has always been the ultimate goal for me to achieve in all my renders it is usually renders that achieve them that I put in my aptly named big inspiration folder up here in notion that I look into from time to time to inspire and remind myself where I want to be in the future but photorealism is a very tricky thing you know actually no it's it's not photorealism it's us it's human beings human beings are just too good at telling what is real and what is not we're just too good at telling when we are looking at photographs taken from a real camera as opposed to renders generated from a computer right you're just too good at it we are just too good at it [Applause] foreign [Music] istic than photographs right so don't you think we should be implementing them somehow in our renders I think so so I tested out this fresh new workflow that I had been thinking about a lot in the past week or so and after a lot of trial and error and a lot of effort and patience put into it I think I have come up with a really interesting video idea for you here today so let me just give you a brief summary of everything we're going to be doing here today and just basically list out all the advantages I found while working with this workflow the first thing I noticed is your scenes are going to get super optimized with this method since a lot of the heavy lifting will be done by the photograph itself and hence your 3D scenes will automatically become extremely lightweight and super responsive and with extremely low render times as well not just that you're gonna learn some really important principles about color grading and image blending and lighting that you've never had to tackle before in your all-in full-on 3D scenes principles that you can take back to those full-on 3D scenes and apply them there also you're gonna finally become an actual VFX artist because we're finally going to learn how to composite 3D animations into these free stock footages get a tiny little taste of what is like making high quality Hollywood level VFX for once we do that by learning simple motion tracking and object tracking techniques which if in case have scared you in the past I promise you I'm gonna burn that fear away in this video because they scared me too but oh my God is it easy to do that in blender it is literally a combination of what five buttons or so and you'll instantly have a fully tracked footage in front of you in seconds it's crazy how simple it is we'll talk about all my failures along the way too and not just shove all my successes in your face like most YouTube tutorials do because I feel like they teach you a lot more than successes do at the end of the day so we'll take a look at them as well and all in all just learn a new approachable way to tackle photorealism and not just photorealism we learn a new way to tell fresh new stories as that's what the ultimate ultimate goal is in my opinion even above photorealism so yeah that's what we're gonna be doing today so let's find only get to it we're gonna start with the still render learn some really important fundamentals and then get to the fun part and do the same thing in a stock footage later on so let's start with this photograph for now and the first thing we need to do is understand the perspective of this image understand what kind of camera this photo was taken with and we can do that very easily using an amazing free software called app spy once we import our photograph into F spy we are treated with these X and Y axis all you got to do is align them with the axis of the photograph you chose and that's it you're literally done you can enable any of these 3D guides in here to check if everything is aligning well or not if in case you're having trouble aligning the X and Y axis press shift while you drag these endpoints to get a zoomed in version of the Crosshair location if in case your image is still misaligned enable this from third vanishing point option to get access to the z-axis as well to align the perspective even better so yeah do that and save your F spy file and to bring it all back to blender all you got to do is install the F spy importer add-on which will enable this F spy option within the import menu in blender using which you can import that file that you just saved and as soon as you do you will be treated with a new camera in your scene that will very conveniently set the photograph as the background and will obviously set all the important parameters like focal length based on the perspective you just plotted in F spy at this point you should have a perfectly aligned camera in your scene and you know how we know it's perfectly aligned is because when you summon a plane into the scene and scale it or grab it in your scene it will just line itself perfectly with the floor right out of the box which is great so we move on to the next step which for me with experience and practice with this workflow now has become to add a sphere into the scene and place it where I plan to put my subject and then make the plane below it a shadow catcher so it can actually mimic the floor within the photograph and then comes the most difficult part of this process which is finding the right kind of hdri that suits the lighting in that photograph and I think a lot of people over complicate this part but lighting at the end of the day just does two things it uses bright regions namely highlights and dark regions namely Shadows so all you got to do now is rotate and scale and displace that hdri till you're perfectly aligned with the shadows and highlights of that photograph which is definitely easier said than done but that's where the sphere helps you can clearly see the bright and dark regions on it so the eyeballing aligning process becomes much easier what I initially did was I imported the 3D model directly into the scene and tried lighting the scene using that but that over complicates the sliding process trust me reading the highlights and shadows actually becomes harder than it has to be so A Primitive shape like a sphere is the way to go in my opinion here so yeah this is where you scramble through your hdri folder and find an hdri that fits your photograph this is where you put all the hard work in you can actually even use the sky texture that is available within blender it's way more versatile than any hdri available in the market but it definitely needs a little extra patience than usual but irrespective of what you choose to do you're eventually gonna find the perfect hdri and when you do your blade literally done eighty percent of the work now it doesn't matter what kind of model you put into the scene now it's gonna fit right into it I started with this amazing 3D model of a robot made by Nina upon sketchfap that just fit perfectly into this scene not just that I also found this giant snave made by Al diesel that also kind of worked out with the composition of the scene and to add some chaos to this Serene photograph I also found a flashbang made by Nikolai that also kind of looks cool in here so as I said nail the lighting down in the beginning and after that it's just about having fun with it all but here are also a few tips that will go a long way in this workflow first is don't just put a simple plane in there as the floor or as the shadow catcher to be precise make sure you mimic some of the environmental aspects as well especially if the shadow of the CG object is going to fall on some part of the photograph like for example here in the robot if I rotate the Sun a little bit you'll see the shadow is bound to hit this concrete flower box and if the shadow doesn't adhere to the shape it's falling on it's gonna break the illusion again the human eye is just too good at picking up of these aberrations so an easy fix would be to just model a cube in there to mimic the shape of that box doing just that will help sell that effect way more than it did before so you gotta keep that in mind and the second tip would be to match the color of the Shadow catcher with the color of the actual floor it's mimicking because that can impact the color of the Shadow that's falling on it which is pretty important if you think about it and if you have the patience project the texture directly onto that plane if you want that's going to yield an even more accurate result but just the color will do fine as well and the third tip would be to use the Hue saturation nodes on your model textures when I said that if you nail the lighting down your models will just automatically fit into your scene without a problem I was kinda lying to be honest you're still gonna have to do some housekeeping using this Hue saturation note this is how the robot look by default when I brought it into blender a little dull a little dark for my taste so I plugged in a hue saturation node in there and cranked up the saturation and value fields to integrate the robot a little better into the photograph so yeah keep these tips in your mind if if you do end up giving this workflow a try and yeah it's not just for landscape photographs you can make anything with them like here I put a tiny person in there on the keyboard which I think is a cool idea and again you're not limited to these specific still images you can do all of this on a portrait photograph as well like I did here with this young gentleman I installed the face Builder add-on that allows you to align CG faces with real faces and that opens up a whole new space for this workflow you can achieve some really crazy results with this free add-on so that's another interesting aspect of this whole idea and yeah one more thing try and get as close as possible to the photograph here in blender itself because you can't expect to just deal with it later on in a photo editing software like Photoshop try and get as close as possible here in blender itself with the lighting and the colors and then take it to photoshop to just nail the illusion home in the post-processing stage the levels filter and the Hue saturation filter are your best friends again put a black and white filter on your image and then use the levels filter to match the black and white values of your CG part of the values in your photograph your eyes alone are more than enough to blend them and then match the colors of the two images using the Hue saturation filter it's gonna take some tweaking and some patience to get there and you will only get better with practice so just keep at it until you reach there because once the images look well Blended you can put a color grid over the whole thing now add some noise filters and some dust particles and some overall blur on them to gel them even better together and that's it that's all there is to it so all you gotta do now is go pick a photograph from any of these free stock photograph websites like unsplash and pixels.com and just make something cool something you have never thought you could before by the way as a beginner pick a photograph that is actually easy to understand if you pick photos like these that have a thousand different light sources and ambiguous Shadows it's gonna be really tough to replicate them here in blender and you know how I know it will be tough because I made that same mistake myself I downloaded a bunch of photos I thought would be cool for a final render but didn't think a lot about how the lighting was in them or what the perspective of the image was and hence I feel a bunch of times because I picked the wrong kind of photograph not that you can't remake them in blender it's just gonna take a lot more time and effort to get it right and I lack that kind of patience so I suggest you pick photographs that are easy to understand easy to break down in the beginning so you're not discouraged by this whole workflow before you even start giving it a try but just in case you're still struggling with the whole picking the right photograph thing and you're just not able to nail the lighting down no matter how hard you try I've got one more excellent solution for you on polyhaven.com when you go to download an hdri you get this 3D option on the left which you can use to pan around the hdri but also you can select an angle and save that hdri image at that particular angle which then you can bring back to F spy and do all that alignment stuff that we did before and eventually have the same situation going on in blender but with the perfect hdri already ready for that particular scene because you snapped the picture from it it will just give you the exact required lighting and the perfect values with no effort at all so that's another way to approach it they're actually paid and neurons like hdri maker that allow you to project an hdri on a dome and thus do the exact same thing within blender with even more Precision so that's another available option for this workflow so check it out if you're curious talking of curious let me tell you all about today's sponsor skillshare let me start off by telling you everything that I've come to learn in life through this amazing platform I've taken classes about cinematic color grading and resolve I've taken Advanced Lightroom courses Advanced Photoshop courses concept art courses and not just digital art oriented classes I've taken piano courses and music production courses screenplay courses and so much more what I've done is I've put the skillshare app right in the same folder as all my entertainment apps on the phone so whenever I'm drifting off to these apps for some procrastination I more often than not end up in skillshare and I'm not gonna lie I'm a curious person I like to know everything there is to know about everything and skillshare just feeds that Curiosity to no extent there are thousands of perfectly curated short and snappy courses for literally anything you can think of public speaking productivity and time management Finance management marketing or even organizing and decorating plants in your house or making the perfect cup of coffee in the morning it's an endless ocean full of knowledge and information for the Curious ones out there so check out skillshare from the link in the description below the first thousand people to use that link will get a one month free trial of skillshare so check it out before the deal Runs Out okay let's move on to the interesting part of the video now which is doing the exact same thing but with a video footage for a video we don't need any external software like f spy blender is more than capable of doing that internally through its motion tracking tools so let's start with this airplane window shot I found on paxus.com and try to track that so to track a footage you gotta open up the motion tracking workspace within blender which is super easy to do once you're in there you just gotta open up your clip and in the end panel you'll clearly see the FPS of the clip so make sure your project FPS is the same as that next you gotta do two things first press the set scene frames button that sets your project keyframes to whatever is the length of the clip you're using and then press prefetch so blender can load that clip into memory so that you get a nice and smooth playback throughout the process now there are a lot of fields in here and all of them have different kind of purposes but for the purpose of your first tracking project let's just focus on the important things starting with this tracking settings panel that actually has some presets for you if you'd like for blender to set everything up for you but let's do it all manually for now and only focus on the most important fields in here starting with this motion model drop down this setting is just so blender can understand the clip a little better for our clip here today we can clearly see that there's no perspective change going on here there's no complex movement happening there's no skew there's no crazy rotation and scaling happening in the video it's a very simple straightforward camera motion so we can just choose location from this drop down because that's the only thing that's happening in a video clip now these other options are for when there are a lot of location and rotation and scale changes happening in your clip not just that it has an option for clips that have a clear perspective change and also there's an option called app find that is for when there's just a lot of changes happening in your clip including skewing and perspective changes along with all the location rotation changes as well within the clip so set this motion model based on what I just explained next step in the tracking workflow is to find high contrast points within the clip like this tiny black dot or this tiny little screw here on the turbine and you can do it all manually by control clicking on any of these high contrast points but why do it manually when blender can do it all for you so you simply press the detect features button here and blender will find all these high contrast points for you but it's definitely not going to be enough we need a few more tracking points at least eight if I'm being very specific so we open up this tiny little menu that just popped up and set the threshold field here to something like 0.001 and distance to something like 75 to get a lot more tracking points in our clip we're just making the blender AI more sensitive to high contrast points through these fields and also don't forget to check this normalized checkbox in here as well what that does is it makes sure that these dots that you're tracking don't get lost in situations like save the sun suddenly Shines on them and they get a little brighter for a few seconds the normalized checkbox make sure it doesn't lose the tracker just because the dot lost its initial dark value so check that checkbox for sure but now if you've got to this point you just gotta press the track forward button in the timeline and blender will just track all these dots for you and once it's finished you can play the footage and you'll clearly see that most of these tracking points stick to their position perfectly but if you open up the graph below you'll see there are clear spikes in here for some crackers that have lost their way in the middle which can happen the computer doesn't promise a perfect result so it becomes our responsibility to delete them so we just find all these outliers and delete them with the button X so we don't have any further issues going forward now that that's done it's time to go to the solve tab where we finally finish things off you're gonna see a lot of stuff in here as well but we're just gonna focus on the important things which happens to be these refined check boxes we need to check the focal and checkbox for sure in the optical center one too because different camera models can have different Optical centers and focal length of course so definitely check them right away there's also these keyframe A and B fields just above them and there's no need to go too deep into what they are and what they exactly do but all you gotta know to do is set these values to wherever your clip has the most movement or perspective change you're giving blender the most difficult part to solve through these keyframes because if it can solve this difficult part it can obviously solve the easy Parts as well so in our clip we can see that the maximum movement in the plane is between frame 1 to 60. so we set these fields to 1 and 60 and there's nothing more left to do but press the solve camera motion button this is when blender finally starts to understand the perspective of the video clip once you press the button blender is gonna do its thing and generate a solve error value that tells you how well the tracking process went lower that number better is your track the most ideal number for a 1080p footage is below 0.5 pixel and for a 4K footage is a value under 1 pixel and we're really lucky to get a number already below 0.5 pixel here mostly because our footage is super stable but in most clips that won't be the case this value could be as high as 20 pixel in some cases and it is then our job to bring that number back down to at least below 1 pixel now how do we do that it is pretty simple remember how we went to the graph and deleted those outliers that had huge spikes that's exactly what we're going to do now as well but on a much more granular level using these refine tools that blender has if you click on this clip display drop down here and click on the info checkbox you see that each of these markers have their own solve error value and our job through the cleanup tools is to delete the markers that have really bad solver values all you got to do is press on filter tracks and then open up this tiny little menu that just popped up and enter a value like 15 pixel so blender can select all the tracks that have a solver above 15 pixel none do in this track so we go down to 10 pixel zero track is there as well so we go down to five and now we have some trackers selected our job is to delete them just like we did with those graph outliers before and press the Sol camera motion button again and observe that solver value again you gotta keep repeating this process until you at least get to below one pixel and that's it that would be the basics of how to track a footage in blender now all that's left to do is just go to the bottom of the scene setup panel and press set as background and set up tracking scene so blender sets the whole scene up for you and yeah there are still Fields like set origin and set floor and set wall under the orientation tab here that are also really useful in a lot of instances but let's keep it simple for now if you have issues with your track I'll link around three really good tracking video tutorials in the description so you can dig deeper into the subject later on and I know I said this is all going to be so super simple but I'm sure all this tracking jargon still looks too intimidating to anyone who's never done it before but trust me you do it once and you'll know what I'm talking about when I say that this is one of the easiest things that I've learned in blender for a while now so please give it a try if not anything else in this video because if you do at this point you should have a fully tracked footage and you just gotta repeat everything we learned in the still imager interface find a good hdri that suits our clip align the Shadows place the shadow catcher bring in the subject play with its Hue saturation values just do the exact same things we did before to blend it all together and yeah I didn't mention it before but you can also use the exposure and Gamma values in the color management app too if you want extra control for this blending process and also as I said before once you have the lighting all set up it's just about having fun with it now try all kinds of different sculptures and creatures and whatever tickles your mind at this point because the difficult part is already over and yeah what we did in the previous process was solve the motion for the camera as you can see here in the end panel if in case you have a situation where you know all the sensor details and focal length of the camera and you just want to track an object in the scene there's also an option for tracking specific objects within a clip here which follows the exact same workflow but just for an object instead and once you do that all that's left to do is parent your CG element to any of these object trackers that come up to blend the overall motion of the two elements together and make sure to find a good tutorial for that topic as well in the description below if you want to delve deeper into that as well but I guess that's it for this breakdown I think a point worth mentioning before we end this video is a better way to try out this workflow is to not use these stock photographs and clips we randomly found on the internet but actually go out and shoot some pictures and videos yourself but come on I know it you know it most of us are just too lazy to go do that but hey it's still worth mentioning also as I said before choose clips that you understand choose Clips with simple directional lighting that you can easily replicate in blender especially when you're starting out don't go choosing crazy ass stock footages that are super hard to track try and keep it simple and one more important thing actually the most important thing in this workflow is to make it about the story and not the photo realism per se photo realism is a catchy YouTube term but this video is more about exploring a new storytelling tool there's no point in doing all this tracking and plotting and all this hard work and at the end of the day just placing a cup and a saucer in the middle of a table and nothing else it's a cool image but it doesn't tell me a story don't just put a rock in the middle of a forest and call it a cool render try and tell a story with these photos and videos a giant turtle or a crab on the edge of a cliff tells me a story a tiny guy on a keyboard tells me a story even a giant banana outside an office building tells me a story not a cup and a saucer or a rock in a forest though don't let the photo or your CG element become the focus of the final render make it about the story and that's when you will have succeeded in achieving what I wanted to convey in this video so yeah go have fun with it send me the results if you end up making something cool I love it when you guys do that so my Instagram is totally open for that subscribe to the patreon if you want to request a specific kind of video thanks to the people who have already subscribed subscribe to it I'm indirect to you for showing such immense support to such a tiny little Creator here on YouTube you have no idea how much it helps in continuing to do this for a living check out the skill sharing too do not miss out on a great deal but for now this video is done let me know your thoughts in the comment section below correct me in case I said something wrong and I will hopefully catch you guys in the next video bye foreign

Channel: stache

Views: 120,917

Keywords: photorealism, photorealistic render, cinematic render, how to make photorealistic renders, hyperealism, motion tracking, object tracking, how to put object in video, how to track a footage in blender, motion tracking tutorial in blender, fspy tutorial, blend 3d object with photo, blender 3d object with video, blender 3d object and image, blending 3d and 2d, vfx tutorial, vfx tracking tutorial, free motion tracking in blender, realism in blender, realistic 3d render, realism 3d

Id: zIUCoFPjJ2A

Length: 21min 42sec (1302 seconds)

Published: Thu Jun 15 2023