Path Tracing // Ray Tracing series

hey so guys my name is the channel welcome back to my Ray tracing series physically based rendering and materials we talked about that last time make sure you check out that video if you haven't already because today we're going to extend that and talk about a very very exciting concept called path tracing so what is path tracing and why is it seemingly so hard to Define I mean I did a lot of Googling before this to try and get a clear-cut definition of what path tracing actually is and there are so many different opinions for what that term actually means if we just talk about Ray tracing a lot of people Define Ray tracing as you have a ray it originally is usually at the camera it comes in it hits something and then you return a color and that's it some people call that Ray costing and Ray tracing would mean that well you don't just stop there you keep going because if you just cast a ray and you hit something you return the color that's Ray casting you've cast your array you've hit something or you haven't you return a color tracing okay so you trace it you kind of keep going and so that definition would mean okay I trace my Ray I hit an object and then I maybe reflect it somewhere and I hit something else and that's Ray tracing but in the world of Graphics in general to say something is Ray traced can literally mean as simple as I sent array somewhere to determine some kind of information so for example Ray Trace Shadows might mean that okay I've hit a spot I don't know if I'm in a shadow or not so I'm gonna Trace array to the light source meaning I'm literally just going to draw a line from the surface to the light source see if there's any obstruction and that's it Ray Trace Reflections could mean again I'm gonna Cast Away from the surface like reflecting it from The View Vector along the normal get that reflection Vector but then actually cast a ray out to see what I hit and I'm going to take that into account when I return my final color right Trace Reflections but then how do we separate right costing from Ray tracing my point is it becomes a little bit muddy when we start talking about terminology so rather than trying to give you guys what would be objectively the correct definition which I'm not sure really anyone has agreed upon I want to give you guys the Cherno definition of all of this stuff so yeah the way I look at it is Ray casting Ray tracing they are pretty similar they're pretty much the same thing maybe I would consider Ray tracing to be a little bit deeper meaning that it can bounce around a little bit but at the end of the day what I Define Ray tracing to be is I've cast array somewhere I'm going to hit a surface I'm going to collect some information from that hit and I'm going to keep going probably depending on the settings of my renderer if I have five bounces well I'm gonna bounce five times that's Ray tracing and that's what we're doing if we look at our application as it is right now now we're casting Rays from the camera we're hitting something we're reflecting them somewhere because I think we've set it to five bounces here it is bounces equals five so five times will hit something will reflect we'll hit something else we'll reflect again we'll keep kind of bouncing until either we hit the sky or we just can't bounce anymore because we've only limited it to five bounces that in my view is Ray tracing what we're doing here is Ray tracing but there's another fancy term it's called path tracing so what is that exactly well if we take a look at this image again and in fact if I just kind of freeze it here you can obviously see that this is extremely noisy because what's happening is when our array comes in and hits something what are we doing at the moment well we return the color obviously of that particular pixel right so based on the material and the lighting conditions whatever what color that pixel is but then we bounce somewhere else and where is it that we bounce well in a random direction right we come in we hit like This Is Us coming in we hit this pixel here and then we might bounce that way here's the code our right direction is just our current redirection reflected along the normal and then based on the roughness we add some kind of Randomness to it so in other words if this kind of blue sphere over here that's our floor was completely smooth if I just make the roughness zero you can see we don't really get that noise anymore because it's predictable where it will go 100 of the time but then if I increase the roughness to like 0.5 or something we get a lot of noise why because that Ray is being scattered it's not simply coming in and being reflected by this green kind of normal and then going that way and that's our perfect kind of deterministic reflection bounce New Direction whatever you want to call it every time it's now going in a range and it could go anywhere here and it will go a random Direction essentially obviously within a limit and depending on the material roughness it will go in an new random Direction every single time and that's kind of what causes all of this noise that's why this isn't smooth it's because for each pixel it will pick a completely random Direction but just one and it will just go in that random Direction and that will be it then next frame it will choose a different random Direction because the direction is random so every frame is going to be a new random Direction but if we scale it back a little bit why are we choosing random directions I mean again they are random within a cone essentially within an area it's not going to go backwards in this case it's going to be within some kind of area whose size is going to be determined by the roughness at the moment but still like why is it random does that make sense are we just trying to randomly generate an image here like what's going on well we did kind of touch on this last episode but basically the thing is if we consider a surface such as this table lots of light is hitting it from like every possible different direction there is some form of light hitting it I have a huge light over here and yeah that's hitting it but then also the wall behind me is contributing a little bit there's a window very far off that way that's contributing a little bit probably so my point is the whole environment is kind of just all contributing to the color of this table and by color I guess what I'm really talking about is what our eyes perceive when we look at it now that's the real world and if you contrast that to what we're actually doing here with our algorithm and inside our application we're doing it in reverse we're going from the camera we're shooting array so pretend the camera is my eye I'm shooting array I'm hitting this table from my eye and then now I basically need to evaluate an infinite number of directions light hitting this point from an infinite number of directions which isn't really possible so what do I do well instead I basically choose a random Direction now at the moment because we're kind of modeling this material that is based off of roughness we're not really doing Global illumination and I don't really want to get into that just here we will very soon I promise but at the moment we are just focusing specifically on reflection but what I'm trying to get out here is that we clearly need more than just a single reflection Ray we need technically like an infinite amount or near an infinite amount of them but the current setup of our Ray tracing application is we hit array we reflect how many arrays one we send out one Ray we don't just send out a billion and remember we're doing five bounces so what does that mean if I hit something else now I need to reflect again in a random Direction potentially and that's just for reflection specifically rough reflection reflection that isn't like a perfect mirror because that really is kind of one Ray what about global elimination what about light just from everywhere hitting these things these rays are seriously starting to add up that is where path tracing comes in because basically what I've just drawn here hitting this table going a random Direction hitting that going a random direction again that is called a path that is like one path that we follow and what we need to do is just evaluate lots of them because think of it this way if I choose to go in one random Direction this time why don't I choose to go a different random Direction next time and then average out the result because isn't that slowly converging on what would be the result of millions of rays hitting me and the answer is yes yes you will begin to converge on realistic lighting now there's a few things here that really need some consideration first of all the way that we're doing it here is very very Elementary we're just literally picking a random Direction like of course we're offsetting the reflection along the normal so it's not like completely random but still the way that we're picking this is kind of random but I've deliberately set it up this way because I think it's really easy to understand and it just makes everything so much more simple and then I think in the future it's also going to be very exciting when we start taking Concepts that we already know and we know the whole kind of mechanics of this and we start introducing real kind of physically based algorithms and all that stuff for which by the way we're going to need a lot of math and uh do you guys know what a better get your math yeah I know a guy brilliant the sponsor of this video I've talked about brilliant a lot in this series because they're sponsoring like the whole series why because this series ties in really really well with what they offer and what they can teach you because what brilliant is is an amazing website filled with lots and lots of courses on various stem topics and they have so many math courses that are extremely well made they're very Visual and they're very engaging they'll quiz you on what they're teaching you to make sure that you retain the information and learn and they've got courses about like everything if this kind of Ray stuff is confusing you like why what are vectors why are they reflecting what's the dot product they have courses that cover all of that stuff all the way down to this everyday math course that's perfect for beginners and their courses are also super mobile friendly so if you don't have much time you just want to learn something on the go it's perfect for that as well the best thing about brilliant is that you can get started for free just go to brilliant.org the channel check out their courses for yourself and if you like them and want to keep going but have been amazing enough to offer the first 200 of my subscribers 20 off an annual membership just check out the link in the description to get started huge thank you to brilliant as always for sponsoring this video so going back to this little diagram that I drew if we actually just get out of this and we watch what's happening here you can see that the noise pattern so like basically where like these black areas are it's different every frame why because we're evaluating a different path every frame and if we kind of watch it closely it almost resembles like what the correct image looks like so all we really need to do here is just kind of collect all of these samples and just accumulate them into one result and that's what we're going to take a look at today as our kind of introduction to path tracing so what I want to do is inside the renderer I just want to go back into the header file and then aside from having this image data I'm going to duplicate this and I'm going to make a new array that's going to be a vek4 and it's going to be our accumulation data this you can see I'm throwing in a floating Point format it's going to be for floats which is way more data than this but the benefit here is that because they are 32 bits per Channel it can be HDR we can basically hold way more data than we otherwise would in here now I'm going to go ahead and jump into the CBP file and then over here where we resize that's where we also need to make sure that we resize our accumulation data so I'll duplicate these two lines I'll delete the accumulation data here and then I'll allocate this kind of new array of width times height okay perfect next up we need to kind of keep track of what frame of accumulation we're actually on so the idea is every time we move the camera it's going to basically reset everything and then when the camera is still it's just going to keep collecting samples keep tracing paths but for that to work it's important to know what frame we're on since kind of the beginning of this accumulation and so what we have to do is inside the renderer we have to add the frame index variable which I'm going to set to 1 by default now y1 why not zero the reason I'm setting this to one is because we're actually going to use this as like the number of paths we've evaluated in order to then average them out so in other words we'll have to divide by this number kind of like working out an average if you have one sample you can just divide that sample by one but if I want the average of two numbers two and four the average will be three I do two plus four divided by two because I have two samples same kind of story here now I need some kind of API around this so that externally I can control it so for example if the camera moves I need to reset it so let's go ahead and write a function called reset frame index and that's simply going to set frame index to one and then while we're here I also want to create a struct here called settings which are going to be our render settings and really all I'm going to do here is just add this accumulate setting here and basically if that's on we're going to accumulate samples and if it's off we're not so if it's off it's going to behave exactly like it does at the moment that might be useful sometimes if you don't want it to accumulate you just want to fly through your scene for whatever reason you just wanted to construct the frame from scratch every frame instead of accumulate over several frames and then we need a way to access this and also actually we need to Store settings so let's go ahead and install the settings over here and then I'll also add a function here to get settings and that will just return our settings Okay cool so what's gonna happen then inside render this is really where we do all of our rendering so obviously at the end of this if our settings accumulate is on then we're going to do frame index plus plus otherwise we're just going to set frame index to one I want to make sure I reset this just in case we stop accumulating and then like we're stuck on frame index 10 for example we need to go back to one now since we've set up this API let's go back to our main file over here Walnut app and then what I'm going to do is just add some UI around this so when we have our render button I want to add another button here called reset that will just call the reset function inside our renderer and then maybe above that I'll just make a checkbox called accumulate and then that will be tied to renderer get settings dot accumulate so we have a way to disable accumulation and also reset from within here okay let's actually implement this accumulation and Trace some paths so all we really have to do is if we look at this function this is what calls per pixel and returns a color and then what do we do we clamp it to make sure it's in that range and then we convert it to rgba and store it inside our kind of final image data buffer so what I want to do is when we get the color the first thing I want to do is store it as is doesn't matter if it's not within this range because our accumulation buffer is capable of storing floats so it doesn't need to be clamped to that 0-255 range so let's paste this in I'll change this to be accumulation data X Plus y times final image to get with and instead of just setting this to color I'm actually going to add color to it now what's currently in there well if it's frame index 1 there should be nothing in there because we're starting again from scratch so that's the that's kind of step one we need to make sure that if we're in here and frame index is one we basically set this to zero now it's a little bit annoying to do this this way if you think about it because that means every iteration has fallen for every pixel it manually has to like set it all to zero and since it's just zero there's a much easier way to do that what we can do is actually move this outside of the for Loop and then I can just mem set the entire buffer to just be zero so if I just pass in accumulation data over here I pass in zero for my value now it does say int and these are floats but a float of zero is just all zeros in the memory there's no difference between the memory representation of zero for a float or an INT so that will be fine and then the size well the size is going to be final image get width times final image get height times the size of glm vac 4 which is going to be 16 bytes so 16 bytes per pixel so that means that if it is frame index 1 it will completely reset that buffer it'll clear that buffer to zero and then over here when we go through all of this let me just get rid of this when we Trace our path and we get our pixel color we're just going to add it into that buffer it will be adding into zero if it's the first time and that's it otherwise it will add it to whatever is already in there so the previous color basically now if we're adding all of this color together aren't we just going to get a really bright image and the answer is yes yes we will that is why we have to average out this color or normalize it or whatever you want to call it before we actually set it into our final image but it's totally fine that this is accumulating that's actually one of the reasons why we made it a float in the first place instead of just an INT as in a float per Channel instead of an INT per pixel because we want it to be kind of in a wider range so that it can actually exceed one so what we can do here is just pull out that color we can call this accumulated color or something like that that will just be whatever this is so for the first frame it will be the same as color but otherwise it will obviously be the full kind of accumulation of it and then what we want to do with accumulator color is divide it by the frame index so for frame 1 that will be one obviously and that will mean that we might as well not do this because we're just dividing by one but otherwise it's going to divide by two divide by three divide by 4 and we are actually going to get our kind of average color it's going to give us the kind of combination the accumulation of all of those samples that we've done all of those paths that we've traced and they'll kind of all contribute to the final result and then with this accumulated color we can then clamp it to be between zero and one after we've done this division because now it should be in that range unless we're in an HDR pipeline which we will explore in the future but since we're not at the moment and we're just storing this as one byte per Channel as an integer we're going to then convert accumulated color into rgba and store it back in image data and that's it that's our full kind of pipeline at the moment so let's go ahead and see what this looks like okay cool I mean that already is accumulating and looks very cool let's shut off accumulate and we're going to see how noisy this looks without accumulation and then I'm just going to check this checkbox and check this out as I just leave it running it's actually going to get cleaner and cleaner because it's going to evaluate more and more parts okay it's been another like 10 seconds so let's just zoom in a little bit you can see how much cleaner that looks I mean that's a pretty decent zoom level and it just looks very very clean especially when we're zooming down now if I play with the roughness here obviously we're going to have to reset this when we make any changes because we haven't hooked it up here but if I just play with the roughness a little bit so let's just actually put the roughness to one and reset it and see what that looks like because that's a really good kind of example of just how noisy this can get like I've turned accumulation off now and you can see how noisy this is like there's just so much noise going on but then I check accumulate and check this out cleans it right up and of course we just have much more information because we're evaluating a random path every time this is 0.5 roughness so yeah looking pretty clean and that is the power of accumulation now if I move my camera now though you can see that it almost looks like nothing's happening and then after a while we might almost see kind of two images as it begins combining both of them because of course I haven't reset it so let's see if we can do something about that if we go back to Walnut app you might remember back from the camera episode that our camera on update function if we look at it it actually has this moved Boolean that we keep track of and every time we move whether it be like via rotation or just was ASD QE the keyboard it will set move to true because we then need to use that to actually recalculate the view in the right directions well how about we just make this update function return a Boolean and well we can return false if the right Mouse button isn't pressed because obviously with our cameras are in use otherwise we'll just return moved and then if I go back to the header file and I will just set on update to return a Boolean up here as well we now should be able to to basically just say that if camera on update returns true that means that it has been updated some things changed in the camera which means that we need to go to the renderer and reset that frame index so now I should kind of get that automatically you'll still probably want to make like all of these parameters over here in this corner actually reset that frame index as well as well as like resizing and all of that but if I just move this around you can see that as I'm moving it's all just incredibly noisy but then as soon as I stop it cleans up there you go moving moving moving noisy noisy noisy then stop and you can see that it starts accumulating and we get a much cleaner result over time this is kind of looking cool as well this reflection which used to look like this but then now if we let it accumulate it looks really smooth and our frame time is about 130 milliseconds or so on this laptop which is about 8 frames per second not very good however now might be a good time to actually start talking about multi-threading and optimization and making this thing a lot faster on the CPU before we eventually move it to the GPU now the final thing I want to mention is that when it comes to accumulation and how long it takes us to go from a completely noisy image to something that's way less noisy but maybe not perfect and then eventually when we just like if I left this running for an hour it would probably look pretty Flawless however are we really doing the best job when it comes to sending out these is Bounce raise and the answer is absolutely not we're just sending them out randomly this is where the quality of your ray is really going to be important it's almost like quality over quantity here we can just brute force it and send out a lot of random rays and eventually we will convert on a correct image true however what if we were a little bit smarter about which way we chose to send out our array instead of just being all random about it because at the end of the day the quality of your samples the quality of your bounces the quality of your rays and parts does play a major role here as well and there's plenty of things that we'll explore down that Avenue in the future such as importance sampling but that's it while I've been talking this is cleaned up even more looks absolutely beautiful hope you guys enjoy this episode if you did please don't forget to hit the like button really getting into some exciting stuff I would love to know what you guys think we should work on next should we do some optimization and actually get this running at like like 30 frames per second also I'm pretty sure we could probably achieve that or should we get into like some more graphics features such as Shadows Global elimination we could do some soft Shadows with area lights that kind of stuff let me know in the comments section below also don't forget to check out brilliant using my link in description below and I will see you guys next time goodbye [Music]

Channel: The Cherno

Views: 48,409

Keywords: ray tracing, cherno, thecherno, ray tracing series, ray tracing c++, ray tracing tutorial, raytracing, tutorial, programming, gamedev, graphics, rendering, vulkan, opengl, photorealistic rendering, path tracing, rtx, c++ tutorial, optimization, performance, rendering equation, open source, GPU rendering, math, maths, ray tracing math, explained

Id: NIpC53vesHo

Length: 23min 21sec (1401 seconds)

Published: Thu Nov 10 2022