Tech Focus: Cyberpunk 2077 RT Overdrive - How Is Path Tracing Possible on a Triple-A Game?

thank you [Music] in 2018 when Nvidia announced the Turing GPU architecture capable of real-time Hardware accelerated Ray tracing it was an announcement that promised Ray tracing as the future of game Graphics the idea that at some point in the hopefully not too distant future Ray tracing would be the primary way we light worlds found in AAA games in 2023 with cyberpunk 2077 overdrive we are seeing one of the first fulfillments of that promise where the near path Trace lighting of the overdrive mode propels the visuals of the game into the stratosphere it might seem like Destiny fulfilled when I put it like that but it's really anything but forget not just four years ago the height of real-time path tracing in games was Quake 2 RTX in four years we've gone from that to this how is this even possible how did we get from Quake 2 RTX into 2019 to cyberpunk 2077 overdrive in 2023 I will answer those questions and more in this episode of tech Focus how did we go from Quake 2 RTX to cyberpunk 2077 overdrive in just four years there are many Trends here that enabled this to happen but one of the easiest to point out is the obvious one and that is the intense increase in performance for Hardware accelerated Ray tracing on the GPU the most powerful GPU running Quake 2 RTX at its release in 2019 is the RTX 2080 TI It managed to run the game rather while at around 1080p capable of 60fps with room to spare yet turn up the resolution to a higher standard like 4K and you can immediately see how heavy the path tracing is on the GPU at 4K here the RDX 2080 TI typically runs in the upper teens and low 20s four years later modern gpus are powering through those Ray tracing calculations much better the RTX 4090 which occupies a similar spot in the full Graphics stack as the 2080 TI shows astonishing gains over that RTX 2080 TI is typically running a bit more than four times the raw frame rate and processing a single frame in a quarter of the time this is one of the key enabling factors of cyberpunk 2077 there's a much higher base for Pure Ray tracing performance which shows up very well in pure Ray tracing workloads like Quake 2 RTX this 4X gain comes from a number of sources in the hardware itself there's of course more Ray tracing cores and Shader cores and a higher operating frequency but some of it is architectural as well for example each Nvidia architecture past touring has respectively doubled the triangle intersection testing rate in the RT core so ampere for example can test two times the amount of triangles in the same amount of time as Turing you can see this in practice when you compare an RTX 3070 running Quake 2 RDX versus an RTX 1080 TI and most rasterization titles or even hybrid Ray tracing titles the RTX 3070 and rcx 2080 TI will run neck and neck or even the RTX 2080 TI will pull ahead yet in pure Ray tracing workloads like Quake 2 RTX here there is a different Trend where the RTX 3070s performance is a great deal better than that of the RTX 2080 TI and that really all comes down to the fact that ampere is architecturally better at Ray tracing than touring the Ada Lovelace architecture found in the RTX 4090 for example further increases triangle intersection testing throughput by two times over ampere as you can imagine better Ray tracing geometry performance is going to be a key reason why older RT gpus May struggle in cyberpunk 2077 while more modern ones flourish so an RTX 2080 TI might have been able to handle multiple traces gracefully against low poly character models found in Quake 2 RTX yet it can struggle with multiple traces against a character model in cyberpunk 2077 which has many thousands of times more polygons another technical architectural Advantage found and more recent gpus is a greater amount of L2 cash this is a general Trend among all GPU vendors but Intel's Arc and nvidia's Ada Lovelace architectures in particular have disproportionately higher amounts of L2 cache than other similarly powered gpus or their most recent predecessors for example an RTX 3090 has 6 megabytes of L2 cache while an RTX 4090 has 72 megabytes of L2 cache Intel arcs a770 has 16 megabytes of L2 cache while its primary competitors in the Ampere or rdna2 families have a great deal less this is one that's a bit harder to quantify with some performance numbers on screen as you can't just turn on and off L2 cache in the GPU from the consumer's perspective but basic research findings in the field and Common Sense indicate that path tracing which uses a lot of incoherent Rays as in rays that shoot around in random directions while they decrease simdi efficiency simdi is the manner of process used in gpus there are a few reasons for that but low cash efficiency is one of the top factors so having more L2 cache is a boon to Performance to counteract that inefficiency there's essentially more space on the GPU to fit denser and more detailed geometry found in a world like cyberpunk 2077 to Ray Trace against a last architectural change making Hardware better in 2023 at Ray tracing in general is found for the moment exclusively in Ada Lovelace gpus and that is scr or Shader execution reordering like a greater amount of L2 cache this is another method that is used to counteract the general simdi inefficiency issues that path tracing induces basically in a game with high quality Ray tracing or path racing there are a great variety of shaders potentially for all materials in the game world for example in cyberpunk a car can have different materials for its Metals for its glass for its clear coat paint for the dashboard for the leather seats and much much more for realistic rendering it's very important that all these materials are respected and shaded correctly when light is being bounced the problem is that the Rays Bounce Around the scene in a rather random way and will access these shaders in a haphazard manner which will decrease GPU unit utilization while it occurs this can then severely decrease performance scr works against this by bundling all those disparate shaders together preventing the scattered nature of the data being accessed on the GPU cyberpunk 2077 overdrive reportedly uses this although like the L2 cache differences I was trying to explain earlier it's hard to get a grip on the exact real world performance implications in this application because I know of no way to turn it on and off but generally this is going to be another way more modern gpus are more adapt at Ray tracing than those of the past [Music] the hardware getting better in raw power through architecture and speed though is only one part of that equation which enables cyberpunk 2077 overdrive to actually exist in its current state the advances on the software side of how Ray tracing Works in real time is perhaps just as if not even more important here there have been great efficiency advancements in the last four years which have made it so that the Rays that are being traced on the GPU are being traced in a smarter and more effective way basically developers have learned ways to shoot a similar amount of rays out into a game scene while getting a much better visual return for a similar amount of performance one of these key advancements comes from rester reaster is a silly sounding word that stands for spatio temporal Reservoir resampling what does this mean well I will get into it in a bit but before that I need to describe the problem that riester is trying to solve modern game worlds like cyberpunk 2077 have a lot of light sources in them at night time in night City there could be hundreds of Lights in a frame heck there could be thousands of Lights in a frame or even worse and traditional rendering without Ray tracing this is already a big performance problem and many techniques have Arisen since the mid-2000s to enable many lights with better performance deferred rendering forward plus rendering clustered forward rendering and so on and so forth Ray tracing or path tracing has a similar many lights problem where performance can become a big deal when tracing many lights see in Ray tracing to get accurate results you are sampling in a way which is very noisy you're sending out Rays into the scene in a kind of random way to find out where lighting is and due to this Randomness the results you get back are incredibly noisy so you need many many Rays classically to get good looking results that is why getting raytraced Lighting in cyberpunk is such a challenge there are tons of light everywhere and you could round a corner and set suddenly see a whole host of new lights that are completely unpredictable a traditional path Tracer would require many rays and definitely not be performant in real time by the time it started to look good it is for this reason why Quake 2 was comparatively easy to trace Lighting in when it released in 2019 in Quake 2 there are just a few light sources in any given scene and the levels are small and pre-packaged in such a way where it is easy to find and Trace lights when new ones are about to come on screen and start affecting the pixels that the player can see the style of tracing used for Quake 2 RTX is completely untenable for good visuals or performance in something like cyberpunk 2077 that method really wouldn't work for most modern games especially big open world ones like cyberpunk 2077 so new methods had to come into existence that did not exist in 2018 or 2019 to make cyberpunk 2077 possible with its overdrive mode that is where restore comes in or in a its branded form of rtxdi the algorithm works by reusing the paths of light from previous frames or from the current frame to accurately fill in the gaps in the noise for important local light sources take a look at this example from the original restore paper from 2020. here is a previous state-of-the-art rendering for many lights in this particular scene here notice all the salt and pepper noise across the entire image and it took 20.2 milliseconds to render this would require a lot of denoising to get it to look right now here's a rester rendered image of the same thing as you can tell there's a lot less noise in this image and it also runs faster to boot rendering in 16.7 milliseconds versus the 20 of the older method so the rester image here looks quite a bit better and runs quite a bit better by essentially reusing rays in a Smart Way images like these ones you're seeing on screen from Easter in comparison to an older method would be much easier to denoise in real time for something like a game and restart is what is in use in cyberpunk 2077 with rtxdi which is handling all the direct lighting and shadowing for local lights neon signs and such without the Advent of rester cyberpunk 2077 would either require tons of more Rays to have a coherent image or require tons of more denoising to make it look like anything that you would recognize as a city either way both of those things would require quite a bit more performance and without rester perhaps cyberpunk 2077 overdrive would it even be possible in real time at all in 2023 thank you Beyond optimizations to Ray tracing algorithms in the year 2023 games in general are also doing their rendering quite differently than before due to image reconstruction and this is another element that is making cyberpunk 2077 overdrive possible right now back in 2019 when Quake 2 RTX launched dlss2 did not even exist the entirely different technique of DLS S1 was what was on offer and it was in general not impressing too many in the PC World in 2023 dlss2 is now a compelling machine learning based image reconstruction technique and image reconstruction in general has become an established pillar of modern PC gaming and it is changing the way Gamers can experience GPU heavy titles like path raced ones for example on an RTX 4090 if you were to currently max out cyberpunk 2077 you would be hard limited to around a 1080p internal resolution when looking for a healthy 60 FPS but with advances in machine learning we can now take that 1080p image process it in real time for a performance cost and reconstruct up to four times the amount of pixels and you'll see a greatly improved image quality for that performance cost and on top of that you can add machine learning assisted frame generation to further enhance the presentation in 2019 when quick 2 RTX came out frame generation was essentially unheard of for games the last compelling demo of the technology was nearly 10 years earlier when LucasArts demoed a version of frame generation for the Force Unleashed 2. in 2023 that is now a viable technology and we can take that 4K dlss image that I showed earlier and we can amplify its frame rate to increase perceptual fluidity with dual ss3 on Ada Lovelace gpus so things like frame generation and image reconstruction are making heavy Ray Trace experiences much more fluid and and detailed than they possibly could have been just four years ago so cyberpunk 2077 overdrive is the synthesis of a lot of separate elements that have evolved over the last couple years there's Hardware advancements Ray tracing algorithmic advancements and image processing advancements all of which make the experience possible as we see it today so where does the technology go from here here I think looking at the visual and performance limitations of cyberpunk 2077 overdrive will give us good hints as to where the next advancements might be happening one of the first limitations in cyberpunk 2077 overdrive that I notice is how it deals with forward rendered elements so transparent things like glass these elements are why I describe cyberfunk 2077 as near path traced as these type of materials are still not fully handled via Ray tracing like the opaque World geometry tends to be so if you look at opaque Reflections they look really great such as this reflection puddle on the ground here which does not use transparency and there in the reflection you can see really compelling lighting you can see Reflections in Reflections the global illumination is there the material response looks great in that Reflection Glass Reflections are conversely missing this realism so if you look in them like you can see here you can see the same Hallmarks of rasterization as the glass geometry itself and the Reflections in that geometry are still being lit by similar methods that the game employs when overdrive Ray tracing is set to off this makes the shading of the objects in the reflections for transparent things less realistic than the rest of the world in cyberpunk 2077 overdrive this is an open area of research but I think there's other hints in other titles as to what could happen quick 2 RTX for example had some very compelling thick glass rendering with proper refraction and caustics same with the Unreal Engine 4 title chernobylite which had some great looking raytraced glass refraction and shading I imagine this is one area that may improve with further patches to cyberpunk 2077 overdrive as yes cyberpunk 2077 overdrive is still unfinished and work in progress as Nvidia and CD projek Red are describing it as improving over time still another limitation in the current rendering is the amount of path traced bounces that are occurring currently two bounces for specular and diffuse lighting that is a lot of bounces and is actually enough from a perceptual realism perspective but there are still potential lighting conditions where more bounces could help for example in Minecraft RTX I set up a maze like entrance to this structure here where the deeper you go into the structure the more bounces of diffused lighting are needed to actually have lighting show up at the furthest end inside that entrance you can still see some light as the most extreme bounces are showing up there definitely a lot more than two bounces with its current setup cyberpunk 2077 overdrive could not represent such an extreme lighting condition in Minecraft RTX this is possible due to a technique of surface caching which is caching diffused Lighting in the game World itself so you can get some of these extreme lighting situations in Minecraft RTX it has its limitations though surface caching typically is limited to diffuse lighting so opaque Reflections like those found in metal would not have this many bounces they just have one only rough materials such as concrete will show off the surface caching behavior I'm not sure if cyberpunk 2077 will use surface caching like Minecraft does or other titles do as it is an imperfect solution to a degree for one like I said earlier it can't do specular lighting bounces so only get diffused lighting being affected by the surface cache in a world like cyberpunk 2077 with all of its metals and glass maybe that's not enough for realism similarly current surface caching techniques can lead to Greater lighting lag you can see this if you block off light in Minecraft RTX it can take a while for the light energy to disappear from your view when you do so rather based upon what I hear from Nvidia I think cyberpunk 2077 may use a different style of cash technique called neural Radiance caching this machine learning based technique was presented originally in 2021 and I think it would offer some interesting benefits for cyberpunk 2077 specifically for one the neural Radiance caching can include specular information in that cache as it doesn't have any assumptions about what type of materials are being cached against so you could see multi-bounce Reflections being handled by the neural Radiance cache this is something other techniques can't do really and this is really good for a world like cyberpunk which has a lot of synthetic materials metals and glass it would also of course provide diffused lighting information so extreme shadowing conditions like I showed off earlier in Minecraft RTX could then be accurately represented in game in cyberpunk 2077 with less noise and lag that is another thing that neural Radiance caching could help clean up noise because right now in areas that are mainly indirectly lit you can see a good amount of noise in cyberpunk 2077 Overdrive surfaces can look like they're swimming in lighting information using cached lighting information from something like a neural Radiance cache could help alleviate this noise issue completely a last limitation in cyberpunk 2077 overdrive that gives hints to the future is in performance right now the game is extremely heavy that makes sense as it is a nearly fully path traced AAA game yet I still think there's some room for improvement on the software side for example right now the game does not take advantage of omm which stands for opacity micro Maps this is an asset format that can be read by Ada Lovelace gpus which speeds up the tracing of alpha tested geometry like vegetation for example right now if you go into areas of the game world with a lot of vegetation you can see how they can be much heavier than those areas without vegetation I imagine omm could be added to an update of cyberpunk in the future but beyond that simpler speed up I'm not sure what other optimizations could be added in one thing I found interesting in making this video is that I saw all that cyberpunk 2077 overdrive runs at Native 4k on an RTX 4090 with a similar frame rate to how an RTX 2080 TI runs Quake 2 RTX at 4K so perhaps we're going to see a similar situation here with cyberpunk 2077 overdrive perhaps now the game is very expensive at 4k on Modern gpus but in four years time maybe we'll have gpus that can render cyberpunk 2077 overdrive at 4K much like a 4090 runs Quake 2 RTX at 4K that would be awesome to see and something I would look forward to but in the meantime I hope you enjoyed this Tech Focus about cyberpunk 2077 overdrive how it came to be and what it might evolve into in the future if you did enjoy it please hit that like button and subscribe to the channel if you're already subscriber hit that little bell in the corner to be informed as soon as digital Foundry posts a video if you want to help us out support DF on patreon to enable more videos like this and get access to years of our content in high quality for download other than comment below follow on Twitter as always this Alex bring you farewell foreign [Music]

Channel: Digital Foundry

Views: 259,456

Keywords:

Id: vigxRma2EPA

Length: 22min 14sec (1334 seconds)

Published: Sat Apr 22 2023