AI is transforming the world and computer
graphics is no exception. Five years ago we introduced DLSS, which revolutionized graphics
with better speed and better image quality through artificial intelligence. Since then, the
AI model behind DLSS just keeps learning with new capabilities such as Frame Generation,
increasing rendering speed by up to 4X with excellent image quality. Today Neural Rendering
takes the next big step forward with DLSS 3.5 DLSS 3.5 improves Ray Tracing with a new AI model
that is more accurate and more beautiful than traditional rendering methods. Let me show you how.
First, we need to understand how a Ray Traced image is computed. The game engine has the materials and
geometry for the scene but that doesn't tell us how the scene looks because we haven't computed
how all the lighting interacts with the scene. To compute the final image with Ray tracing
we send Rays into the scene to interact with the lighting and geometry, but we can never send
enough rays into the scene to understand exactly how the scene looks because there are so many
pixels and because the rays don't distribute themselves evenly throughout the scene. There's
always holes in our understanding of every scene. Even offline Ray Tracers have to deal with
this problem and we do this by using denoisers. There are multiple kinds of denoisers for every
frame that are combining information across pixels in the frame by blending them together and
across multiple frames by accumulating information in order to come up with our best estimate
of how the scene looks. Denoisers have a few common challenges. Denoisers accumulate pixels
from prior frames in effect stealing rays from the past in order to increase detail but they
do so at the risk of introducing ghosting and removing dynamic lighting effects. For example,
here you can see ghosting that's introduced when the denoiser grabs information from the
past frame in the wrong place. And you can also see in this example that the global illumination
effects in this image were removed by the denoiser. Similarly, reflections can have lower detail
because the denoiser blends information across the frame. The detail in this reflection was reduced
because the denoiser blurred pixels together These days Ray Tracing is followed by upscaling
and that makes the job of the denoiser even more challenging, because the denoiser naturally
removes high frequency information in order to make a smooth image. Denoising is a difficult
task and upscaling is a difficult task. With AI, we have the opportunity to bring them together
and train one model that's able to use all the available information to solve both problems
together. With DLSS 3.5 we are introducing Ray Reconstruction, which runs on all RTX GPUs to
provide the best image quality for Ray Traced effects. By incorporating additional inputs from
the game engine and a new AI model that does both super resolution and three reconstruction at the
same time. DLSS 3.5 is trained on five times more data than DLSS 3. This was necessary because of the
diversity of Ray Tracing effects that the model needs to recognize and work with. We've trained
this new DLSS model to recognize many different Ray Tracing effects to make smarter decisions
about temporal and spatial information reuse, and to retain all the high frequency data
that's necessary for high quality upscaling. Ray Reconstruction is smarter than denoisers.
the DLSS AI is trained on a huge data set of images created using an offline rendering process
with far more computation than could be available in real time. The AI then recognizes certain
patterns that correspond to effects such as Global Illumination and uses information from
it's training process in order to reconstruct a more realistic and dynamic image. DLSS Ray
reconstruction generates higher quality Ray Traced images. For example, here we're comparing
"DLSS Off" to "DLSS 3.5", and you can see that DLSS reduces ghosting and improves the dynamic lighting.
You can see reflections can be much sharper using Ray Reconstruction even in movement. Creative
applications have a wide variety of content which is challenging for traditional denoisers
because they need hand tuning for each scene. As a result, you get sub-optimal image quality
when previewing a new scene in a creative app. With DLSS Ray Reconstruction the AI recognizes
all types of scenes and so you can get much higher image quality when you're previewing
a scene before committing to a final render. Altogether DLSS gives you several AI powered
options to increase performance, enhance image quality or both. It's all possible thanks to the
specialized tensor cores in every RTX GPU. Let's see how it all comes together. We have a scene from
Cyberpunk 2077 in RT Overdrive Mode. It's beautiful but it's not playable without AI to improve
the experience. We start by enabling DLSS Super Resolution. DLSS Super Resolution reconstructs a
4K output from a much lower resolution input and provides a huge performance boost and great image
quality. But we can do better. Next, we turn on DLSS Frame Generation which analyzes sequential frames
in order to create additional frames that further increase smoothness. Finally, we can turn on DLSS
Ray Reconstruction which further improves image quality for Ray Traced effects, and in this scene
also improves FPS just a bit. The reason that can sometimes happen is that we're replacing multiple
denoisers with one AI model. In general the speed of games using Ray Reconstruction is going to be
about the same as the speed of games without. Five years ago we started a revolutionary journey
to redefine graphics with Neural Rendering and Artificial Intelligence. DLSS has come a long
way in five years and today's most immersive and realistic experiences now rely critically
on the power of AI. Yet, the transformational impact of AI is just getting started. We can't
wait to see where we'll be five years from now.
