Dear Fellow Scholars, this is Two Minute
Papers with Dr. Károly Zsolnai-Fehér. Today I am very happy. Do you
know why? Of course, because we are going to talk about an incredible
AI light transport simulation paper. You see, these simulations normally take
place by creating a little virtual scene, and we would like to see what this virtual
scene would look like in reality. We can do this by using a technique called
ray tracing, which simulates the path of millions and millions of light rays to
create a beautiful image or even video. This technology is also used for simulating the
path of soundwaves in some modern speakers, and even in Apple’s new Vision Pro headset.
So, millions of rays, right? Sounds great! However, look. This is anything but a
beautiful scene. Why is that? Well, before we simulate enough light rays, which may
take from minutes to hours, we will get a noisy image. This image will get less and less
noisy over time, however, we don’t have hours of time per image. In a video game or any kind
of virtual world, we prefer real time results. So, what do we do? Well, we start noise
filtering. These are techniques where we can show a noisy image. And,
magically, they clear up this noise by imagining what this image could
depict. Some of them are incredibly good, and many of these AI-based techniques are so
fast that they even work for video. Loving it. However, not so fast. Unfortunately, if we
add volumetric effects like haze and smoke, things get a lot more difficult. Look. This is the
noisy input with smoke. And a previous technique does…this. Whoa. That is not great. Well,
do not worry, because this technique doesn’t yet understand smoke, just hard surfaces.
So let’s see what this previous technique with additional training on volumes can do. Ouch.
Well, smoke looks a bit more like smoke now, but unfortunately, the whole scene got destroyed. So,
is that it? We can perform incredible ray tracing on these beautiful scenes, but only without
haze smoke and similar volumetric effects? Well, don’t despair quite yet, hold
on to your papers Fellow Scholars, and have a look at the new technique. Wow! This is way better than the previous technique.
Actually, it is not perfect, but so much closer to the true simulation result. This is the one that
would have taken up to several hours to compute, and this is what we now get almost immediately
from the new AI. My goodness! And once again, this is compared to the previous method
that was trained on volumes too. And this was not just by chance, when comparing other
cases, the new method also comes out ahead. So, if the new one is so good, this previous one
must be from super long ago, right? Some ancient paper. Well, is it some ancient paper? Nope.
Not at all, this is the Neural Temporal Adaptive Sampling and Denoising paper from 2020. Such
incredible improvement in just one paper. Wow! And hold on to your papers Fellow
Scholars, because here comes the best part: all this runs now not in hours, and not even
in minutes. What you see here runs in real time. It can really take these incredibly
noisy sequence of images, and my goodness, it reconstructs these beautiful videos from it
that are closer to reality than ever before. I absolutely love this. It almost feels like
we are living in a science fiction movie. And can that really be? Yes, it gets even
better. The technique decomposes these images into surface and volume components, so
it can separate the opaque objects in the scene from the smoke itself. Is that good? Well, to
say that is good news would be an understatement, because look! It can even reimagine the
scene as if the lighting was different, or we can even play with the smoke plumes
themselves. So, from now on, we will likely be able to enjoy these beautiful games with
simulated smoke. Bravo! What a time to be alive! Also, training this neural network took only
24 hours on a single NVIDIA graphics card, that is insanity, and that just needs to happen
once, and then, everyone in the world can enjoy it for as long as they wish. And it didn’t
even take looking at millions and millions of noisy and clean image pairs to train
this network, it only took a bit more than 10,000 images. That is not a lot. Proper learning
from very little data. I am out of words. And when watching all these beautiful results,
if you feel that this ray tracing thing is pretty cool, and you would like to learn more about it,
I held a Master-level course on this topic at the Technical University of Vienna. Since I was always
teaching it to a handful of motivated students, I thought that it’s not right to have the
teachings only be available for the privileged few who can afford a college education. No. The
teachings should be available for everyone. Free education for everyone, that’s what I want.
So, the course is available free of charge for everyone, no strings attached, so make sure to
click the link in the video description to get started. We write a full light simulation program
from scratch there, and learn about physics, the world around us, and more. If you watch
it, you will see the world differently. Thanks for watching and for your generous
support, and I'll see you next time!
Real time AI-manipulated video is going to be insane with AR.
Everyone will be hot, and cities will be forests.
I cannot listen to more than a minute of this guy's videos. I just can't. Why, does he, talk like, that, aaaand, aaaaaaand aaaaaaaand
edit: WTF I just found a video of him presenting at a conference and the flow of his voice is COMPLETELY NORMAL AND NATURAL IN IT. So he chooses to speak like this on purpose in his videos? Idk why but this is seriously messing with me
I’ve never gotten nauseous from someone’s voice before, weirdest feeling ever, car sick like
This guy's voice sounds AI generated...
Research done by Nvidia makes it harder for other companies to catch up.
Holy crap this guys voice is annoying and distracting. He needs to change the way he speaks.
Just mute the video. I would hate to talk like that
I love Speedy Gonzales' videos!
That narration was reallyyy haaard tooo listeeen tooo