Advancements in AI with DLSS 3 in Unreal Engine 5.2 | Inside Unreal

around the world Houston reports a good deploy to ensure no recontact with the International Space Station since that's orbit will decay in a few weeks where it will then enter the Earth's atmosphere and burn up hi assistant Houston this is Commander Diaz do I need to breathe close to our trajectory evening Commander negative on that clear sailing as far as we can see down here it was any cause for alarm you know we'd see it too your crew members can keep sleeping tight well I'm seeing something out there I can't make it out but whatever it is it's getting closer [Applause] [Music] okay yes copy Commander Diaz do you copy Houston you're not going to believe this I'm picking up transmissions on the ham radio that sound identical to the suits that experiment in the debris it's an oil and spacesuit I'm not sure I'm hearing you right repeat that Commander suit set I'm seeing suit set you're mistaken Diaz is that rear of the atmosphere and burned up years ago it's impossible yeah I know it's impossible but I know what I'm saying it suits that it's come back and it's not just in orbit it's hit it right for the ISS UE I wasn't making any sense I'm in the sky I need to alert the club yes [Music] [Music] [Applause] Commander Diaz do you copy damn hi folks did you hear the news starting with Unreal Engine 5.2 The unreal editor for Mac OS is built and distributed as a universal binary providing native support for past generation Intel based and current generation Apple silicone-based Max in a single package learn more about these Tech developments at unrealengine.com feed find out how metaverse entertainment use metahuman and Unreal Engine to create a natural believable and Charming virtual K-pop band creating a new wave in the genre that has garnered over 21 million views learn the choreography of the development of Maeve on the feed while you're there be inspired by the developer interview with dog how games and how it overcame Monumental challenges to development and stayed true to its original ambition for level zero using their Ingenuity perseverance passion and the tools the studio skillfully leveraged within the Unreal Engine ecosystem cute monsters treasures and a franchise that has enfralled fans for 36 years the latest in that franchise Dragon Quest Treasures developed by Square Enix and Jose discover how they utilized unreal engines blueprints sequencer and more to bring the spirit of treasure hunting to life stepping over to this week's Community Spotlight with this vibrant and stylized environment from senior 3D environment artists Salvatore Gambino and the team behind the upcoming game from eluvium Overworld in this Fantastical setting you can see how they have strived for a cohesive vertically designed Monument of a map let them know what you think of brightland steps over on their art station speaking of stylized environments that reach for the heavens environment and lighting artists Francisco Rivas has unveiled their latest exploration into stylized pieces creating everything from scratch utilizing zbrush quixel mixer blender and Unreal Engine to realize their inspiration learn more about living in the sky on the forums follow and feel the road of merchants traveling across the land and Seasons with a futuristic twist take a trip to senior environment artist Victoria zatharotnia's art station to dive into how they created adventures with Comfort using Maya substance zbrush and Unreal Engine for the art station challenge medieval back and forth thanks for watching this week's news and Community spotlight [Music] hello everyone and welcome to inside unreal a weekly show where we learn explore and celebrate everything unreal I am your host Tina and today with me I have a couple of the incredible team from Nvidia who are going to be going over some truly incredible things if I might say so myself so to start us off um Richard why don't you tell us a little bit about yourself oh um Richard I think you might be muted potentially that's what Chad is saying to me soon enough does that work um is that it yes I think so there we go no did we tell me right now okay we're good now I can be heard it's working we're there yes yes you can be here okay it's okay it's normally me that's muted it's in fact chat right now is even like wow Tina it's not you for once yeah oh no worries no I thought we had this working a minute ago but uh if it's working now that's cool okay well yeah so uh my role at Nvidia uh my name first off my name is Richard calgill I'm uh the RTX evangelist at uh Nvidia uh I specialize in the Unreal Engine so my background is in game development um I spent a lot of time as a lighting artist and environment artist um so uh that that's that's sort of my specialty and um that's that's what I tend to understand the most and uh yeah today we're going to talk about uh uh DLS S3 in uh the context of uh various unreal five projects awesome thank you so much for going through that twice I really appreciate it and then um Zach would you like to tell us a little about yourself as well sure my name is Zach um I come from a background of visual effects worked on Films commercials music videos all that good stuff uh have some game development experience working in VR and now I'm currently at Nvidia working alongside Richard as a product marketing manager whatever there we go the Dream Team yeah awesome well I know today we have so much to dive into and look into but I believe that you brought a Sizzle for us to see at the beginning of it all right we did awesome let's go ahead and pull that up shall we thank you [Music] [Music] foreign I feel like I shouldn't say this because I feel like I'm not allowed to play favorites but the Nvidia says those are some of my favorites they really are thank you well yeah a lot of people well yeah a lot of teams a lot of people worked on all that content you see there um I mean you know it kind of visually summarizes a lot of what we're trying to do at this company right is which is to Advanced Graphics especially uh the realm of real-time Graphics so yeah yeah it's but it's it's it it really Pops doesn't it it's it's a nice to watch yes it definitely does so um before we dive in for anyone who might not be familiar do you mind kind of walking through what dlss is and what they can look forward to in regards to the upcoming third installment of it yeah of course um uh you can switch to my screen uh if you want I'll make this visual too um I have uh the uh unreal five uh City sample um running I think I think one of the best ways to to talk about the lss3 is to is to show um what it gives you like you know what are we trying to achieve and what is it going after um uh uh DLS S3 if you want to get technical is um basically uh three Technologies um uh there's a super resolution aspect that's the upscaling and uh image enhancement um a a method that dlss uses so you know we call that super resolution excuse me and um yeah that uses deep learning models to uh uh uh uh like I said upscale and uh improve the final image output quality um uh and uh another key component of DLS S3 is frame generation so that's uh basically the doubling of frame rate by um uh uh using again AI algorithms to generate um additional frames um so and that's a fairly complex operation but we we can get more into those details as we go um and then the third component to this is a technology called a reflex and uh reflex is all about um uh reducing system latency uh the act of uh generating frames using deep learning um and um uh you know sort of sort of uh creating new frames out of thin air means that the system is going to introduce a little bit of latency in uh what reflex does is claw back that that latency Edition to hopefully get you to like a a net zero loss of of latency and I can demonstrate that here uh with this scene so um yeah here I have the the city sample um uh this is running on a 4070 by the way so it's it you know it's it's high-end Hardware but not the highest in Hardware and you can see that it's getting you know I'm at Native resolution um no DLS S3 and it's getting almost 60 frames per second this is pretty good um uh uh but yeah it's uh uh that's where where the frame rate is at without DLS S3 um now um I have this on the tab key so I'll I'll swap to it I'm going to turn on dlss super resolution just the super resolution part of DLS S3 and you can see that took my frame rate from UH 60 frames per second to about 80. so you know about a 33 increase so that's pretty cool and then by turning on the full the lss3 super resolution plus frame Generation Um I take my frame rate up to 140 in this case uh the amount of Gain Is Gonna you know vary by the project uh you know some will be a little bit more a little bit less but this is about a you know uh 250 percent increase in frame rate um uh and you'll also notice in the top left corner um I'm tracking overall system latency um so if I switch back to Native rendering right that's my system latency with sub 60 frames per second rendering and then full dlss3 is about the same maybe it is going to depend like you know it might be a little bit more a little bit less depending on the scene and and the project and so forth um but you can see that the system latency is very close to Native um when you know everything is added up together so uh but yeah like at a much much higher frame rate oh yeah any any questions at this point or any I mean because you know the scene is really cool right and it's fun to just run around in it um yeah I mean the first I think we can all agree that this is basically sorcery that you just performed in front of us um so we're working on our Blackmagic skills yeah yeah they're they're top-notch yeah you're doing good yeah a very difficult scene to to render and simulate right because this is um this is the uh the the smaller version of the the city level I'll just zoom out here um you know this is It's a fully simulated environment right with I don't know how many you know hundreds or thousands of of vehicles and and you know characters all walking around um you can see them you know down there in the distance uh yeah yeah and it's uh you know the frame rate's holding um latency isn't dramatically increasing you know or decreasing it's holding at about Native levels um I'll switch back to Native yeah I can see that if you're okay with answering real quick um I think everyone's wondering what your overall setup is I know you said you were on the 40 70 but do you mind also letting us know um what resolution it's at and all that kind of good stuff yeah it's I'm in 1080P at the moment but that's that's really just for the broadcast uh um I mean you know I I you could you could run this at 1440 and it would scale reasonably well right like uh with this Hardware you're you know you'd still be above you know 100 frames per second uh at 14 40. um I don't know the exact frame rate or what it would be um uh but but it's it's it's right around there yeah I think the biggest thing with when when we decided to use this demo is we wanted to have something that everyone has access to so we didn't want to use any special cases like we were special we have access to this Hardware or these you know we we optimize the scene this you know anybody can grab the city sample and um you know a 4070 card especially as well is uh is a kind of a nice point for for a lot of people to uh to be able to get their hands on right uh to be more specific about the hardware this is running on a a core I9 system but it's an older CPU it's not the latest one I I don't know the exact model um but it's got that uh uh and the 4070 is an Fe you know 12 gigabyte card so um uh uh fairly you know um yeah nothing like it's beefy right the 1470 is pretty beefy but it's not like it's not the highest end card that's currently available mm-hmm I also I do think it's worth pointing out when we were talking about this beforehand you mentioned that this is this is even the smaller map yes it's possible even at an even larger scale there is a crazier yeah uh the larger betray which is I don't know how much bigger it is it's like it seems to me like it's at least eight times this area it's just enormous um I uh to be honest I wasn't sure if it was going to uh successfully compile because that I mean that is it's so extreme right but that's that's sort of the point of the larger map is that it's it's meant for like you know really high-end hardware and that sort of thing um but yeah this one uh you know as big as it is it still fits very comfortably on a uh on a 12 gigabyte card um oh yeah I was uh but it always makes it just a little bit nervous right because this is this is um this is a nanite scene uh you know it's there's billions of triangles being rendered um you know it's fully Dynamic uh the uh the amount of simulation going on is uh crazy uh and um I uh once I saw that it was working I I I turned on um Hardware retrace uh Lumen so it's got uh uh raytraced uh reflections Hardware raytraced Reflections and GI going on but that's the only part of it that's uh using Hardware retracing um uh the rest is just uh uh the default scene basically yeah awesome um also just to clarify for anyone in chat who's wondering um so dlss 3 is not currently out yet but this is the this is upcoming right right yeah we're we're in um we're in beta internally um and we're we're close to release but uh not yet we don't we don't have a ship date or release date yet no exactly did you want to add to that at all no yeah you said it um yeah the release is upcoming but the team is working hard and excited to get the product into the hands of everyone so that they can you know start building and optimizing games which is what everyone is waiting for yeah I can um if you want I can I can walk through some of that um uh that setup and procedure just a little bit it's good for people to see exactly you know sort of the nuts and bolts we're we're uh we have everything pretty uh pretty locked down as far as what the release is going to be so um you know it shouldn't it shouldn't uh shouldn't change too much yeah absolutely I'd love to walk through that if you're if you're up for that I think that would be fun um yeah so uh uh uh the way it works is um uh we have a I'll start with this um from the marketplace um you could search for for the LSS and that'll come up with our our dlss you know plug-in product page right and on there there's going to be an external link that'll take you to the Nvidia website where you can download the package um so yeah when you uh click that button it'll load this page here and um it's fairly straightforward there's some you know you can download uh the plugin for for UE uh there's a uh an end user license agreement to uh click through uh you know uh uh oh that's for there we go that's wrong there's the right spot so you know once I click on that then I can download the um uh uh the uh you know the version of uh dlss for whichever version of unreal I want uh we support all the way back to 426. um uh but this you know when we release this will be updated with a 5.2 zip file that can be downloaded as well yeah and so when you download the zip um it should look on the inside it should look something like this there'll be an installation guide at the top and then some folders uh we have a sample project um which is really good to go through it gives you a lot of good examples of like you know how to set up blueprint scripting um sort of uh you know some uh some good proper setup uh uh and uh then we have the plugins themselves uh like I said DLS S3 is is multiple components right it's not just one plug-in there's uh there's uh dlss this is the super resolution part of it uh streamline um contains um both the updated frame generation and reflex um and this is an interesting topic by itself uh is this this folder is called this plugin is called streamline because that's the technology we're using to deliver that um Nvidia has a technology called streamline which is it's really meant to um take multiple uh uh uh and different Technologies and make it easy for the for you to integrate them into your engine and and bundle them together in this case we're using streamline to uh bundle uh frame generation and the latest version of reflex into one plugin um and and make it very uh easy to integrate and use um uh so so that's what that is uh and then we have a sort of a couple of um support Technologies um uh but still important uh NIS or Nvidia image scaling is uh you could think of it as a dlss without the um deep learning aspect right so this is meant to be a very backwards compatible and very cross-platform uh image upscaler but it's important to uh the LSS super resolution because uh actually it has a NIS has a very good um sharpening method and when you you know if you as a developer when you're when you're building out your interface and you have like say a um a uh an image sharpener slider for the end user to use what you're actually doing is is calling NIS to do that um uh so so you know if if you want users to be able to adjust a sharpness that's how you go about doing it but it also gives you a good fallback right So for anybody who doesn't have RTX Hardware um they could you know you could detect that and um have them fall back to NIS is one of their options um uh and then uh dlss uh movie pipeline support is is just a sort of a supplemental plug-in to the LSS that allows you to use the LSS super resolution with the movie render queue system um and that's it that's all it does but if you're you know if if if you want to um uh you know use mrq to Output uh uh you know offline images say with a deep learning anti-aliasing that's another part of dlss um which is a a high-res version of dlss called dlaa and uh yeah uh uh if you wanted to use that with with mrq this is would be the plugin to enable um I'm not an mrq expert I'm not going to get like too heavily into that one but but basically you know there are all these um uh plugins that you can uh install into into your version of unreal when you get the package um and then we have a documentation folder which just you know lays out everything right there's the sort of the key things here for for most people is going to be the quick start guides uh they they have very good information um you know for uh most use cases and and tips and some some installation um uh procedures and things like that uh the quick start guides are are very high level information and then there's deeper documents like the um uh the programming guide and uh if you're interested in the legal stuff there's the the EULA document um uh and uh and so on so uh uh and I can show some of that real quickly here like here's the uh the frame generation uh quick start guide um if you just skim through it you'll see that uh you know it'll it'll uh give you some tips on uh troubleshooting uh known issues um like maybe some common problems you might run into like if it's just not working out of the box uh why that might be uh so some things you can you can step through and then some instructions on setting your up your preferences and so forth plus a lot of common uh console commands and and C bars and um uh things like that but yeah so yeah that's basically the package when you when you get it um and uh to install is actually really simple um you just want to go into your your plugins right here and you'll you do need you need to do a manual install you're downloading the zip do a manual install um but just go to like say your um wherever you've installed uh 5.2 at go to ue52 uh engine uh your plugins folder and um in here if you don't have a Marketplace folder already you just make one um that's sort of the official install location for anything you download uh through the marketplace and here I've I've already you know dropped the plugins in um sorry Richard perfectly not interrupt um might be frozen if you don't mind um just restarting that real quick so you can make sure we're following along with everything you've shown us okay uh uh okay awesome it's um it's a it's a text message kind of week you know sure yeah everything always works yeah always perfectly smoothly without incident every time [Laughter] um while we're getting that set up if you don't mind there's a couple questions I might throw your way um okay one of the first ones is about kind of a difference between DLS S2 and dls's three and they're they're kind of wondering what some of the big key differences are going to be between the two um if you don't mind kind of going into that a little bit yeah of course uh the lss2 was uh primarily the super resolution technology um uh but there are other important differences I mean starting with the lss3 we also have uh improvements to the uh underlying uh deep learning model with uh new presets that like uh different uh uh sort of optimized deep learning models that you can choose from as well as just the the algorithms themselves are uh uh dramatically improved in quality over dlss too um uh uh and and there are some other other differences like DLS S3 does over-the-air updates um that's that's a new Option um for developers so over the years is pretty neat like you know in the in the past with DLS S2 uh a developer would have to you know if we issued a new some refinement some update to dlss they would have to get that latest version of dlss and include it into their um you know their their code and then recompile a binary and and you know issue an update right another dot One update um over the air updates you don't have to do that anymore you just check the box and then as we um uh issue updates to the lss3 users take advantage of them so as we refine the model and you know do speed improvements and so forth um it would just be automatically there and the developer would have a lot less work to do uh now if if a developer doesn't want to do that if they want to lock in like what they have um just it's it's you just turn over the air option off and you don't make use of it but um like I said by default it's on and you know we think most people will want to use it yeah yeah awesome um I think we've got screen share back up and running if we want to hop back into that as well cool yeah um um let's see yeah you can go ahead and share my screen again if you're not already okay yeah so um I just you know installation stuff it's not very exciting um so I don't want to spend a lot of time here um it is necessary I just I just want people to be aware of of these things like um especially especially the documentation and the sample project once you get your plugins installed uh into the right location um which I'm not sure if that part got cut off but that was uh just plug-ins Marketplace directory you drop your your plugins in here um I think in the documentation we actually just for what it's worth we say uh uh you know put it in a Nvidia subfolder just for organization which you can do you can you know take all these and you know oh oops it's not gonna like that because I currently have the unreal editor up you know for you can you can create subfolders in the marketplace folder just just in case you got a ton of plugins in here right um uh and that's up to you however you want to organize it but um uh yeah that's where the plugins go and then um in the uh dlss3 ZIP file we have a a sample project called the alysses 3 test or ngx test for older versions of unreal and let me load that up real quick I'll show you what that's all about this is just a great way to make sure that everything is working like it's supposed to um you know this is this is a project where um you know if I if I go to my plugins uh and I search for blss you'll see everything in this you know particular project is already enabled um and I can uh I'm gonna oh so you know uh certain aspects of uh the lss3 will work in uh different viewports a little bit differently um so you can you can preview dlss super resolution in any viewport but uh frame generation will only work in either Pi or Standalone build or like in a you know shipping release build or something like that it's not going to work in the viewport it doesn't work um and under certain conditions like that so uh but but if you're in the editor and you want to preview your scene under ideal conditions uh the thing to do is to you you could use Pi but Pi has a few shortcomings where it doesn't um like uh properly it doesn't unless you like like a like like a full screen very easily or it doesn't let you set screen percentage um and preview that so a standalone game is going to give you uh the best uh DLS S3 like full editing experience like previewing all the capabilities while you're in the editor and it's got a couple other advantages too right when you run a standalone game it's a separate process so if anything was to go down with the system it's not going to take down your whole editor write us a separate thread so that's that's that's kind of a good a little bonus thing um yeah okay so here I've uh loaded up the uh VLS S3 test scene and you can like I said you can just very quickly confirm everything is working obviously this is a very very simple scene right and all we're doing is just confirming things work um uh but like if I've got the latest OS you know Windows 10 Windows 11 installed and I've got my NVIDIA drivers up to date everything should be working just fine um and we can confirm that here so you know I can I can take the LSS I can uh um I can switch it just to like standard anti-aliasing um you know and see what that's see what that's like or I can compare that to uh NIS Nvidia image scaling uh compare that to dlss you know and see what like the different quality you know modes and how they look um uh like I was saying before dlaa is essentially the LSS without the upscaling aspect right so it's using deep learning algorithms to pull out details and enhance the image but while running it at 100 and so why would you do that it doesn't sound like that would net any performance gain and it doesn't that's the point of dlaa is to give you essentially the highest quality like machine learning based anti-aliasing and image enhancement you can get without like you know say uh you know upping your screen percentage past 100 right like going to 150 or 200 percent uh dla will give you uh uh you know one of the highest quality images you can you can possibly get at Native rendering um and uh so that's that's what that is and then you see there are the different quality modes or uh the LSS some of this isn't I'm in 1080P right now so some of it's not going to look it's not going to look very good at a really low resolution um you know in fact uh uh uh at 1080p um uh we have an auto so dlss has an automatic mode where you can just set that and it will set the right um Quality level for the uh the target resolution you have um at 1080p we we recommend not going below quality level that's like you know it's either quality or or dlaa at this low of a resolution right um but we can look at that see what quality looks like um I can turn on a frame generation um and here you can see what the you can see what the outputted frame rate is it's very it's obscenely high right with a simple scene like this um tells you how many frames you're presenting uh uh you know in this case you know we're for every one uh uh uh traditionally rendered frame we're generating an additional frame um so yeah if I turn that off it goes down to 300 frames per second turn it on goes up to 400 plus um uh uh and you know over here on the side we have uh some additional uh debug information I can find some of these other bits and I I know this may not be very viewable at the you know at 1080p but here you can see like basically the the host frame per second which is the your sorry your base frame rate and then your the the frame the generated frame rate which you know uh DLS S3 is making and then the kind of scaling you're getting out of the scene right so I mean you could expect and we put this in the we'll we'll have this in the documentation you can expect that frame generation you know will give you a performance multiplier you know anywhere from you know uh at least at least one and a half times but it could be over two times so I've seen it just kind of depends on the scene and the content um uh and you know sort of how you're stressing the system but uh you you could you could yeah be anywhere from like 1.5 x to 2.2 plus x uh uh performance multiplier just with frame generation alone um but yeah this this is a a good little uh testing just to make sure that okay I've got the plugins installed everything is working I'm set to go like if it works here it should work everywhere else hey Richard do you want to kind of mention the uh that stat info window if people don't see it what's the uh the shortcut to bring it up oh which I couldn't hear everything he said the stat window in case people are wondering if uh if they can't get to it oh sure sure um let me get back in into this scene just real quick so I want to show that we do have a uh there is a um uh a toggle to turn uh this on and off you'll see it at the bottom there the control shift home uh shortcut um if you want to if you want to get rid of that this um this debug information on the side by the way will only show up in like a development mode or in the editor it won't show up in a shipping game even if you've got the option turned on um that's just it's really just for your development purposes um and uh will be the settings in the project so under uh yeah settings project settings you scroll to the bottom there's an area for plugins and from here you have preferences for both dlss and dlss frame Generation Um and you can set those things up like uh you know do I want to be able to see uh uh the LSS in my editor viewport and I've got a toggle for that and you'll see it'll do a quick refresh right but as I turn that on and off so you can see um what the visual differences would be like or for frame generation there's a a load debug overlay setting and that's how how that debug overlay um toggle works right you can set it there yeah with this is the one detail out of all these things where if you Mouse over here you'll see that you need to uh um if this is off by default you need to um your uh oh sorry you need to do an editor restart in order to get it to load but then once you do that it'll work from from then on forward excuse me so but yeah there's some interesting uh uh project settings uh here in the uh uh so the preferences that you can set uh including the allow the over-the-air update like like I said this will be on by default but you can turn it off if you want um and uh you have independent controls on that for both uh super resolution and frame generation so you can set over the air updates for one and not the other if you really want um and you'll notice uh this is one of the cool new features in dlss super resolution is you have these uh these presets that you can configure um so uh all this is is uh uh uh as we've been developing dlss super resolution we've come up with a different sort of uh optimized machine learning models or different resolutions and different configurations um and the we think the defaults are very good you shouldn't need to mess with this but if you really want you can change the presets to one of the other ones now uh like you'll notice that they have sort of opaque name sorry it says preset a b c it's like what is that all about um if you go into the uh let's let's see let's go into the oh let me pull up the documentation for that so many windows all at once oh yeah I it's it's uh there's a lot to it but yeah um so there's a uh a dlss programming guide in in the documentation um that details exactly what those presets are um in fact if I just load this up this gets very very detailed right this this document is largely if you're a coder and you're um you know your managing your own code and managing that integration and you want to do a lot of custom work um but it's got a few interesting nuggets for for Content people like the preset uh values so if I in this document if I search for preset um I can go to the preset section and uh you'll see here we've defined what these what these are um uh uh yeah these were just basically different uh uh deep learning models that produce slightly different results like one will have a slight advantage on um like lower resolution inputs one will have a an advantage on fast moving images um some of them are technically outdated right like preset a it was the older I think dlss2 um a trained model uh that we were using previously um the latest version of dlss3 defaults to preset D for perf balanced and quality and preset f for Ultra performance and dlaa but like I said if you want to customize that change one model to another one this defines what they are and then that's where you set that setting uh just a fun footnote you'll notice preset e is unused I think it's like you know in the course of of doing um you know machine learning and and deep learning training you you try out um like like a certain AI algorithm uh and not all of them pan out so I think that one was was uh put to pasture for some reason uh so that one's blank but um you know uh uh basically uh yeah that that's just like you know defines like how you can uh um uh change these and and control them like what they do and so forth yeah um but yeah anyway so um yeah you can set all your preferences for the plugins from here and then we do a couple of local overrides right like if this is for your project we for your you know you're working on a team and uh uh the the project is set up a certain way but you can do local overrides if you want things um just a certain way on your machine so yeah um but yeah that's that's the test project and um you know you can see I've loaded things up and everything was working um I just want to call out one other thing in this project uh and this is largely everything I'm showing you here is applicable to the current dlss2 as well so um you know I mean DLS S3 it adds frame generation and improved um you know uh super resolution and and uh you know a newer version of reflex but the 90 of what I'm talking about also applies to the you know current released version as far as like how you interface with it and set preferences for it so um it's a uh it's going to be aware of all that if you if you weren't aware those things were there um uh yeah so so this interface that that lets you select and you know control all the aspects of DLS S3 um we have a you know a a good blueprint example here that's commented um that uh if you're curious about any of the bits like how we are controlling the a modes or setting a resolution scale um uh it's all right here um uh including the you know the specific uh dlss blueprint scripts um you know you access those in in a logical way that you'd expect you know like I can search for blss and get my dlss functions or um you know because frame generation and the latest version of reflex are housed in streamline I can search for streamline get all those functions technically you'll see little little tidbits here like uh uh frame generation in in code is referred to as dlssg and that's because we hadn't yet it was it was named in code that way before we had a product name right eventually it became frame generation but during development I believe it went through you know different sort of code names and the lssg was just one that was decided on at some point um awesome yeah uh but you'll see stuff like that so they're fairly easy to find and fairly logical and there's not too many controls um but like if you if you want to do you know like really Advanced scripting with good fallbacks a lot of this is just very copyable right like this is ready to go as far as just like you know take it put it in your project integrate it to your um your uh your your existing UI and hopefully that makes your development life a little bit easier um but yeah I just strongly encourage people to to check out this project um for uh very good and uh proper scripting setup absolutely yeah best practices and all that yeah um and yeah from there uh I think I just well um I I did want to show one thing in the documentation that was pretty interesting stuff um I believe in the frame generation quick start guide we have a we have an image here talks about what DLS S3 is doing this is also on our website um but it's just it's just a good image because because you might be thinking to yourself okay I get it like it's you know we're upscaling and rendering but exactly when is the frame generated how does that work um this this visually outlines that pretty well right so if you're um rendering your image at just hypothetically fifty percent uh input resolution performance resolution right and then upscaling that if you're rendering the the scene the frame at one quarter of the total pixels and then upscaling that um for your super resolution image and then the frame generated image happens after that where frame generation works by actually generating this um the first image and then it generates the third frame and holds while it generates uh the in-between frame and then displace them and we do that because uh there's a bunch of information it needs from you know this next image right in order to generate the the in-between image um you know motion Vector data and a bunch of other stuff so it it it's it's it's looking ahead holding generating this Frame and then displaying and this is not uncommon in graphics anyway right like it's very typically when you're rendering images to screen you're not um you're not generating the image and then rendering it immediately there's usually like a one to two frame hold anyway and exactly how much is held back is sort of uh uh uh I don't know it's like sort of an area of uh ongoing uh uh work research Improvement that sort of thing um so this is not this is nothing too crazy um uh but you know because of the uh uh uh added there is uh some added latency that this introduces that's where the reflex part comes in to uh claw back some of that latency reflex basically works by um reducing what we call a GPU back pressure um if you look at your total system latency uh there is there's a chain of events going on right like you there's there's latency when I uh click my input device whether it be a controller or a mouse or a keyboard um you know latency from that to the computer to the CPU um from the CPU to the GPU as that gets that information gets processed um and then there's there's latency from the GPU to this to the display that's a whole round trip you know going through multiple components that sort of thing and we can't control all parts of the PC right but what we can control and influence is the the GPU portion of it so um that's that's really just what reflex does is its whole intent is to look at um any place where there's some inefficiency in the in the GPU chain and shortened that that that total time that round trip a little bit um and uh like I said you know the the the working goal here is to um you know hopefully once you've got frame generation you know doubles your frame rate but maybe maybe add some latency and then reflex closet pack and then you know the the net effects should hopefully be that you're not making things um any worse for yourself um uh it you know it's good like I said it's going to depend on the project some you might add a little bit of latency some you might actually net improve latency um but uh yeah that's basically DLS S3 at a high level and and um everything it's it's it's doing uh you know to generate um uh uh the image as you see um and oh and this is interesting too in the quick start guide there are some just some comparative images here let me pull them up full screen um just want to show this real quick um this is uh and this is a 4K uh Native image so 4K 100 resolution um uh dlaa in fact uh and you can see it's it's uh only running at 16 frames per second but that's that's this is an extremely complex scene it's actually using uh nanite and like retraced everything multiple I think about two dozen light sources very complex shadowing um Lumen like the pretty much the whole kit uh and uh you know so not a not a great frame rate now but but keep in mind 4K resolution right there's there's a lot going on here um this is the same scene um uh running with a super resolution dlss super resolution uh in Ultra performance mode um so just to flip back and forth between those two images because they they look nearly identical there are minor differences I mean it kind of depends on of course it's going to depend on exactly uh how you nitpick the details but if you look at here's the native image the laa and then here's the the 4K or the 4K Ultra performance image running at about four times the frame rate of the Native image um you know that's that's a pretty good jump um uh and then this scene here same thing but now with frame generation added and uh you it's it's a little hard to see in uh you know in the screen share but this is running at 100 frames per second so uh this this is a really good example of a scene like this where you know you you start off with something that's nearly impossible to render right this um in fact I can pull this up in real time so you can see uh literally what I'm I'm talking about so yeah you can see there's uh uh you know every light in here is a light source there's um a very complex almost uh path tracer-like uh you know quality Shadows going on because there's a lot of very complex Ray tracing um this is a scene in many ways that's that's not designed to run well it's just running at a you know a very high quality um uh uh but um you know if I turn the FPS counter on uh and uh I believe the LSS is already already present now I'm not going to go down because I'm in 1080P I'm not going to go down to 33 resolution that's I think that's a little too low for 1080p but I will try 50. see what that quality looks like um you can even compare it to say you know 100 percent uh Native resolution right visually so that's that's 100 1080p wow and you can see 23 FPS and here I'll go to Performance mode 50 percent it takes a moment for it to refresh um there you can see the you know the kind of uh performance increase you can get out of that and uh you know even if that's not your this is just one of the cool things about um uh you know taking advantage of dlss from an editing standpoint because it even if if this is not say the final resolution that an end user sets for their for their game maybe they want a higher quality it could at least help you from an editing standpoint right like you well you're moving the scene around and there's like really complex shadowing going on I can set my screen resolution down maybe to a lower resolution than I I even thought was possible and use it to do uh like really fast complex scene editing um uh uh you know make my um my development workflow a little bit better so um yeah there's lots of potential uses for uh um you know super resolution technology that's in dlss3 I've actually been talking for a while I don't know if we need another break or need a uh you know if there's more questions oh we have a ton of questions but I've just been enthralled with everything you've been going through so cool awesome um yeah we can we can dive into a few of those in the meantime but thank you so much for walking through those different scenes and I think especially that last one really helps to put in perspective the insane difference you can get in frames and all that with I mean that scene was beautiful no matter what you did to it so right right yeah it's uh I mean you know it's it's it is one of those seeds that like I said it's not really tuned for performance there's a bunch of things you can do to some a scene like that to make it run faster but it's it's in its current form it's really meant for high quality right like not it's not quite path Tracer quality but it's it's in the ballpark of or getting close to so yeah it's um but but that's that's that's sort of an extreme example now if if you're um as a game developer you probably would create a more optimized scene um and you know be probably a lot more mindful of those things and uh get potentially even much higher frame rates um than what I'm getting here on a 40 70. uh uh and of course there I wasn't even fully taking advantage of the full dlss3 right I was just using the super resolution part um uh you know that that scene will go like you saw it'll go from like 16 20 FPS over a hundred um we need to take advantage of the full DLS S3 um yeah so yes it's possible to go much further than that especially if um if you're starting with more optimized content from the from the beginning yeah absolutely mind-blowing honestly um all right I've got a few questions I'm gonna throw your way um some of them might have some slate repeat information just in case I know we had a few people come in a little bit late who might have missed some of the information in the beginning so I figured we could kind of rehash some of that but um the first one I want to ask is um would you say this is closer to machine learning AI rather than say what people might take as generative AI since I know we've kind of been splitting AI into different categories as it's been advancing yeah I think so um I I mean I'll admit I'm not like a I'm not a neural learning uh systems expert I'm not a a machine learning expert but um these are um uh uh dlss is uh is is it's a series of you know uh deep learning algorithms that have been trained on a wide variety of images to understand how images should look once you um uh upscale them and you start with a lower input and upscale it and then how to it understands um how to uh enhance the output and and produce a correct looking result in the end um and you know that's is literally uh years of training and um untold amounts of data have gone into making that system uh uh Broad and you know um uh open enough that it can that you can feed it any type of scene any type of image that can produce the correct result on the other side um so I mean you know people remember when it started you know sort of dlss1 or pre-dlss1 it had to be trained for that specific game um and that was a lot of work just by itself and then sort of the big innovation or dlss2 was getting it to the point where um it could be um sort of the uh you know more Universal where you could just throw any kind of uh any kind of uh input on it and it would uh produce the right result um uh so yeah uh obviously you know dlss3 is looking to take that further this but yes I wouldn't characterize as generative AI right that's that's another uh thing that's happening now that's um uh fairly new uh but um you know who knows what the future holds as far as that goes yeah I mean I feel like this in and of itself is kind of an example of there's we're getting to No Limits um mind-boggling yeah for sure awesome uh next question we have is from Miguel AR and they were wondering if dls's three is I know you mentioned that it'll be backwards compatible with different engine versions will it also be compatible with previous GPU versions or would this be exclusively for the 40 series uh the the super resolution and reflex updates um those work with older RTX cards um uh uh the frame generation part is 4D series only yeah makes sense I feel like it's hard to we always want everything to be backwards compatible but it's it depending on what it is sometimes it's just not possible yeah I mean I can I can touch on that a little bit uh I don't know Zach if you want to chime in on this too but this um it's it's an interesting topic because um uh uh frame generation is you might think of it as just software but it's more than that um it's relying on the uh uh uh uh there's a hardware accelerated component to it that is uh exclusive to the 4D series they have um uh [Music] Optical flow accelerators like I was gonna say Optical I was like right on the tip of my tongue yeah the uh is they have Optical flow accelerating uh Hardware that's about three times faster I think in Hardware than the 30 series that Hardware is present on the 30 series but just runs uh physically slower and and so um for that reason the lss3 frame generation is 40 series only um and that's where we're at today yeah makes sense to me um next question we have um which is kind of an interesting one it's sort of a hard question for me to ask you but um what are any parts of dlss that you think can continue to be improved um even past what's being updated with three oh all of it any and all of it it's always possible uh is is that's there's there's a whole I mean there's very interesting topics there because uh um we've already seen reflex is not that old of a technology we've already seen it go through um a series of improvements I think the the version of reflex that got mainlined in unreal is version 1.6 uh the version we're shipping you know when we when we released the updated plug-in here I think is version 2.1 um and we're always seeing improvements across all of our Technologies whether it be Memory optimizations performance optimizations quality improvements um so uh from a imaginative standpoint I don't think anything's off the table um in fact I mean one of the that's that's part of the reason why I loaded up the Grandma's Kitchen scene because with dlfs 2 if I showed that scene um running in 4K Ultra performance mode it wouldn't look uh nearly as good with the older algorithm uh right but starting with dlss3 we made um some pretty important advancements to the quality of ultra performance at 4K Ultra performance was originally with dlss 2 was originally targeted only for 8K displays and I think officially our documentation still says that um but uh starting with dlss3 there were improvements to that where we started to talk more about the idea that you could get suitable quality with ultra performance out of 4K displays or 4K images and I feel like that's been reflected in like user feedback from say a portal RTX right portal RTX is one of the one of the first titles to take advantage of uh the lss3 um uh broadly like it's it's got all parts of it included and I I saw a lot of uh like user comments people talking about how surprising that they put it on a 4K display in Ultra performance mode and uh getting like a shockingly good quality out of that um and that's that's an improvement to the algorithm right I mean originally dlss didn't even support 33 Ultra performance that was part of the original um scheme and then it came into the lss2 I think around 2.4 or something like that and we've seen it improve again and again to where we're at right now it's insane the difference that has come even just between these two iterations of DLS so far um so crazy to think where it can end up in the future I I really do think sky's the limit I mean we don't know but you know it's uh um is it this is a um this is a a you know a big Focus for NVIDIA and there's a lot of people dedicated to it um yeah yeah awesome um next question we have is from 2D James and they're wondering is this compatible with and display across multiple nodes and if so what are the limitations known uh that's a good question I don't know the answer to that I haven't personally used end display so I'm not too familiar with it I get the idea of it but I just I just don't know the answer sure um what DLS S3 also work for VR uh the the super resolution part uh does yeah um uh NIS is also an option for a VR uh the the frame generation part uh doesn't um um but there are certain things certain areas of research that are interesting to us right like um how could we make VR better how can we make things like movie render queue better um uh so yeah these are I'd say they're areas of interest for us in areas that we want to keep improving definitely um next question is from I'm gonna Slaughter this and I apologize um Silver Cross I believe um they're wondering what happens if you add Nvidia reflex into the mix does it improve latency and UE overall uh yeah um uh reflex is always an option like um uh super resolution and reflex can be used independently of each other you could say I'm going to make a game that only uses reflex and I don't care about any of the other dlss stuff um uh the only hard requirement we have the way this works is if you enable um uh frame generation reflex is part of that it has to be turned on you cannot use frame generation without reflex they go together um uh that's that's sort of what the Streamline plug-in helps to deliver and make sure that two seemingly unrelated Technologies um uh cooperate and um are sort of Auto activated so I think it's there is it's technically not possible to run frame generation without reflex you wouldn't want to do it anyway it wouldn't provide a good experience um uh but but um you know if someone wants to just like I said just use the super resolution part or just use the reflex part um that's certainly an option to them awesome yeah and uh I was I was going to point out that uh especially if you're doing like um if you're doing a a a heavily like you know competitive multiplayer game or even as an end user right uh you might want to turn on dlss super resolution at the lowest input resolution that you're comfortable with and then turn on reflex uh as well that'll probably that that combination will probably give you um the lowest possible latency maybe not the highest frame rate but the lowest latency and that might be exactly what you're going after that circumstance yeah seem to be extremely use case specific yeah it totally depends on the the type of you know project or game or whatever that you're making like there's you know a different different strengths of weaknesses and and use cases depending on um you know how you want to optimize things awesome um next question is from woland and they're wondering since this um is closer to machine learning if they need to train the AI on the specific game that they're creating for using dlss or if it's a more automatic process it's automatic uh there's no there's no training involved anymore um I mean we we train the model internally uh uh you know and that's where like I said you can just have the over-the-air updates enabled and you get the latest model automatically um but uh yeah nothing nothing for the developer or end user to do awesome um let's see this is exactly what I was talking about before we're through um through everything you went through some of these questions have been answered through the ways you know I'm finding ones that we have already addressed um uh Matthias is wondering will you keep updating dlss2 for the 30 series and lower well yeah I mean okay so if you look at it very technically with the steel ss3 um a set of plugins when when we release this update um uh it is technically it's called dlss3 super resolution like if you look in the in the plug-in version number it's 3.1 if you want to get technical but that works on all RTX Hardware that's super resolution plugin Works um you know across the entire line so uh uh you know it'll continue we you know uh uh the 20 Series 30 series um they're designed to run uh dlss super resolution and reflex um you know so the I don't I wouldn't expect that would change um over jump rally is wondering does DLS S3 allow a different ratio between real frame and generated frames like creating a frame every three real ones for example uh no but that's an interesting uh topic for sure an interesting an interesting thought process yeah I had never thought of that personally until seeing a question which is why I wanted to ask because it made me curious as well yeah there's always interesting um rendering optimizations conceivably that could be done in an area like that right because in in VR you look at things like ovated rendering and um you know a variable rate shading and things like this where you're trying to optimize what is rendered within the frame or or have a variable um resolutions you know from one frame to the next um so you know it's easy to imagine lots of different possibilities uh but that's not a current thing um streamings is wondering can these settings I think the the settings you had gone through in the example project be exposed in which UI through simple nodes or does that need more in-depth coding um I I'm not sure I fully understand the question you need try one more time I think so yes okay so so I ask it again I'm sorry um Can the settings be exposed in which UI through simple nodes or does that need more in-depth coding um well you changed the uh yeah the presets if we're talking about the presets you just change those in your um uh uh your your uh system preferences your project preferences um uh uh and and just set them there and yeah of course if you if you if you set it to you know from whatever the default preset is let's say you like preset a That's the older deal it says two one let's say so you you set you set it back to that one because you'd like maybe some result maybe it works for your project um uh then as that and he said over-the-air updates as if you have that set it will update that version that preset in your build you know in future updates automatically at least until you you know maybe if you decide to change your mind as a developer right and go back and um switch to a different preset but uh if you if you set it up that way it would just be um uh automatically improving um from there okay um next question is from uh you risk games and they're wondering what is the memory footprint like when running with dlaa and frame generation uh they do they do each have a cost um uh let me actually load that up that might be I mean trying to think of where I saw that might be an art because that does get a little more technical that might be in our programming guide yeah if you want to share my screen I can show you some of those expected costs awesome got it pulled up okay yeah so um if you look in the programming guide uh uh this is the um uh the memory cost for uh super resolution uh you can see that there there is some you know uh expected uh these are GPU costs by the way so some some GPU overhead involved but I mean the good news here is that you're also reducing the size of your frame buffer you're reducing the input image so even though uh the LSS super resolution has a an inherent cost to it typically you'll see your overall memory go down um uh you know that's not guaranteed I'm not saying that that's an absolute certainty but um because you're rendering a smaller you know input resolution image typically you'll you'll overall you'll you'll lose or you'll you'll use less memory as far as the cost for frame Generation Um I it kind of depends on I think the uh the size of the currently rendered image it's I've seen it typically be anywhere from uh uh you know it's two to maybe 500 megabytes um but again it's it's like uh if you're using the full uh DLS S3 package with super resolution um you might be you know you so you're you're putting some additional memory costs there and then probably saving memory in other places awesome um all right I think we've pretty much caught up on most of the questions so if anything is popping up that y'all would like for me to ask make sure you throw that in chat right now um in the meantime I have a a question from me that I'm curious about um and it is for both of you so Zach I'm gonna force you force you to talk you've been awfully quiet today but um in your opinion what is the most exciting change with DLS S3 I think for me it's it's definitely frame generation you know it's like when I think about when I think about that the first thing that when I heard about it the first thing that came to my mind was it's like hallucination it's it's the stuff of movies right when back in the day when people were like I want to make this but now it's it's getting closer and closer to reality which is it it's really amazing and mind-blowing at the same time when you really ground yourself and think about what's actually happening yeah I think that's the biggest thing at least for me the biggest impact uh the thing that I thought about when I first heard about it and seeing it in action is it looks it's definitely I definitely resonate with a lot of people when they say it's it feels like magic you know I've heard like Voodoo uh it does really feel like that because back in the day everyone's saying oh yeah I'm getting 60 FPS this is so great but now we're doubling you know those frames plus plus yeah it's right doubling your primary I think that's the thing that people are I mean working in film for me before 24 frames per second then we're going to like you know like 48 48 and we're thinking okay wow this is pushing it but in in real time in games you're going way beyond that [Music] um yeah what about you Richard what do you believe is the most exciting update uh I I don't know I I personally got really excited when I saw the the quality improvements the ultra performance um just because I know that that by itself is potentially a big boost and that's not going to be like you know not every user is going to want that I totally understood right like uh some people definitely want like uh dla level quality or nothing right like they but we we provide that option we definitely encourage developers to build that into their into their app so um you know it's uh I I feel like the full range of options here is is pretty powerful you can use frame generation with dlaa you can use frame Generation Um you know with uh dlss at a lower resolution like you know you can sort of uh optimize and tune and configure the uh the the the the range of choices to like like suit what you want like the the sort of the the uh the the trade-offs and benefits of of various quality and um performance outputs and so forth so all that stuff is is really cool uh yeah but yeah I mean you know I would have picked just frame generation if Zach hadn't already taken it because I mean it's hard to argue with with that yeah sorry you stole it from me but it kind of that's an obvious yeah I took to I took the steak off the table before you're gonna give me the beans and the peas I I I I I don't know like if you're just if you're just like in just Raw like you know performance you just want to like ultimate Max frame rate uh then you know going with like frame generation with ultra performance and uh uh Reflex on plus boost that's one of the that's one of the options with with reflex you can you can be on and then on plus boost which is like you know maximizes uh uh any remaining GPU power to reduce latency as far as possible um you know that combination can be pretty killer right like you get your um your latency way down your frame rate way up um you know and uh you know especially if you're like I said if you're on a 4K display have a surprisingly um like high quality image like you know uh this is this is actually an area where I feel like people need to um just be willing to reevaluate your expectations with with the LSS as since it's been a constantly evolving thing you know it's progressed quite a bit from version one to version three um and you know it continues to improve so uh uh you know if you had a certain like dlss will never look right unless it's like you know quality resolution or you know higher you know with version one um uh reevaluate that uh you know for DLS S3 you might be surprised yeah definitely um awesome there's there's a question here it's not it's not technically related but I am actually very curious about um your thoughts on it as well but this is also from over jump rally um and they are saying speaking of phobiated rendering in your opinion why can't we have something similar fixed in a non-vr game for example like in racing games you only look at the center of the screen for most of the time and things like that I mean just my opinion listen Yeah Yeah well yeah I mean you know I I'm impressed by the idea of ovated rendering um uh of course you know doing that on a desktop would require some uh pretty accurate eye tracking um and it is a newer form of rendering I kind of give um you know development of foviated rendering a little bit of a break in that sense right because um if the concept has been around I'm not sure exactly how long like maybe a close to a decade but it's still sort of at the same time it's kind of in its infancy right like from a practical standpoint not too many people are really doing it yet so um maybe that maybe that is the future right like maybe that's how we render on a desktop screen um uh certainly be cool I wouldn't be against it especially if it I mean you know as long as it works and and runs great and you know that's fantastic um I I did if if we want to go through it I did have one other um scene I wanted to load and that was it was just to step through um uh uh at least quickly like how just to show like how you can set up um dlss3 for the first time on a on a art complete scene this doesn't have to take that long but I thought it might be interesting from the standpoint of like blueprint scripting and and just a few setup details that we haven't covered um absolutely yeah if if you want um go ahead and share my screen alrighty um we've got it pulled up now okay yeah so this was a scene um from the marketplace oh hold on there's just this one right here the abandoned Homestead uh I thought this was pretty interesting just in the sense that well it advertises itself right at the top as being nanite Lumen so it's the you know and it looks looks pretty cool nice and modern um uh the developer of the scene said that um uh you know it's fairly optimized but maybe not the most optimized and that that piqued my attention right because um you know a scene like this I could see it being very challenging to render uh with all the foliage and um you know High poly artwork and so forth mm-hmm oh yeah uh I have it here um I said we can do this I hope very quickly um I'm excited I feel like this is gonna we're just going back into Wizardry with this one I can feel it already oh yeah here uh let me pull up a stat FPS so this is the scene running without any deal of this on it right so that would be lss3 this is native rendering um you know it's running I guess I guess I don't think there's anything outrageous about the scene in terms of um you know like uh what this uh developer made um uh it's definitely uh you know a high quality art complete scene there's a lot going on here um oh I think it's it's normal and expected that something of this detail level would uh uh you know not run necessarily 60 frames per second out of the box uh I did I did do a couple of quick additions just so you know just for full disclosure here uh when I saw the scene uh initially I turned Shadow casting on for all the foliage so I think some of this didn't have Shadow casting before and you know it's a fully Dynamic scene so um you're you know make it a little bit more challenging and um I turned the lights on the interior if there's multiple light sources on the interior here and I set them to um be uh raytraced so that we would be push it a little bit further some some big wrecked lights and fill lights I think he's doing that just to boost Lumen just a little bit um oh yeah um Okay so anyway this is you know 5.2 Branch I've already put my plugins in we did that before so I can um go looking for the dlss plugins but it's kind of a cool I don't know if everybody knows about this but when you're searching for something um there's like a neat Boolean you can do you just do one of these um horizontal slashes search for multiple terms here I'm going to look for a dlss and Nvidia and or Nvidia um because I want to enable frame generation this is the Streamline plug-in but it's in in the plug-in name it's called the LSS frame generation and you can see here it says that it's both frame generation and reflex bundled with streamline um and then here's the super resolution for dlss and dlaa and then I'm going to enable NIS as well because like I said it's good fallback for machines that don't have DLS capability but it's also your sharpening control for dlss um so yeah let me restart that and hopefully I don't hit any like weird Shader compiles or anything and everything just loads you know I gotta let you know Richard I've worked in and with Unreal Engine for years now and I never knew that you could do a flashy additional search I just learned about that one I was I'm like wow really okay and apparently it works in other things too like in like the level you know if you've got like a hundred sub levels here you can you can use it here as well yeah it's pretty neat um it's like a it's a cool cool like coding coding um and or to narrow down your search yeah and I don't know if it's an unreal fivism or like if it wasn't there in previous unreals but if it is unreal five and I'm all for it I've been impressed by a lot of the um little refinements um okay so uh we can just check things I've you know enabled the plugins um very quick check feel the dlfs super resolution plug-in is there um I'm going to load the debug overlay you know I might need to do one more restart if this was not on by default but um just FYI uh in any point I can I can very easily turn dlss super resolution on and off in the viewport by you know clicking this checkbox here you can see it do a quick refresh whenever I turn it on or off um and that's just you know for previewing purposes um but yeah so anyway um same thing as before I've got the plugins enabled so uh uh no new scripting yet I'm just going to uh relaunch it again in Standalone mode sort of get things configured the way I want yep okay make that full screen oh and yeah so it's complaining about landscape physical materials very common and unreal for something like this right I'm going to uh disable the table all screen messages yeah so we'll get rid of that so it's just not plugging our eyes okay so I've got vlss enabled but I'm not taking advantage of it because just by default it uses whatever 100 resolution or something um you can test you can test for that right like I can go um our screen percentage right test and then return without putting in a number right it'll tell me what the screen percentage value is what it's currently using so I can go okay well I don't like that I want to use quality mode because I'm in 1080P so I'm going to set 67 percent right and oh let me uh let me see if this works yeah I'm going to turn on my uh GeForce experience overlay it's got a performance overlay tracker that you know shows me uh the frame per second and the render latency in the bottom corner um uh you know you set those preferences yourself by you go alt Z and then performance and then there's a little uh whatever this disk icon the settings icon and then you can set like what kind of thing what kind of details do I want to show you can go like heavy on details like how much memory am I using um in this case I just want to know what the latency is and what the frame rate is um and then to pull that up at any point is alt R so altar to send it away altar to bring it back oh yeah okay so I can see that going from [Music] you know 100 resolution right the Z-Force overlay takes a moment to update I've gone from 47 FPS uh and then you know quality and you can see that like I mean I just my opinion you know people can have different opinions I I don't see a visual difference between you know uh uh at this resolution I don't see a difference between using quality or full resolution it looks the same to me um any any differences are very subtle uh but it's hard to argue with that performance increase right I've I've increased my frame rate by 50 by making that choice um so that's good uh and then you can um enable us so we've got streamline going frame generation is ready to be used so I can um you can you can actually just type this is one of the cool things about ue5 you can just type partial words to get like full console commands so I know I know I can type um blss and space enable and that gives me both my like enabled dlss super resolution and my enable dlss framework Generation Um and then I can just set that to a value of one right and then we just let it refresh for a moment and there we go and by the way that when I did that reflex automatically came on right it's like I said it's not going to it's not going to let me turn on frame generation without also enabling reflex so it simultaneously increased my frame rate while reducing my my latency um uh again I could if you want to see what the difference is it's like what is the render latency about 20 milliseconds a little over 100 frames per second I can go back to uh turn off frame generation right and we can see where so it'll give it a moment to update so we can see that the the latency increased by I don't know what that is one to two milliseconds a very nominal impact in this particular case uh but I went from 60 ish fps to 100ish FPS yeah oh that gives me like yeah so and not not a bad performance boost right yeah and keep in mind too I'm running in Standalone game now Standalone a standalone game mode gives you a pretty good sense of quality and performance like for the for our final shipping build but not it's not the same thing this is kind of like running in development mode so it's entirely possible if I went and compiled this into a shipping code I'd get a further boost out of it I definitely saw that with the uh City sample like where in in Standalone mode in the editor it would be about the same it would be like 100 frames per second on this Hardware um but then in a shipping build it was like 130 140. so that's not uncommon um you have to figure you know running in the editor there's a certain amount of overhead whether it be a memory or just running debug code or whatever it is that is you know not showing you like necessarily the final quality I've definitely seen I'll I will say though I've seen it the other way too in some projects where like you run in Standalone mode to get a certain frame rate then shipping code is like the same so it just kind of depends project to project I haven't figured out a Rhyme or Reason to it um but it's uh it's just something to keep in mind that this is not necessarily your your final in-game performance um but this is pretty good I went like what did I start out at I went from 40-ish like 45 to 102 so about it's about overall it's like the same um uh uh jump I got with the city scene uh you know very similar this is like Lumen and nanite where it went up by about two and a half times about 250 percent by using the full DLS S3 um so okay I feel good about that so um I'm going to do some very simple not advised generally for game development level scripting right this is this is just get going I guess we're this very sort of hacky but just if I wanted to generate a um like a test build that I run for myself and just to confirm everything is working or or maybe if I just want to make this simpler for my own development without having to put in the console commands every time um I'll show you uh you can do a search for uh uh you know you just right click search for blss and you can uh you'll notice there's only one enable command there's an enable the OSS there and then I can do a search for streamline that the lssg mode right and then I can do um I'm going to set my I'm going to set reflex uh you can search for reflex or you can it's also under a streamline right reflex streamline so I can set reflex mode and I don't have to do this but I'm going to do this because there is the option of on plus boost like I said by default when reflex is enabled it goes to enabled right just on mode um but you know we think it's it's good to give users the option in the menu to for uh boost like I said boost is all it is is just if there's any like leftover GPU power like even a percent or two it attempts to take advantage of that that unused utilization to squeeze out any more um render latency uh uh you know that's possible now that might come at a cost of perhaps some frame rate right like maybe you lose five frames per second but you gain five milliseconds that's a user preference right um but this is why we say just default to enabled and that's that's the way dlssg works or sorry frame generation works that uh defaults to just uh you know uh Reflex on um so um yeah but like of course if you're building out your UI you want to and we have this in our UI guidelines document um uh where you know it talks about these sorts of options um so let's see there's all of that I want to do a console command to execute the screen percentage it's the over my own thing in this case 67 I know 67 is quality uh I mean the different modes are uh 67 for Quality 58 for balance 50 for performance and 33 for um Ultra performance and if you set it to 100 it uses dlaa that's automatic [Music] um and just because I'm not completely crazy just to show you this is possible I'm going to set a little very quick Branch logic uh uh I can test uh is dlss supported right oh wait no that's that's testing a specific mode whoops you can do that too yeah so this is a very simple Boolean just to test does the user have an RTX card do they have the latest drivers are they on the whatever the latest version of Windows that supports you know this feature um if if it passes yes to all of the above then it executes and that's just uh you know right if it's false I can just say you know don't do anything or all or use NIS instead but here I'm going to enable dlss turn frame generation on all set at the on plus boost just to try it and set the set that oh and um let's say I want to sharpen the scene a little bit too just real quickly um I have it the NIS plugin enabled and like I said NIS controls your your scene sharpening so if I'm looking at this you know I don't know if I maybe this feels a little blurry to me and set um I can just start typing NIS Sharp I notice the uh the console command and the value is zero to one I think it defaults like 0.2 or 0.3 I'm not sure we have that in the documentation but I can just try like um I don't know 0.5 see what that looks like I might feel a little bit over sharpened to me that's that's good too of course when you're doing something like this to look at it in full screen or like whatever your target resolution is um so maybe I wanna yeah I'm gonna I'm just gonna go with that I'm not looking for Perfection here but I just want to sharpen it a little bit so um yeah I don't have to use a console command for that either I can um search for the NIS sharpness uh blueprints function I do a value of 0.4 again normally you you know on a user interface you'd hook this up to a sharpness slider and give the user the choice and you know set a good default for your project but that's just basically how you control that and that you know that should work so let's see I'm just going to run it Standalone mode print in action well a little bit of live scripting hopefully I did everything right but if if this works I should be getting um you know 100-ish frames per second or close to it yeah give it a moment to sort of catch up and update I just you know it doesn't refresh right away but yeah looks like we're we're working at my chart I can tell a little bit of sharpness added to it I feel very comfortable with that I'm I'm kind of I'll admit I'm kind of somebody who likes over sharpening all of my no I know most people don't but just me my personal preference I'm kind of I kind of like that that look um I know I'm I'm the lunatic but Yanks the sharpness slider up too high um uh but yeah that's that's that's basically it so that that that works like I could then you know uh uh build this and and pass it off to my friend or or you know uh to a teammate or something and have them sort of you know there's no control here right there's no interface but it it's a uh basically working dlss3 implementation um that's and that's really it uh yeah I just wanted to show that that um you know to take a uh art complete scene and how quickly you can uh uh add the lss3 to it very straightforward um very easy to get that set up thank you for walking us through that really appreciate that yeah cool awesome well um for anyone who might have missed earlier in the Stream the dlss3 does not currently have a set release date but I feel like you can confidently say soon TM right it's not yeah it's not it's not soon with like you know whatever years away or anything like that we're we're in we're in beta right now we're we're squashing bugs so as soon as we have a release date uh we'll announce it awesome um for anyone who's interested in following and getting the latest news in accordance with any of that is there anywhere uh that you would recommend they do so Zach might have good suggestions on that one yeah we do we have um you know we usually have our technical blogs that we'll publish where people can go to and our team is very working very hard to put out social media as well um if you go to Nvidia Game Dev on on Twitter follow that I think Richard has it up on his um little uh dialog box there but anything that comes out will be announced there and you know working with you folks as well helping us uh get the message out that's uh that'd probably be the best there's also some links in the description I think on the page in the forums on the UE forms that uh initially went out for this session there's three links that people can find there that will lead to the first one is our Unreal Engine page uh for NVIDIA so it shows all the technologies that Nvidia is working on in conjunction with you all and and Unreal Engine everything from you know Lighting Technologies from our dxdi to you know to dlss it should all be there all the links to how to get set up as well so that probably be the best place to to find that all we also have um you know our own webinars as well we don't run them as frequently as you do because uh it's uh it's not a big band it's just a two-man show but along with a few other with a few other important people in the background uh helping us out but uh it's called level up with Nvidia and that also we generally tend to have featuring any new announcements or technologies that we have well you know you can tune in we'll definitely have sessions for that and can also happy to come back on here as well to to kind of parrot that message when when the time comes yes please do always love having you guys on thank you yeah yeah we're I mean we're usually on the uh on on our own web show we frequently talk uh on real topics um you know so far it's been like 80 at least eighty percent of the time uh uh uh because a lot of our game development focuses in the unreal space so yeah I'm sure we'll be talking about um uh this and other stuff in in more depth as you know we continue to develop these plugins awesome very much looking forward to seeing where it all goes and playing around with it once it's released but with that in mind thank you both so so much for coming on today I know you're both busy people so I appreciate you coming on and taking the time to walk through this with us and show us some of these really cool scenes and these incredible capabilities that dlss now has so thank you so much really appreciate it awesome thank you for having us yeah yeah uh thank you thank you everybody for the questions and so forth we'll try to answer everything yes yes absolutely thank you everyone who came and watched the show it wouldn't be what it is without you and your participation and your involvement and you just being here week after week so really appreciate you being here I recognize a lot of you I get really excited when I see your names repeat pop up again week after week so I really appreciate you coming and hanging out and enjoying development with us and all the cool facets that come with it so thank you for being here um we have another episode of uel FG tomorrow we're gonna have on some of the team that created straight lights so make sure you come hang out for that one very very cool beautiful project wonderful game um I'm going to be playing part of it and I suck so that'll be fun um come hang out and watch that um if you missed yeah yeah it's gonna be it's gonna be a good time if you came in a little bit late um and you missed any of information no worries whatsoever we post all of our streams in video format that can be viewed On Demand on both our YouTube and twitch channels at unrealengine.com um you can also find all of the latest news and keep up with us on all of our socials at Unreal Engine as well and last but certainly not least please come say hi in the forums where you can keep up with us meet other developers join the community overall happy to have you there so if you haven't already check that out as well but with that thank you one last time for both of you for coming on really appreciate you being here and I will see you the rest of y'all later all right bye everybody thank you foreign

Channel: Unreal Engine

Views: 26,265

Keywords: Unreal Engine, Epic Games, UE4, Unreal, Game Engine, Game Dev, Game Development

Id: lruUkEIRvQ8

Length: 116min 30sec (6990 seconds)

Published: Thu Jun 01 2023