GAME ENGINE DEVELOPER Reacts to ROAD TO PS5 (Part 3)

hey wassup guys my name is Jonah welcome back to the third and final part of my reaction / review of the road to PS 5 video I'm super excited to finish this presentation today so let's just jump into it and take a look at what else is in store we just go back a little bit I guess we're offering cow consumption always which is at about 33 minutes and 15 or so seconds the best job link will be missed or eclis our process for setting CPU and GPU frequencies has relied on some heavy duty guesswork with regards to how much electrical power games will consume and how much heat will be produced as a result inside this is really interesting as well because I used to work at EA as most of you know and specifically I spent most of my time working on mobile games and mobile games have a very similar kind of set of requirements to this in in in in terms of power consumption you know thermal issues all of that stuff is very very much a big deal in that space it's interesting because it's not really just a matter of can this device play this game at like 30 or 60 frames per second its how long can it do that for before it either overheats or because it because of thermal throttling like the CPU or the GPU has to like scale back its clock speed there's all of these kind of surrounding issues not even just about can the hardware physically do this it's is it sustainable how much power consumption is gonna use and how much heat is it going to going to introduce so this is the console it's interesting to hear that they're basically battling with the same issues and how much heat will be produced as a result inside of the console power consumption varies a lot from game to game when I play god of war on my ps4 Pro I know the power consumption is high just by the fan noise but power isn't simply about engine quality it's about the minutiae of what's being displayed and how it's counterintuitive but processing dense geometry typically consumes less power than processing simple geometry which is I suspect why horizons map screen with its low triangle count makes my ps4 pro heat up so much that's interesting cBS consoles has been to try to guess what the maximum power consumption during the higher console lifetime not B which is to say the worst case seen in the worst case game and prepare a cooling solution that we think will be quiet at that power level if we get it right fan noise is minimal if we get it wrong the console will be quite loud for the higher power games and there's even a chance that it might overheat and shut down if we miss estimate power too badly mm-hmm PlayStation 5 is especially challenging because the CPU supports 256 bit native instructions that consume a lot of power these are great here and there but presumably only minimally used or are they if we plan for major 256 bit instruction usage we need to set the CPU clock substantially lower or noticeably increase the size of the power supply and Finn mm-hmm so after much discussion we decided to go with a very different direction on PlayStation 5 we built a GPU with 36c use mind you our DNA to see use our large each has 62% more transistors than the ceus we were using on PlayStation 4 so if we compare transistor counts 36 our DNA to see use equates to roughly 58 PlayStation 4 C use it is a fairly sizable GPU then we went with a variable frequency strategy for PlayStation 5 which is to say we continuously run the GPU and CPU in boost mode we supply a generous amount of electrical power and then increase the frequency of GPU and CPU until they reach the capabilities of the system's cooling solution it's a completely different paradigm rather than running at constant frequency and letting power vary based on the workload we run at essentially constant power and let the frequency band vary based on the workload yes so that that's solving that whole problem of like what happens when we have a very light scene we don't have much actual computing power required because we don't have like a particularly dense piece of geometry as he was saying because in those cases what happens is the PlayStation still going kind of at full blast because that's what it's that's how the hardware works but here what he's saying is that during the lighter load C they can actually reduce the frequency because they don't need that processing power and they can maybe boost it in other areas supposedly but in heavier scenes they can kind of increase that frequency so that's that's an interesting way of looking at it's kind of like an in Reverse and it's ultimately probably going to provide just a nicer general solution we doesn't tackle the engineering challenge of a cost-effective and high-performance cooling solution designed for that specific power level in some ways it becomes a simpler problem because there are no more unknowns there's no need to guess what power consumption the worst case game might have as for the details of the cooling solution we're saving them for our teardown I think you'll be quite happy with what the engineering team came up with ok how fast can we run the GPU what's this teardown thing ones that happen CPU the strategy the simplest approach would be to look at the actual temperature of the silicon die and throttle the frequency on that basis but that won't work it fails to create a consistent playstation 5 experience yeah that's also like difficult to do because throttling frequency is based on temperature that's how games will eventually like drop framerate and it will just start taking every frame will start taking longer because they have to dial back the clock frequencies because the device is like overheating that's not generally a good thing to do it's it's it I mean it's either that or just overheat and then shut down the whole console but in general obviously it's not an ideal solution it's definitely something we used to battle with all the time with various like Android devices it wouldn't do to run a console slower simply because it was in a hot room easily rather than look at the actual temperature of the silicon die we look at the activities that the GPU and CPU are performing and set the frequencies on that basis which makes everything deterministic and repeatable mm-hmm while we're at it we also use AMD's smart shift that doesn't change the fact that the room might be hotter though and I'm not sure how the hardware is going to adapt to that but generally like you know looking into the activity and being a little bit more informed I guess about what's actually going on with the hard way is obviously a smarter way of doing all of this technology and send anyone using to the GPU so it can squeeze out a few more pixels the benefits of this strategy are quite large it creates a more balanced overall system running a GPU at two gigahertz was looking like an unreachable target with the old fixed frequency strategy with this new paradigm we're able to run way over that in fact we have to cap the GPU frequency at two point to three gigahertz so that we can guarantee that the on chip logic operates properly they're getting up to the point where it is running too fast for the logic to actually work out is 36c use at two point two three gigahertz is ten point three teraflops and we expect the GPU to spend most of its time at or close to that frequency and performance similarly running the CPU at three gigahertz was causing headaches with the old strategy but now we can run it as high as 3.5 gigahertz in fact it spends most of its time at that frequency that doesn't mean all games will be running at two point to three gigahertz and 3.5 gigahertz when that worst case game arrives it will run at a lower clock speed but I just wanted to say that like I don't generally like looking at numbers like this because I don't know in my experience like it looking at at like clock speeds and frequencies and teraflops and all of that stuff it's it's fine and it might look good on paper but when you actually start programming for that hardware and you start stress testing it and you have an actual full game that you've built up which is a difficult thing to do it's not something you can get to within a few months of just having this new ps5 hardware when you finally get to that stage you may notice problems and and again numbers are fine it's cool that it meets certain specifications but to be honest like that means nothing until you can actually see that in practice so I don't know what would just you know this is just my opinion and my thoughts but when I look at these numbers it's very difficult for me to be like oh though it is a lot better like when I look at SSD spades and it says like you know five gigabytes a second or whatever like that that's a that's an amount that I'm sure that the console could sustain and I can actually you know picture myself streaming five gigabytes per second that's like something tangible but a clock speed and you know a teraflop count I just for me personally at least is a little bit harder to look at that and be like oh I can draw this many triangles oh I can make a game that suddenly supports this because until I start programming for that hardware and I see what other people have done and how sustainable it is and all of that stuff it's very difficult to just you know see kind of a specific how should I say this a specific upgrade in a particular department even though obviously the numbers are the numbers are there but there's a bit more of a disconnect I feel between those numbers and what's actually capable of being achieved on that platform not too much lower to reduce power by 10% it only takes a couple of percent reduction in frequency so I'd expect any down clocking to be pretty minor all things considered the change to a variable frequency approach will show significant gains for PlayStation gamers the final of our three principles was about finding new dreams it's important for us on the hardware team to find new ways to expand or deepen gaming and that's what led us to a focus on 3d audio as players we experience the game through the visuals this what it like as a musician as well like I really like what here and audios always like something that I very much notice in games and that I really enjoy so this is exciting to me as well even though I spend a majority of my programming time working on graphics and rendering like related features through audio and through the feedback we received from the controller such as Rumble or haptics personally I feel a game is just dead without audio visuals are horse important but the impact of audio is huge as well at the same time the audio team on a game project has to do a lot with a little for example on PlayStation 4 there's fierce competition for the Jaguar CPU cores audio typically ends up getting just a fraction of a core that's not much of a computational resource particularly when you consider that the visuals run at 30 or 60 frames a second but audio processing needs to happen at almost 200 times second so it's been tough going making forward progress on audio with Playstation 4 particularly when PlayStation 3 was such a beast when it came to audio the SPU's and cell were almost a perfect device for audio rendering simple pipeline algorithms could really take advantage of asynchronous DMA and frequently reached a hundred percent utilization of the floating-point unit there's unfortunately nothing comparable on PlayStation 4 probably the most dramatic progress in the PlayStation 4 generation has been with Virtual Reality the PSD our hardware has its own audio unit it supports about 50 pretty decent 3d sound sources and this provided a hint as to where we could go with audio as well as some valuable experience not to oversimplify but here were our goals for audio on PlayStation such a crazy animation that's cool the first goal was great audio for everyone not just a VR it's funny how the audio like part of this presentation has it's like amazing visuals users or sound bar owners or headphone users that meant audio had to be part of the console it couldn't be a peripheral the second goal was to support hundreds of sound sources we didn't want developers to have to pick and choose what sounds would get 3d effects interesting to listen to we wanted every sound in the game to have dimensionality and finally we wanted to really take on the challenges of presence and locality now when we say presence we mean the feeling that you're actually there you've entered the matrix immersion is like it's so important for imagine like if you're trying to immerse your player in your game and the audio isn't is an on point and their music or the ambience and just all of the kind of sound design is not on point it's very difficult to do and that's why I like visuals are great and all but if you don't have things like you was talking about haptic feedback for controllers and you know the actual audio sound design as I mentioned that stuff goes so far in terms of immersion I'd go to say that if you're not dealing with like a virtual reality headset if you close your eyes and just listen to the sound design that can immerse you a lot more that you looking at a TV and just or a computer screen and being like oh cool these visuals look real it's not of course something we thought we could perfectly achieve the idea was that if we stopped using just a rain sound and instead use lots of 3d audio sources for raindrops hitting the ground and all sorts of locations around you then at some point your brain would take a leap and you'd begin to have this feeling this feeling of real presence inside the virtual world of the game this has the capacity to affect your appreciation of the game just like music in a game does for sure the concept of locality is simpler it's just your sense of where the audio is coming from to the right of you behind you above you this can immerse you further in the game and it can also directly enhance the gameplay to use dead space as an example I know old school you're fighting enemies in fairly dark spooky locations back in the day if you played the game using the TV speakers you could tell that there was one last enemy growling and hunting you down but it was difficult to tell quite where that enemy was with headphones you could tell that the enemy was somewhere on the right which lets you deduce if you couldn't see it that it must be somewhere behind and to your right but with 3d audio with good locality the idea is you know the enemy is precisely there and you turn and you take it out so how do we know where a sound is coming from in the first place well all those bumps and folds in the ear have a meaning evolutionarily speaking based on what direction the sound is coming from sound waves bounce around inside the ear there's some constructive and destructive interference and the result is a change in volume the phase of the sound also shifts depending on what path the Soundwave took to reach the ear canal these volume changes and phase shifts are different for each direction and also vary depending on the frequency of the sound head size and head shape also impact the sound in a similar fashion it's also like the environment also matters a lot because sound reflects off of a lot of different materials and that can definitely create some kind of phase phasing issues and all of these things that in terms like it your brain will eventually construct that into something way that the sounds changes based on direction and frequency can be encoded in a table called the head related transfer function or HR TF here's part of one the vertical axis is the frequency the horizontal axis is the direction front back left right and the color gives the degree of attenuation of the sound at that frequency the HR TF is as unique to an individual as a fingerprint is in fact you're looking at mine right now here is how we measure NH RTF we've taken hundreds of people through this process we put a microphone in the subjects left and right ear canals and then sit the subject down in the middle of an array of 22 speakers they're basically measuring the acoustical properties in terms of like the actual physical acoustical space inside someone's ear to determine how they hear which is we then play an audio tape from each speaker as we rotate the subject in the course of ten or twenty minutes we're able to sample the HR TF at over 1,000 locations using an HR TF when rendering audio creates unparalleled quality but it's computationally expensive the simplest way to use an HR TF is to process a sound source to make it appear as if it's coming from one of those thousand locations we sample unfortunately the processing has to be done in frequency domain rather than time domain so there's multiple fast Fourier transforms needed for every sound source for every audio tech that's a lot of multiplies this computational complexity was the determining factor for our strategy and we had to bite the bullet and design and build a custom hardware unit for 3d audio collection that's what I was expecting basically we're referring to the hardware unit and the proprietary algorithms we run on it as tempest 3d audio tech the meaning of 3d audio and technology should be pretty op with the visuals as for tempest I feel it really reflects our goals with audio it suggests a certain intensity of experience and also hints at your presence within it we're calling the hardware unit that we built the tempest engine it's based on AMD's GP new technology we modified a compute unit in such a way as to make it very close to the SPU's in playstation 3 remember when I said that they were ideal for audio so the tempest engine has no caches just like an SPU all data access is via DMA just like an SPU our target was that it would have more power than a CPU thanks to the parallelism that a GPU can achieve and then it would be more efficient than our GPU thanks to the SPU like architecture the goal being to make possible near 100% utilization of the see use vector units where we ended up is a unit with roughly the same Cindy power and bandwidth as all eight Jaguar cores in the PlayStation 4 combined if we were to use the same algorithms as psvr that's enough for something like five so the basic with vectorization everything they're just talking about how they've designed a unit that's capable of doing a whole large amount of multiplies which there's like Fourier transforms and all that stuff required to do all the audio so it's a very it's basically like a powerful like multiplication chip that's capable of doing essentially what the audio what the audio needs thousand sound sources but of course we as VR that's enough for something like 5,000 5,000 but of course we want to use more complex algorithms and we don't need anything like that number of sounds it would have been wonderful if a simpler strategy such as using Dolby Atmos peripherals could have achieved our goals but we wanted 3d audio for all not just those with licensed sound bars or the like also we wanted many hundreds of sound sources not just the 32 that at most supports and finally we wanted to be able to throw an overwhelming amount of processing power at the problem and it wasn't clear what any peripheral might have inside of it in fact with the tempest engine we've even got enough power that we can allocate some to the games to the extent that games want to make use of convolution reverb and other algorithms that are either computationally expensive or need high bandwidth but the primary purpose of the tempest engine remains 3d audio now 3d audio is a major academic research topic it's safe to say that no one in the world has all of the answers and the set of algorithms that has to be invented tuned or implemented to realize our vision for 3d audio is immense for example use of HR TFS and games is quite complex before I talked about making a sound source appear as if it's coming from one of those thousand HR TF sample locations but for high-quality 3d game audio we have to handle other possibilities the sound source might not be at one of the thousand HR TF sample locations so we have to blend the HR TF data from the closest locations that we have sampled the sound source might be moving which needs very special handling as that blend keeps changing and that can cause phase artifacts in the resulting audio or the sound source might have a size to it meaning it should feel as if it's coming from an area rather than a single point there's also whole categories of approaches to be handled 3d audio can be implemented using individual processing of 3d sound sources but alternatively ambisonics can be used for 3d audio ambisonics speak somewhat like the spherical harmonics used in computer graphics and finally there's audio devices the player might be using headphones or TV speakers or have a higher-end surround sound set up with six or more speakers all of which need different approaches that's a lot of variations it's nice to have the computational resources of the tempest engine but it's clear that achieving our ultimate goals with 3d audio is going to be a multi-year step-by-step process having said that headphone audio implementation is largely complete at this time it was a natural place for us to start with headphones we control exactly what each ear hears and therefore the algorithmic development and implementation are more straightforward for TV speakers and stereo speakers were in the process of implementing virtual surround sound the idea being that if you're sitting in a sweet spot in front of the TV then the sound can be made to feel as if it's coming from any direction even behind you virtual surround sound has a lot in common with 3d audio on headphones but it's more complex because the left ear can hear the right speaker and vice-versa we have a basic implementation of virtual surround sound up and running we're now looking at increasing the size of that sweet spot which is to say making the area you need to be in to feel the 3d effect larger and we're also working to boost the sense of locality head yeah so this stuff this stuff is super interesting because they're basically talking about emulating a whole surround sound set up for like a TV or something where you just have kind of two speakers in front of you how can they make that feel like surround sound meaning happen they make that feel as if you've got speakers all around and in which case you could actually like literally send sound to particular speakers that you know are behind the player to make it feel like something's coming from behind rather than in front they're basically emulating that for a two kind of front facing speaker situation which is super cool and I really would be interested to to hear exactly how this stuff sounds it reminds me a little bit of like the the Nintendo 3ds and how they were able to kind of you know deal with like a 3d display that didn't require glasses phone audio is the current gold standard for 3d audience basically just based on tricks and stuff but it's super interesting on PlayStation 5 but we're going to see what we can do to bring virtual surround sound to a similar level after which we'll start in on setups with more speakers such as 6 channel surround set which is always it's not at the point where some of the PlayStation 5 games in development are extensively using these systems one of the game demos allows you to toggle between conventional PlayStation 4 style stereo audio and our new 3d audio I listened with just an ordinary pair of over the year headphones and wow I could feel a difference 3d audio has that dimensional feel to it conventional stereo audio feels smashed flat by comparison the improvement is obvious it's probably like going from mono to stereo and then now going from stereo to 3d you so interested a big advancement by super cool have I entered matrix does my brain believe I'm really there like I was talking about earlier when I explained our target a presence well the answer isn't no but you've probably caught on to what's missing here namely who's HRT F was being used it wasn't mine it was the default HRT F the audio team analyzed the hundreds that they measured and chose the one they felt with the closest fit to the total game playing audience this shows a a piece of the default HRT F on the left and my HRT F on the right you can see that the general features are much the same but the details are quite different soon they'll be asking people to go to a lab they won't like the the next-generation audio they'll have to rock up at a lab to get their ears analyzed with the default HRT alpha now because 3d audio sounds pretty great when I use my HRT F though the audio reaches a higher level of realism which is to say that when using headphones and my HRT F I occasionally get fooled and even think a sound is coming from the real world when it's actually coming from the game yeah that's the level you want to be at the level that makes you take off your headphones and be like what oh oh that was part of the game corollary to this is that there are a few people whose HRT FS are sufficiently far from the default HRT F that's the red dot here that they can toggle between ps4 style and PS v style audio and not sense much difference I've had a few people describe the PlayStation v 3d audio is sounding like a bit better stereo audio presumably they're the ones at the very edges of this diagram which means what HRT F you're using is key I'd like everyone to be able to experience what I'm experiencing but obviously it's not possible to measure the HRT F of every PlayStation user that means HRT F selection and synthesis are going to be big topics going forward as the tempest technology matures at PlayStation 5 launch will be offering a choice of five HRT FS there's a simple test where you pick the one that gives you the best locality that's just the first step though this is an open-ended research topic maybe you'll be sending us a photo of your ear and we'll choose a neural network to pick the closest HRT F in our library maybe you'll be sending us a video of your ears and your head and we'll make a 3d model of them in synthesis amazing HRT F audio game to tune your HRT F will be subtly changing it as you play and home in on the char TF that gives you the highest score you know that it matches you the best this is a journey we'll all be taking together over the next few years ultimately we're committing to enabling everyone to experience that next level of realism hopefully I've been able to illustrate a bit about our design and decision making process today and why PlayStation 5 has the feature set that it does now comes the fun part we get to see how the development community takes advantage of that feature set and be promoting for the completely unexpected will it come from audio ray-tracing the capabilities of the SSD or something else I guess we'll find out soon enough thank you for your time today there you go wow that's so much stuff here to talk about um yeah like what he said at the end I think I'll just kind of mention this in closing that what he said at the end about now it's time for the development community to basically take hold of this and we'll see what what they create and I'll probably just be completely unexpected and that's what he's hoping for it's gonna take a long time until we realize the full potential of this new platform just like it is with any new platform and just like it has been with every single PlayStation and other console you know throughout history it's it's definitely gonna take a few years to mature I'm really interested to see what they're able to achieve now I know for a fact that like companies like PlayStation Xbox and you know Oldham all those kind of consoles they are working a lot more closely with developers now than they ever have before which is really good it means that they can hopefully start cutting down as he mentioned earlier in the presentation that kind of time to first triangle where no one has any idea how to get started with a new platform that's being cut down hopefully we'll see a more rapid progression in platform maturity from something like this but in general I honestly I'm super excited to see what people are going to come up with for this platform I think that the possibilities are going to be very much endless and we'll be able to see a lot of really cool content that probably wouldn't have otherwise been possible I hope you guys enjoyed this reaction and just my thoughts on this Playstation 5 presentation it was a lot of fun to look through this I feel a lot more informed about the next generation of consoles which is really good so now that we've finished watching this road to PS 5 presentation what excites you the most what do you think will make the hugest impact in this new generation of consoles leave your thoughts in the comments section below you guys enjoyed this video please don't forget to hit the like button and also subscribe to this channel if you are new for those of you who have been part of this channel I'm going to get back to working on my game engine called hazel and making videos about that as well and who knows maybe one day hazel will support the PlayStation 5 as well thank you guys so much for watching I will see you next time goodbye [Music] you

Channel: The Cherno

Views: 218,874

Keywords: cherno, ue5, unreal engine v, unreal engine 5, epic, unreal engine, reaction, demo, game engine, game engine dev, rendering, graphics, amazing, ps5, playstation 5, mark cerny, ssd, audio, power usage, sound

Id: ESQyAhooBLM

Length: 29min 34sec (1774 seconds)

Published: Mon Jun 01 2020

Reddit Comments

[u/mikesaintjules]

When looking at Cherno's video reactions to all this, it really made me believe this was the right decision for Sony to make by releasing this video first. Combining this with the forthcoming teardown one after the other would have been too much information for the average person to process (if they understand at all).

Having games be the next event and then the teardown somewhere afterwards makes sense.

[u/TheMeMan999]

Finally! 😃

Actually, compared to part two, this one came rather quickly.

[u/OpticalPrime35]

It's funny because I've watched the video about 8x now, Cerny's talk not this video, and I generally didn't go into the audio section. My focus has been mainly on tbe crazy I/O hardware and the new GPU features and such.

But while watching this I actually closely listened to what Cerny was saying about this custom audio chip and it's capabilities and damn.

As anyone who has played PSVR knows, the audio is really, really good at times. And this chip can handle 5000 of those sound sources on its own? GOOD LORD

Now of course the teams that REALLY go crazy with audio will create larger and more complex stuff for the chip to handle, but even at their most complex the chip can still handle many hundreds of those types of computations. Really is insane. Basically means there is no hard limit to the PS5 audio realistically. Very few teams will be creating tens of thousands of sounds for a videogame and those that do, well .... Good luck fitting much else onto the Blu-Ray disc lol. May have to have like a Blu-ray disc for Graphics and one for audio.

Immersion is going to be so incredible this generation. With the seamless gameplay, 3D audio and much more advanced controller, Immersion seems to have been a key design philosophy for this console. Which is exciting as hell

[u/AhabSnake85]

someone please explain to me the difference between memory bandwidth and i/o throughput

[u/scootuk]

After watching this video it's got me wondering, I know they are just rumors at this point, but if it's true that the PS5 will support PS3 games, what are the odds that the tempest engine will play some part in that? Mark Cerny made note of how the tempest engine works in a similar fashion to the SPU's in Cell so I don't think it's mad to think that IF Sony were to achieve such a feat that they would do so by utilizing their own custom hardware.

[u/camthorn]

I loved the presentation of the the tech behind the PS5. Mark Cerny has done a terrific Job and I could listen to him all day, same thing with John Carmack I still dont believe that 90 percent of PS fans arent interested or bored by the the tech that fuels the PS5.

[u/[deleted]]

I could listen to this guy all day.

[u/geraltseinfeld]

Hey guys, it's The Cherno!

[u/Blackdeath_663]

i don't get why people are so excited about these videos, hes basically just pausing a video and saying yup thats cool because of the exact same reason stated in the video. very little additional insight or information offered.