Cutting an NVIDIA RTX 4090 in Half with a Water Jet: The Science of Cooling

standby today we're cutting a cooler from an RTX 4090 in half [Music] the cooler originally came from one of these this is a water jet and it's a high-powered machine that uses grit and water to blast a clean cross-section straight down anything we put under it today we're working directly with nvidia's thermal engineering team to cut that RTX 4090 cooler in half but don't worry no actual boards were harmed in this and we'll get to see prototypes never before shown to the public to learn more about the real science behind video card cooling Vapor Chambers fan design and thermal engineering this interview is heavy in the scientific details and that's thanks to this guy Malcolm Gutenberg thermal engineer at Nvidia who didn't hold anything back including the math when talking to us as you go to bigger fans pressure isn't proportional to damage of the 1.6 its proportional diameter squared and RPM squared okay so you actually lose pressure capability and it ditches the marketing to really dig into the advanced science of modern computer cooler design we'll be talking about fan prototyping play blade designs count density fin pitch cooler analogs for acoustic tuning and looking at resin cast Cross sections of heat pipes and VCS and more so let's get started before that this video is brought to you by Squarespace we use Squarespace for our own GN store and juggle complex multi-piece orders all the time with it Squarespace makes it fast for us to roll out new products with detailed Pages full of galleries videos and descriptors it's also useful for your own resume sites for a photographer or projects portfolios or for starting your new small business idea there's never been a better time to try and start your new business than right now and we can vouch that Squarespace makes it easy visit squarespace.com Gamers Nexus to get 10 off your first purchase with Squarespace hey everyone so we have a special video for you today I'm joined by Malcolm from Nvidia and Malcolm you do thermal engineering yes sir thermal engineering and he brought all of this really awesome stuff uh so we're gonna go through some prototypes of things I think right like fans yeah we got a bunch of cool stuff here we got fans Vapor Chambers the new 4090 here and uh do you want to do you want to just we'll tease this do you want to briefly Explain how yeah yeah so so essentially we took a water jet to our 30 90 and our new Vapor chamber on the 4090 right we'll walk you through some of the differences there and yes not allowed to see into the vapor chamber this is the 490 itself assembled and it disassembles into some of these pieces not all there's not this many fans on it some of these pieces but the rest went into designing it uh so today is going to be entirely engineering focused I'm excited because I we do a lot of testing of thermals it doesn't necessarily mean I understand how a company got to the design it got to and I definitely don't understand all of the different properties and things like fan design Vapor chamber design so that's what Malcolm's going to help out with let's start with these two pieces uh and then kind of get into this this water jetted yeah yeah yeah sure so walk me through what is this what am I looking at yeah so this is probably the earliest version of 40 90 that we had so essentially these were very early acoustic and flow prototypes so that that's initially what we wanted to see is how the fans performed oftentimes the fans are the longest lead time item and we really want to make sure that because a lot of things can go wrong in fan design right is that because you're still kind of figuring out the whole rest of the card as you're working on them yes sort of that but also there's a lot of thing we simulate these fans obviously uh very thoroughly but there's some specific frequencies that you can't necessarily count for and some have impacts just because of the impedance they're operating at so it's important to make sure that you know what the sound quality will be before you go and Tool a fan yeah and everything so uh these are acoustic and flow prototypes obviously not functional heat pipes right but the overall impedance of these is very similar to uh the final so not necessarily you're not using it for like thermal load no no it's just basically for checking noise and uh and just the airflow character exactly okay exactly I think there's two of these in the world one this year right another one in Santa Clara yeah so um so then this leads eventually after I get some revision and stuff to the final design so this is the 4091 uh I wanted to get to this kind of early in the discussion because it's so cool it kind of ties everything together yeah so uh first like I mean this is It's a cross-section cooler right except this is the cleanest cross section of anything I've ever seen because I tried to cut a vapor chamber open once and I used a Dremel and it basically like liquefied everything uh so it just kind of melted you couldn't see too much into it and then my next step to try and see into it was take a drill and step drill it out and that also was not great so this I guess let's um let's talk through vertically like what it is they're looking at uh I'll let you talk about like heat pipes yeah sure sure yeah so we got all the I mean yeah chopped it right into we got the vapor chamber here so that's where you see this Gap here so that's all Vapor chamber right and it's larger than the uh 3090 Vapor chamber so you can see 30 90 Vapor chamber was about this amount of the card and then we have the heat pipes on this side this is the full uh west side of the cart so the entire area contacting the Chip And Memories yeah and to help people with the orientation here that's GPU contact is in this area if it weren't cut in half and then I guess you've got memory contact here exactly exactly and then we got six heat pipes six heat pipes as well the interesting part is the evaporator design is completely different than the vapor chamber so I think this is an area which we spend a lot of time optimizing the evaporator but the regular user should never see inside the evaporator right right well something has gone category graphically wrong so we actually implemented a new type of uh Wick inside the vapor chamber itself so essentially you have two forces in a vapor chamber you have the evaporation and you have the rate of evaporation and then you have the mass flow rate back into the evaporator and essentially as soon as the rate of evaporation is greater than the mass transfer rate back into the evaporator you run into dry out and essentially your thermal resistance skyrockets and your thermal performance well tanks I guess yeah so is that because it is it is no longer condensing back is that what's happening uh it might not have enough capillary Force to get back it could I mean there's a tons of reasons why you can run into dry out one of the things that's always different with a new generation of chips is the heat flux of the chip so essentially the power density in a certain part part of the chip whether it's in a Computing workload or a more memory heavy workload can light up different parts of the chip right and you'll get higher localized heat flux and there's basically two ways that uh Vapor chamber can dry out you can either have too much total power and higher rate evaporation than back to the condenser but you can also have localized areas of dry out okay that seems really hard to troubleshoot a very difficult yeah uh because at that point I mean I guess for NVIDIA it's a little a little uh easier since you have tools to access sensors on the die maybe exactly yeah that could help but I guess we should maybe some background info there too so for when you're working on designing these um if you're at a stage where you already have some silicon you can work with there's effectively a network of sensors across the die it's that accurate and then uh it's stuff like for a user you'll see maybe Junction temperature or like Edge temperature depending on the vendor or just GPU temperature right like yeah as if it's all just one thing yeah yeah and that's I don't know exactly how that if it's just averaged of all of them but for you guys when you're working on designing it you can tap into all the different thermal sensors exactly so we can look at uh the north south temperature Delta which usually gives us an idea of how the Tim is contacting the drive uh East-West as well we have a ton of sensors and some are directly in the hot spots some are in other you know processing parts of the die right um so we have all that information and uh that's a great way to check if we have dry out basically if we have one that's way hotter than the average then you know something is up right right so why um for walk me through combining heat pipes on Vapor chamber sometimes you'll see just one solution yep on on a product um where do you draw the line when you're going to start using leveraging both oh that's a really good question that's a really good question so a vapor chamber is essentially just a 2d heat pipe you know so it's it's the exact same principle um but it's basically really effective at both qmax and obviously spreading the heat so the overall qmax of a vapor chamber is much higher than uh individual heat pipe if you directly put a heat pipe really you're limited by the amount of Wick you can have to push water back to the evaporator area is Wick when you use Wick in this context are you talking strictly about a physical Wick or are you talking about a like a physics characteristic uh so like when I talk about Wick it's just this you know either centered powder or on our new one basically we notice that this is uh the 3080 VC for reference here right so essentially we have these heat pipes touching directly over the evaporator area and we found that a lot of the condensation happens directly above because that's where the Heat's being drawn out that makes sense yeah east side of the card so unlike this version which has sintered powder Wick and basically centered powder Wick is just the median in which it uses capillary Force to push water back right um we used a mesh type wick on the top so essentially the difference between a mesh and a center powder Wick is the overall uh porosity of the material okay that makes sense porosity is about typically equivalent to the poor radius squared okay so as you go to slightly larger pore radiuses which you have in a mesh you increase the permeability and essentially you allow water to travel right back to the evaporator much easier and then we still have centered powder on the evaporator area itself it can usually be thinner and has a higher capillary pressure than a wick mesh type Wick and we'll drop some footage into where uh just to get into like internal heat pipe structure is very similar in terms of like what you use to Wick things so we have some footage from a factory of how they make heat pipes and the centered powder it just as the name explains it is literally a powder cot copper correct yeah and uh at least for the factory we saw they kind of dump it into the heat pipes and then just shake them a lot and then they heat it up and um Alternatives like you're talking about with like mesh or weave materials in we were trying to get a shot of it earlier it's really hard to see but on the 40 series card I think you were explaining to me that part way down the vapor chamber uh you're you're explaining what it is we're seeing in there exactly yeah so as you it's it's a little difficult to see so on the condenser side we use that mesh type Wick with the higher permeability and essentially we have a higher flow rate of water back to the evaporator I like to think about it as like a blower versus axial fan okay you know blower is typically high pressure low flow rate axial fan high flow rate low pressure right so you can think of sintered powdered mesh as more like a blower very high pressure capillary pressure good water retention in the evaporator area to avoid dry out good thermal performance in the evaporator the evaporator area itself is like 80 to 90 percent of the total thermal resistance of the vapor chamber so optimizing the evaporator is really the key um so yeah and so basically axial fan mesh blower so there's like these copper columns in here and uh they I guess it sounds like Nvidia change the design for this generation for 40. yeah so a couple reasons to have the copper pillars at all one is structural stability like this is incredibly weak you know copper like I can bend it with my hand yeah without the pillars in it yeah exactly so you need to have because we do put a lot of pressure with the dye on the vapor chamber uh and especially since the dye kind of does this pumping motion when you heat it up right and uh cool it down so we put a lot more pillars in the inside right above the GPU die but the main reason for it is because essentially the water is condensing directly on top of the evaporator and we can Wick the water directly back down to the evaporator area so the qmax of this card compared to the you know critical heat plugs you know the point of which the card at which the card dries out is way higher on this one like like not going to be a problem not going to be a problem yeah even if you're overclocking the thing basically as far as the electricals will take it right it should be able to handle it um so we're pretty proud of the uh the new new VC design yeah how about um so the the bumps here at the memory contact yep uh that was not on this design you see it's just a flat plate this is extremely common we see this on basically every video card I've ever taken apart um I've I think I've run into this once or twice but uh can you walk me through design decision here yeah exactly so you can see it's actually a different notched pedestal on the North and the East the west and the South so basically we want to make sure that the pressure on the die is very uniform so we find a lot of the time whenever someone says oh this Tim is not good right it's usually just because you have uniform like contact contact yeah so essentially we have non-uniform memory around the GPU and each memory tin pad provides a force back on the heatsink right so typically if you look at the temperature sensors let's say you had no uh no balancing no Notch pedestals and you just kind of let it run the north can be like you know 5 to 10 degrees hotter than the South just because you have non-uniform Tim contact so we were talking too about how some of these components in at least in the past if not this design you will sometimes almost use the component as a standoff not necessarily because it needs the cooling but uh the component itself that is but because of what I guess the movement just to make sure that the pressure is equal so we'll put it a lot on the South Side here to make sure that we have perfect Tim contact on the GPU itself right and then uh yeah if you do have perfect Tim contact the total thermal resistance of that so let's say for example you made it infinitely conduct conducted for zero Bond line thickness essentially your your gain net gain in terms of the whole card would be like three to four seat okay like this Tim is really good especially if you have uniform Tim contact is that what this yeah that is over here no this is PTM 7900 so essentially uh it can get down to about 25 to 30 microns Bond line thickness in the middle but we have a curved die so it's slightly larger at the edges the other interesting thing about this Tim is it's a phase change material so it'll squeeze out initially but then it's got enough viscosity when it is liquid to avoid being pumped out so basically what we do is we run these cards through oh and that sorry to interrupt but uh Tim pump out exactly is a an issue we talked about in another video I'll link it below so you can get details on that I'm trying to like to find some of the terms as we get oh no worries no worries no worries so anyway yeah but basically Tim pump out is the GPU die is slightly curved and then when you heat it up it relaxes slightly so it creates this pumping motion right and if you have uh you know very liquidy material it can pump out after you know a few few hundred Cycles sure and then if it's your performance yeah especially the hot spot will get really bad before we jump into fans here how about this stuff that you've got kind of in resin I guess yeah it's I think I've only seen stuff in resin like this if it's like in a science lab of an insect you know like so what do we have going on here this is heat pipes cross sectioned I guess yeah exactly so there's a bunch of different stuff here basically I did cut these two apart for the video here but all of this stuff is just random stuff whenever we have a problem with a vapor chamber yeah we just chop it open see what's wrong so for example here you can see the vapor space itself now the vapor space is extremely isothermal usually like when you're modeling a vapor chamber we model you know the discrete bits right we model the the shell we model the wick and we model each pillar you know and the conductivity we assigned to the vapor space is it can range but usually it's like nearly a million watts per year it's like effectively isothermic yeah yeah all your losses are typically the biggest resistance is in the wick you have this like Wick and liquid film parallel resistance that's dominates the total overall resistance and uh essentially you want to maintain that Vapor space so you have that isothermal capability and that's really the reason you use a vapor chamber in the first place so so when we're designing we usually get uh you know CNC heat sinks very far back to make sure that everything's kind of working as expected right uh especially with the vapor chamber it can drive a lot of the total thermal resistance um usually it's probably our second biggest thermal resistance the first is the airflow right getting the most airflow gets the most benefit right let's uh let's do that let's talk about airflow so these are the final fans those are the final fans yep um what is the uh the high level like most most critical thing for us to know before we get into some of the prototypes about the decisions you made these fans are really like we spent a ton of time this generation making sure that they're absolutely perfect right so a few well nothing's ever perfect but these are by far are the best fans we've ever made yeah first thing we used a different bearing type so previously we used a two ball bearing went to fluid Dynamic bearing overall there is some acoustic benefit to going fluid Dynamic bearing basically you miss out on all that metal on metal makes sense yeah um but the big benefit is also lifetime uh the overall lifetime of fluid Dynamic bearing is much better than a standard two ball okay yeah um so I mean the big difference the blade structure is completely different and what's interesting is these fans look wildly different yeah but different flip it around here right okay yeah almost identical right because the pressure the impedance of the push and the pull yeah let's yeah grab this final is actually pretty similar I mean we can adjust the impedance almost by changing the fin pitch and uh you know how many fins and density yeah exactly fin spacing so so we found that usually we can find a acoustic Optimum point in the PQ curve and corresponding to some flow oh PQ is pressure versus flow rate so every fan has a PQ curve and usually you can find that Optimum point and we're obviously going for very entire operating Point basically yeah exactly exactly so realistically these fans are very similar the push is slightly higher impedance overall because it is blocked by the PCB and right you know you have to change directions um but the fan geometry is fairly similar the blade design is completely different than last generation these two are um I guess the comparable right and that means yeah comparable exactly so size obviously different as we talked about uh blade count different yeah so a couple things so first off size is bigger so you may think that size is always bigger but there is an Optimum Point uh this time we were limited by how big the actual card is so bigger was better basically the the there's very defined Affinity laws for fans so as you increase diameter if you keep the RPM the same flow rate will be proportional to diameter cubed okay but if you keep the RPM the same then it'll be louder by and there's a another Affinity law for that one it's not as simple though it's got logs and stuff okay so essentially we can do the math and figure out ISO Acoustics what our benefit is in terms of airflow so essentially it works out to be diameter to the 1.6 so uh isoacoustic so the other thing is as you go to bigger fans pressure isn't proportional to diameter of the 1.6 its proportional diameter squared and RPM squared okay so you actually lose pressure capability um but the overall net benefit in terms of cooling sorry I'm just looking over your shoulder at PR over here in the background what's going on right now is he saying stuff we can't say he's famous flow rate goes up and more flow rate is what cools the card um what is it you're you guys are really trying to identify here I guess ultimately as easily set as possible it's best thermals at a given like acceptable noise level exactly yeah when you're actually trying to achieve that I mean where do you start right like there's a lot of variables like a hundred parameters that you could change on a blade shape and we try and explore as many as possible that the problem is unlike the diameter Affinity laws you know that are easy to spit out and sound smart these the height and the blade number don't aren't so well defined so essentially as you change the thickness or the blade number you pretty much have to change all the different parameters and you're dealing with a a whatever the heatsink itself is and so that's why like to the comment earlier of you know you can't make it perfect the the entire reason that fans are still changing when it seems like one of the most sort of mathematically yeah so like mathematically I guess you could do as much as you want to make it perfect but then you throw all the variables at it and that's why you still Advance over time yeah exactly because there's still more to do so yeah and there's some interesting things you can do you know now that our simulation capacity is kind of growing and we're simulating more you can actually incorporate the whole heatsink and then figure out what the interactions are between the heatsink and the fin right it's uh you know ever of all yeah yeah all cfd it's actually really interesting how you model fans in particular so in a thermal design you you can just there's basically three ways to do it the one you can just put the PQ curve in and put no blade parameters that's the least computationally expensive just slap it in there see if it's good you know yeah then the second way I find really interesting is basically you put the actual blade geometry in there and you instead of rotating the fan you rotate the air around okay okay yeah so it's called a moving reference frame and it gives you a pretty good idea of the overall airflow but what we've been able to do this generation is do a full sliding mesh simulation so essentially it's a transient simulation and you do a moving reference frame for every like tenth of a rotation yeah and do or a tenth of a degree of what I'm hearing here is NVIDIA is its own customer for this type of stuff yeah for computational uh Power to do the simulation yeah yes because that's it's it's a lot of it's cool working for a company with like infinite Computing resources you know yeah yeah you could like specifically look at different frequencies and stuff so let's say you get a fan back and it's like oh 800 Hertz is really bad you look in and Contour where the you know High acoustic levels are right there's a little more we're going to talk about too in a separate video I think so we'll we'll save this for the next one but talking about like compression of the thermal pads and stuff like that um and uh there's a couple other topics along the way we'll get into especially with some of the Tim choices and the electrical so uh for now I think probably at least most of you have learned something interesting but yeah write a comment just like you've done in the past when we've done these types of engineering discussions let us know uh what aspect you're most interested in and thank you very much Malcolm for joining no problem my pleasure very cool card yes it was great to learn about and you'll see more in our testing so check back for that thanks for watching we'll see you all next time

Channel: Gamers Nexus

Views: 649,113

Keywords: gamersnexus, gamers nexus, computer hardware, nvidia rtx 4090 founders edition, nvidia rtx 4090 video card, nvidia rtx 4090 tear down, nvidia rtx 4090 review, nvidia rtx 4090 thermals, nvidia rtx 4090 power, nvidia rtx 4090 cut in half

Id: g4lHgSMBf80

Length: 27min 21sec (1641 seconds)

Published: Thu Oct 06 2022

Reddit Comments

[u/Ar0ndight]

Quite the contrast between this video and the ARC cooler analysis

[u/IC2Flier]

Sometimes I wonder what fresh hell Nvidia's marketing is thinking. They do this (which is fucking amazing), but they also embroil themselves in BS that alienates a YouTuber.

Regardless, this video is yet another example why GN is Tier 1 on Youtube alongside Buildzoid and der8auer.

[u/[deleted]]

[removed]

[u/CarVac]

I want to know why they went to pull fans. Does that just work better?

[u/glenn1812]

I genuinely love gamers nexus. As a kid who didn't know much about PC tech I've Learnt so much from them the past 2-3 years from their fantastic reporting.

[u/AnOnlineHandle]

This title is too traumatizing for me to watch the video.

[u/NewRedditIsVeryUgly]

They talked a bit about the computational resources they have when designing the card... I was hoping he'd ask how on earth they don't have the best cooling design out of all the partners every generation.

They can simulate so much that even by brute force they should have an edge over everyone else.

And to put the size of this cooler to scale: even when the tall engineer was holding it, it still looked enormous. Just look at it https://youtu.be/g4lHgSMBf80?t=1259

And some of the AIB cards are even BIGGER... what the hell is this thing.

[u/DarkCFC]

Misleading title. It's only the cooler that's getting cut in half. I expected a GPU die, sorta like derBauer did with electron microscope.

[u/paul_tu]

"For science" makes everything you do look better.