- We have looked at a lot of
ballin' GPU over the years, whether it's the six titan views we had for the six editor's project, three GV100 Quadras for
12K Ultrawide gaming, or even this unreleased mining GPU, the CMP 170 HX. There are not a lot of cards out there that we have not been
able to get our hands on in one way or another, except for one. Until now, the Nvidia A100, this is their absolute top dog, AI enterprise high performance compute, big data analytics monster, and they refused to send it to me. Well, I got one anyway Nvidia, so, deal with it. Just like everyone's
gotta deal with my segues, smart deploy provides out of
the box windows imaging support for over 1,500 computer models. You can deploy one windows
image to any hardware model with ease and you can get
free licenses worth over $500 at smartdeploy.com/linus. (upbeat music) The first two questions on your mind, are probably why we weren't
able to get one of these. and what ultimately changed that resulted in me holding
one in my hands right now. The answer to the first one is that Nvidia just plain doesn't seed these things to reviewers and at a cost of about $10,000. It's not the sort of
thing that I would just, you know, buy. 'Cause I got that swagger. You know what I'm saying? As for how we got one, I can't tell you. And in fact, we even blacked out the serial number to prevent the fan who reached
out offering to get us one, from getting identified. This individual agreed to let us do anything we want with it. So you can bet your butt, we're gonna be taking it apart. And all we had to offer in return was that we would test
Ethereum mining on it, send a shroud, that'll allow
'em to actually cool the thing and reassemble it before we return it. So let's compare it really
quickly to the CMP 170 HX, which it is the most
similar card that we have. It's the silver metal and it's not ripped for my pleasure. Regrettable.
- [Jake] Alright. - And we actually have one
other point of comparison. This isn't a perfect one. This is an RTX 3090. And what would've been maybe more apt is the Quadro or rather they
dropped the Quadro banding. But the A6000. Unfortunately that's another
really expensive card that I don't have a
legitimate reason to buy and Nvidia wouldn't send one of those for the comparison either. So the specs on this are pretty similar. We're gonna use it as a standin' since we're not really looking at any workstation loads anyway. So the A100. This is a 40 gigabyte card. I'm gonna let at that
sink in for a second. And the craziest part, is that 40 gigs is not even enough for the kinds of workloads that these cards are
used to crunch through. We're talking enormous data sets to the point where this 40 gig model, is actually obsolete now, replaced by an 80 gig model. And these NV Link bridge, connectors on the top here, let's go ahead and pull these off. These, there we go, are used to link up
multiples of these cards so they can all pull memory and work on even larger data sets. Now the diet, the center of it, is a seven nanometer TSMC manufactured GPU called the GA 100. We're gonna pop this shroud off. We're gonna take a look at it. It has a base clock of just 765 megahertz, but it'll boost up to fourteen ten. That memory runs at a whopping one and a
half terabytes a second of bandwidth on a massive 5,120 bit bus. It's got 6,912 CUDA cores and, what is it? 250 watt TDP. Woooh. She's packing. - [Jake] Oh, you're
just going right for it. - I'm going right for. - [Jake] Oh geez. - This is Linus tech tips. - [Jake] And basically every part of this is identical to the CMP card. - It kinda looks that way. I mean the color's obviously different. - Yeah, but it looks like the clamshell is two pieces in the same manner. There's no display outputs. The fins look the same. - Now here's something
the CMP card specifically didn't even contain the
hardware for video in code, if I recall correctly. - Yeah, this doesn't have anything. - Okay, so it's not that it was fused off. It's just plain not on the chip. - Not on GA 100, yeah. - Okay but, - GA102, which is like 3090. - Yes. - Does have it.
- Ooh. - And A6000. - Okay, you ready? - Oh God! So yeah.
- Hey. - It's like exactly
the same on the inside. Same junk power connector. - Wow. That is super junk, check this out guys. It uses a single eight
pin EPS power connector, which you might think is
a PCIE power connector. So here, look, I'll show you guys. This is an eight pin, like normal GPU connector, but watch, cannot go in. But if we take the connector out of our CPU socket on the motherboard, There you go although, the clips are interfering a little bit. I mean, what the heck is going on here, ladies and gentlemen? - You need more power. - Yeah exactly. - So you can combine two
PCIE connectors into that. - [Andy] Can't remember
how to get it outta here. I see the fingerprint of the technician who assembled the card though. - I think we have to
unclip this part first. Oh, there's a little screw, right? - There's a little screw. - Haha, third type of screws. - [Andy] Yourself.
- Didn't see that one, nerd. - [Andy] You're a nerd. - [Jake] Your face is a nerd. - [Andy] Your but nerd. - [Jake] Whoa. - It's not coming off, Jake. - What? You gotta like tilt it out, buddy. Whoa, whoa, whoa. Don't pull the cooler off. - See? It's like it's caught back here. - Hey ho. Hey, how you doing? - Jesus. - Stressful. Look, maybe if we break it, you'll actually have to buy one. - I don't wanna buy one. That's not the goal.
- What? - I thought you put your
hand up for a high five. I was like, "well, what
are you talking about? I don't want to buy one." - Why not? Whoa, what is going on here? You see that? - It looks like there was a
thermal pad there or something, but there isn't, its like greasy. - It actually, no, look at it closer. It's not greasy. It's, you see how this is brushed almost. Or looks somebody sandblasted it. - That part's not. I don't remember that on this card. - Alright, so the spring loading mechanism is just from the bend of the
back plate, that's kinda cool. - [Jake] So I checked the CMP thing. Doesn't look like it. - [Andy] I wonder why they
wouldn't have like a map. - [Jake] This doesn't look brushed at all. What did we, last time we twisted? - [Andy] No, I don't think we did. - Yeah we did. - [Andy] I'm pretty sure
I just rimmed on it. - [Jake] Oh God! No. You were against rimming on it. And then we were like,
just twist a little. - [Jake] Oh. God. Ah. It has an IHS. It looks basically the same. - [Andy] Yeah. - [Jake] We're gonna have
to clean that off and see there's not much alcohol. - [Jake] No, I like to go in dry first. So yep, that's the same thing, alright. I mean, this isn't the first time Nvidia has used the same Silicon
in two different products with two different capabilities. We see the same thing with their Quadro lineup
versus their GForce lineup where things will just be disabled through drivers or fusing off
different functional units on the chip. What I wanna know then is besides the lack of NV
Link connectors on this one. - Well, they are in there. They're just not accessible
and they probably don't work. - Right. What is the actual difference in function between them? (Jake sighs) - Well, this one doesn't
have full PCIE 16X, - Right?
- It does less memory. I think it has way less transistors, but it is still a GA100. - Yeah, so the transistors are there. - Yeah, they're probably
just not functional. Let me see what the chip
number is on that one. - Yeah, 'cause were we not even able to find a proper Nvidia.com
reference to this one anyway. So we're just relying on
someone else's spec sheet. So the transistor count
could just be wrong. - Okay, so this is so
the CMP card was a GA. - Look at this guy? - Yeah. What a weirdo. GA 100-105F. And this is a GA100-833. - If it's a GA100, I guess it could be a different GA100. I don't know. - I mean, it used to be back in the day, you would assume that it's
just using the same Silicon as the GForce cards because
Nvidia's data center business hadn't gotten that big yet, but nowadays, they can totally justify, an individual, like new guide design for a particular lineup of enterprise product. - And interestingly enough, the SXM version doesn't have an IHS at least it seems that way. But the SXM version is
also like 400 Watts. And this is like 250. - [Andy] Yeah, totally different classes of capabilities, alright? Let's put it back together then, shall we? - I got your new goop.
- Goop me. - I brought two goops. - We're going for the no look catch. - Oh yeah baby.
- Yes. X marks the spot, baby. My finest work. - Maybe it'll perform better now. - [Andy] Probably not. (Jake laughs) (Andy truck signals) We're backing it up. (Jake chuckles) - [Jake] Cool story, bro. - [Andy] Thanks. Thanks bro. - Where's our back plate. Did you take it? Oh shoot. - Yes.
- Black. I thought it was gold. I was looking for gold. (Jake laughs) - [Jake] Aren't we all.
- I don't know about you, but I found my gold. - What's that? - Yvonne. - Shut up (chuckles) - Alright. Alright. Let's get going here. Which one do you wanna
put on the bench first? - What do you mean? We're not gonna compare to that thing. It doesn't do anything. We don't need this thing. - But here we go, boys. - We can't put this in the first lock. 'Cause we don't have a display output. - You like the bottom one?
- Yeah, - You're a bottom? - Sure. - This, okay. This is how you flex IT style. Now you might have noticed at some point that the A100 doesn't have any sort of cooling fan. It's just one big fat, long heat sink with a giant vapor chamber
under it to spread the heat from that massive GPU. So Jake actually designed what we call the shroud donator. It allows us to take these two screws that are on the back of the cart for securing it in a server chassis, because that's how it's
designed to be used. So it's passive, but there's lots of airflow
going through the chassis, and then lets us take those screw holes, and mount a fan to the back of the cart. It's frankly not amazing. (Jake chuckles) - What? No. That is aerodynamics at its peak. You should hire me to
work on F1 cars, okay? - Yeah. Not so much. - Yeah. It only blows probably
more air out this end from the back pressure than
it does on this end. (laughs) But it's enough to cool it, I swear. - It is.
- Yeah. - Let's go ahead and turn
on the computer, shall we? - Oh yeah, so a couple
interesting points here. It wouldn't boot right off the bat. You have to enable Above 4G decoding. And then I also had to go in
and I think it's called like 4G MMIO or something like that. I had to set that to 42. - Okay. - The answer to the universe. - Yes. Thank you. And they are both here. A100 PCIE 40 fricking gigabytes. - I installed the like game
ready driver for the 3090, and then I installed
the data center driver, and I think it overwrote it, but the game ready driver, it still showed as like active and you could do stuff with
the A100 and vice versa. So it's probably fine. - Now, interestingly, the A100 doesn't show up
in task manager at all. - [Jake] Did the CMP, I can't, - [Andy] remember.
- No, no. I don't think it did actually, anyways. - What do you wanna do in Blender, classroom? BMW? BMW's probably too short. - Yeah. Let's do classroom. I think BMW on a 3090 is like 15 seconds or something like that anyway so. - That's also like the spiciest 3090. - [Jake] That you can
get. Yeah, pretty much. It's just so thick. Why would you ever use it? - Because you wanted, - Is it even doing
anything like (chuckles) - Here's one reason, 'cause you can do classroom renders in a minute and 18 seconds, that's why? - Okay. Well, what about the A100? You didn't plug the fan in, hey. - Oh whoops. How hot is this? - Probably warm. - Fortunately it hasn't
been doing anything. Time to beat is a minute and 18 seconds. So let's go ahead and see how it does. - It feels like this is the intake. I mean it's hot. So like,
- Oh yeah. But it's going. It's going Jake. It's going. You did good. - It works enough. This should be like, this is all. - This should be way faster.
- Way huge GPU, right? - [Andy] It's actually slower. - [Jake] How much? Not by much. - It's like a few
seconds, but it's slower. - So it's worse in CUDA. What about Optixs? So the interesting thing is this card doesn't
have Ray Tracing cores. The 3090 does, see you'd think that Optixs would only work on the 3090, right? - Do you want me to just try the A100? - Yeah, sure. It's still GPU compute. - I mean you gotta give it
to it in terms of efficiency. For real though, even running
two renders to the 3090's one, the average power consumption
here is still lower. - [Jake] Yeah well, and
looking at while it's running, it's like 150 Watts. - Yeah. - [Jake] Versus 350 or
whatever it was on the 1390. - Yeah, ready to go again? - [Jake] Yep. - Okay.
- [Jake] Oh my God. - Man, this thing is fast. - What's the power consumption? - [Andy] Holy bananas. - [Jake] 353. Still like just, I want one of these. This thing is sick. (Jake laughs) It's way faster. - Yeah. There's no question. We don't even need to. - It's gonna be like thirty seconds. - Yeah. Not even close. - So do you wanna know why? - I would love to know why. - You said it earlier. You just weren't really thinking about it. This has half the CUDA course of a 3090, it's likes seven thousandish I think. - Right, so it's just full of
like machine learning stuff. - Yeah, so it has basically
half the CUDA cores. So the fact that it is even close is kind of crazy in CUDA mode. But in Optix, what I found out is Optixs will use the Tensor cores for like AI Denoising, - [Andy] But nothing else. - Which you'll see in there. So I think it's falling back
to CUDA for the other stuff. - [Andy] Got it. - But the 3090 has Ray
Tracing and Tensor cores so. - Right. - It just demolishes (chuckles) - Where's the thing
where you can select apps and then tell it which GPU to use. Yeah, here we go. No, so it'll not allow you to
select the A100 to run games, even if we could pipe
it through our onboard, or through a different graphics
card like we did with that. - [Jake] It doesn't have DirectX Ray - Mining card ages ago. No DirectX support whatsoever. - [Jake] Let's check it in GPU-Z. - So way fewer CUDA cores. You can see that we go from over 10,000, to a lot less than 10,000. Pixel fillrates are actually higher. I guess that's your HBM2 memory talking. - [Jake] One point five
Gigabytes per second. - What's a 39, One point five terabytes per second. It's like - [Jake] 50% or more - 60% almost. - Holy banana. - But what about the supported tech? Yeah, so we can do CUDA, OpenCL, - [Jake] PhysX (laughing) - Sure. - [Jake] We should set
it as the PhysX card. - Dedicated PhysX card. All the rag dolls everywhere. - [Jake] And OpenGL but not
Direct anything or Vulkan even. - OpenGL. Now that's interesting. - [Jake] Go to the advanced tab. 'Cause you can select like a specific DirectX version at the top under General. Like well, the DX 12. What does it say? Device not found. It's the same as the mining card. It'll do OpenCL. So we can't mine on it (chuckles) - Alright. I mean, should we try that? - [Jake] Yeah, we could
do mining or folding or. - Sure, I have a feeling
that's gonna kind of suck for that too. - There's not.
- Like AI in mining. - I don't think so. It's still a big GPU dude. - So you can't. - Well suck is relative, right? Like for the price you'd never buy. - I think it might be better
than the CMP card though. Just a little bit.
- Shut up. - I think so. So the only thing you can adjust, I think this is the same with the CMP card is the core clock and the power limit. You can't mess with the memory speed. - [Andy] And you can move
the power limit only down it looks like. - [Jake] Yeah. Top is the 3090, bottom is the A100. - [Andy] Wow. That is a crap tone faster than a 3090. - [Jake] It's pretty
much the same as the CMP, but look at the efficiency. - 714 kilo hash per watt. - [Jake] And I bet you if
we lower the power limit to like 80, it's a little bit lower speed. Maybe we can go, I don't know. We probably don't have to
tinker with this too much. I mean, it doesn't draw that
much power to begin with, I guess.
- Yeah. I think it's pretty fricking efficient right outta the box. - I mean the efficiency is better. It's a little bit better, but before it was doing 175 mega hash roughly at 250 Watts, so it's pretty pretty good. 3090, you can probably do like 300 Watts with 120 mega hash. We're running the folding client now. I've had it running for a few minutes, and it's kind of hard to say. The thing with folding is, based on whatever project you're running, which is whatever job the
server has sent you to process, your points per day
will be higher or lower. So it's possible that the A100 got a job that rewards less points
than the 3090 did. It does look like it's a bit higher, but you can see our 39. This is like a little, like comparison app thing is 31% lower than the average. So it's probably just that this job doesn't give you that many points. - Got it. - The interesting part is the 3090's drawing. 400 watt. - [Both] 400. - Holy shnikes. - [Jake] A100 is drawing. - 240. (Jake laughing) Man, that's efficient and performance per what? Maybe gamers don't care that much. Actually we know for a fact, gamers don't care that much. In the data center, that's everything, because the cost of the card, is trivial compared to the
cost of power delivery, and cooling on a data center scale. - Especially when you have eight of these with a 400 watt power budget, like you would get on the SXM
cards in a single chassis, times 50 chassis, like that's a lot of power (chuckles) - Let's try something, machine learning. - Unfortunately for obvious reasons, most machine learning or deep learning, whatever you want to call it, benchmarks, don't run on windows. So instead I've switched over to Ubuntu and we've set up the CUDA Toolkit, which is gonna include our GPU drivers that we need to even run the thing as well as Docker and the
Nvidia Docker Container, which will allow us to run the benchmark. We're gonna be running
the ResNet-50 benchmark, which runs within TensorFlow two. This is a really, really common benchmark for big data, clusters and stuff. Except our cluster, is just one GPU. In a separate window, I've
got Nvidia SMI running. It's kind of like the Linux
version of MSI Afterburner, but it's made by Nvidia, so not quite, but what it's good for, is at least telling us our
power and the memory usage, which we should see spike a lot when we run this benchmark, I took the liberty of
pre-creating a command to run the benchmark. So we're gonna be running with XLA on to hopefully bump the numbers a bit. We will do that for the A100 as well. So no worries there. It should be the same as well as using, what do you want? Look, he left cause he
didn't have time for this. And now he's back. This is the world's most
expensive lint roller. (Andy chuckles) I even don't remember what
I was saying, damn it. Distractions aside, we're
gonna be running with XLA on. That'll probably give
us a bit higher number than you would normally, but it is still accurate and we're gonna be
running the same settings on the A100 as well. So no concerns there. We'll also be using a batch size of 512 as well as fp16 rather than fp32. So if you wanna re-create
these tests yourself, you totally can. Let's see what our 3090 can do. Look at that 24 gigs of
VRAM completely used. God, I don't know if
there's any application aside from like Premier
that will use all that VRAM. I'm sure Andy can attest
to that (strained laugh) Okay, 1,400 images a second. That's pretty respectable. I think like a V100, which is the predecessor to the A100 does like less than 1000. So the fact that a 3090, which is a consumer gaming card can pull off those kind
of numbers is huge. Mind you, the wattage, 412 Watts. That's a lot of power. It'll be interesting to
see how much more efficient the A100 is when we try that after. The test is done now, and the average total images per second is 1,400 and 35. It's pretty good. I've gone ahead and added our A100 so we can run the
benchmarks on that instead. And I'm expecting, this is gonna be
substantially more performant. So it's the same test. I'm just gonna run the command here. Gonna wait a few seconds. We got Nvidia SMI up again. You can see that it's
just running on the A100. The RAM on the 3090 is not getting filled. We're just using that as a display output. See, all 40 gigabytes used. That's crazy. (Jack laughing) If we thought the 3090 was fast. Look at that Andy. That's like a full 1000 images more, we're getting like 2400 instead of 1400 and the icing on the cake. If you look at Nvidia SMI, we're using like 250 Watts instead 400, while getting like almost
double the performance. That is nuts. - Probably the coolest thing about this whole experience though, is seeing the Ampere architecture on a seven nanometer
manufacturing process. 'cause you gotta remember while none of this is applicable
to our daily business. What this card does do, is excite me for the next
generation of Nvidia GPUs. Because even though the word on the street is that the upcoming Ada
Lovelace architecture, is not going to be that
different from Ampere. Consider this, Nvidia's gaming lineup is built on Samsung's
eight nanometer node, while the A100 is built on
TSMC's seven nanometer node. Now we've talked a fair
bit about how nanometers, from one fab to another, can't really be directly
compared in that way. But what we can do, is
say that it is rumored, that Nvidia will be building the newer ADA Lovelace gaming GPUs on TSMC's five nanometer node, which should perform even better than their seven nanometer node. And if the efficiency of improvements are anything like what we're seeing here, we are expecting those cards to be absolute freaking monsters. So good luck buying one. (Jake laughing) Hey, at least you can buy one of these. We've got new pillows, that's right. This is the, what are we calling it? - [Jake] Couch ripper. - The couch ripper the couch rip. It's an AMD themed version of our CPU pillow with alpaca
and regular filling blend. And this video is brought
to you by our sponsor, ID agent. 90% of data breaches start
with a phishing email. So you can reduce your
organization's chance of experiencing a cybersecurity disaster by up to 70% with security
awareness training. That includes phishing simulation, Bullphish ID by ID agent is
a Phish simulation platform that transforms your
biggest attack surface, into your biggest defensive asset. You can add every employee
to your security team with security awareness training that empowers them to spot
and stop Phishing threats. You can automate training campaigns and reporting for stress free, consistent training that gets results. Choose from a rich set of Plug and Play Phishing campaign kits and video lessons
accompanied by short quizzes, or you can create your
own fishing campaigns and training materials easily. Bullphish ID provides
effective affordable one-Stop phishing resistance training that fits any business and budget. Get two months for free and 50% off setup at bullphishid@it.idagent.com/linus If you guys enjoyed this video, maybe go check out our previous video, looking in more depth at the CMP 170 HX. - [Jake] I like this silver better. - If we were smart, we'd be mining on this, but we're not that smart. - [Jake] Well, you know, mining is bad.
