The Storage of the Gods: A Deep Dive Into the Intel Optane P5800X

this video is brought to you by asrock and the z590 velocita this is a z590 board it's got some pretty killer features 10 usb ports at the back we've got intel and killer knicks as well as the wi-fi 6e which is supported by the killer double shot pro software which i recently did a video on you should check that out it has real tech alc 1220 audio with the nahimic audio drivers and the usb type c front panel connection is a 20 gigabit connection this board supports up to ddr4 4800 it's also get some premium features right in the box one of them an extra vrm fan i mean this is a 50 amp vrm solution which is more than adequate for any kind of ordinary stuff that you're going to be doing with 11th gen rocket lake cpus pci express 4 in the nvme and the x16 slot but if you do find a 10th gen intel cpu on sale or a really good deal well that'll work just fine on this board too we actually selected this board to use as our new storage platform test so we're going to be testing pcie 3 and pcie 4 storage on this platform ssds like the intel p5800x and with all that this motherboard comes with a gpu holder bracket good job asrock and thanks for sponsoring this video all right we're here to talk about something very special the intel p5800x one of the most confounding pieces of technology ever that i spent my own money on at least this particular one although i am working on intel to give me a second one because i have some ideas to do some really cool stuff but i'm gonna need more than one of these and uh they're expensive but they're good i mean intel knows what they have this is the technology like the gods of old descended from mount olympus and her face just forged this with their you know awesome forgery skills i mean that in the best possible way and zeus used his lightning skills from from on high to zap the lightning into the rock and thus was born optane which is an entirely different storage medium than nand flash is insanely exciting you know there was this is not the first one it was prior generation there was this weird star citizen marketing campaign where i think there's i wonder if the star citizen credits in here are still good and then you know like the enterprise grade p 4800x so this was for the enterprise you didn't get star citizen this was for the enthusiast which is basically the same thing just packaged up a little different and a little cheaper than the enterprise drive as yet there is no enthusiast equivalent of this and yet it's pci express 4 and insanely fast let's do a deep dive [Music] look i always say i'm a big fan of optane you just had no idea how much of a fan of optane that i really was this is the 800 gigabyte model it's the one that i picked up the 400 gig is just a little too small and the 1.6 terabyte was just a little too unaffordable at three and a half thousand dollars over a thousand dollars a terabyte when nand flash storage even the good stuff you know from samsung like the high-end stuff with a custom controller because samsung's always get the spicy you know custom engineering yeah it's on the order of like 150 dollars a terabyte not 1500 a terabyte it's 10 times more so yeah i'd say intel knows what they have here this is really really good stuff you see these used in the enterprise to do things like database right caching and online application processing transaction logs for database servers things like that and the reason for that is these are so fast you can do quality of service engineering on them to be sure that read write operations are basically guaranteed in a short order you can also use an array of them to make even more specific guarantees about workloads and uh time delta and things like that the reality is that this is 3dx point this is not nand so it's a different technology and this is a pci express 4 device that saturates the pci express 4 by 4 it's four lane interface at up to eight gigabytes per second intel actually rates these for like seven and a half gigabytes per second but in my testing i can peak in a perfectly perfectly ideal scenario i can get this thing up to two and a half million iops one device two and a half million apps that is insane uh intel rates it for about one and a half million iops in a purely 4k you know random read scenario and if you get a 70 30 mixed read write they rate it for two million iops you can really cherry pick stuff and get two and a half million iops so kudos for intel for not going with the two and a half million number but to give you an idea so a lot of the testing i had initially done on threader for pro it takes about four and a half cores on threadripper pro for truly random i o to saturate this because four threadripper cores can't keep up with the random i o on this thing now to be sure one core can saturate this thing with a sequential one core can handle a sequential seven and a half gigabyte per second transfer but when we're talking about random ios and servicing two million operations per second on something that's only running at three or four billion operations per second we have you know low five figures worth of cpu instructions per core to deal with that so we're going to need more cores to share the load because of the sheer number of cpu instructions necessary to handle that workload so i've done the full benchmarking on this the crystal disk mark numbers the latency numbers the number of iops things that you can expect on a desktop performance the benchmark numbers don't really tell you the whole story the latency it's like with nandflash uh the latency when you go to look up something is usually on the order of 100-150 microseconds worst case scenario on this is like 60 microseconds its best case scenario is about six microseconds so it's about on average about halfway between the latency of main memory and nand flash of course main memory is you know ephemeral you lose power what was in my memory is lost that's why you see optane in a dim format and they finally have a controller and matrix and all of that stuff that can actually keep up it's been a while you know intel was a little late to the game with pizza express 4 for these a little surprised but this is each device can saturate pc express 4 which is nuts completely nuts and i've got ice lake in the house to do testing on the island side of things so in fact doing the testing here has broken my bench for doing ssd testing so i'm actually switching to rocket lake to do i o testing because it requires less cores to fully saturate the interface on something like this and this is the absolute high end of what you can expect so software wise how can you make this go fast that's where this story kind of falls apart intel expects you to use this as the whole drive or for your application to know what obtain is and to be able to deal with it so for something like microsoft sql server or postgresql you can store the replay logs on this you can store transaction logs on this and the database server intrinsically knows how to deal with that it knows how to deal with a different block device for those kinds of storage it's not built for optane specifically uh but it knows how to deal with a block device and the separation of things in the block device and so that works really well things like vmware if you're running an all-flash setup with vmware vmware knows specifically how to deal with optane if you're running vsan um then your and your capacity tier is also flash then it will use something like the optane cache if you're using an obtained device as your cache tier it'll use it for right caching basically and these are exceptionally good at that the endurance of this 100 drive rights per day so 800 gigabytes i can write five times 800 gigabytes to this drive per day for five years and it's still under warranty these things are basically indestructible this is the one storage meeting where the price has actually gone up over time i mean i don't know if you noticed but like even our our plucky old p 4800x ssd has basically held its value really well and they're still kind of pricey on ebay even though these don't saturate the pci express 3.0 interface the gauntlets have come off as i say and the gods have descended from mount olympus and brought us the p5800x it's that good and yes you absolutely absolutely can use this as your primary storage drive but here's why that's dumb windows is huge it's like 40 gigabytes by the time you get the windows side-by-side stuff and all that kind of stuff but the pathway that your specific workflow uses in all of that code to boot up the machine is only on the order of like five six gigabytes eight gigabytes at most so you really want something sitting between your operating system and your storage medium that's going to move all the crap that you're not actually using that you're not actually depending on for speed off of this onto something more affordable like nand flash and there's not really okay well there are enterprise products to deal with that but they are wildly expensive um like the inmodus enterprise software it's very very good about dealing with this and it's available on like linux and stuff like that that's an option for pedestrians you know pleb tier i just want my windows 10 workstation to run fast and i'm not willing to run virtual machines primo cash primo cash is pretty much it beyond the benchmarks what you would see with this drive is really kind of nuts and we need to cut to a desk benchmark window for me to show you so when we were talking about h20 on the laptop you know h20 is a combination optane nand flash device where intel's doing that but intel is doing the software part of it with rst i really like that product it's really good only applicable on intel platforms primo cache is applicable pretty much everywhere and when you use this drive on say an amd platform it will work fine it shows up as a u.2 device so if you're doing the database workload or you're doing anything like that works fine if you're using primo cache something like that or it's totally fine if you want to use this on an intel platform with intel rst guess what nope they're not going to touch that with a 10 foot pole it's not been qualified it's not been tested they do not want it to work you should you're using it wrong you're using it wrong but i showed you on a laptop how you could launch a bunch of programs simultaneously so if you are crazy enough to use this just as your boot drive uh you can and it's fine on team red rt blue or even on arm for that matter uh and it will be insanely fast and the technical reason for that is a large large part of the technical reason for that is that most of the user interactive applications stuff that you experience that's that's low q depths you know the computer's only getting one to four things at a time from storage and this storage is exceptionally good at that but unlike old octane it's also exceptionally good at streaming as well so things like loading a level so if you bought this for star citizen star citizen is going to be as bananas going from this to this as it was when you went to this from a nand ssd it's that insane and the numbers are on the level one form if you want to check that out the drive doesn't slow down as it's loaded so like with the samsung 980 pro it starts out at 1.1 million iops it will get a little bit slower as it fills you can see that in the fio benchmarks if you're running something like the mushkin gamma their new ssd with the new feison e18 controller which is really really an incredible piece of engineering from fysen they've done a great job with that still fundamental limitation is nand flash nand flash is not going to take care of you in terms of those kinds of low latency things you can there's ram on the card and some things can be cached and there's a lot of software optimization you can do but at the end of the day it's just not as fast so you're kind of limited in what you can do that drive starts out at 1.7 million iops oh well that's within striking distance of optane right no you still have the really high latency so you're going to be waiting a long time relatively on those q depth one to four things to come back and that's what makes that interactive experience so good this is our new benchmark machine it's based around rocket lake the 11 600k why rocket lake well it's got pci express 4 but i encountered some anomalies when i was testing on the 5950x platform i mean that's a 16 core platform versus a 6 core what's going on uh it's interesting enough that i'm going to do a separate video on it but the amount of ios that you can do while only using one cpu core was quite a bit higher on rocket lake than it was on the 5950x now for real world benchmarking loading games stuff like that it wasn't really much of a difference that i could tell between the two platforms when you're sort of i o constraint i mean even using a p 5800x is like your steam drive it didn't really seem to make a lot of difference so our rocket lake pci express 4 ssd test platform is based around the asrock velocita z590 and i'm running uh linux off of a sata ssd we also have windows that's running off of a sata ssd so that we can just throw in a pci express 3 or 4 nvme or u.2 in this case because i've got the u.2 carrier card for the p5800x and it is running at pci express 4 speeds so that we can do the full monty of benchmarks now intel rates this drive at 1.5 to 2 million iops depending on what you're doing but if you're using ridiculously small 512 byte sectors and you create some other not exactly realistic real world testing scenarios artificial benchmarks let's say you know four or five million iops is possible and you're talking about three million like upwards of two three million iops being possible per core on rocket lake running at about five gigahertz so that's pretty interesting this is a pretty interesting test platform we're going to do a separate video on that but boy howdy that is fast that's just another data point that says the p5800x is in a class by itself it's literally from another planet in terms of being in a class by itself we had to set up a whole new test bench if you want to take a look at the fio scripts and other stuff that we're contemplating for a standardized 2021 ssd test bench i've got a thread up on the level one forums definitely come and contribute i've got fio files for read mixed read write and write workloads that i want to kind of have a standardized setup just to give you an idea of how insane the p5800x is here is a side by side with a four nvme array of the samsung 983s these are also enterprise-grade ssds this is eight terabytes of flash in this array and look at these performance numbers yeah the streaming score is a little better than a single p5800x but otherwise the p5800x completely demolishes that array it's really truly breathtaking performance all right so with all the gushing out of the way what are the bad things well the price as i've said about five times now but also these are only available up to 1.6 terabyte capacities so in an age where i can get this same form factor well actually it's a little bit more bulbous it's like this uh 16 terabytes 16 terabytes readily available 45 terabytes available on special order 1.6 terabytes kind of paltry but like i say for your particular workload you really don't need a lot but you do need software that will look through all of the stuff that you're actually using and make sure that everything on here is stuff that you're actually going to need those windows help files that you never load if you use this as a boot drive it is an absolute waste because all those help files you're never going to look at are at the ready but you're never going to load them so they never needed to be fast in the first place you need software like primo cache to deal with that or some kind of enterprise caching software and for the enterprise it's basically there for sql server it's basically there a linux you know you can even do this with lvm block cache this drops right in for lvm block cache it's really great a lot of people are using optane for their zfs intent log because the right endurance is so good those are options but even at 1.6 terabyte capacities it's a little on the small side if you're going to use it for things like your zfs metadata special device you're back into you probably need to use the 16 terabyte nand flash for that if you have particularly large pools because you'd be surprised how quickly you can go through 1.6 terabytes for the special metadata devices i mean just you know 250 300 terabytes of storage and you're you're entering that one ish terabyte of uh uh storage for metadata for most volumes that would store it depends on the files that you're storing a whole bunch of other parameters i don't want to make blanket uh descriptions but generally that's how i'd size it so yeah will we see 10 terabyte opting no not as long as the price is decided not even facebook and google can afford it but it really is basically alien technology deposited in our hands it really is that good don't take my word for it you know check out other reviews check out other stuff but for that use case it's really it's really incredible if you have anything that you think that would work well for testing this reach out let's run some tests because i've got this in test machines i've also got many of the older ones not just the the 900 series but also a bunch of these 375 gig p4800xs so we can do some fun stuff let's get to it i'm going to lose level one i'm starting out and you can find me being really excited about the p5800x and optane i'm signing out and i'll see you later you

Channel: Level1Techs

Views: 26,782

Keywords: technology, science, design, ux, computers, hardware, software, programming, level1, l1, level one

Id: fINeI6Xm6t0

Length: 18min 4sec (1084 seconds)

Published: Thu Jun 10 2021

Reddit Comments

[u/Arbabender]

The more I read/see about Optane/3D XPoint, the sadder I get that it's mostly dead in the prosumer space.

A moderately sized Optane cache drive in front of a decently fast NVMe SSD sounds like the dream.

[u/JimShirt]

3D XPoint seems like the easiest way to increase the speedyness factor of a computer, but its not really adopted yet... I want this!

[u/HilLiedTroopsDied]

my 280GB 900P optane is feeling old now.

​

Intel we want consumer PCI-E 4 drives!

[u/greggm2000]

I'd love to have one of these. Not willing to pay what it costs though, especially after Chia.