IT WORKS!!! - Six 8K Workstations, 1 CPU Finale

on our last episode in this epic series you guys saw the not quite 100 000 pc finally finished and ready to concurrently edit six 8k red videos but we cut the video short just before we loaded up our editing software adobe premiere and then made our editing team actually try to use the thing but don't worry because right after this word from our sponsor that is exactly what we'll be doing [Music] if you've already seen the rest of the videos in this series you can skip to where we test drive the system but if you're new here's a rundown this is the 100 000 pc or 68k editors one cpu project it started out as a sparkle in my eye more than a year ago when i began to wonder if we could take the seven gamers one cpu concept our seven user virtualized gaming rig and morph it into something actually productive like for editing our videos from there we spent months part shopping online troubleshooting broken hardware and testing different configurations in an effort to build a single machine powerful enough for six concurrent users and without the traditional virtualization headaches like a lack of hot pluggable usb so let's run through it now i couldn't get the quadro gv100 graphics cards that would have brought this build up near the target 100 000 mark but i was able to get my hands on six nvidia titan vs which have essentially the same horsepower just at about a third the price think of it as kind of the audi r8 to the lamborghini huracan for cpus the plan was to use the two xeon platinum 8180 processors that intel graciously sent us so those are 28 core 2.5 gigahertz behemoths except that i dropped one of them off the table and broke a memory channel we can't have one of these 32 gig ecc memory modules from kingston not working so we ended up having to buy a matching set of qualifying sample cpus on ebay definitely took an l on that one in order to reach our target of two usb ports for each editor one stop systems provided us with this crazy pci express expansion board which took one slot of our motherboard and turned it into eight that gave us room for the six usb cards that we'd need one for every user as well as a 40 gigabit melanox network adapter now mounting it wasn't particularly difficult but it did require cutting a massive hole in a case that you literally can't even buy anymore which sucks in its own way for storage we ended up using some fancy m.2 to u.2 to pci express adapters to plug in our dual pci express 960 gig optane nvme ssds these bad boys have excellent random i o performance which is important to keep program loading snappy when you've got multiple users hitting the drive at the same time so we combined all that with a bit of water cooling hardware from ek some power supplies from evga talking 3 200 watts total and some hard drives and we've got a recipe for a seriously badass system now the problem is when you play with this kind of semi-experimental setup you're bound to run into some issues so in our case that manifested in our first choice motherboard from asus which after days of crash troubleshooting ended up having a bios defect that they say they've fixed now but we haven't verified that caused some of the pci express slots to not bifurcate or split properly as a result our virtual machines kept crashing thankfully with the help of linux guru wendell how's it going we got it diagnosed and the guys over at tien sent over something that just worked right out of the gate so we switched over to that giving us our sort of sexy but in an industrial way and definitely not one hundred thousand dollar final build we've already made sure the vms work and the cooling system is sufficient so all that's left now is to sick our editors on it and see if it can actually handle the load so now we're at the moment of truth we are taking it into the editing den and for better or for worse we are having our editors try all at the same time six editors to edit 8k footage off a single machine there are a couple of things we need to do first though one is i need to pull this power supply out oh including its mount thanks jake we're having fun we're having fun already and we need to swap out the 10 gigabit network card that we have in here now for this 100 gigabit network card now we're only going to be using it at 40 gigabit we're going to be plugged into our dell switch that's in our server room using a uh a fiber optic cable that melanox very helpfully sent over to us but the reason that we need to go from 10 gig to 40 is because even though 10 gig is lots for a single editor editing 8k video if you have 6 editors times 8k you are well in excess of 10 gigs so 40 gig should be plenty one other small upgrade i'm making here is both the one stop systems daughter board card and our 100 gigabit nick are really designed to have like server airflow over them so we're just going to throw this fan on here did we figure out the final price tag for this thing uh no it's like 30 or 40 grand it's pretty good it's a lot less impressive but yeah well we didn't end up putting quadros in it that would have pushed us to 100 grand you know what the craziest part of this is this looks like a relatively economical solution compared to the new mac pro you buy mac pros for six editors seriously seriously the base spec and you're at like 36 grand already this is about competitive i think with the base spec it has more ram better graphics card 64 gigs of ram you'll have your own titan v it's water cooled look how horrified all these people look so guys remember how cool it was when you each had individual editing workstations oh new paradigm a single tower and everybody just plugs into it it's a great idea good [Laughter] so some of the cabling for this setup is admittedly a little bit janky i'd like to take a moment to bask in the elegance it definitely froze of our solution here uh okay i'm gonna have to restart it give it yield restart if you didn't pick up on this at any point prior to this moment in this series this is not actually a solution that we're endorsing this is a super super clutchy weird stupid hack around for lulls now all we need to do is enable vms uh that's quite strange oh logitech software go away please i can't deal with this right now oh wow go away please ulta4 control delete is that going to help me okay task holy crap you don't get to go on top of task manager what about this one uh okay well that's a lot better you no not like that i mean i guess technically we can move forward with one monitor okay so theoretically we've got a 10 gig virtual link let's go ahead and we might as well see if it works at all right okay we just broke one gigabit per second so we're up sitting around 1.5 to 2.5 gigabit per second while we're loading in all the files here i've had better at least one displayport cable is dead out of three clearly when show needs to start at some point so and also this timeline is super super super chunky it's very slow i mean is premiere only spazzing out when i try and go full quality maybe i'm just getting too ambitious here here we go i think this is a network problem i don't know now playing okay seems to be playing normally oh this is hang on back at the beginning come on come on okay took a while to load let's try loading up oh there's a bit of a hitch there f oh there we go there's a couple drop frames at the end there anyway so let's try one quarter just because we knew this wasn't going to be like the most powerful workstation i always work on one quarter oh no just pause just off and just wait a little bit and now go and it should be fine yeah that's that's literally you just have to wait the premiere whisperer yes yes i don't know what's going on yeah see it's like weird super weird you know okay maybe unraid doesn't like handle this well [Music] that was not successful but there's good news so we spent the last week tinkering with it and it looks like resolving our performance issues could be as simple as enabling a feature called huge pages now under normal circumstances when you've got virtual machines running within a host operating system in order for those vms to know where to find their memory there has to be this table or a map of where their virtual memory corresponds to the physical memory of the system the problem is that the default size of these you know little table entries is only four kilobytes which is tiny so you've got 384 gigabytes of memory you end up with so many entries on this table that it can't all be contained and instead of just referring back to the map for where the memory is the cpu has to calculate it instead which adds an enormous amount of cpu overhead so with huge pages we take those four kilobyte chunks and we make them two megabyte chunks which drastically reduces the size of our table and therefore our cpu overhead now by default unraid doesn't have this feature enabled they had to send us over a special build because they didn't observe any performance difference between regular pages and huge pages but they probably didn't test with 384 gigs of memory and six vms running a ram intensive workload so we've got it and we're ready to carry it back in there and give it a shot right i'll carry it more towards the middle okay yeah definitely not late regarding taking that side uh oh you want it to go in this way actually that's whatever oh that's what i was trying to hi everyone here to ruin your day straight up for the second time so we had some issues with our super long displayport cables last time some of them didn't seem to be working right so we're going slightly less ambitious one monitor per editor this time around and we've pre-tested these cables wait you plugged in the other end of this though right oh well this is just broken [Laughter] so our latest problem appears to be cable slash monitor related we ran an hdmi cable to dennis's station and no problem okay editor two taran let us know if it lights up hey you know what i want i want dvi back i loved dvi you know why because it worked so we don't have enough long hdmi cables but what we do have is couplers and at this stage in the game i think it's more likely that two hdmi cables with a coupler is gonna work than these stupid long displayport cables there we go it's on yeah i got it okay all right we got six sex machines out some yeah nobody move i mean it's it's working like i like the responsiveness so far okay good can i get that sound bite uh no no i got it we heard that good you're gonna fire up after effects what are you trying to do to us here you say your machine is amazing no go ahead just do whatever you got to do okay so dennis is hitting it full ham cpu usage at around 70 oh can you even like mouse bro um we don't know we've never tried this before and no one has ever tried this before terrible actually are you talking about your popcorn chicken or about the editing experience 1.8 gigabit there you go it just had a bunch of stuff stored locally oh okay yeah 3.2 gig right now 5.4 what are you even doing why don't oh after effects is still in the background yeah dennis you're you're killing you're killing the server and everything no no it's fine it's fine it's fine this is what we want to see okay so dennis's is running okay now yeah everyone make sure you're doing something as heavy as you can right now let's see how much we can make it hurt so we're at about 46 47 55 59 68. okay how bad's it chugging kids who's vmware hoffman what are you doing what are you doing so guys with everyone hitting it hard how bad is it how would you compare it to your regular workstation um it doesn't really play for on premiere on this machine but does it on your regular one i don't play for my video okay so i can't really come but like it's i think it's the same pretty much the same okay ed how would you compare to your regulars do you even edit video anymore you know what forget it runs on google or it's better it's briefly better than what what i know because his desktop sucks yeah his isn't as good as the other ones oh okay okay pella how would you compare this to your regular machine i'll go one down can't do it perfect you'll go one down nine nine out of ten score okay alex passable passable it works for one screen that's okay people seem to be forgetting other than the monitors other than the monitors i can't no i'm not there's no way i'm subscribed what i'm not doing other than monitors well though because we could solve we still monitors is solvable at around four hundred dollars per editor well it sounds like we just didn't want to spend the money it sounds like you need to well we're not actually deploying this whoa technicalities come on out of 10 out of 10 performance responsiveness yes 8.5 8.5 hoffman how we how did we do i can't tell really you can't tell the difference oh they're both like lagging this is an upgrade over the three-year-old systems i think i'm going to stop fooling around because this is two full days that the editors haven't had their transcode server they're finally here my 24 pcie ssds for my new storage server things like timeline scrubbing are going to be much faster on these systems than they were on the old ones [Music] so we're looking at a 40 gig connection here and we're gonna have everyone play back um phantom footage everybody ready go ahead and hit it let's see how hard we can make it hurt 8 12 13. oh just 14 15 gigabit per second ouch all right so the last thing i'd like to see is everybody just grab a giant file off the server and copy it to your desktop when i say go okay i want to see how close we can get to 40 gigabit because theoretically we should be able to saturate that connection windows file transfer though still three two one go oh oh how much space is there they're only 100 gig things okay you gotta find something under 30 gigs okay so i wanted to have four octane drives in the machine but remember we had to change motherboards six months ago okay three two one hit it three twelve nineteen twenty one so would you what are you guys getting approximately each three four five hundred megabytes a second not too shabby not full 10 gig but not too shabby either oh we're done that's it six editors one cpu is complete and i think it's fair to say mission accomplished boys because all of our editors described it as eight and a half to ten out of ten like their usual editing experience on you guys have 69 50 x's right is that right yeah i think so they have 6950xs with 64 gigs of ram and then they've got uh titan x pascals if i recall correctly or quadros oh you you stole a quadro that's fine doesn't matter if you guys dislike this video you can hit that button but if you liked it hit like get subscribed or maybe consider checking out where to buy the stuff we featured at the link in the video description massive shout out of course to tien kingston nvidia intel um evga uh case labs rip one stop systems ek water blocks uh who else hooked us up on this thing corsair sent us over no knock to a sentence over fans corsair commander pros uh cable uh cable mod cables yeah the ebay seller that i bought now i paid for those those don't they don't get a shout out so that's it uh yeah seagate hooked us up with some hard drives on here which we could have set up with local storage oh also down there is our merch store which has cool shirts like the hoodie i'm wearing um and our community forum which you guys should totally join

Channel: Linus Tech Tips

Views: 2,542,421

Keywords: 6 editors 1 cpu, editing, premiere, adobe, after effects, final cut, virtualization, 7 gamers 1 cpu, gaming, computers, nvidia, intel, 28 core, cpu, gpu

Id: 1yFQd4MaKK0

Length: 18min 28sec (1108 seconds)

Published: Sun Jul 28 2019