We’re running out of internet - Steam Game Caching Server

we have a desperate lack of internet because we are about to host over 200 gamers over at lab2 for the whale lan party and we are stuck with an internet connection that is slower than the one at my freaking house i mean okay fine it's not that slow but still two gigabit down split up between 250 people that's only like eight megabits per person that's one megabyte a second you're not going to be downloading warzone if you're on that kind of speed fortunately we have an ace up our sleeves oh it's a big sleeve because today with the help of intel who sponsored today's video and a portion of our whale land party along with supermicro who sent over this server uh ubiquity i guess gets a shout out infinite cables we are gonna build the ultimate steam battle net origin windows update epic game store any game you can think of caching server to give us local high-speed copies of as many games as possible and we're going to test this thing with over 200 gamers to see just how hard we can push it it's going to be freaking epic and steam and good old games i battled that damn it that's not in the script [Music] caching game downloads and updates on your local network is nothing new the basic principle revolves around dns or domain name system so when you go into your browser and you type in any website let's say lttstore.com your computer contacts a dns server which converts that text based url into the numerical ip address of the website that you want to go to that way your machine knows where to send the request now the caching solution that we're going to be deploying today sits in between the dns servers like google's 8.8.8.8 or cloudflare 1.1.1.1 and it will actually redirect those requests to the nginx web server software that is running on it engine x then checks the request against a list of cachable content in our case windows update steam origin battlenet and basically all the other game stores and if it matches it will cache any incoming content from those stores onto the machine storage that means that the next time someone wants to download that same thing say at a lan party where everyone wants to download the same game at the same time nginx will automatically redirect that download to the local cache of the game rather than download it from the internet now for a home gamer with a single system it's not going to matter much as you likely aren't re-downloading the same games very often but when you've got 200 plus systems on a small connection the benefit is huge now if you're an og you'll probably remember the video that we shot deploying a solution like this in our office nearly four years ago but quite a few things have changed since then for one the process of deploying a steam caching server has gotten a lot easier and a lot more robust gone are the days of running a separate docker container for every service that you intend to cash it is all built into one with a sweet piece of software called lan cash creative name and two this time we actually have 200 plus people and their computers at an actual event where we can coordinate a test of our caching server and see what sorts of wild speeds we can push from it we're getting a little bit ahead of ourselves here though why don't we talk a little bit about the hardware this is the super server sys510p dash wtr from supermicro it's actually a fairly simple and modest one used server up front we've got four three and a half inch bays which by default are configured for sata drives but can be upgraded to sas or nvme like you see here these are gonna be spicy and then around back and then around back it's equally boring there's the two full height and one half height pci express gen 4 by 16 slots alongside dual 10 gig networking management some usb and dual redundant power supplies let's pull one of these boys out those are long but only 500 watts that sounds pretty weak these days but it's actually perfectly reasonable since this system is only designed to house up to a 270 watt cpu along with the four storage drives up front there's no gpus or anything like that since all we need to run our game caching is a beefy cpu and storage so it's perfect for our use case the machine came from the factory at supermicro with an intel xeon gold 6314u ice lake processor that's a 32 core 64 thread 205 watt chip it's not the beefiest cpu that you can install on this machine by any stretch of the imagination but it is plenty for what we're trying to do since the caching process is actually fairly efficient we also have eight sticks of registered ecc memory and since linux our operating system of choice here will cache our frequently accessed files in system memory our best bet is to cram this system full of as much ram as possible to take some of the load off of our storage drives fortunately we've got a whopping 256 gigs of 3200 mega transfer per second sk hynix memory to fill up all of the available slots the truth is our ram is not really going to be a major factor for us because we went so flipping overkill on our storage originally our plan was to throw four of kyoccia's ultra badass cm6r 30 terabyte gen 4 nvme drives in a software raid 0. unfortunately we were on a bit of a tight schedule getting the server here in time for the lan in fact we are shooting this on wednesday the lan party starts on friday and because of the rush a small oopsie was made on our order and while the sleds say nvme the back plane that it's connected to is only plugged in via sata so they overnighted some cables we should have those for the actual event but in the meantime we had to get a little bit creative we're going to have to wire up just two of these ssds right now get creative oh god really this is how we're doing it oh and then here's the other one it gets better it's not that bad you know i feel like we had a real role reversal moment here yeah yeah get creative here look you can take one of the backplane uh molexes out there and then i got a little a long boy yeah it works great see this look at that it's mint like from the factory what jake just handed me is a pci express by four to u.2 adapter card which is fair enough then we've got this other one which is a pci e by four to mini sas hd uh but the pcie version of it that then goes to u.2 and then plugs in with this molex connector to the power that used to go into the back plane jake don't yell at me yell at super micro no i don't want to yell at super micro they did us a big solid put that in the yeah doing this for us so i don't even whatever super micro you're chill you're chilling my book oh god oh god no no it's not in the slot at all well it's in the spot it needs to be there there we go yes mission success how many ltt people does it take to put in an expansion course a couple three two to do it and one to film them because we make videos a few yes that's a joke oh wait we gotta put some screws in that first you want screws in it what here i got some screws what you don't even know no no put some screws in it seriously i don't even need screws it's like a ten thousand dollar ssd what they're 30 terabytes but the script said we were going to use them so we are using them which one the 30 terabyte keyocci is yeah oh bloody hell tape what um i don't understand well no this grip said we were going to use them we were going to use and it sounded like we were we're only going to use two we're only going to use two for the whole land these things read at like seven gigabytes a second i mean that's fair that's really really fast how much network would we need for that and you gotta remember that when you're downloading games that's a highly sequential activity yes having multiple users hitting them at the same time is going to add some randomness to the requests but it's still relatively sequential compared to if we were running this as a database server or something like that just do this for you it's fine okay i mean that is like your job yes all right fair we do have one small problem it's cool that this server has dual onboard 10 gig network cards that would have been sufficient if we were going to use it just internally here but since we've got 200 people hitting it oh and since ubiquity provided an enterprise xg24 network switch which happens to have dual 25 gig uplink ports well i think you can see where i'm going with this no you can't it's right here this is a dual port 25 gigabit melanox connect x5 card which we should be able to link together for a whopping 50 gigabit combined throughput you know we should update your job description to do things for me properly we don't do things properly here this is lines meaning no no no we do we do things properly i mean this is going to work right i mean hopefully i haven't tested anything so you haven't we already pre-cached three and a half terabytes nice but they said we should pre-cache as much as possible so we're gonna try to pre-cache like 40 terabytes i have never used command strips okay so this side's peeled really yeah okay well i just put holes in my walls like an animal if we were going to be deploying this long term then it would be better to use a nas operating system like trunas but since we are just using it for the event we're throwing ubuntu server 20.04 on it you know boot it up i do i'm convinced this guy likes tinkering with gaming technology like this more than actually playing games whoa let's see if he's any good dude dude you played mw2 with me dude i smacked everybody it's fine i played a lot of mw2 no one calls it mw2 i do when you have to say it so much because you play it so much it's not even faster m w u 2. mw2 modern warfare 2. it's the same number of syllables yeah but it doesn't mean mw2 modern warfare 2. that takes way longer ah marginally longer once the os is installed and we're ssh'd in it's best practice to install any available updates then we can go ahead and install docker io and docker compose these are the only prerequisites required by landcash the pre-made caching solution that we're going to be using that comes nicely bundled in a docker container before we download land cache though we're going to want to set up our ssd raid 0. digitalocean has a really nice tutorial for doing this on ubuntu server on their website so we're just going to copy paste all the commands from there well let's see if our array imported that i definitely didn't make earlier it did not oh it did look so i've already cached 3.3 terabytes okay of 56. raid 0 baby do we even have enough of this to do we even have enough time to fill that um maybe here with our storage ready and docker installed we can run the download commands from landcash.net edit the config and we're off to the races now editing the config is where things get a little trickier it's not that bad well it's trickier than just copy pasting yeah yeah well the main things you have to do are set your system ip this is so lancash actually knows what ip address you're using if you don't i think it actually binds to all the ip addresses and it might still work but probably just set it anyways then you're going to set your cache root which is where everything gets stored yeah really want to put this in the right spot do that if you don't put it in the right spot it's just going to fill your boot ssd and you're going to have a really bad time when it gets completely full your system will probably go super unresponsive to the point where like it will take forever to enter requests and even i've had that happen before when you have zero bits of free space on an ssd it crap sometimes you can't even uninstall stuff because you can't like write the log file for the uninstall it's really bad bad time then you'll want to set your cache size as well as your cache index size for the ram and what is the ratio one gig to 256 megs per terabyte per terabyte i set up the equivalent of 60 terabytes we will need to up that when we add the other drives the last thing you'll want to do is set your time zone it's not mission critical but it's nice for logs so if you're like having issues and your your logs are in a different time zone you're like what the hell is going on we set that to vancouver that's where we're at okay that's all we got to do man we can just click start systems on our network won't magically start using the cache they don't even know it exists right so we actually have to go on the system and point it to it and the easiest way to do this if you're just testing is to go into the settings and go enter your ip address don't put 8.8.8 in there that could confuse things just just just put your land cache it will forward everything else unless it doesn't in which case having a secondary one would a lot of the times windows doesn't like properly fail over anyway are you serious so remember when our internet went down the other day it's because the steam cache was offline and the internet was fine just nobody had dns resolving and in the dhcp server i had 8.8.8 as option two but they just window sucks okay if you're doing this as an actual deployment you'll want to set that ip address for the dns server in your router otherwise you know setting it on all the different computers individually is going to be an absolute nightmare especially if the ip ever changes okay it's set we're good i'm firing up steam uh right disk management yeah i put a seven terabyte on here oh cool um wow you'll have to delete that it's probably like a zfs part two later buddy yeah put d we'll call it big d big d why are you an officer he's in offline mode you don't have to when you when you go to install the game it'll prompt you what the hell confused why doesn't have internet what just happened what even just happened we had internet we had all the internet we've run out of internet where's my series of tubes darn it damn it dale yeah this is sponsored by intel we should have an intel nick in there when that actually works if we use the intel mic it would probably be fine we should just go get one this is not working oh it's something hey jake have you seen oh you you spoiled it what well i was gonna i was trying to ask you on camera i have seen you seen what logistics did to all of our intel nicknames like the adorable little content i sort of instructed them a little bit on it i love it okay let's just leave it we have to install drivers come on let's go steam steam screen record oh god here we go this computer got to get my you don't need to do it just click install no click install we'll have to add one nice ui oh really i hate it i mean it's it looks pretty i don't know if it actually is simpler wow you totally could have just done this no i couldn't it wouldn't have been here yes it would it would it'll say do you want to install a new library i don't believe you okay go see look quick it's gonna go too fast and you're gonna miss it hold on where is it where is it that's not that fast yeah that's not that fast this is like oh did you remove the dns no uh i think you might have no it's there it's not using it no that's it yeah it's configured properly uh just remove it and then add it again ipconfig slash flash dns no oh forward slash flush dana nope flush what are you drunk like you have to put a space buddy there flush dns oh one word there you go he said slash slash no it's just like what is slash slash i should forward it slightly he said slash slash whatever anyways okay okay let the record show that i need to leave go to steam damn it oh my god oh my god plug into the intel card you don't even have drivers what difference does it make well i can get them i can't even you're plugged into the intel card i can't get drivers how am i supposed to get drivers for this guy 2.5 gig man this is plugged into 10 gig why don't we just go to my computer upstairs which is also plugged into 10 gig that is legitimately an option if this doesn't work immediately that's where we're going to find i'm glad this is going to be such a pain-free experience for the land attendees oh it will be oh there you see like this the cpu in this system is like oh my god yeah we had a 1200 ks downstairs which is what we really wanted to use because that thing is fast look my ssd is just 960 pro is just at 100 percent this is this fast look it's riding at 1.2 gigs a second something to consider is that not every game is set up exactly the same in terms of the way that the files are compressed so some of them are more cpu heavy some are less cpu heavy um counter-strike global offensive might not be the best benchmark but we are getting 200 and 230 megabytes a second right now oh i do have a way to demo this that's more better um so the steam cache pre-filling thing just downloads to ram and then just deletes it right because you don't want to download the things you're pre-filling right that doesn't make any sense no one at the lan is going to need to go any faster than that anyway because they're going to have two and a half gig max to each station right no giggle only gig let's try black ops 1 this one is historically like pretty good download there 3.4 4.5 5.3 gigabit 5.4 now we just have to see how it scales to 200 people it's pretty much done six gigs well no the data is downloaded it's like what it's doing right now is probably just unpacking it because it's like yeah we would have done that on the fly but you downloaded me too fast way to go hot shot your goal then is to start this no i just need to select one game and then it will already be cached arc survival evolved enter it's already going to be cached this is going to download a 10 gig like on the dot you think so yeah look nine point there you go the cash because this is already cached and this is just downloading to ram there you go nine gig and the cpu is at what 18 only whatever this thing's gonna be able to do like probably 30 yeah maybe 40. that's outstanding yeah some stuff went down the setup here is absolutely wild but we're finally ready for the big test check this out we had to build a router okay the router was only one gig capable we have a two gigabit connection so this is just made of another server got our surveillance setup got our main switches here and this is it the steam cache clayton server we're ready for the big moment okay so we've got 150 people who ended up coming to the land and we need them all to go right now one two three whoa that is a lot more than the two gig internet connection we've got freaking awesome this thing absolutely saved our bacon and the most wild part is that the cpu usage is not even that high like i'm i'm pretty surprised at how easily this handled this even with just a couple of nvme drives like it's barely even working at it freaking love this 261 megabytes every second here comes mine get over here okay my speed though thanks intel for sponsoring this video and thanks for providing the steam cache it absolutely saved the day everyone was able to download games i mean i was downloading games at two and a half gig basically the whole time because i brought myself a little usbc two and a half gig network adapter but everyone else was probably at one gig for the most part absolutely no hiccups at all if you guys enjoyed this video you can check out a classic where we built ourselves a pf sense box way back in the day but one that was designed for gigabit internet not two gigabit internet it's a while ago but the principles are still pretty much the same

Channel: Linus Tech Tips

Views: 2,969,315

Keywords: steam cache, intel xeon, intel zeon, caching, game cache, gigabit, downloading games, fast speeds, game downloads, supermicro, speed, internet, DLing games

Id: wWltASCJO-U

Length: 21min 9sec (1269 seconds)

Published: Thu Aug 11 2022