(metal clanging) - Oh, my God.
- Wow. - This thing is disgusting.
- What happened? - [Jake] Ugh! - Looks like it was in a construction zone and an apocalypse, and
then it got snowed on. Oh my God, it's a good thing
we haven't turned it on 'cause there's just metal
server rack hardware just sitting on the motherboard here. Look!
(guys laughing) Oh, I think it might short out, boys! Why are we even using this thing? - Well, we're not right
now, that's for sure. - [Linus] Ugh! - Oh, no!
- I can't even lift it all up! - Ew!
(guys laughing) - Ugh!
- Ew! - Ugh, I don't even wanna touch this. (coughs) I got a mask on
and I'm still coughing! (metal clanging) - You gotta be wondering why
we would use a server like this when we have a shiny new
backup server like this one right over here. This guy holds the backup for our main storage server, Whonnock, which has all the data, for all of our ongoing projects
for all of our channels. So thanks to this, if
we somehow lost a piece or even all of that
data on the main server, say to drive failures or even
a fire in our server room, we'd have a local copy of
it (makes poofing sound) ready to go for our editors. But what if the worst
case scenario happened? Say there's an earthquake or a tsunami, and our whole building is destroyed, that is where our remote backup-- (coughs) Sorry! Our remote back up over 300
kilometers away comes into play, except for one small problem. For the past two years, it
hasn't actually been working. And unfortunately, we've
used up all of our favors from server manufacturers so we're gonna try to use
this thing to replace it. We have no idea if it still
works so good luck, everybody. (Jake chuckles) Good luck learning about
our sponsor, KiwiCo. KiwiCo ships out crates that
contain hands-on projects for kids to help develop
their creative confidence and problem solving skills. Learn more at the end of the
video or at the link below. (upbeat music) First order of business is let's get Auntie Dale server outta here so that it doesn't get corrupted. - Corrupted by the dust.
- That's the thing, you dust one server next to another server now you got two dusty servers (chuckles). - Oh, boy, this thing is-- Oh, my God! - [Linus] You want me to take that end? Well, I'm just asking. I'm just saying I play a
lot of badminton, you know, I lift a lot of 80 gram weights over and over and over again. - This side's way nicer!
- Yeah, yeah, you're good. Don't worry.
- Let me just-- - Oh, okay.
- Sorry, just gotta scratch my back a little bit here. - You know, we're sort of in a hurry. - [Jake] Oh, yeah? - So let's just clean the
server now then, I guess. Oh. - [Jake] There is wood in here. - Yeah, it might be sawdust of some sort. Okay, we're gonna need
to take this outside for the main event.
- I agree. - The cleaning bit. What is some of the stuff in here? What is that? (chuckles) What is that?
- Some template for something. It's plastic, too. - It's so dusty.
- Wait. - Those of you who have
been around long enough might actually recognize this
as the very first Storinator that we ever rolled out. Got that silkscreen logo
on the front from 45Drive. So those are ancient
SATA enterprise drives. - These ones are different. I put these, okay, this was my test server like two years ago for two months. - Well, you took really good care of it. (Jake laughs)
- Construction started then it's just been actually two years. - Oh, you know what? That's probably why this thing's in here.
- Yeah! A Vertex2!
- Oh, my God! So I've had this Vertex2
since I worked at NCIX. I don't know why we circled this. (Jake laughs)
It's a 60-gigger. - Oh, the motherbo-- I didn't even look in this thing properly. Holy! (chuckles) - Yeah, that's--
- Ugh! - [Linus] That's gonna be a problem. - [Jake] The PCIE slots and the RAM slots, are those even gonna work anymore? - [Linus] I hope so. So this is a triple redundant power supply.
- It's huge. - Yeah, it's enormous. And it uses up like, I don't even think we can put a different power supply in here. That's another thing, too,
is we wouldn't normally try to roll a server with
such outdated hardware because of concerns
about power consumption. This is not efficient. Pretty sure this is a DDR3. - I think it's DD4.
- I don't think so! - No? 1.5?
- It is! Whooee! Either way, we're not that
concerned about power efficiency 'cause we don't pay for
power in our rack space at that data center. - Thanks, iTel! - Yeah, (chuckles) thanks, iTel. (laughs) Hey, the vertical parts aren't that bad. That's how you can tell
that it was falling dust. - [Jake] Just a little
bit of drywall, you know? - For those of you wondering
what the heck went wrong here, part of it is that this
thing's been sitting around for a couple of years,
but the other part of it is that it was accidentally left in a rack that was inside a construction zone. Now the construction workers
very thoughtfully put, like, they actually built
a box around the rack, thinking that they were protecting the delicate equipment inside. I guess they didn't understand that having a big opening
in the top of it meant that the dust would go in. (chuckles) - [Jake] Anyways, whatever, we can fix it. Hopefully.
- Yeah. The fact that we haven't powered it on with this dust is good, though. - Yeah.
- If we powered it on, there's almost definitely
something conductive in some of this dust. So this thing has 64 gigs of RAM. That's fine for an offsite backup. - I was thinking about adding more RAM but I really don't think we need it especially if we're gonna run Unraid. I think maybe we should
vacuum it a little bit first. - No, we can't vacuum it
'cause there's static problems. We can't suck, we have to blow. - Okay.
- Okay. - [Jake] I think this
thing's gonna be enough for reference here, let's just... (loud whirring)
It'll be fine. - All right, here we go. (loud whirring)
Ah! (upbeat music) (loud whooshing)
- (laughs) Alex is like, "What is going on?" (upbeat music) (loud whooshing) - It's like new, baby! - [Jake] Oh, wow! - Right?
- Way more convenient than getting an air (mumbles).
- Yeah. I've had things like this before. - [Jake] Usually the
battery-powered ones suck but this one-- (chuckles) Complains about vacuum being too staticky, sticks Swiffer in. - [Linus] Look, we don't have
to put this in the video. - We're putting this in the video, 100%. Oh, oh, come on, are you kidding me? - [Linus] Well, I want
it to be clean, don't I? (Jake groans) - [Jake] Do you at least have
your Livestrong bracelet on? (Linus laughs) Wait, wait. How are you taking this
off with a Phillips-- The mounting hardware for
this CPU cooler was lost. And so I ghetto-found some
that are the right size and put some washers on there. It works. That cooler, what does
the acid tank, 0264. We're up to 10,000 now. - Oh, wow.
- Yeah, this is old. - Here's a little
spring-loaded mounting screw. It's just sitting here like this. (Jake laughs loudly) - What is that even from?
- This is not from this computer. How is this in here? Thanks, I hate it. I just need some paper towel. Who keeps moving the bloody paper towel from the PC build corner? I don't like Kimwipes! They don't have the absorbency of (beep). - Yo, you want some IPA, bro? Brandon brings the paper
towel back! (laughs) - Oh, well. Wait, so you found paper towel, but then you didn't actually-- - [Jake] You walked in 10 seconds after we found paper towel. - Do you have to make
noise while we're shooting? - I just wanna-- There we go. - You know, you're doing
four different things. You're watching TV, you're playing a game, you're texting on your phone, you're fricking get it to land on its thing.
- Oh wow that was pretty good! Oh!
- Oh! - Wait, is this isopropyl?
- Hold on. Should I (mumbles), no, it's not. - [Jake] Here, catch. - [Linus] What is this thing? What is this? - [Jake] Do you need some tweezers? - Oh, it's just like a little bit of wire. This isn't even a kind of wire we should be cutting in this office. It's not network or anything.
- Ethernet, is it not? - No, look at the color, it's weird. - No, no, no, this is,
this totally is ethernet. But also why is it in there? Like in the socket? You should blow that off. 'Cause this is dirty, you
should blow it off first. Here's a...
(loud spraying) - [Linus] Oh, yeah, that'll do. (loud spraying) - [Jake] Clean the CPU
before you put it in. - Why do we have 200
sub to LTT wood chips? - Sub to LTT! - [Linus] Huh, how much
work did you put into this? - Oh, okay. - [Jake] (laughs) Oh, yeah! (Jake laughs)
(loud clattering) Can I help you at all?
- Yeah, sure. Why don't you install some drives? That'd be helpful. (Jake grunts loudly) I mean, we don't even
know if this thing posts. That might be good to know before I even bother with this, actually. I'm just gonna keep installing SAS cards while you do your thing, okay? Here I am installing not one, but two SAS cards while you're
just slacking over there. Look at this, I'm even
gonna have all the screws in before you get the cables in. - [Jake] I'm already done. - [Linus] Oh, you gotta put
more than one power cable in this one.
- Well, can't you tell it to shut up?
- No you can't. - I can.
- What? - [Jake] I am God. Oh, redundant power
supplies have a mute button. - [Linus] Oh, well that's pretty good. - Yeah. You ever have a data center
if you had a hardware failure and it just won't shut up
until you fix it? (chuckles) - Well, I thought that was the
point of a hardware failure like being noisy. - [Jake] Yeah. - Oh, wasn't done! Wasn't done! - [Jake] Whatever, Linda. - [Linus] I thought you
said it's past the thing, is totally working! Why would not you not tell me? - Remember Albion that died? - Okay, that's fair.
- Just give it a sec. I think it's fine. - That was a good posting sound-- - [Jake] It's good, it's good. - So we're using our
Ironwolf Pro 12 terabyte but then--
- Not all of them. And then these drives
are from Auntie Dale. - Are you gonna let me talk or not, okay?
- Did you know these drives are from Auntie Dale? - So with a normal rate, you got the stripey stripes across the multiple drives, okay? With Unraid it actually
puts the entire file on a single drive, so you're limited to single drive right speed. That's the biggest problem with it. For us though, it's not a problem because this is gonna be offsite with only a gigabit connection. For our purposes, this is
gonna be more than fine. It's day two and I've got
an update for you guys. We actually took out the Ironwolf Pros 'cause we realized
another temporary server, Clover Server, was full of EXOS drives. So now we've got 22 of
them, which is all we need for our offsite backup of Whonnock plus snapshots of older versions of it so that in case something
accidentally gets deleted, we can (makes clicking sound) go back and get it real quick styles. We also went and updated
the firmware on our HVAs, updated the bios on the board, updated the IPMI firmware. And this thing is hummin' so well, I'd say it's a pretty
good hummer, you know? 'Cause it doesn't have the greatest power consumption characteristics
and it's heavy. - [Jake] Is it like an H1 or H2? - It' a hummer. Oh yeah, we also added a
couple of 500 gig cache drives. These are running in RAID 1 and they're not super high performance. They're just MX500 SATA
drives from Crucial but they're high enough performance that if we wanted to run a VM or something like that, then we could and realistically they're
way faster than we need since it's a one gigabit
connection to this bad boy. Now all we gotta do is put the-- Where'd they go? Where the heck the, oh, yes. Ah, yes. The old 45Drive mechanism
for covering up the drives. - [Jake] Did they update that? - Oh, it's so much better. - [Jake] I don't actually mind this one. - It's fine. These rods are really stupid. Ow, dammit, my finger. - [Jake] This thing is so cute. It actually looks pretty new. - I know, it doesn't even look that old and decrepit anymore! So you know where this
was originally deployed? - [Jake] No. - In the hallway at the Langley House. - Really?
- Yeah, you would like trip over it in
the top of the stairs there. Now I can show you what that looks like in the Unraid software. We're doing a parity sync right now so you can actually see
pretty much all of our drives are reading at their maximum speed of about 200 megabytes a second. The reason it's doing that is because we did change out
those drives earlier today and so we need to rebuild the parity data. Unraid does still have parity protection, it's just written individually
to two dedicated drives. So we can lose up to two of
the 22 drives that are in here without losing any data whatsoever. - [Jake] And look at that capacity! - [Linus] I was gonna say
another nice thing about Unraid is it's got a nice easy to
understand graphical interface. So you can see here, we've got 240 terabytes
of offsite storage. This is kinda fun. It's an older system but
kind of still checks out. It's doing 3.9 gigabytes
per second internally, just reading all the
data off those drives. - Those poor HVAs are just rippin'. - They're going, boys. You think if I wiggle wobble
this thing around a bunch that read speed will go down? - [Jake] Seems like a bad idea. - (chuckles) It's all fine and good to have a bunch of storage but we're not gonna be
copying things manually so what are we doin'? - Backups are only good if
it does it itself, right? You don't wanna rely on human error there especially us two. Unlike Whonnock server, which does a live file backup to (mumbles) which we built a few days ago. I guess you guys haven't really
heard about that one yet. Anyways, sensible Jellyfish Fryer, we're calling it an Auntie Dale now. It's great, yeah. Because this thing is running Unraid and we don't really expect
the performance to be great, running a live file sync to it probably isn't the best
idea even with cache drives. So instead, we're gonna be doing snapshots and now because our backup
server is running (mumbles), we could do a set of fast snapshots but we actually are gonna use
a tool called our rsnapshot which is based on rsync. A file transfer software that's very ubiquitous
amongst the Linux community. The fantastic thing about rsnapshot as the name implies is
it takes these snapshots but they're also de-duplicated so say we have our 30 terabytes
of base footage on Whonnock, and we add a terabyte a day, once it does that big backup and say, we're doing hourly backups
and we wanna keep 24 of them so we'd have a whole day's worth of 'em, rather than just making
a new copy each time, it creates a new hard link
which is basically like a map that's pointing to those existing files that are already there. So that way, when we do
our new hourly backup, it's only gonna add the say, 500 gigs from whatever ingest we just did and then it will just point
to the existing files. - Now the question becomes,
what about deleted files? Well, that's fine, too,
because what it'll do is it'll take your new
snapshot and they'll say, "Oh, hey, that's not there anymore." So we're just going to throw that out but we're gonna leave it
in your previous snapshot just in case you need it later. And then what'll happen
is you set a policy within the software and say, okay, I wanna keep these snapshots for, let's say hourly for a week,
and then I wanna keep them-- - That's a lot. - Is that, okay.
- I was thinking hourly for 24 hours.
- Sure. - And then it takes,
once that 24 hours is up, it recycles the oldest
one as the daily backup. - So that's one of the
reasons that this needs so much more storage
than the Whonnock server that it is backing up because it's actually gonna be backing up everything that was on Whonnock's server over the last probably month or two. - But when you think about it, say if we do one or two videos a day, realistically, we're not gonna pass a terabyte a day of new footage. - I dunno, David, can we? - Maybe a little bit more. Okay, whatever, doesn't matter. We'll say it's a terabyte a day. 30 days of that, that's 30 terabytes plus the 30-ish terabytes
of Whonnock we usually have, should be well within the 240 we have. - You give those guys a hundred terabytes, they fill it. - He's been doing pretty good so far. - The good news is, that Unraid allows easy
expansion of your Unraid arrays so if we just ship a
MasterCard into hard drives-- - [Jake] Here's some more Ironwolves! - We can take our
sophisticated rods off here, and it'll be fine. - Now to show you guys what
this looks like in practice, we've set up a test directory with a sort of 50 gig-ish project And we've started the
rsnapshot hourly backup. And if we look in our backup thing here, we got this PDFs tech
quickie, and look at that! It's getting transferred. - [Linus] Yeah! - It's pretty slow 'cause it's gigabit, but it is at least going. I think Turbo Ride is on right now so it's like got all the drives spun up but we wouldn't leave that on in practice. - So there it is, in spite
of what it looked like at the beginning of this project, our offsite backup server is ready to go. Meaning we have a 3-2-1
backup solution in place for the first time in (inhales loudly) years! (clears throat) Practicing what we preach just like I practice these segues. KiwiCo is our sponsor for today's video and they're a monthly subscription service to help make STEM topics fun and interactive for kids of all ages. They believe that if you
learn small things today, they can turn into world
changing ideas tomorrow. So each month, you'll get a themed crate to learn specific topics. Each crate is designed by experts and contains everything
you need to complete it which means you don't need
to run off to the store to get extra supplies like
glue or scissors or whatever. The crate that we're showing here is at a Kiwi level that's
for kids ages five to eight that'll teach programming and robotics. Another one is the Eureka level
for your teens ages 14 plus that highlight basic
engineering and sound practices. KiwiCo is a great way to keep
your kids occupied for hours and you can check it out
today at kiwico.com/LTT where you'll also get 50% off
your first month of any crate. If you guys enjoyed this
video, you might also enjoy, oh, maybe a lower capacity, but higher performance one. New, new Whonnock server. - Whonnock 3.
- Man. The rollout for it, couldn't have gone any better, actually, and that thing is flippin' fast. We're gonna have a playlist where you can watch all the
new, new Whonnock stuff. Can you make a playlist?
- I can make the playlist but it's gonna be called the Whonnock 3 playlist.
- Link it below, new, new Whonnock.
I love storage porn disaster recovery is amazing I was wondering why they didn't have a secondary site