- Hey, it's me Destin. Welcome back to Smarter Every Day. There's a lot of stuff that goes into making this YouTube channel and some of it's more
complicated than others and there's a lot of
behind the scenes stuff that you don't see. This is one of those types of things. I wanted to learn some stuff
and I called in an expert and he happens to be a
personality here on YouTube. His name is Linus Sebastian
from Linus Tech Tips. So the thing you might not
realize about Smarter Every Day is I have a problem and it's because of this thing right here. So as you know this is the
V2511 high speed camera. This is the workhorse of
how I do slow motion video on the channel. But the problem is you
got two types of memory on this camera. You have volatile memory on the camera. Think of it like RAM on your computer so if you shoot something
you got a certain amount of RAM that can save on this local camera and if you ever lose power, you're done. There's non-volatile memory on top and this persists if
you lose power, right. So this is 512 gigabytes of memory but the problem is after every shoot I have to do something with this, so I have a problem. Let me show you what I did
to try to fix this problem. So about, I don't know, a year ago, no, six months ago I sent
this particular, come on. Come on, come on over here. About six months ago I sent this tweet out to a guy named Linus Sebastian. He does this thing where he puts, I don't know, servers in people's houses
and I called him here. So Linus, I need your help, buddy. This is pretty serious. - I'm here. All right, yeah this is really serious. This is bad. - First let me show you my hard drives. Can I show you this? - Yeah, okay.
- Okay, let me get the hard drives and show it to you. - I don't know if I can handle this. (Sebastian laughing) - I'm sure you've seen people
with more than this, right? - No, I think this is even
more than Justine had. - Okay, so this is not
all of it, there's more. Do you want me to just stick with this? - No, I wanna see, bring on the carnage. - Okay, let me go get the old stuff. - So let's start doing some
inventory here, I guess. 434 gig, so we're at two
and a half terabytes. That's a one terabyte. Three and a half. - So this is the old stuff. I call this the deep vault. - Five and a half. This is an eight terabyte drive. - Yeah, it is.
- And it's literally ducked taped
to two other drives. - That's how I roll, man. It's pretty serious. And so, I literally go to the spreadsheet, look at this. So I have an entire spreadsheet. - Go have a look at this. - So what I do is I just come in here, I Control + F and I'll say, like, Mars rover, and so if I go to hard drive A27, so go find A27. (Sebastian laughing) This is how it works, man. - [Sebastian] Oh, God. - [Destin] So A26. 21, okay so somebody's moved A27. (Sebastian laughing) - [Sebastian] So we're finding a flaw in the system already here. - We're hosed, we're looking for A27. So A27 is not where it's supposed to be. So you see my problem, right. So these aren't even indexed yet.
- We all see your problem. - [Destin] These are-- - [Sebastian] (screaming)
You just dropped it. - [Destin] Yeah, I did. But it was on carpet so we're good. So it was my understanding you shipped a 45 Drive server here. - Yeah.
- And we're gonna install it and it's gonna
make this problem go away. - It might not be enough. - Seriously? - Have you added up the capacity of this? - I have not, no. - 'Cause I'm at like 25
terabytes here already. - Yeah.
- I shipped you a 160 terabytes server. So here's the thing, you could make the argument
that our data hoarding really doesn't make a ton of sense because we could just
grab the necessary footage from a YouTube upload, splice it into our new
video when we wanna refer back to it and bippity boppity, off to the races. Now, with Destin, his case was a really
interesting one for me because he actually
does not have the luxury of being able to go back
to the previous video and grab it without making any
very significant compromises, because this kind of high speed footage the data rates are so
incredible and the amount of runtime for what could be a very, very short clip wouldn't make sense to upload in the YouTube video at all. So he's gotta pick and choose his battles. So he's throwing out a ton of frames whenever he's showing you, whether it's a hummingbird's
wings in realtime, he's throwing out a ton of
data in the actual video that you guys watch, except for the small segments
that he decides to show you. So everything else here could be necessary if you ever wanted to go
back and look at something in more detail. So we gotta get this server rolled out. - You're genuinely worried, aren't you? - Yeah. - My tweet said you would
fix my problem, dude. - Wait, your tweet-- - Help me Linus that you're my only hope.
- Your tweet says I'll fix the problem so now it's my liability if it doesn't, is that right? So you put words in my mouth. - You never actually replied to this. - No, I didn't. - You don't even care. You don't even watch Smarter Every Day. Let's just fix my problem, I don't care. - [Sebastian] All right,
so let's come over here. So Destin was-- - [Destin] Yeah, we installed it. - [Sebastian] Yeah. - [Destin] Any more light always is good. The nice yellow light. - Oh God, he just tripped over the drives.
- Sorry. It was only like 20 terabytes or whatever. - So this is great. We actually had this
server shipped down here. It must've been like-- - [Destin] It's a while back. - Two months ago or something like that. It took me a long time to get down here. I'm sorry, because clearly the problem is, I would've gotten on the next plane-- - [Sebastian] It's a big deal, dude.
- If I'd known it was this bad.
- Yeah. - Now, have you installed
the drives in here yet? - [Destin] No, I haven't. - And that's why this drill is here. - [Destin] Yeah, we're
gonna take this out. - Undo them out, got it. (machine drilling) - [Destin] Cool. - Look at that. (cover banging) (Destin laughing) - [Destin] Yeah. - I love this video so far. - [Destin] Are you being sarcastic? - The shenanigans are real. No, I love it. This is great. So Seagate actually
shipped you 12 terabyte instead of 10 terabyte drives, which is what I thought they had sent you. - Okay. - So that's another 30
terabytes of raw capacity. - You just rip them open like that? - This is good. Yeah, yeah. You just kind of shuck them. - Like corn? - Yeah. Yeah, yeah, basically. - I think you think I'm joking but I actually don't know all this stuff. - So this is just a friction
mount here on the side. - It's a spring? - [Sebastian] But give
it a little more pressure than you're comfortable with. There you go. - That's it?
- That's all there is to it. - So this is easy. - This is easy. So the way that RAID works
is it's writing the true data and it's also writing what
are known as parity bits. So if I had, let's focus on these
four drives right here. I'm running this in, let's say, a RAID 5. - Okay. - That means that for every
three pieces of real data, and this is a
oversimplification for the sake of it being easier to understand, I'm writing a parity bit that is, think of it as kind of like
an algebraic expression where this parity bit is on the other side of the equal sign of these three. - Got it. - So if I were to lose any
one of these four things I can rebuild the other one. - By resolving the equation. - It's a way of rebuilding
what you had lost, right. - [Destin] Right. - But we are not running parity in a conventional sense here. So we're using a software called Unraid that rather than striping the parity bits so that they exist across multiple drives it actually uses an entire single drive, or two drives if you prefer to be able to lose two drives to write
all of the parity bits to. So that comes with some
advantages and disadvantages. The disadvantage is that it's meant mostly for archival storage. It's not as fast to write to. - Okay. - But the advantage is that in the event that you set it up with
a single parity drive and you lose three drives, let's say your house floods or something and it floods up to here, all this data is still good. - Okay. - Whereas if you had
the parity bits striped, if you lose more than your threshold, all the data is gone. - I see. So let's say just for
sake of easy numbers, let's say they were 100 terabytes here, which there's more than that, but let's say there were
a 100 terabytes worth of hard drive space, that's not how much server
space I'm gonna have? - Nope. - How much will I have? - You would have anywhere from 80 to 90, depending on what you prefer. - And so that other part is
just for the parity bits? - That's right. - This is a big deal for me man, I mean. - No, it's great. You need it more than anyone
else we've deployed one of these to so far, I think. - Are you serious? - Yeah. - It feels like it. - This is a dire situation. - [Destin] This is another video for you but this is like a really significant, I'm not gonna say emotional, but there's a lot of
stress tied up in this. A lot of my life has been protecting this and I've got some in like
a safety deposit box. How long's it gonna take
to slurp all the data? - [Sebastian] A long time. - [Destin] Really? - You've got a big, big data hoarder problem here. (Sebastian laughing) - [Destin] This isn't data hoarding. (Sebastian laughing) - There it is. (Sebastian shouting with excitement) - [Destin] Did it work? - Oh, yeah. We can do this later but I just wanted to make sure all the
drives are showing up. Three, four, five, six, seven, eight, nine, 10, 11, 12, 13, pal, perfect. - [Destin] Is it good? - And the SSDs are here too. - [Destin] Is it good enough to high five? Okay, just making sure. They're all formatting, wow. You can hear them all clicking. - [Sebastian] Yeah, they're going. - This is nothing. You do this all the time. You don't care about this at all. - [Sebastian] Oh no, I think it's cool. I think it's cool. I'm sharing your cool experience. (Destin laughing) - Okay, so once we set up our RAID here, even in the event that
we actually did lose more than two drives, because it's on RAID
the only data you lose is the data on the drive
that physically failed. So that's cool. And in addition to that, because we're using Ironwolf Pro drives they come with a five year warranty and they come with a
data recovery warranty. - Thank you. I appreciate it. - Oh, you're very welcome.
- This is a big deal. This is Linus from Linus Tech Tips. They do great videos like
they hooked me up here. This is a big deal. Thank you for that. - My pleasure.
- And your partners were 45 Drives and Seagate. - And Unraid. - And Unraid. So thank you to you three
companies that did that, really appreciate it. Go subscribe to Linus. Linus does a bunch of, like everything, right? Everything from reviewing keyboards to like everything.
- Pretty much. - You do everything.
- To touring the world's only commercial quantum computer manufacturing facility. - Really? - Yeah, we did that. - Go check his stuff out, it's amazing. I like your channel. - Thank you. - Yeah, I've watched your
channel for a long time. Big thanks to Linus for
coming out to Alabama. This was a huge problem I had, this data storage issue. Thanks to Linus for working
with these companies, his contacts, not mine. You had 45 Drives who made
the enclosure and the server, Seagate who donated the hard drives and Unraid is running the
software on that server. I'm very thankful and I'm also interested in seeing how he does in the next video. I'm gonna do something
kind of interesting, I'm gonna pluck Linus
out of his RGB Keyboard gamer system, sandals
over socks kind of world and I'm gonna introduce him to this man. - I'm Luke Talley and at this time in 1969 I was a senior social
engineer at IBM in Huntsville. - Luke Talley is an amazing individual and I want to just
juxtapose Linus who's up on all the new tech with
Luke who had to figure out how to make memory for computers. I want to put those two together
and just see what happens? And it is fascinating. I hope this video that you just watched earned your subscription
here to Smarter Every Day, but more importantly, I hope you will stick around and subscribe and ring the bell so that
you can see that next video. Because Linus and Luke both teaching me really interesting things about one of the most important computers
ever built by human hands is a fascinating trip. We went to the US Space and Rocket Center. All that's in the next video
so please consider subscribing if you're into that sort of thing. If not, that's no big deal. Anyway, I hope you enjoy this. I'm Destin, you're
getting smarter every day. Have a good one. Thanks Linus, bye. Let me show you a trick. This is a trick I learned about computers. - Yeah. - So I have a pen and paper and sometimes I write things down.
- There we go. No, I remember the IP. (Sebastian laughing) This is a better way.
I'd love to know how long it took him to ingest all that footage and if it was large enough.
Just about 5:30 in and things are starting to get away from Linus. Darn gravity.
I use unraid, so a few things I feel are worth passing along (keywords bolded for skimming):
You mentioned on Patreon that you aren’t moving the data over, I’d nudge you to actually do that. Worst case scenario, you move the data back onto your endless external hard drives.
You should join the /r/homelab and /r/DataHoarder communities. For less than you probably spent on this you can get fully enterprise hardware that you can have ample room to grow in to.
Love Linus tech tips. He is one of the reasons I love technology so much. Was he here visiting in Huntsville?
[removed]
AWS Glacier might be an okay option for the older stuff. It would take forever, but it would be very cheap once posted.
With that kind of data, you'd definitely qualify for some AWS SnowBall action https://aws.amazon.com/snowball/
[removed]