What is a "Homelab"? (And a high-level overview of mine)

hey Jarvis tell me a joke child dead why does the Sun rise in the east and set in the West dad some it's working don't touch [Music] hello again everyone and welcome back to my channel the concept of home lab is not something that I've ever covered on my channel before honestly I just didn't really think there'd be an audience for it but I guess I was wrong because the Jupiter Broadcasting Network they basically came out with a self-hosted podcast where they covered this type of thing and you know it seems to be popular at least in my circle so I guess this hobby that I've been doing since probably 2006 or 7 might actually have some merit because it has an official podcast Wow so I figured I would make a video about my home lab the types of things that I run and see if you guys are interested and if the numbers show that you are maybe this is something that I will cover more often on my channel but what is a home lab what does that even refer to well basically it means running your own servers I mean servers are generally something you see in a data center right so why would you want to have these loud racked items in your home well it's actually a lot more popular than you think especially like I said we have a podcast now but people do this for various different reasons it could be you know for example Plex a lot of people like to host their own media Plex is a huge thing other people want to run their own website but they might not want to pay for hosting another really popular reason to run a home lab is for learning I mean you will learn more about servers and networking running your own servers then you would learn in a week doing something else I mean the whole thing is what self hosting is that if it breaks you get to keep both pieces but that's the most exciting part because if it breaks you get to fix it and when you fix it you get to learn and learning is you know it definitely is a great reason to do this some people will actually run their own servers because they want to get certified and they don't want to use these virtual labs they want the real deal on real hardware and those are people that go above and beyond comes to learning so what I'm gonna do in this video is I'm going to show you my server room first and then I'm going to show you some of the tweaks customizations and automations that I run as well as some of the software that works behind the scenes on this channel the hope is that I'll get an idea just how big of a demographic this subject is I'll look at the analytics but I also hope that if you are into home lab maybe something that I'm doing will inspire you or give you other ideas for yours that's part of the fun when you run home lab is to see what everyone else is doing to inspire things that you might want to implement on yours so let's go ahead and check out my server room and by server room in my case I mean server and gaming room because this room in my house serves both functions classic gaming is one of my hobbies and that's actually one of the reasons why I create videos on retropie but you didn't come here to hear about that I have entire videos that are dedicated to retro pie like this one now on my server rack I actually have four servers here I have two PowerEdge are six tens and I have one PowerEdge are seven ten the are seven ten is actually running FreeNAS that's probably the most important server of all of them in my case since I have a youtube channel I generate a lot of content and when I say a lot of content I mean a lot of files and the storage requirements for a YouTube channel can actually be quite huge and I wanted something that was powerful and up to that task now in the Linux community we have something called open media vault which is a network attached storage distribution that is very common but I wanted to use freenas because it does have first-class ZFS support and that was very important to me the two are six tens run proxmox proxmox is my virtualization solution and I only have one of those servers on right now they are in a cluster but it's not required for both of them to be online and nothing on my proxmox servers are mission critical to the point where they need any failover or anything like that I mainly only use the clustering and fail over as demonstrations in my videos on this rack as well I have 8410 so if you don't already know when it comes to Dell servers t means tower and R means rack the T 410 is not powered on very often that's actually my testing than I use from time to time it has no dedicated purpose it's probably only turned on four or five times out of the month this little apparatus right here this is a cloud lit case it's something that I found on Amazon and what that allows you to do is actually attach seven raspberry PI's almost as if they were blade servers and you can even put a switch here at the bottom which makes it look pretty cool four of these raspberry PI's here are in a kubernetes cluster it's something that I started playing around with that I hope to create a video on pretty soon and basically I have one PI as a master node and then I have three pies as workers they're all Raspberry Pi fours I also have a test PI which is actually a Raspberry Pi 3 and that's just for one-off experiments basically to see how things work on a Raspberry Pi so you know just testing I also have my development server on here which is basically where I keep my team up sessions it's a Raspberry Pi that stays on all the time and on there I have my code for various projects that I work on and I have a bastion that allows me to get to this instance from wherever I am so I can easily work on my code anytime I'd like to and everything on there is backed up via get the switch is a unify switch which is something that I've found to be the best in my use case it handles VLANs very well I have PF sense as my router / firewall which is not pictured here and I found that PF sense and unify actually work very well together I have the same VLANs defined in both and you know it couldn't work any better it's just a perfect combination in my opinion and I have other shenanigans going on in my server room as well who knows what you might see here hey Jarvis run a speed test starting speeches now I will be back with the results in a moment your download speed is 93.0 several megabits per second and your upload speed is 23 point 1 6 megabits per second so I want to switch gears real quick and talk about software customizations and also the servers that I'm actually running or the applications that are running on them so I'll open up my laptop and we'll take a look ok so here on my Galago Pro you can see the first of three tabs that I have open this is pfsense again this is my firewall slash router it's essentially what makes all the networking routing and firewalling happen in my case this is a net gate SG 3100 I love this device it's very fast and I couldn't be happier with it you see here that I have a handful of VLANs set up for various purposes and if I scroll down for each of the VLANs I actually have a graph so if I have any network contention at all I can usually figure out where it is based on the VLAN at least it'll narrow it down to the network and then I can figure out which device it actually is but honestly nothing can really take up too much bandwidth on my network because I throttle everything every subnet slash VLAN has its own maximum speed setting that's far below what my ISP provides and another benefit of having graphs is that if I see for example this graph right here going crazy after bedtime then I know my kids are on some website or watching TV or something so I can see all the activity here same on the guest network but I don't really care much about that because again that's just for guests and if I scroll down you can see that I have an Internet of Things VLAN right here and you would think it actually looks really busy right well it's not really even using a megabit right now and I have that VLAN throttle to maximum of five megabits so nothing on that VLAN can use too much traffic in fact the IOT VLAN it can't access anything else in the entire house or any other network all it can do is get out to the Internet and back that's it the Internet of Things devices are completely segregated from all of my other things now switching over here to proxmox you can see that this is my proxmox cluster I'll see if I can make this bigger for you guys it probably should have did that on the other tab given it'll be a bit easier to see you can see here on the left I have my two proxmox servers again these are both Dell PowerEdge are 610 servers and like I mentioned one of them is powered off and the reason being you can see that the primary one isn't really all that utilized it's only using about 22 gigabytes or so of its 64 gigs of ram and it's idling below 10% that's pretty decent now when it comes to VMS this can actually go crazy because i have a virtual machine let me go ahead and this over here called videomatic which is this one right here so if I go to the console see if it works it takes a minute to come up so I'll go ahead and log in basically what this allows me to do is I can open a project file for my youtube channel because all of my footage is synced to all my other machines via sync thing including this one right here I can't show you the process right now because I actually use X to go to interface with this and I didn't set that up on this laptop yet but essentially all I do is open X to go it'll have the same screen kdenlive will come up I click open I'll have all my project files in footage files on there because sync thing makes sure that every single piece of footage that I record is synchronized to this VM and then once I load the project file I'll hit the render button and then it'll go ahead and render the video for me what I like most about this is the fact that I don't have to worry about my laptop or my desktop whatever device I'm using having all of its CPUs bound because of a video rendering task which can actually take quite a bit of CPU since everything is offloaded to this server and this server does the rendering work for me then I don't really have to have a busy computer and for hardware you can see that I've allocated 12 virtual processors or 12 cores to this virtual machine so while the usage is you know right now it's actually a bit higher there might be another job running this will actually go quite high when I am running a rendering job which will make this percentage quite a bit larger but still not enough for me to need both hypervisors or both VM hosts turned on which is why I only have this one on at any one time now real quick I have a few I'll mention I have a couple minecraft servers here I have Plex for media that's a very common thing on a home lab I also have these three staging machines I have one for a boon to one for Debian and one for arch these are actually ansible test machines ansible is the configuration management solution that I use I'll talk more about that in a minute these three machines are constantly checking out my ansible git repository specifically the staging branch and if there's a commit to that branch it'll run that configuration and then it will let me know if there's an issue so I guess you can think of this as more of a poor man's CI CD solution and then over here on the last tab I have FreeNAS probably not much to go over here but I'll just show you something right here I have a set of pools or actually I have one pool here I have a bunch of data stores so you can get an idea how I carve out my storage which is one of the things I like about ZFS and you can even see right here I have 698 gigabytes of YouTube files or you know similar to gigabytes these are gibi bytes but similar basically a lot of content there and that's not even everything that's just everything I have an archived yet which is why I was saying that you know having a YouTube channel you need storage so what I'd like to do now I'll go ahead and close this is shift gears and talk about my ansible configuration so I'll open a terminal go ahead and make this bigger so now I'll SSH into my dev server this is my Raspberry Pi the one that's always on and inside here I have a git folder and inside there I have my ansible configuration so if I list the storage here I have several files and directories I'm going to show you what actually makes this really awesome in my opinion so if I go into roles list the storage you can see that I have four roles right here I have one called base I have a role for Raspberry Pi a role for servers and workstations so this ansible configuration manages all my servers and workstations and PI's as you can see the base role is applied to everything that has my user account in it and all of my dot files like my vim configs and T MUX configurations things like that basically things that I want to have on every server laptop desktop all of my devices another benefit to having a base role is that if there is a security vulnerability in the wild and I need a patch for that I can simply commit my change to base and all of my machines will get that fix so I could fix everybody all at once now I'll go back a level here then I'll go into the hosts virus directory and I'm going to randomly just bring up one of my hosts config files so I will go ahead and open up this file right here I name most of my machines after Final Fantasy 6 Esper's which is where this name came from and you can already see that I have some configuration here and what I like about this is I have everything modularized so here for application selection for example I can set true or false for any of these items games include Steam for any games that I basically install on a typical basis Firefox is here it's true now you might be thinking you know that's not really a big deal because Firefox is just a package you can simply install it well actually this will download Firefox from the Mozilla FTP and it will make sure that it's installed it'll also make sure that it has a desktop file so it'll show up in the applications list but what it will also do is every time ansible runs if mozilla has updated firefox it'll automatically update it for me which means i'll often have the latest version of firefox before Ubuntu and Debian or whatever distribution I'm using even has a chance to package it the same is true for Thunderbird it's the exact same thing the Thunderbird and Firefox task books that basically are activated if these are set to true will remove the distribution version so that I'll only have the vanilla Mozilla version direct from their servers google chrome again true-false sync thing which is what i use for all of my laptops and desktops and even some servers that will install that so basically have pretty much everything true if I scroll down though say I have VirtualBox and the Vivaldi browser as well but what's really fun is the desktop environment selection so basically I can set true or false to any of these items right here and for example if I set Nome to true which it is I could literally install just Ubuntu minimal with no X at all and if I set Nome to true it'll build it all the way up to Nome being a working thing it'll install a gdm it'll install a gnome shell it'll even set my gnome keyboard shortcuts and wallpaper the same is true for the mate' option right here it'll do the exact same thing complete with the desktop wallpaper and shortcuts but with the MOT a desktop environment instead now I could do a video more about this if you guys would like but I'll show it off to you I'll show you what this actually looks like so I'll log out of here and what I'm going to do now is SSH into one of my testing servers so let's just pick lubuntu staging so now I'm on the Ubuntu staging VM now ansible is already set up on this machine it'll check in every now and then and if there are any commits to my git repository it'll automatically run them so basically I don't have a central ansible server I use ansible pull and each of my nodes is instructed to only run ansible if there's actually been changes to the repository which avoids unnecessary CPU usage if I commit a change to my git repository for ansible it'll run on all of my machines and I'll also get a report via a telegram bot that I'll tell me exactly what happened if it was successful if it was unsuccessful so once I do have ansible setup I could simply run this command right here which is simply provisioned I could do this on any of my machines to force and ansible check in at any time but what if I don't have danceable setup at all what if this was a brand-new machine well simply all I'd have to do is run curl deploy slash bootstrap and pipe that to sudo bash now in this case deploy is the name of the server and bootstrap is the ansible bootstrap script which is hosted by Apache this command can only be run on my home network this is not accessible outside of my network so while I normally don't recommend piping anything to su do bash unless you check it out this script is actually something I maintain myself and you can only get to it if you are on my network but simply all I have to do to set up a new machine is install Linux whatever distribution once it's installed i simply run this one command so i'll go ahead and run it right now even though it's already set up on this machine you and now it says running the first bootstrap I'll minimize this then I'll tail follow VAR log ansible law and here you can see that the ansible provision is already well on its way right now it's actually copying over dot files and it's going to do a bunch of things so I'm not going to hold the video open just for this but you can see it's doing some actual work so I know I've glossed over pretty much everything here this was a high-level look at my home lab but if you want to know more about any particular component let me know in the comments below and I might actually do a dedicated video on that as soon as I get caught up with the others now I won't do any videos about freenas or pfSense reason being my friend Tom Lawrence already does an amazing job with that content and he's the guy I recommend you go and check out check out his YouTube channel if you want to know more about those but for anything else I'll definitely consider doing videos in fact I already have some proxmox videos on my channel that you can check out right now so what do you think about home lab is this something that you would like to you know know more about is it something that I should be creating content on or do you think it's a terrible idea let me know in the comments below and then in the meantime I will get back to working on some other videos I have coming in the pipeline so subscribe to my channel if you haven't already done so and I'll see you soon thanks for checking out my video I really appreciate it if you found it useful click that like button and if you haven't already done so make sure you subscribe so you'll see the latest content as soon as it becomes available if you want to help me out there's links down below for my patreon page as well as links for purchasing my Linux books and also my affiliate store which has a listing of linux compatible hardware that i've actually tested personally thanks again for watching and I'll see you in the next video

Channel: LearnLinuxTV

Views: 94,348

Keywords: LearnLinux, Linux, Tutorial, Review, Howto, Guide, Distribution, Distro, Learn Linux, operating system, os, open-source, open source, gnu/linux, homelab, home lab, self hosting, self-hosting, self host, home server, freenas, proxmox, ansible, automation, mycroft, ai, what is homelab, home computer lab, homelab setups, best homelab, technology home lab, networking, servers, virtualization, information technology, it, self-hosted

Id: UpaOd_9ZSAI

Length: 23min 36sec (1416 seconds)

Published: Tue Feb 04 2020