Raspberry Pi 5 Mastery: Create a Powerful Home Server/Home lab

[Music] hey robot makers how you doing hope you're having a good day so far so do you want to learn how to build your own home server or home lab using a raspberry pi5 then this is the show for you so let's dive straight in my name is Kevin come with me as we build robots and other things lost my thread now bring them to life with code and have a whole load of fun along the way right let's get over to our our slides we've only got a few slides and then we're going to get right into the demo today like we did last week cuz that seemed to work really well so yeah this is about building the ultimate home lab and Home Server so I'm going to be building this Home Server I've got a brand new Raspberry Pi 5 over here just mute that for a second um and we're going to basically open this up we're going to load this up with uh raspberry pios and then we're going to install Docker on it really quick using a script I've created the ultimate Docker install script you might say we're going to uh why are we going to use a rasby pi 5 you might ask you might have quite plenty of other computers lying around that you could do this on as well well for me it's about the speed and the memory of the pi five and also the fact you can now have the mvme for dis space as well as you can have plenty of room to install all your different apps and try stuff out in your home lab uh and also serve things to your family with your Home Server so how we're going to do this we're going to do this with Docker and some really cool home apps I'm going to show you today so let's go let's get into this shall we so what's the difference first of all between a home lab and a Home Server and then I promise we'll get into a a bit of a live demo so Home Server runs apps designed for use in the home so that they might stream music videos it might um host audio books things like that your family photo album file storage like a NZ it can do home automation through things like home assistant uh family calendaring recipes and meals we'll have a look at one that we've created on there whereas a home lab is more for Learning and exploring technology so you're going to be building and testing different things out you can bring these up on Docker tear them down and it's it basically doesn't leave a trace behind it's very experimental in nature and the the cost in the space is maybe a bit more elaborate than a Home Server because you might use more than one computer so one of the things I'm looking to do very soon just behind me there is a four raspberry pies flickering away there that are in a little rack and I'm going to replace those rasby Pi 4S with some raspber Pi fivs so this is what basically made me think about the the home lab and the Home Server element to it because we run our home server on our home lab so that's kind of where we're coming from so let's get straight into it shall we then okay so what I'm going to do I'm going to get over to um a browser and I'm going to show you uh let's just open up a nice new empty browser let me get over here and I'm basically just going to go to home.mac do.com so this is a website I'm going to show you how I've done all this today so this is a website that's running some software that's called Homer we're going to have a look at Homer you can just see the uh the name of it there h o m a r r uh and this is a free piece of software that you can install on the computer and create dashboards really easy to use dashboards and it could do a lot more than I'm probably going to show you today but I just wanted to sort of give you a flavor of the kind of thing that it can do so I've loaded up some apps that my home server is streaming out or making available to anyone on my local area network and because I've got some rooting as well and I'll show you how to do that later on we can actually expose these to the internet securely um and mean that we can access these anywhere on the go and even on our mobile phones so the first I'm going to have a look at today is something called that's called mey so I've got um a quick link there food. kmch.com for this one and I'm just going to sign myself in on here just with a password I've got saved okay let's get myself logged in and what we can do on here is we can have all kinds of different uh uh meals that we've set up on here so let's have a look at some uh classic pesto for example so this one we can we can say that we've made this we can see what the ingredients are we can see what the steps are and things like that and even have a little picture and we can put all those together in like a family cookbook so that we can uh share that out with them amongst our family so I think this is like a really cool app you can even build out things like meal planners shopping lists you can have a timeline of when you added various different things and who added the the recipe and so on so yeah I really quite like this one this is a nice simple one to add to uh Home Server now I'm a bit of a a geek so I like to do techy things such as home aut uh home assistant that kind of thing so one of the things I did in the early days and it's still running on my Raspberry Pi's is node red so if I click on that button there we can get straight over to my node red instance and you can see I've got all the kinds of flows set up on there to grab data from various different sensors some of them are Pimon enviral sensors some of them are things I've created myself these things like a subscribe counts the bathroom fan for example we can even control a robot from this um this is a burger bot so basically it just listens to an mqtt broker and that means that it can listen to messages and then react to them and we can even create like a dashboard which is what these these things down here do so we've covered the um node red in a show quite a lot of detail previously so I'm not going to show you how to set that up but this is something that just sits on my Raspberry Pi instances and uh means that we can do all that kind of clever stuff in the background so that's node red uh now quite a few of you might have already set up home assistant um I've got this running in a Docker container as well and that also means that I can tear this up tear this down very simple bring it up bring it down I'm going to show you how simple it is to do that in a moment and look at this I've got this home automation I've only got a few things set up on here but um it found my Elgato lights in the studio so if I hit this button here you'll see I go all dark a little bubble in the corner there so that's um home automation it's working over zigg Wi-Fi and so on you can even see my HP printer there see I need some uh color ink being sent to me at some point and again there's plenty of videos out there about home assistant I think I did do a show on that quite a while ago as well so I'm not going to show you how to set up home assistant this is more about how you can install these apps really really simply and I'm going to show you the trick for doing that in Docker shortly and the weather station so uh in the UK at the moment it's actually quite windy there is a storm is it iser or something I can't but it's cold Alex yeah something like that begins with I and yeah so it's quite windy so if I look at uh this weather station that I got upset so this weather station is the Pimon um weather station that they sell they sell it with um a hat that goes onto a Raspberry Pi and you also get the all the anemometers and Equipment goes outside rain sensors um gas sensors all that kind of good stuff well the gas sensors are actually a different sensor you sure what they are so um I bought this kit from the Raspberry Pi store this weekend this is an enviral plus from Pimon this is like an air quality sensor so that's a hat that sits on your Raspberry Pi and they also do another module that goes with that which can detect particulates in the air this PMS 50003 so that s goes hand in hand with that I do have an early version of that over there but um at the moment it's not bringing any data in so I need to look at why that's not working but anyway the point here here is that uh we have all this weather sensor data and unless you know the exact URL or something it could be hard to remember where it is so our Home Server as well as hosting all this information storing it all in an influx database so that we can graph it really nicely we're using grafana to uh show that data we can easily get to that and have a shortcut for everybody else to to click on from uh either their mobile phone and all they have to remember is just the home. kevmac do.com for example now down here we've got a little notepad area so you can um you can sort of click on this and start editing some text you can leave like notes for the fridge that kind of thing I've got a shortcut there to kevs robots.com if I click on that we'll get over to Kev robots.com and I've also got a link to my true naaz server so I have a BSD server that's running true naaz just down to my left and that's got I think about five or six terabytes of storage it's where I shove all the uh the videos once I've um I've finished recording them and I've got a couple of other things on here and I'm going to show you there's a there's an error there we need to fix that in a second I'm going to show you how easy that is to do so my 3D printers I've got two 3D printers I've just put a shortcut to one of them on there I've not actually got it running at the moment and you can notice there's a little red dot down there and that means that it's currently unreachable whereas these other things are reachable because they've got that little Green Dot so it just does a little HTTP check to see are these things up and running or not I've also got a link to I think the uh the Discord server there as well so this little um um nodule here what would you call it a node or a tile maybe the tile's best um this currently isn't pushing any dat dat this is looking for a component that's called Dash so let me show you how you edit these dashboards real quick um first of all you can actually switch the theme so if you want it to be like a dark theme a lot of people prefer that but I just thought for this live stream it's actually a bit easier to see uh if it's this lighter theme so if I click on this little um pencil thing here it says we're now in edit mode and you can see we've got these little cogs in the corner and we can click on this little button here to add a tile so we can either an app we can add a widget or we can add categories so I've got a category down there which is for one for home one for 3D printing and if we go back up to there we can add in a widget and one of the widgets is something that's called Dash it's actually called dash dot um just to make it a bit easy you see the little full stop there uh so Dash dots we can add in a dash uh node just by clicking on that button there you can see up here we've now got a dash tile and if we click on this little Cog we can then edit the details of of that so it basically just wants to have a URL uh pointing to a dash server so I'm just going to type in a number let's just see if um I can't remember if I've actually got Dash running on this one or not but we're actually going to set that up in a second so if I just do that I don't think it is running on that particular server yeah so that one isn't running similar to this one here so we're going to have a play with that in a second okay so let me show you how we can install that real quick so if I load up here this is actually one of my new Raspberry Pi um Raspberry Pi servers Raspberry Pi computers Rasberry Pi 5 and currently I've all that I've got installed on this one is a couple of Docker containers which are running code and I've also got this clustered pie clustered Pi is the name that I gave to that little cluster of raspberry pies behind me and there is also a website which is uh clustered pi.com it's if I type it in clustered pie.com can't if it's Dash pie or just that it might be Dash pie anyway if we want to actually grab the code uh that's got all the stacks in it Stacks are a um a Docker convention for if you want to have lots of different services or maybe just one service running then you can create a stack so I've got a whole bunch of stacks one of them is called Dash so let's get Dash up and running so if I do CD Dash so you can get all these just by doing a git clone of this uh repository I'll show you how to do that um in a second once we set up a new Raspberry Pi so if I just do LS there's only a couple of files in here let me just do LS DL just to see that so the docker compose let's cat let's have a look inside that Docker compose file because that has all the magic in it so it's only got a couple of lines in here first of all we got this version number 3.5 and then we've got a service so the service is called Dash the image so this is kind of like a pack packaged up image of all the dependencies all the the program code everything that's needed to run Dash uh is contained within that at that um so this is actually in a repository so Docker has a repository hub. do.com and in the um what's that Maro dash dot it's going to pull the latest version of that and it's going to run it unless it's been stopped so it'll Carry On Running if you reboot this it'll start the service up as well which is really cool it's going to run as privileged which means it kind of runs as like the pseudo user and the reason for that is because Dash is going to pull some CPU and memory usage things so it needs access to that sort of Kernel level stuff and it's going to export this or expose this on ports 3001 so that's the external Port that it's going to map the internal port to and the reason we do this mapping is because you might actually want to have something running um on Port 80 rather than Port 3001 and Port 80 is what we call HTT TP Port 443 would be https for example um and then we've got these volumes a volume is um like a file system but it's contained within Docker as a single file so it means it's really easy to copy an entire of stuff around thank you for subscribing John we just uh go over here and uh just switch that over there there we go and all we need to do to bring this up so um let's just type that back in there so docker comp POS up and I'm going to do I'm going to just do Docker compose up to begin with and this is basically going to bring that entire application up and start running it right away so you can see there it says it's listening on Port 3001 and it's grabbing all this kind of information so if I now go back over um to our um our dashboard over here let's see if we can get this uh to pull that information through so if I go over here let's see if we can get that to pull that information through so I think this is actually running on 232 on Port 3001 I'm going to save that out U did I have the right HTTP yep that looks correct let me make sure that server is actually y 1.2 232 and it should be able to see that information coming through so that's 232 and 3,001 let's just save that out and for some reason it's not happy with this uh I know why that is and that's because I'm running on my I've actually connected through https service and because of that it's saying that this this um is sending data to http rather than https which is the secure one so if I actually just switch over to uh the local version of this so if I just go to I think we're on uh 7575 there we go and let's just see if we can get this to pull the information through there we go so we can now see that Dash is now pulling through that information um which is it's showing the CP usage and also the ram we can actually go over to that machine in and actually look at the uh the dash kind of proper so this is what Dash looks like it's just a nice little uh utility that shows you what's going on there we can see all the different cores rasby pies have four cores so we can see that it's basically idling it's not doing very much at all you can see there the the mvme dries got uh basically a lot of stuff free on there and we're basically not using very much memory at all and there's the network bandwidth as well so I quite like Dash um to be running on all my different servers because it basically doesn't take up very much memory and you get this lovely little dashboard that you can just connect to uh and you don't have to use 3001 you can map that to whatever you would like so that was really really simple now that's currently running um on here if I just do um stop and we go back to here you can see it's basically unable to connect to the back end because we've stopped it so we basically just do- D for run detached run in the background and only stop once I tell it to stop so if I now go back to that you can see it's back up and running again no problem so that's how easy it is to run some of these things in your home server and your home lab just by using Docker and creating these composed files so that's what I wanted to show you there I'm going to go back now to my slides so we can have um a bit more of a look at some of these apps so we looked at home assistant now there's one that's really cool that's called jaalin if you do a lot of media streaming and you want to do that inhouse without going out to Netflix or Amazon Prime or Disney Plus or Discovery or whatever million different things are out there you can stream if you've ripped your Blu-rays and your DVDs to a disc you can use jelly finin to do all the discovery of what those IM those videos are that music is and stream it out from from uh your raspby Pi home network we've just looked at dash dot which is that nice little app for showing you monitoring servers engine X is something we'll be using um to do some complex rooting on our networks I use this to host um the kevs robots.com and also clustered Pi websites and this also does some load balancing so because I've got four servers each one of them is running an instance of the uh the website Kev robots.com and when you hit the server you're hitting one of those servers and the next person that comes in to look at the the server hits the second one and then the third one it kind of does it in a round robbing technique and engine X does that load balancing for me we've just looked at Docker we're going to look at how to install Docker very shortly I'll do that as a bit of a live demo in a second and we can use painer to manage Docker painer is a really nice interface it's a web interface so rather than to remember all those command line things you can just point and click and see all your your images all your volumes and everything within Docker um just on web page so I used pora quite a bit we looked at melee which is that recipes um program this is really nice and Homer is the dashboard that I use to store all those clickable buttons those clickable tiles and make it accessible for everybody who isn't as it Savvy as me on our home network so if you think I'm missing some apps that should be on here definitely put a comment in the the video below or let me know what you want to see on here I know there's a lot that relate to jelly fin there's like sonar and um other r um plugins and they basically just do a lot of housekeeping for your uh your ripped video content so yes that's really really cool so Docker Home Server and home lab Essentials these are the things I think we absolutely should have on here so just going to talk about Docker for a second just for those people who don't know what this is so Docker is a platform for developing shipping and running applications uh it enables you to separate out your applications and infrastructure so you can very quickly deliver and deploy software it's very easy to develop it so I've created a a search engine for Kev robots.com using Docker using some of these techniques we're looking at here so with Docker you can manage your infrastructure you can manage your networks this is why it's a home lab environment uh the same way that you manage your applications and you can also do this all in code so you can actually save all that configuration tear it down Build It Up um in a new area just by using that code and that configuration and you can take advantage of docker's methodologies for shipping testing deploying code very quickly and you can reduce that delay between writing code and getting the application running and I can attest to that I wrote a search engine in less than an afternoon um basically just using Docker so there are some differences between containers and virtual machines so I just wanted to cover this off so if you think on the left hand side there we have these containerized applications so application ABCD these could be uh the med or the home assistant each one of these can be in a separate container then we've got Docker that coordinates kind of orchestrates all those applications we have the host OS and then we have the infrastructure that that's running on whereas on a virtual machine you have an application which is running on a guest operating system and all that stack is being emulated in a hypervisor and then you might have a second virtual machine that also has an application and a full operating system and then there's a third one so there's a lot more bloat in there there's a lot more um emulating of the guest OS that's not really needed so you can do just as much in a lot less memory a lot faster using containers so this is one of the reasons we we look at this so containers are this abstraction layer that sits between package code and all the dependencies and you can have multiple containers running on the same machine they all share the same OS kernel and they are all separate from each other separate processes all running in user space and they take up less space than virtual machines they're typically tens of megabytes rather than uh gigabytes and they can handle a lot more applications require few vmms compared to yeah compared to a VM a VM VMS are a physical abstraction of Hardware sorry they are logical abstraction of physical Hardware turning one server into many servers they have a hypervisor that allows multiple VMS to run on a single machine and each VM includes a full copy of the operating system the application the binaries the libraries taking up tens of gigabytes so VMS can be quite slow to boot up so this is where Docker really comes into its own that's why I've chosen this so a container is a Sandbox process on your machine that means it's separate and isolated from all the other processes on your host machine and that means that they're also they're also secure and that isolation leverages things such as kernel Nam spaces CG groups these features have been on Linux for a very long time and they're the thing in the background that makes that isolation SE and separation possible so Dockers work on many of these capabilities made it appro more approachable and easier to use than things like a VM so how does it do this in the background so we have a client this can either be portainer this could be a remote API or a dashboard or just the docker command line then we have the host itself so in this case ours is a Raspberry Pi 5 we have several containers that are in there which run all the application code an image is a a a container that isn't running so an image is kind of of um what the container will run so images you might have one image for say Ubuntu and then you can have several containers all running that image so but they're all running their own instance of it and a registry is a kind of collection of all the different images that are possible that you could have you can have a local registry or you can use the docker Hub or third party registry um to store all those images ready for you to download and that just means you don't have to have all the images installed on your machine you can basically just pick and choose which you uh require so hub. do.com is where they mostly reside so things that they need to run the kind of resources um a Docker needs for a container to run is a volume so this is a logical storage for each container it's like its own file system you can have a network you can have different types of networks several networks you can have isolated networks uh shared networks uh specific IP addresses which is quite cool uh and basically this is the IP address to connect to the outside world and then we have ports so ports is that number that we have like Port 80 for HTTP and that allows traffic to flow between different machines so ports a bit like a something you can plug into I guess so to install Docker this is the script uh I put together to install it so you can just go to get.com uh and there is a script that they have ready for you to download there but I found that that requires you to do like pseudo Docker you have to run it as a super user so you can get round that by basically running this script so we do a Pudo apt get update that will update the operating system we do a pseudo app install so lib Fifi fi Dev uh lib SSL Dev Python 3 and python pip that's just some dependencies that we need and then curl will download the uh get Docker script and it will output that to homey get doer. sh the only thing to watch there is that home Pi whatever you call the username when you create your Raspberry Pi um setup I used to call it Kev so that might change there then we have this change mode plus X that basically makes the script executable and then we can run that get Docker script and that will install Docker once it's been installed we then want to add the pi or the Kev user in my case uh to a group that's called Docker and we want to give that uh permission to run and then we can do this system control unmask do ER and then we can change the the uh file permission On The Run Docker do sock and that's kind of they sock is a socket and it's a way of other applications can communicate with Docker to get things like stats and so on and then we do pip 3-v install Docker compose I'm not sure why it's got the- V that just gives you the version but this will install Docker compose so we can do that bring up and bring down then finally we just do system control start Docker and that'll just start the Docker running so we're going to do this in a second live and then we can do various different things with Docker so we can we can list what volumes we have we can create a volume and it basically just creates like an empty file ready for a full file system to exist in there for that particular Docker container we can remove a volume just by doing Docker volume RM which is like remove and then the name of the uh volume and you you only have to type the first couple of letters of it for it to recognize it it's got quite a nice kind of Auto complete there you can also bind rather than having a volume you can bind a file system uh to a mount Point as well if you wish to do that so uh you can basically do this by doing Docker run and then d-mount and then what the the source is and what the target is within um the docker instance so that's one way of doing it as well I use that sometimes if I want to expose files from outside of the docker um to the file system within it and then you can basically just run Docker you can do docker pull that will pull the image down to your local machine getting started is one of their test applications and you can do things like Docker image list to see which images you have on there and if you want to tidy everything up you can do Docker image prune and that will get rid of all the unused uh container unused images on your machine just to get rid of some uh get some space back couple of other things as well we can do Docker run dasit so this makes it interactive over the TTY console that's what that stands for so interactive terminal uh you can do Docker image RM to remove a specific image you can do Docker container ls- all if you do Docker container LS it will only show you those that are running whereas if you do Dash all it will show you the ones that are not in use as well and then Docker volume LS will list all the volumes Docker attach is a way to attach your console to a running container and you can also just stop the container running when you're within it by doing exit uh and that's uh just a way of stopping the container running to disconnect you can just do the contrl p contrl q and that'll keep it running in the background and then we got Docker PS that basically just shows you which um containers are running if I just switch over to here for example I do Docker PS it's not the prettiest of outputs but I can see on there I've got do Dash dots running I have the register running I have kroots which is the uh uh website and also we have pora running on on there as well that's running on 9443 so that's running on 2323 so we can have a quick look at that actually we go to 9443 and it does require https and I think I'm at an admin on here I always forget my password for this what did I put for my password on this I can never remember that um we'll ignore that one for now okay let's get back to um to here and yes so if you like what I do you want me to make more of these types of videos please give me a like give this video a like drop me a comment let me know if there's any apps you'd like to run on your home lab and um also if you've not subscribed to the Channel please consider subscribing it really means a lot to me I do go live every single Sunday at 7:00 GMT or British summertime whichever we're in at the time um so hop over and join me if you can on there so let's set up a home server and home lab shall we so what I'm going to do I'm going to do this kind of live now um so that's a it's a different machine Let's ignore that one and let's go over to our desktop I'm going to try and do this um live so I've got a brand new Raspberry Pi 5 here this is an unboxed one I bought this one yesterday from the the leads Raspberry Pi store so you not seen one of these before it's quite basic inside there's our Raspberry Pi we've got some uh information there about how to set it up now I have already installed onto this flash um onto this micro SD card uh the raspberry pios because it does take a little while to do that so I'm just going to push that into the uh slot just there and what we need now is some power and also I'm going to plug in this video so we can actually see what's going on as well so I plug this in here plug in the power and then if I quickly switch over to this one here this will once it uh has done a quick post check this will change from these uh these little bars to let's see if we can get there we go Raspberry Pi desktop so the first time it loads up um I think it has to expand the file system so I'm not sure if it just did that a couple of seconds ago um I also just need to plug in a keyboard I'm just going to do that now as well let plug in a keyboard and I'm going to let it use the the Wi-Fi connection as well so these colored bars just mean there's no video coming from the pi just while it uh it boots up right just going to give that a second wonder if we can get me in the corner actually let's see if I can uh add in an extra camera on here so me just pick let's see oh I didn't want the screen share that's the wrong button I wanted the camera let's see there we go I put me in the corner over that okay so this is our Raspberry Pi desktop I've just got a regular keyboard and mouse plugged into the Raspberry Pi and this is the first instance of this Raspberry Pi running so it's not got any updates or anything installed on it I don't even know what IP address it is so let's have a look over here just hover over this and I can see this connected to my home network which is good if I just hover it should tell me 112 is the uh the address there right so what I'm going to do now is I'm going to grab those clustered Pi files if I do git clone https colon github.com Kevin melier oops and then clustered piie like so so that should grab if I've typed that right yes the clustered pie code should download it onto this Raspberry Pi and then we can start put bringing some of those um Docker instances those Docker images up on our brand new Home Server that we we are creating now as we speak our home lab and Home Server so it just takes a couple of it's quite a big um oops keep knocking the micophone it's quite a big repository this that's why it takes a couple of seconds so if I now do change directory go into clustered p uh and let's go into stacks and let's have a look what we've got in here so we've got a whole bunch of different things that we can install now before I install these we need to install do so let's go into um I'm going to go into the playbooks so before I discovered Docker I used anable quite a bit for installing things and one of the things on here is Docker so I'm going to actually use this Docker install uh script let's just let's just have a look inside it first so if I do Docker Das install trying to do auto complete here Docker in no not Docker prun Docker install if I type it correctly then it should auto complete Docker install sh so this is what this script is going to do it's basically just going to make sure those packages are installed it's going to do a pseudo app update um it's going to remove this uh python config paer for some reason and if we haven't got the get Docker it's going to grab that uh it's then going to add the user like we look before that user mod uh and it says Pi so I need to change that that's the first thing I was going to look at let's just change that and the other the other instance of the word pie I keep pressing the wrong key there uh let's just change that to Kev because I've logged in there as Kev let's change that there I think we need to also just change that one to Kev as well I'll show you why that is in a second if youve not following along let's make sure there's no other instances of that no that looks good good and then that's fine that's fine and that is fine right let's just contr X Y to save okay so let's now run that file so the think to run it I basically just need to do that CH mod plus X to make it executable and let's just type in Docker install. H so if I now do LS you'll see it's got green and that means that file's not executable so we can now just do SL do install okay and has that worked let's do LS let's see so we should have a get Docker if that's run properly it seemed to be very quick when that run I expected that to take a little bit longer so let me just make sure that did actually work Docker install interestingly it's changed it back to the previous um let just try that again not sure why that didn't work Docker install told this is a live uh install so some things can go wrong sometimes that's really weird okay we'll have to do this the kind of L hand way then so let's just open up another terminal let's just get this uh file in one window and then run it in another window so we can basically just copy and paste the bits that we need I'm not sure why that didn't work I think it might be detecting that it already exists maybe uh okay so oops let's just scroll that back down okay I just want to get to the top of the uh so somebody saying should it run a pseudo let's let's try that Wayne that might be a good idea let's try pseudo install Docker H get rid of that there as well let's try Just sh no such file why on Earth can it not find that okay I'm going to do it the way that I expect them so let's just do cat install Docker one more time I I created this to try and make things easier and obviously this is just backfire there so Docker Dosh what is going on there so let's just change that again plus X Docker install.sh that's cat Docker install.sh there we go so I'm basically going to open up another terminal here next to it and basically just run the lines one bit at a time that's probably the easier way to do it I'll I'll find out why that's not working an update the scripts so that that works you should be able to just run it straight off so the first thing we're going to do is a pseudo app update PSE sudo appt update that's usually quite quick because it's just grabbing the Manifest of the updates um you do usually do an up upgrade after that but we're not going to do that now we're now going to do a pseudo app install and we need to have the following packages let's find them so lib uh ffi Das Dev we need lib SSL dodev we need Python 3 that's quite a big install and we also need python let's just move that out the way there Python 3 pip Python 3 pip okay let's get it to install that yes please and it's going to grab those and it's going to install them so this is where the Raspberry Pi 5 is quite good because it's quite quick at doing that see it's done it okay what we need to do then next is we need to we can skip that about the python config PA so that's not a problem we need this curl command here so let's just grab that so this curl command is basically going to grab uh the content of this uh let me just copy that there I've got a bit of a like a drunken cursor going on here there's a slight lag cuz we're watching this over a capture card and um that seems to to make it look like a drunken cursor so it's it's difficult to uh control through here if I if I was directly plugged into a monitor I think it'd be fine but because I've got it through the capture card it's a bit tricky there we go it's kind of messed up a little bit the front of that let's just get rid of that 200 and then run that and that's that'll be really quick to command to to to run so why can't it ex why can't it do that let's just move that slightly blocking the end of the command line there ah it's got a little funny character at the end there let's get rid of that so failure writing to Output destination maybe that's why there is an issue so let's see where we are all right we're in we're in a directory where we can't normally run things well let's go into documents for a second documents there we go and let's just run that command again there we go Cur ah and it's because this is another instance where I changed it on the script and then it's it's reset it back so that curl command is doing home SLP it needs to be home slev that's the problem there I always think it's important to show you me making mistakes because um it just keeps it real okay there we go so if ially do we'll see that we've now got in the um directory beneath this one so if I go back One Directory there we go get Docker so we can now change that like we did before change mode plus X and then the get Docker okay and that will make it executable it should be now green and we can now do get Docker like so and that will now go and grab Docker run the get Docker install script from Docker themselves it just takes a couple of moments for it to do that and then there's only a couple of other steps that we need to do after that which is mostly um just fixing some permissions so that it can run V says wishing I could help po Kev i' created this script i' not actually tested it out I wanted to show you open opening a rasby pi 5 and if I unbox that it kind of ruin that little surprise there as well wouldn't it so that's why I didn't run that that's my excuse so it doesn't take too long to install Docker and then we can start up upping some bringing some of these applications up that we have in those Stacks so while that's installing over there let's just grab the next command that we need to run so it's just this user mod one so I'm just going to grab that so pseudo user mod AG do Kev might be able to type it faster than I can cut and paste from here here copy we go okay that's now installed so we can now just paste that command we've just uh grabbed paste that there okay and then we need to do the pseudo on mask just grab my keyboard that work quicker than I thought it would do so pseudo and then system sorry user mod we've got user mod so it's system control CTL unmask Docker I always imagine it's like a Scooby-Doo moment unmasking Mr Johnson the fair ground keeper right so we're now going to do pseudo and then change the mode so this is going to mean that um other applications can then check they can read and write so 666 is to do with Unix file permissions uh I was thinking about this this morning in the shower figuring out what the sixes are I think I think that's a read and write but not execute so read run and then Docker do sock and a sock is like a a socket for Docker um so it's like an interprocess communication so let's just now scroll that down a bit more to the bottom and I think the the next one is install Docker compos now I have had some problems on a Raspberry Pi installing this so if we do um python three let's see if it will let me do this it probably won't let me do this because the new way that uh raspberry pies have python environments um sort of sequestered off so if I do pip install and then Docker Das compose because Docker compose is actually a python script which is quite cool okay yes so see there it's uh not allow me to do pip let's try pip three I think is it- M let's just try pip three on its own and once we've got that we can then finish the job if if this one doesn't work we can do pseudo appt install so yeah I got this message before basically it's not happy with some component in there so let's just do pseudo apt install Docker compose instead and that seems to do the trick so yes to install all that I think has to do like a full python install but again it's quite quick on the raspberry high five to do this um cuz it's so much faster so 50 up% through 60 up% and then it's just going to do the install of that which is pretty quick as well I think okay so while that's doing that let's just go over here and go back and have a look at some of these Stacks that we've got on here so these Stacks are ones I preconfigured that I've been running for quite some time now so we Dash is the one that we looked at a couple of moments ago it Eclipse mosquito is a cool one that's for creating an mqtt broker we've got home assistant there we've got um the whole Kev robot stack we have masteron mongod DB MySQL engine X node red paina post what's that post gql and we've also got it's got Mouse in the way there um the weather code I think that's for um the weather kit that I've got there uh random facts which is a new python service I created the other day um so we can have a look at that random fact one actually let's go into that one let's have a look what's in there oh there's nothing in there I mustn't have updated that one yet okay let's go to um what would be a cool one to run let's try home assistant let's see if this one works the way I expect it too okay me just go and check back on the other window there yep that's all done so so that should mean now we can do the docker compos I think the last command for this was we basically just started the docker service so if we do pseudo system CTL start Docker I think that's it yeah you can also enable that service and that means that if the service stops it will restart it so I think if you do enable that will make sure it survives a reboot so there we go and if we need do if we now do Docker PS uh it's saying that we need permission now we've added all the permissions in there so I think that might basically just need to have a reboot so let's just reboot the pi uh in it six we'll do that it just take a couple of seconds for it to take that down and then bring it back up again these bars are just my capture card saying that there's no HDMI video coming through so once this has come through we can then um bring up um an instance of something like home assistant I'll not set it up I'll just show you that it this is this is how easy it is to do it and you'll see it's uh downloading some of the packages and things like that as well all right let's go into terminal let's do Docker PS perfect so the fact it says that container ID there's nothing running there but it does mean that Docker is accessible now as a regular user so if you go back into that clustered Pi folder so cluster Pi stacks and then we're going to go into the home assistant one and then let's have a look what we've got in here so there's just one Docker comp post file let's have a quick look at that and it's just got a couple of things in there so there is the home assistant service it's going to create a container called home assistant it's going to grab it from the uh that particular GHC doio repository uh it's going to create two volumes one which is mapped to home Pi clustered Pi home assistant config so we'll just need to edit that to be Kev instead of Pi uh it's going to going to continue to restart it unless it's stopped it's running that privilege mod mode and it's also going to try and run it on IP address 150 um now that might clash with an existing one so let's see what happens on there but um I'm basically just going to edit the file uh using Nano just to change that one line so instead of being home Pi it's home Kev because that's the username of the user that has everything installed there so if I now do Docker compose and then up up- D let's see what happens so it's not happy about that um um Network so it says there basically just do Docker Network create so we can do that create and my Mac VLAN active is a bit of a um a deeper dive thing we'd have to cover off in a different show I think but it's a type of network that Docker uses uh and it means you can have like other IP addresses um that you specify uh which is what that 150 thing is all about right if I do that and I try again I like the way it helps you out with this so there you go so what it's doing now is it's pulling all the different parts of the docker container because Docker containers contain almost like different pieces of the jigsaw different um subsections that make up the container so one of them might be like Python 3 so if a new version of Python 3 comes along um you could update your container to have that latest version so it kind of splits these things out so it's downloading these now once it's downloaded all these different things it will then bring up home assistant really quickly so it's just down it's just down to how fast your broadband connection is so I'm streaming and downloading at the same time so streaming up but download downloading at the same time so my bandwidth must be like maxed out at the moment once it's pull these I think it then has to extract them because they're kind of like a zip file that they're encrypted and compressed um for security so once it has downloaded them it will just just expand them extract them out they go extracting so you'll see all these little things sort of say extracting and once the P's complete that means the image is ready ready to be used so once the last one's done it can then start running it and running it with the configuration that we have in this home assistant. config file that I previously built um so let's see how far that gets if it works it works if it doesn't we'll move on and I'll show you some other things about how how I've set up the uh the DNS and the routing that and so on for to make this home um server work with home.mac docomo thing to look at so I think this 400 Meg file is quite a chunky one there so that might take a little while to do I think I need to get some DW on dw40 on my chair I've got a squeaky chair going on there so while that's going to downloading I'm going to flick over to the slides and just carry on just because this is going to take a little a little while so arav is asking is this a four or8 gig model so a very easy way to answer that on uh Raspberry Pi's if you want to check yourself if I just go over to here and run a another terminal if we type pin out and then me just move this across here and then if I just scroll up on here you get like a little image of the Raspberry Pi so it's a Raspberry Pi 5B and you can see there that it's um does it tell us the the memory that's in it if I scroll down I think on here somewhere it tells you the actual spec of the machine there we go so this is an 8 gig version there you go and it's running on the micro SD so if I run this on one that's got the nvme Hat you'd see that it say nvme instead of micro SD I will upgrade all these to have nvmes shortly um it's just hard to get hold of the hats at the moment so that's a quick way to find out if you have for 8 gig version I went for the the eight gigs on all of these hey Darcy how's it going there's no Java on this show I'm afraid so yeah I've got two of these um Raspberry Pi where's the case gone for that one and yeah both of these are the 8 gig version I it just says on the the top there on the bottom there 8 gig cool right so has this nearly downloaded we're about 8% done now aren't we I'll let this carry on and then we'll just complete the install now I think um Maria uh sha is doing a a Docker show very soon or has she already done it I can't remember so I'm going to be a guest on her show I think it's next Monday night no so so not tomorrow but a week tomorrow so we'll be talking about robotics with Maria and she has a channel it's called python simplified which is really cool so if you've not checked out you absolutely have to check that out I'm pretty sure you will already seen Maria's stuff already but she's she's I think she's studying computer science and she's kind of teaching what she learns at the same time so it's a really good reinforcement of what she's learned but yeah she's very good tutor okay so we very nearly installed uh well downloaded this so it's extracting now that means it has downloaded which is cool and there we go so the downloads complete here we go yeah she's good isn't she Darcy love it okay so now it's doing this it's probably just doing a little bit of housekeeping in the background with all those images and then it's going to bring up home system there we go so it's just extracting that other one out there and then I think it's got those other three to go it's very exciting once you've downloaded these images it does cache them in the background so you don't have to download them again unless there's an update to them so I think you do like Docker pull to get the latest version of it otherwise it'll just stay there right so it's now creating our home assistant you can see there that little command and then once it's done that it will tell us that the uh the application is up so there was an issue there and it it was the IP address I said that might happen because the IP address was uh uh 150 so what we can do we can actually change the IP address so if we just go back on there we don't have to download all them again we can simply just come down here let's just change the IP address to be 151 15 151 let's go for that okay let's save that out and let's just do dock up so it doesn't have to do all the downloading again uh it's still not happy about that so it says the IP address is supported on connected to networks uh the other so I'd have to troubleshoot that one we know it would work that's probably a specifically weird one the reason I had a um um a specific IP address is because I wanted to uh to use that so that I knew exactly what that would be um when I connected to it so if I just um hash these out I just comment these out we can probably get it to run uh without that information it doesn't really really need it to be there we can do the same with those three lines there as well let's give that a go and then let's just try bring it up again it's going to recreate it and it's done right so it's now up so if we do Docker PS that's now running o Docker PS even that's now running in the background there we go and we've not expor exposed any ports so it's running but it's running inside that container we can't actually get to it so we would actually have to expose ports 8 8123 I think it is uh have we actually exposed that in the docker compose file sorry run the same command twice it's already up so that's fine let's just have a quick look inside there so we can basically just add in here port or ports and then just add in these lines so we want Port 8 one two 3 to be maap to Port 8123 I think that's it let's just save that out yep that's fine and then we can do even though it's still running we can say dock up and it will recreate it and then it will only swap it out when it's finished building that uh that new container so it's building a new container swapping it out so if I now do Docker PS again we hopefully can see that that Port is now exposed and that means that we can connect to it and um we can see what home ass system looks like so yes so Port 8123 which I think is the home assistant one so if we load up a browser in here so this is within the Raspberry Pi I could I could launch this from the Mac as well if I wanted to did I hit the right one then yes I did right and if we just go to the Local Host on here so I think it's 192.168 dot one dot and what was the the IP address on the machine here 112 112 and then Port 812 3 home assistant there we go so we brought up home assistant it's ready to create my new smart home so if I click on that it will then start doing all the user setup so that's how easy it is to to bring up an application you can basically do that with any of the stacks that are on there so if you find an application you can find a Docker version of the application pretty simply so I'm going to go back now over to the Keynotes and we can just have a quick look at some other bits and pieces so how all this works then this this Home Server and mapping um a domain name so that you can basically type in something like home. km. comom uh there's a few things that are going on there some quite techy things in the background but I'll I'll talk you through how this is happening so uh we have these raspberry pies they're connected to my local router and my router is is connected to the internet and there are some things in the background we will cover these off in a second um that allow traffic to go from h.m. comom to find that being hosted on these Raspberry Pi in the container in the docker container and then push all that information back uh to that web address so that we can see it live so my home router redirects all traffic that comes to home. kar.com to my home network and then Homa can have that link that little tile so when you push that you can then get through to say home assistant so high level that's kind of the the different the the rout that's going on there so to make this work we need a domain name I bought kevm ma.com years ago when domain names were cheap and people didn't tend to buy them but it's actually quite hard to get a short domain name nowadays all the good ones have been taken particularly it's do if you have something like.xyz or or any infinite number of other extensions I think they're a bit easier but the Dooms are quite uh quite used up but you can buy them from sites like godaddy.com that's the the site I use to to buy all my domain names I one of those people that holds domain names and has way too many I think I bought one uh yeah that what was it yeah that's what it does.com something like that I don't know so I own this kevm mar.com and I'm going to use that for my Home Server so that home. kevm mar.com that prefix is a subdomain and we can create as many subdomains as we like and I use cloud to protect my home network so I don't actually expose my pure home IP address um Cloud flare disguises that uh and puts a different IP address kind of in the way of the user so if somebody tries to attack my home network they're basically just attacking Cloud flare and they can protect they can prevent um denial of service attacks dos attacks uh so and it's also free to use as well Cloud flare so that's quite cool gold daddy does charge I think an annual fee or every three years to renew your domain name but it's quite cheap to do that so rooting is something we need to to be comfortable with if we want to rout traffic from home. kev.com through to our Raspberry Pi uh now your home broadband comes with a router and typically that router does like Wi-Fi as well but hopefully also has a wired socket now if you're going to be doing any kind of Hosting you'll probably want to have a wired connection uh to your your devices so my raspberry pies so these ones behind me here they have a little router underneath them a little uh switch and if you want something a bit more heavy duty my home lab is going to be upgraded to this uh AR Ruba switch here which has got like how many is that 24 ports um it's a little bit noisy so I need to sort of find somewhere for that to live it doesn't disturb my live streams but that'll give me a lot more ports than I currently have available so that's going to sit between my my home router and the Raspberry Pi themselves that switch down there and my router is connected directly to my broadband connection which is to my internet provider which is British Telecom and then British Telecom and then enable all traffic um from the internet to sort of travel it between um my raspberry pies and the and the world now obviously you don't normally host things in your home network so you have to do a bit of hacking of your router to make this happen uh but most routers have this kind of capability in there they call this port forwarding uh and sometimes they call it an IDM Z it's like an in um or or DM said industrial demilitarized zone and it's a way of protecting the internet traffic and all Bad actors from attacking your home network so they only allow traffic in that you allow in so the way that you do this is with port forward in so you can see here on my home rout This Is My BT business Smart Hub um I've got a couple of different rules and the two rules that are important here is these ports the external ports 8 and the internal ports being mapped to Port 8 and they're being mapped to the IP address 192168 1.4 so one of my raspby pies behind me the fourth node uh is also running an engine X load balancer and this means that it can take all the traffic that comes from the internet to those put at addresses and then it can then redistribute it to the correct place so my home weather server is only running on one particular Raspberry Pi node not 1.4 so I think 1.3 and it's also running on Port 3000 I think it's Port 3000 the grafana instance so that engine X can actually map those IP addresses it can look at traffic that's coming in and direct it the right way I'll show you how to do that in a second but essentially what you need to do on your your uh home router is set up some Port folding uh so that you can connect Port 18 Port 443 which is like secure um HTTP to the outside world does mean that that machine is then not really protected by your home router so the the firewall is off in effect uh and you can see here here are some of the um this is on cloud flare this little screenshot uh and I've created a a uh aame record which is like the the host name uh food so the sub subnet food. kevmac docomo name there uh has an IP address of that router that's behind me so those are you need one for each subdomain that you want to create and then these other numbers over here these are 1851 199 10853 uh I think they're all GitHub so if you go to kevm ma.com you get like a really basic looking page with some links because it's really old web page but they're actually hosted on github's repository and then there's some other things on there as well such as like the world worldwide web c name is like an alias and that basically just takes you to um that first one there I think these proxy things mean that uh cloudfare they actually cache some of the content so it makes it much quicker so when you're you're accessing kevs robots.com and there's like a great big image file it probably has a local copy of that and serves you that copy rather than hitting my server for every single one of them because that would be really really slow so engine X I was talking about that so I have an engine X server running uh you can actually look at the content of that if you really want to on um looking if you download that clustered P repository that we did earlier in the show uh you can basically look at the engine X stack and that has all the configuration in there so I've got this little snippet here which says listen on Port 80 to food. kevs robots. food. kevmac docomo pass so basically just forward all that information on to 192168 1.100 9925 and that's what that melee is running on on that other so 100 is another rasby Pi 5 I think that's Dev Dev 1 or Dev 2 uh and if you if it hasn't got um uh an Indo at HTML or anything like that this is what these triy files do so it looks for u. HTML the URI itself index HTML or just index HTML so it tries all these different things otherwise it'll just give you a port 404 not found so these things make our services a lot easier to access it's a bit of hacking once and then uh it's it works forever so if you like these kinds of things you want to learn more and I will be putting up a Docker um of course I've half built it I just need to build out some more make it look a bit prettier uh that will be available on C robots.com learn and you can uh learn all about Python and Robotics and so on uh for free so check that out if you've not checked out already and we do have merch you can get these hats we will be selling the uh the things I promise we'll be selling these very soon we just need to work out with our shipper the shipping rates and we also need to get a government E and I number I think it is some kind of economic registration number but yes the robot makers Almanac will be available very very shortly and you'll be able to get yourself one of these be selling those at the maker Central as well this year I think too so head over to C robots.com merch to check out the merch store help support the show and if you've not joined our Discord server then go over to Kev robots.com Discord and you can join the party there as well I know Dary was talking about setting up a Discord server our own it's very to do that so you just need somebody to give you a hand setting up all the different channels and if you want to join me on social media I'm all over social media and you get all kinds of behind the scenes things I did a fun little video on social media about this robotic egg cup which has got this little server that sort of cracks the egg um it's based on Robbie the robot a lot of people thought this looked like Vincent from black hole but it's actually or Dr robotics is it from uh robot Nicks from Sonic the Hedgehog no it's actually Rob Robbie the robot so I'm on uh threads Kevin Mia threads. net I'm on Tik Tok at Kevin Mia 6 U I am on Instagram at Kevin Mia on X kevmac on masterdon kevmac masterdon docial and I'm also on um Blue Sky km. B.O as well so if you're on there follow along and say hi and if you want to get your name in the end credits you can do that as well by going to kez robots.com coffee uh you can buy physical coffee if you're watching the show live you can do a super chat if you're watching this on replay you can do a super thanks which is a little thanks button underneath the uh the main player and you can also join the YouTube channel membership program um which is just the price of a coffee every month by clicking that little join button which is also just below the main viewer window as well if you're watching on desktop so here is some of the supporters I just want to give a shout out to so we have a new coffee bought today from Steve Robinson thank you for that Steve I really appreciate that somebody else who wanted to remain nameless Bottle Coffee and Maria Louis made did as well in the past month and uh we have members on The Bu me a coffee side we've got uh alvro Diaz we've got Mar Le Louise Mayer we have Jeff Johnson Dean Cy Marin Brent Tom shmy and Steve Phillips hi to everybody that's in the the live chat we'll have a chat in a second got so much stuff to show you from the raspby pie shop in a second and then on the YouTube membership side we've got Vince who joined last week so hey Vince thanks for joining aliser wear John Paul Jolly Cassie Dale from hybrid robotics tinkering rocks JDM Johnny bait Bill Hoy oxr 39 handsome Cher lights got Michael and of course we have Tom as well so I think that is everything I've got for you for the main show today so this is the point of the video where YouTube will suggest some videos I think over here uh of videos that you thinks you will find um interesting also to do with robotics python raspby pies so definitely check check one of those out as well and if you're watching this on replay this is the point the video where I'll say thank you so much for watching and I shall see you next time

Channel: Kevin McAleer

Views: 24,059

Keywords: raspberry pi 5, docker, home assistant, home lab, homelab, home server, homeserver, lab, home, portainer, mealie, assistant, nodered, node-red, raspberry pi 5 setup, tutorial, raspberry pi 5 tutorial, how to build a home server, how to build a home lab, how to build a homelab, how to build a homeserver, raspberry pi, raspberry pi projects, home server build, homelab setups, how to homelab

Id: NmpcKv91LLs

Length: 68min 58sec (4138 seconds)

Published: Mon Jan 22 2024