Raspberry Pi Cluster Ep 4 - Minecraft, Pi-hole, Grafana + MORE!

in the last episode i showed you how to install kubernetes on the turing pi cluster running on seven raspberry pi compute modules in this episode i'm going to show you some of the things you can do with a kubernetes cluster in the last episode i explained in a very basic way how kubernetes works it basically runs applications across all your servers to help manage kubernetes there's a handy little tool called qb cuddle sometimes i call it cube control but if we go down that road i have to start talking about why it's called a gif and not a gif and why spaces are better than tabs or why vim is better than emacs and i i really don't want to go down that road richard what's wrong nothing nothing literally it's all good i won't go into a deep dive on managing kubernetes in this series because that could be its own 24-part series but i will be using qbcuddle a lot to deploy apps on the cluster and to check on their status or to make changes to them there are usually many ways to do the same thing in kubernetes and i'll let you decide if that's a feature or a bug and just as a warning there may be a few parts of this episode that are a little over your head and that's okay that that's kind of how it is working with kubernetes the only way to learn is to build test clusters make them fail in new and exciting ways and then start over again i like to think of it like every second you're working right at the edge of your understanding when it works it's magic and when it doesn't well good luck no i'm just kidding i'll try to help you understand the basics and once you start building your own cluster hopefully you can start to understand how it all works together one of the most frustrating things that you run into if you're trying to do things with raspberry pi's or any arm-based single-board computer like the raspberry pi is the fact that many applications and container images aren't built for the pi so you end up in a cycle like this you think of some brilliant way you could use the pi cluster and you search for how to do brilliant thing x in kubernetes or k3s you find a repository or blog post with the exact thing that you're trying to do and you deploy the thing to your pi cluster and it doesn't start you check the status and kubernetes is saying it's stuck in a crash loop back off so you go check the logs and you see exec user process caused exact format error this happens more often than i'd like to admit you see the problem is most container images like the ones you see on docker hub or quay are built for typical x86 computers like for the intel or amd processor you're probably using on your computer right now most are not also built for arm computers like the raspberry pi and even if they are there are many different flavors of arm and maybe it's built for 64-bit arm but not for 32-bit arm that you might be running if you're not on a 64-bit version of raspberry pi os it can be well a little complicated and sometimes if you really want to do something and can't find a pre-built image that does what you want you'll have to build your own container images it's usually not too difficult but especially when you're quickly testing new ideas out where other people may have already done the exact same thing and did the work for you but you can't use it because it's not compatible it can be a bit frustrating to illustrate let's look at one of the simplest hello world examples from kubernetes own documentation the hello mini cube example if i create the hello node deployment with qb cuddle create deployment hello node and then pass it the image and then i check on its status after it's deployed using qb cuddle get pods after a couple minutes i'll see the pod in crash loop back off state and if i check the pods logs with qb cuddle logs i get you guessed it exec format error and not to be discouraging but sometimes when you want to solve a problem with kubernetes on a cluster of raspberry pi's you're kind of inflicting the pain on yourself imagine this scenario you want to run a minecraft server on your cluster now think about all the other people in the world who run minecraft servers how many of these people not only run a minecraft server but run it in a docker container in a kubernetes cluster on a raspberry pi which might be running a 32-bit operating system the amount of people who are doing the same thing that you're trying to do is usually pretty small so like early explorers you may have to do some extra work to get things working google is your friend here because often there are one or two other crazy people like me who are doing the exact same thing and finding their work can help you a lot now for the rest of this video i'm going to show you what i deployed to my turing pi cluster and how i did it so the first thing that i usually do with a kubernetes cluster is i want to make sure that i can know what's going on inside the cluster so i use some monitoring tools to see how the cpu and memory are affected by things that are running on the cluster and the best thing for that in my opinion is prometheus and grafana prometheus collects metrics from all your servers and all the running applications and then grafana can display them in nice fancy dashboards so there's a project from core os called cube prometheus and this is pretty much the standard that people use when they want to run prometheus and grafana and alert manager on their clusters and that's what we want to do so i'm going to go ahead and try to use it i'm going to use git and clone the project to the raspberry pi to the master node and then try to apply it using the quick start guide here so i'm going to log into the raspberry pi pirate at turing master and i'm going to switch to the root user account you can actually set it up so that you could do it as the pirate user or you could do it from another computer too but for consistency's sake i'm just going to do it in here clear that and now i'm going to clone this project using git git actually i'm going to go to my home directory which is pwd which is root so i'm in the root folder now and i'm going to clone this project with git git clone paste that in and it's going to grab all of this repository data then i'm going to go into the readme and look for the quick start guide which is right here and it says basically that there are some manifest that that you can apply to your cluster and it will set everything up for you we'll get into what manifests are and how they work in a bit but i'm just going to try this and see what happens so we'll create the setup manifests i have to cd into the cube prometheus directory and then i'm going to run these two commands so apply the setup manifests and then run this command to make sure that everything's working and that command is not giving me anything promising so i'm just going to try running this command by itself and see what it gives me no resources found so sometimes when this kind of thing happens you'll need to use qb cuddle to investigate what's going on in your cluster so i'm going to see i know from past experience that this cube prometheus project actually sets up a namespace the monitoring namespace and i can see that also if i use qb cuddle get namespaces it'll show me that there's a new monitoring namespace and it says that it was created 42 seconds ago when i created all of these things so you can see it was the first thing it created up here so i'm going to look and see what pods are running in there and kubernetes pods run containers and applications and they're kind of the basic unit of kubernetes application management there's a lot of different ways to manage those pods in kubernetes but i know that pods are the things that actually get the work done so i'm going to get all the pods in that monitoring namespace using qb cuddle get pods dash and monitoring and that's going to show me that there's a pod in here that's stuck in crash loop back off i think i've seen this before let's see what's going on i'm going to i'm going to get the logs out of this pod so qb cuddle get our kb cuddle logs dash f dash n monitoring and then i'm going to see this is going to show me the logs and it looks like there's actually two containers in this pod and there's two so let's see what's going on in the prometheus operator maybe it's crashing so i'm going to say qb cuttle logs dash f dash n monitoring and then the operator pod and dash container and give it that one and it looks like prometheus is actually working okay i don't see any problems here let's check this other container and see what's going on so i'm going to ctrl c to get out of that and i'm going to look at this container there's our old friend the exact format error so apparently this cube our back proxy thing is is not going to run on the raspberry pi and i'll i'll cut this story short and not not make you sit through this whole process but i actually found this problem when i was working on this and trying to get it to work and so i open up an issue on github about this basically the the cube rbac proxy pod which runs alongside the operator pod or container i should say uh it is not compatible with the arm v7 raspberry pi 32-bit operating system that we're using so it gives us this error and we're gonna have to find out a different way now i'm gonna cut a bit of time out of here actually the first time that i the first time i tried doing all this i i found this problem and i started investigating whether i could port this project to something raspberry pi compatible on my own and i i got pretty far i got almost everything working there were a couple little things that weren't plugging in together but then in the midst of all that i found another project by somebody named carlos carlos edp created this cluster monitoring project which is it's basically the same thing as cube prometheus but a cool thing that he did is he made this all work really well with raspberry pi and arm based clusters so any kind of arm single board computer it runs really well with this cluster monitoring project it even gives you things like cpu temperature monitoring which is really cool because you can see if your raspberry pi's are going to be throttling when it's running kubernetes workloads so what i'm going to do is i'm going to i'm going to delete all the things that i just created because i don't want to run the cube prometheus project this is it's not worth the time to try to get that working sometimes when you run things on the raspberry pi you just kind of need to find a better faster way and this is the better faster way because somebody already did all this work for me so i'm going to say qb cuddle delete dash f and then it was in manifests setup so that's going to delete all the resources that i had just created and i'm going to go back a directory and i'm going to remove the cube prometheus project because i'm not going to use it so now i'm going to grab this project which i know is built for the raspberry pi so it should work a little bit better so i'm going to say git clone that then i'm going to cd into the cluster monitoring directory and i'm going to follow this quick start guide so let's see what it tells me to do a cool thing about it is that it has directions for a kubernetes cluster mini cube cluster running on your local computer or even k3s so it's it's tuned for all these different scenarios so i'm going to need to edit this vars.jsonnet file and change a few settings for my cluster so i'm going to say i'm going to look and make sure it's in here so there it is i'm going to use nano to edit that file and i'm going to find the k3s enabled settings so i'll go down to find that here it is k3s enabled i want to set that to true and then i also need to change the master ip so i know that the master ip on this cluster is 10.0.100.164. and i'm also going to change the the suffix domain here to have one of the ip addresses of one of the worker nodes that's going to be used when i access services running on the cluster so i'm going to choose 10.0.100.99 i think that's worker o2 so i change that and then there's also a setting it's not mentioned right here but you can you can set it up towards the top that turns on the arm exporter that gives you data about cpu temperature things like that so i'm going to say true for that those are all the changes that i'm going to do for now there's other things you can change in here and you can read the documentation for that for your own cluster but that's all i'm going to do for this particular setup and i'm going to hit ctrl o to write the file and then ctrl x to exit and now i'm going to run a couple commands to build the manifest for my particular cluster now there's a couple things that you need to install if you don't have them already installed i do on this raspberry pi but you need to run apt get update to update your app caches and then run apt get install dash y build essential and that's going to install the make utility along with some related things and golang to make sure that you have the go language installed on your raspberry pi and i already have them installed so it should say that and then then go back to the command line so now that we have those those dependencies i can run make vendor and so i'm going to take that clear this i'm going to run make vendor this is going to take a little while then i'm going to run make and this will also take a while as it generates all of the different yaml manifest files all right so it's generated all of the yaml manifest files that kubernetes needs so now i'm going to do the same kind of thing i did with with the cube prometheus project i'm going to apply the setup manifests and that should create a monitoring namespace and all of the different things that go inside of it and then i'm going to apply all of the other manifests that are generated and now that those have all been created kubernetes is going to start taking all the things that were just created through all these manifests and start scheduling them on nodes so you can watch that process if you use qb cuddle get pods in the namespace monitoring it will show you what's going on in there and you can see that a lot of these pods are already running one of them's still creating if you want to watch it you can use the watch command and it will let you see that updated every couple seconds so you can see that prometheus is still getting up and running the containers creating it's it's likely pulling down the information to that raspberry pi uh the the docker image that's going to run prometheus but you might notice that there's a lot of instances of some of these things there's a bunch of arm exporters there's a bunch of node exporters that's because this project sets up a bunch of different pods one on every single raspberry pi to do things like get all the cpu metrics get all the memory metrics and get the temperature data and send it all into a central prometheus instance and we'll see that very soon once prometheus starts running sometimes when it's starting up you might see an error the first time it starts but it usually gets into the running state pretty quickly and at this point i can also see if there is a a way to get to these by saying qb cuddle get ingress dash and monitoring and ingress is just a way for kubernetes to give you a nice host name to access your services that are running and we'll talk a little bit more about some of these things in the next deployment that we do so prometheus is running here and prometheus is aggregating all of the monitoring data coming from throughout the cluster into it and prometheus lets you browse all that data and there's a lot of it there's hundreds and hundreds of different things that you can monitor and most applications that run in kubernetes also put out metrics endpoints that can be aggregated in prometheus and it gives you things like let's just do a really simple one up and you can see that these are all one that means that they're up and running so you can monitor whether things are up or down you can graph it and obviously since everything is up and running there's a lot of different things that are just one here but prometheus monitors all these things it's not the best interface for something like a dashboard to see how things are doing at a glance but prometheus is important to get all that data together so that you can browse it and see what's going on in your cluster if you go to this target section you can see that everything is is up and running and sending data to prometheus so prometheus is happy and getting all that data together now to be able to browse that data more conveniently and in a little bit more usable way grafana is a great system for monitoring dashboards and alerting and sending you emails when things bad things happen and we can access it on this on this uh hostname and the default login is admin admin and i don't want to save that password and it'll prompt you for a new password when you when you log in the first time so i'm going to set a new password and then it takes you to its dashboard its dashboard can be customized you can put different things on here that you want you can star different dashboards on it but i'm going to go to the dashboard browser under manage and click on default to show all the all the dashboards that come with this cluster monitoring project and thank you again so much to carlos edp carlos eduardo who has set this up basically saved a lot of my time in trying to do this stuff myself and he set up a custom dashboard that's that's tailor-made for k-3s clusters on raspberry pi called kubernetes cluster monitoring via prometheus if i go here it gives us a lot of great information tells us memory usage on the whole cluster cpu usage on the whole cluster tells me if all the nodes are up or if any of them are down and how long they've been running it gives me graphs of that data so i can see over time how things are running it can even monitor the pi cpu temperature so you can see a few of these pies like the master is 60 degrees celsius because it's doing a lot of work coordinating all these other pods and scheduling things on different raspberry pi's and managing our whole cluster for us and some of the other pies like down here worker 06 is only 47 degrees celsius because i guess it's not got much going on there's also different alerts set up so that if something goes bad like if the cpu usage goes too high it'll start alerting on that or if the temperature gets too hot and the pi starts throttling and there's other graphs down here too but the great thing is now i can monitor my cluster and make sure that when i'm deploying different applications on it or as i'm running applications i can make sure that i'm not running out of memory or running out of cpu or i don't have nodes that keep going up and down which can happen sometimes especially when you're running with raspberry pi's with only one gigabyte of memory on them so now that i have a monitoring system in place i can start to deploy other things on the cluster now that we have a way to monitor our cluster and see how things are running on it it's a good idea to start running some things on the cluster so i'm going to set this dashboard to update just the last 15 minutes just so i can see what's going on the cluster has been pretty stable everything's running well i don't see any spikes in usage or anything because the only thing that's running is the monitoring but i'm going to run some software on it that i'm very familiar with because i'm an open source developer who started off using this software years ago when i set up my first website and that is drupal um i'm going to install drupal on this cluster and see how it works i have a very basic installation of drupal and the way that i'm going to install it is just using some kubernetes manifest that i'm going to create using yaml and i have these up on my github gist site under under drupal.yaml a manifest in kubernetes is basically a way of describing something that you want to run on the cluster and there's a lot of different things you can put into manifest in this manifest there's a config map which is kind of like a settings file or some data that you want to put into a container that runs in kubernetes so this is just a configuration for drupal and a note here this is not this is not the most secure way to do this there's better ways to do this but for the purposes of demonstrating this in an episode today i'm just doing it this way so there's a config map that has a configuration there's a persistent volume claim this this tells tells kubernetes i want to have some space to write files and have that be persistent so even if drupal dies or the node fails i want to make sure that those files still exist so that when a new container is created those files don't just disappear so i'm going to create that and then there's a deployment and we saw some of these deployments from from the monitoring system a deployment basically says i want to have a container running i want to have a pod in kubernetes that runs and if it fails i want it to restart itself and i want to be able to scale it up so i have multiple instances of that container running and i want to be able to deploy updates to it in a rolling fashion so that it deploys a new one the new one is online and then it takes down the old one so kubernetes will manage all that for us if we just use a deployment and you'll notice that i just have replicas one here and that's because this really simple example might or might not work that well if i have multiple replicas with most applications you'll need to make some modifications and set them up a little bit more custom to make sure that they work correctly if you scale them up beyond one instance but this will work for our demonstration purposes and a deployment a lot of this stuff this is kubernetes yaml and there are plenty of memes around about kubernetes yaml usage and initially when you start using it you're going to be kind of drowning in it and it's it's it can feel a little overwhelming but after you start working with different kubernetes resources you kind of understand how it works and it's kind of like the matrix you don't see the yaml anymore but you see the the resources you see a config map and a persistent volume claim and you see a deployment and you see the things in the deployment that come from the other resources then you see services that expose the deployment to the world and then you see ingress that lets people access your application so it's a little bit overwhelming in this video i i don't have time to explain all of the different types and all the different options and parameters but at a very basic level when you get a request for drupal on this cluster once we get this running we'll request this host name and then this ingress resource will tell kubernetes hey when i get a request of this hostname i want to send it to this drupal service on port 80 and then this drupal service which is running on port 80 is going to say hey i'm going to go to the drupal app and the drupal app is running up here and it's going to say hey i want to run this request through a container that's running drupal 8.8 on apache on port 80 and then drupal say okay i'll serve that response and for that drupal also needs a database so there's another manifest for mariadb which is an open source database that you can use with drupal and wordpress and other things and it's going to have the same kind of setup it needs a persistent volume so that the data in the database can stay on the cluster even if the database container goes away or needs to be rescheduled on another node and then there's a deployment that tells it hey i want to make sure this database is running all the time and there's one replica databases can be a little bit complicated to run in kubernetes but most of the time they're not so bad if you if you just need to have one replica and don't need a cluster of database setups and then the database has a service this service is not exposed to the outside world it's only going to be accessible by drupal in the drupal namespace and that's one thing that's really nice about kubernetes is you can say i want a namespace for all my drupal stuff and i want another namespace for my monitoring stuff and i don't want the monitoring monitoring things and the drupal things to interact with each other because the monitoring system doesn't need to see the database that drupal sees there's other things that you can do for security too but namespaces are a very helpful way to organize resources by application in a kubernetes cluster so what i'm going to do is take these two manifests i'm going to take the drupal one and copy it i'm gonna cd back into my home directory clear this and i'm gonna make a directory for drupal cd into that directory that i created and i'm going to create a file called drupal.yml with nano paste that manifest save it with ctrl o control x and then i'm also going to make a mariadb manifest so copy that you know maria db dot yaml paste it save it and now i have these two manifests that i'm going to deploy into my kubernetes cluster and i'm just going to use qb cuddle apply dash f and give it the manifest file drupal.yaml it's going to create all those resources you can actually apply multiple manifest files in one instance and in this case actually i forgot to do what it says up here in the documentation i have to create a namespace for this to run in so i'm going to create that namespace first and then i'm going to apply this manifest and then i'm going to apply the mariadb db.ml and now it's going to create all these things so if i say cubie cuddle get pods in the drupal namespace it should show me that these containers are creating and let's watch that and see oops let's watch that and see how long it takes what it's doing is it's downloading the drupal docker image whatever the image is in this manifest to the raspberry pi where it's running and it's also going to download the mariadb image to the to the node where it's running and we can also check if i go out of here i can say qb cuddle get pod stash and drupal for the drupal name space and then dash o wide that gives a wider output that has more information you can see that it's scheduled drupal to worker o3 and it scheduled mariadb to worker o4 a cool thing that kubernetes is doing for us and i'm going to go ahead and go back here and see when they're ready is kubernetes can schedule pods wherever they can fit so if you have a node that has more memory and you need to have more memory for your application it will move that application to the node that has the memory that you need or if you have certain features like if you're building a bigger bigger cluster with more things than just raspberry pi's you can put a computer with a video card in it and you can do machine learning processing on that computer and tell kubernetes hey this application needs to run on the computer with the dedicated video card so there's a lot of different things you can do with kubernetes scheduling by configuring your applications the right way and you can see here that drupal is running but it's not yet ready that's because in drupal in this deployment we have these these probes that kubernetes uses to know when it's ready to actually serve up requests so until that time if i say qb cuddle get ingress dash and drupal this is going to tell me how i can access drupal through this url and i can load that here until until this says ready which now it says ready kubernetes won't route requests to that actual service that's running in the back end but now drupal's installer's coming up so i'm going to go ahead and install drupal this is going to take a little time and it will also give me a warning when i start the installation because this is a 32-bit operating system and can't store dates that are beyond the year 2038 because of limitations of 32-bit php but i'm going to ignore that error and go ahead and install drupal while drupal is installing you can even go to the grafana dashboard and you can see that some of the pods cpu and memory is going up a little bit because drupal's installer takes a bit of power and a bit of memory to be able to run so drupal's running on worker o3 and you can see that its memory consumption is a little higher as well as its cpu we can also check on the raspberry pi temperature and the temperature's gone up a little bit but we're still well under the throttling threshold so we're safe for now and now that drupal's installed i'll configure the site for the turing pi and there it is we have drupal running on the turing pi cluster drupal is a pretty good cms and a lot of people do like it but there is a cms that's a little bit more familiar to more people and has been around about the same amount of time and it's also built in php and that's called wordpress so i'm going to go to another configuration that i have for kubernetes for wordpress and if you look at it a lot of the things in this configuration are going to be almost identical to drupal it needs a persistent volume claim just like drupal it has a deployment that runs wordpress and mounts that volume claim it has a service that exposes wordpress and it has ingress that lets you access wordpress so let's set that up as well i'm going to go out of here cd back up a level clear this and i'm going to make a new directory for wordpress maker word press and i'm going to go ahead and grab these two manifests and i'm going to make a new directory called wordpress cd into wordpress and then make this file maria dv.aml paste the manifest save it close it then i'm going to get the other manifest for wordpress nano wordpress.yaml paste it save it close it and then i'll say kuby cuddle apply actually i this time i'll do it in the right order i will create the name space and then i'll apply the resources inside of that name space so i'll do that and then qb cuddle apply dash f wordpress and then i'll do the same thing for mariadb and then i'll watch as it deploys them watch qb cuddle get pods dash n word press so it looks like it's creating those containers i'm going to go ahead and get the ingress and see uh and see if i can show you what happens if you try loading this before it's ready so kubernetes is now not routing traffic to this wordpress backend until the point that it is saying that it's ready so you can see it's still not ready and so we're getting service unavailable all right so it says that everything is running i'm going to go back over to the browser and refresh this page and should be giving me the wordpress installer at this point which it is so i'll fill this out and i'll give it a weak password test at example.com and i don't want search index and search engines to index the site even though it's running locally in my house and will not be accessible over the internet at all and now i'm ready to log into it alright so i have a wordpress site running here and if i go here this is the new wordpress site that i have so now i have drupal and wordpress both running inside this cluster and if i look at all the pods so i'm going to say qb cuddle get pods and then give it all name spaces and then say dash o wide i'm going to see where these pods are running kubernetes has distributed the pods across all the different nodes so you'll see that there's there are pods running let's see where's drupal is running on worker o3 drupal's mariadb is on o4 wordpress is on o2 and its database is on o5 so kubernetes is spreading these resources over all the different raspberry pi's to make sure that all the different applications i have running have all the resources that they need to be able to run well and that way if i were to do a load test and load this wordpress site and load the drupal site over and over again those requests would be get getting routed to the different nodes and the resources would be shared and and it's a lot easier than trying to deploy things yourself especially if one of the nodes goes offline because in that case kubernetes can see oh worker 05 is down and after about five minutes or so it'll start rearranging things and putting things onto nodes that are running so wordpress and drupal are nice it's there's some good applications for running websites and things but usually when you're using raspberry pi's you want to have a little bit of fun so in my case sometimes i like to have fun by playing minecraft minecraft is a game that's been around since 2014 or 2015 i don't remember exactly when i think i started playing it around 2015. uh but minecraft lets you play and kind of it's kind of like i like to think of it as legos but in a computer and there's lots of different things you can do with it and minecraft has a server that you can run so that you can play a private game or a public game with other people that is running on a single server so you can have multiple people join in the same game and you can run this on kubernetes and if you search around for minecraft kubernetes like you might do if you wanted to run it on kubernetes you might find that there's a helm chart for it so i'll look for helm in here and helm is a popular thing for running applications on kubernetes and uh there's there's a lot of different ways that you can run things on kubernetes helm is probably one of the most popular a lot of people also just use plain manifest files like we did for drupal and wordpress and you might use other tools like customize or you might use something like ansible and template those files but helm is yet another way to deploy them and to have values that are templated in place in these things and there's lots and lots of different helm charts it just so happens that this minecraft one works pretty well for our needs because it can be made to run on raspberry pi's just like any other kubernetes cluster so to get these running you actually need to install helm first and i believe i already have it installed you can check by running helm and see if it outputs helm's help but i'm going to clear that away if you need to install helm it's pretty simple you can just search for install helm and it will give you this this docs page that says download the right version and then expand that version and then copy the helm binary out of it into user local bin so if you're using a raspberry pi like in this case using hyperios which is 32 bits you would download this linux arm version if you're using raspberry pi os 64-bit or another 64-bit os you would use linux arm 64. so i downloaded that and i copied it into the user local bin folder following these directions and now i should be able to start installing helm charts including this minecraft chart however i'm going to need to customize it a little bit for the raspberry pi so if you look at this it has a command down here first you need to make sure that you have the stable repository that this is in so if you go to helm charts you can see in the readme in the documentation it says you need to run this command to enable the stable repo so i'm going to go ahead and do that now that adds the stable repository that has the minecraft chart and tons of other charts that are in this in the same repository and then after that it wants you to deploy minecraft into your server but one thing that i want to do is make sure it's in its own namespace so this this default command doesn't have a namespace so i'm going to use a namespace i also need to specify some things i need to accept the end user license agreement which this is doing but i also need to specify some other things like i need to make the world size a little smaller to be able to run it well on a slower raspberry pi compute module and i also need to make the view distance a little shorter too that's another thing that you can do in minecraft to uh to make it a little speedier so i'm going to do those things with some values that i can specify in a values file and i have that values file i have those set up already because i set this up before this video and i'm going to create a values file i'll call it minecraft.yaml and i'm going to put these values in it so there's a few other things too that i do that i customize that i set these probes that say whether it's running or not to have a little bit more of a delay because it takes a while for minecraft to generate the world that it is going to run on the server the first time it runs and on the raspberry pi compute module it can take up to 15 minutes on your laptop it would maybe take a minute or less and and on a new raspberry pi 4 it only takes a couple minutes or up to 5 minutes but since it takes a bit longer i had to customize this i also set the version of the minecraft server to run i told it to agree to the license agreement i put it on easy difficulty so i didn't get exploded by uh whatever those little guys are that run around and explode you and i also put a message the message of the day that pops up when you join the server and i reduce the world size and view distance a little bit so i'm going to save this file and now i'm going to run the command to install this chart first i need to create the namespace for it so i'm going to say qbcuddle create namespace minecraft and then i'm going to run this helm install command which is going to install this version of the chart 122 in the minecraft namespace that i just created using all the values in the file that i just created and it's going to call this helm deployment minecraft and it's going to get it from the stable slash minecraft so the stable chart that we installed up here and the minecraft chart that's in it so i'm going to go ahead and run that and it'll take a little time if you get an error about it not being able to connect you might need to make sure that the cube config environment variable is set correctly so that helm knows how to connect to the k3s cluster and apparently i have the wrong name for this so let me choose minecraft.yml instead of yml that's the one thing that i sometimes i use dot yml sometimes yaml i don't know why i do that i should just use one or the other but you'll find that a lot of times in the kubernetes ecosystem a lot of things use yaml a lot to use yml so now everything is running and i can get this service using this command get the minecraft service so i'm going to go ahead and run that and this is going to give me the ip address over which i can access minecraft from any other computer on my network so it's this ip address and this port number 3767 so if i open up minecraft it's been a while since i've opened this on my computer hopefully it doesn't kill the computer by doing updates or something like that i'm going to open it up and try to connect to this local server and it's probably going to fail the first time because minecraft like i said takes quite a while to actually start loading so i'm going to say qb cuddle get pods and minecraft and it says that it's running but it's not yet ready that's because it's generating the world probably right now so while it's doing that i'm going to go ahead and say qb cuddle logs n minecraft and then give it this pod and this is going to tell me what it's doing right now so it's doing some stuff still it's not it's not yet set up i'm going to use dash f and that way it's going to follow the logs on that container and see what it's doing so you can see it's loading recipes it's loading advancements all these other things meanwhile it looks like minecraft itself is running right here and if i try connecting to the server at this time i'm going to try using direct connect using the ip address up here where is it there it is this ip address dot 91 port 3767 and it's going to fail because right now it's not actually responding it's it's still doing uh some of the setup stuff so i'm gonna let this sit and go back and let this sit and we'll we'll see how it we'll see how long it takes all right so that was excruciatingly long i know that it was sped up in the video that you just saw but in real life that actually took a really long time to spawn the area and that's one one thing that i will get into in the next video and i'll mention that in a bit uh is the benchmarking and performance between the the compute module 3 plus and the raspberry pi 4 and other systems like my core i9 macbook pro that i'm recording all of this on but anyway so minecraft is now running and it should be working you can see that sometimes it gets a few ticks behind in minecraft that just means that the server is a little bit overloaded and as long as that doesn't keep going and keep increasing to hundreds and hundreds of ticks it's just going to cause you know no noticeable problems in minecraft itself but now the server is running so i'm going to go in here and get that service again and check what the ip address is and i'll go to minecraft itself which is right here and i'm going to connect to this server 10.0.1 on port 3767 and that should connect us to this minecraft server that's running we'll see if that's the case there's minecraft multiplayer direct connect and it's server 10.0.1.81 for 3767 let's join and this should work we'll see if it actually does and here we are we are in minecraft you'll notice it's a little bit slow uh minecraft on a raspberry pi is not necessarily the fastest thing in the world but i if there's one thing i remember from minecraft before it was something like you're supposed to dig straight down [Music] so minecraft is really fun and it may or may not be the best idea to run minecraft on a compute module 3 plus but it does run and it can be it can be fun still i wouldn't run it with lots and lots of users connecting to it or anything on that pi but on a raspberry pi 4 it might not be so bad or of course running a minecraft server on another kubernetes or k3s cluster is going to be a better option especially if you want to play online with your friends now there's one other tool that i like to install on a raspberry pi on my home network to help me to block ads and to also set up custom dns roles and that's pihole pihol is a really well well supported application that it doesn't have to run on a raspberry pi but it was originally conceived for running on raspberry pi's because it doesn't use too many resources and it's a really helpful utility to help you save bandwidth costs and to save make things load faster because advertisements and tracking things aren't loaded on pages things like that so there's actually a chart another helm chart for pi hole and that's in let's see it's mojo 2600 has this kubernetes chart that uses helm to install pi hole and there's a again there's a couple settings in the in the chart values that have to be overridden to get it to run on a k3s running on on raspberry pi's like we have here so i'm going to do the same kind of thing that i did for minecraft i already have helm running but this is actually using a different repo so i'm going to use helm repo ad to add this other repo and then i'm going to run helm repo update which makes sure it pulls in all the latest information from that repository and then this of course is installing this into the default namespace and i don't want to do that i also want to supply my own values so i'm going to grab the values that i have for this to run it on my network with the k3s so i'm going to say nano and i'll call this pihole.yaml and these values are going to set a few different options that make it work a little bit better on the turing pi cluster and again there are some settings in here like i'm setting the admin password to admin that's not a very secure default and if you're using this in the real world you should probably set that to something else and make sure that you have things set up in as secure a manner as possible but i'm going to set this and i'm going to set these options for the load balancer ip to the load balancer server which i'm using is 10.0.100.99 so i'm going to set those up and then deploy this chart save this and exit and then i'm going to create a namespace for pi hole because i don't want this to just go in the default namespace and once that's created i can deploy the chart into that namespace and i'm going to use version 176 which is the latest version of this chart in the namespace pihole using the values from this this file i'm going to call it pi hole and it's coming from this chart repository and the chart's name is pihole so the same thing here i'm going to say qb cuddle get pods in the pi hole namespace and let's see if it starts running so it's pending so it's it's going to be running these soon so let's go ahead and watch this and see how long it takes all right so now pi hole is running and i've actually added to my etsy host file so i said nano and on my mac i'm doing this i said nano slash etsy help slash host and then i added into that file a line that says 10.0.10 and then pi.hole so that lets my mac know that if i try to access this url it should go to that raspberry pi where this is running and so if i go pi.hole i believe i should see the admin console come up here you can see that it's it shows the raspberry pi statistics so it's funny pi hole shows the temperature of the raspberry pi and if i go over here i can correlate that by the fact that this i believe worker 1 was the one that was setting up pi hole if i go down to the temperature i can see that worker 01 is currently at 60.15 celsius which is uh pretty close to 59.6 so piho is monitoring it and grafana is monitoring it through prometheus and we can see that pihul is running here and i can also log in as admin using of course the password admin which is an insecure default so make sure you change that and i can see all the activity going on now to use this i can access the the load balancer that k3s set up on all the different pies so if i say qb cuddle get service dash and pi hole i can see that there's a load balancer set up for tcp and udp the tcp one's not actually running right now because there's it it interferes with other things that are running on the servers uh ports 80 and 443 so that's going to get stuck in pending state but the udp load balancer is running on 10.0.1.99 and what i can do is on my mac i can start using my mac go through the dns on this pihole server that's running in kubernetes and to do that i just go to system preferences and then network and then for whatever interface that you're using i have an ethernet connection and a wi-fi connection i can say advanced and then change the dns server so right now it's just using my router the default but i'm going to add this one which is 10.0.1.99 and say ok and apply and now my mac is going to start using dns through here and you can see it already is getting queries so my mac is sending web requests now through the pi hole and the pi hole will start blocking things that it identifies as advertisements so if i go in my web browser to let's say cnn then it's going to start loading the page and there might be some things that that pihl starts blocking and you can see it's blocking 24 queries and you can go in and look at the log and see what things it's blocking but a lot of news websites if you don't use it with pi hole and you notice that they're slow and then you do use it with pi holes sometimes websites that take 20 or 30 seconds to load might take one or two seconds because all the advertisements and tracking stuff that happens so pihul is really awesome for blocking advertising and tracking content that makes the web slower and uh makes lots and lots of annoying and and pop-up and all the crazy kind of ads that happen uh show up on your computer so that is pihole and it's pretty awesome it also lets you do things like set up a local dns record so you can refer to your raspberry pi's by a dns name instead of by an ip address and all of those kind of nice things you can find out more about pi hole at pihole.net but that's one of many different things that you can run on a cluster a kubernetes cluster like this one we have monitoring setup that lets us see all of the different nodes that are running and see their cpu and memory usage you can inspect individual pod usage all those kind of things so we have monitoring set up with prometheus grafana and alert manager we have a drupal website running we have a wordpress website running we have minecraft running so that we can connect to it from our minecraft game and have a shared server we have pi hole and there's still capacity left on this cluster so we could install more and more things until we fill it up and if we do fill it up we could add more raspberry pi's the turing pi board itself only has seven slots but you could actually add other raspberry pi's to the cluster all you have to do is use k3s to get a token from the master and then join a new node to it you can add more more raspberry pi's however you want to using k3s or doing it by hand there's just a lot that you can do with kubernetes and running lots of different applications on it and the cool thing about this is all of these applications are scheduled by kubernetes on nodes that have space so you don't have to spend time thinking about where you want to run drupal and where you want to run wordpress and where you want to run minecraft kubernetes takes care of all that for you and if i were to unplug one of these raspberry pi's which is inadvisable because it could corrupt the memory on it if i were to do that kubernetes would after a certain timeout period start taking all the things that were running on that and start moving them to other nodes as much as it can until of course it runs out of memory or cpu so those were just a few of the things that you can run on a cluster but throughout my time working on these different things i learned a lot about the limitations of the compute module 3 plus so i'm really looking forward to the compute module 4 which is coming out later this year it should be a lot faster and hopefully it will have more options available with more ram and it would be really crazy for me to do all this work for these videos and not share it with you time and again you've been very generous in supporting me like on patreon and github sponsors and if you don't already support my work there please consider doing so so to give back i've been secretly maintaining a repository for this project it's still kind of a work in progress but it will quickly and automatically bootstrap all the apps that i just showed in this video on your own cluster using ansible it doesn't even have to be a turing pi cluster and it doesn't even need to use raspberry pi's you can run it on any kind of arm cluster and today i'm releasing that project as open source code on github so you can use it as you like the link is in the description below in fact i ran this automation on my own pi drama cluster that has four raspberry pi 4s with two gigabytes of ram each and then i did a bunch of benchmarks on the two clusters to see how they perform and the results were surprising in many cases the pi drama cluster ran things twice as fast as the turring pi cluster there are a number of reasons for this but the main one is that the pi 4 is overall a much better and faster computer than the compute module 3 plus but how much better and does the new 64-bit raspberry pi os change anything and what are my thoughts on the turning pi overall after having done all these videos well to find out subscribe to the channel in the next video i'm going to give a more thorough review of the turing pie itself and please comment below if you have any questions about the turing pi k3s or anything about clustering i will do a follow-up video soon with questions and answers until next time i'm jeff gearling

Channel: Jeff Geerling

Views: 236,720

Keywords: turing pi, open source, raspberry pi, cluster, k3s, kubernetes, k8s, computer, project, minecraft, pi-hole, drupal, wordpress, helm, deploy

Id: IafVCHkJbtI

Length: 54min 31sec (3271 seconds)

Published: Fri Jun 05 2020

Reddit Comments

[u/hard_cornbread]

I've been doing Kubernetes consulting for three years and still struggle to explain this stuff as well as you do

[u/MaxHedrome]

I’m just glad geerling has started to come out of his shell enough to get the exposure and credit he deserves for his insane robot work ethic

[u/thefrc]

This is a spectacular teaching aid. Thank you so much.

[u/Vlarmitage]

Finally

[u/else-]

Didn’t know about that cluster board. Interesting, thanks for sharing!