Kubernetes 101 - Episode 4 - Real-world Apps

hello everybody welcome to episode 4 of kubernetes 101. today we're going to deploy our first real world application into kubernetes and we're going to find out how a traditional server architecture translates into kubernetes container architecture and i'm going to make sure i'm not muted because that happens a lot and i'm not so that's good hopefully everybody can hear me fine please as with all these live streams feel free to put where you're from who you are and uh whatever else in here but um do know that if you start talking about politics or things like that then we do not care for that in this discussion last week we had a problem somebody was spamming a product about android or i don't know what but yeah that's that's silly so anyway i'm glad to see everybody uh glad that the live chat has uh plenty of great um great discussion that goes on and a lot of times i don't have the time to watch the the live chat the whole time that i'm talking because it's kind of hard for my brain to do two things at once and today you might be hearing some kids stomping around upstairs that's just because they are currently doing some sort of game where they run around the house quite a bit and we have wood floors and it's like a foot above my head so anyways um i'm going to switch over to my screen share here today we're going to talk about deploying a real world application that is drupal now this this episode gets a little bit deeper into drupal itself than maybe some people care to to learn about but the point of this is that this is a real thing that a lot of people need to do in kubernetes you need to take something that's more practical than just one little micro service and get it running inside of kubernetes so you have to worry about things like databases and things like files that get stored somewhere all the things that make it a little harder to work in a cloud-native environment if you don't know what you're doing and these are also some of the things that the reason why a lot of times somebody will switch to kubernetes and then they pay for super expensive consultants because they didn't realize that there are still hard problems to solve in kubernetes if you're taking applications that actually do anything useful in the world so you know if you're just going to disable hello world go applications it's pretty easy to scale this up infinitely in kubernetes but if you want to scale something that has a lot of persistence and a lot of legacy then that's going to be a little bit harder so and the reason i chose drupal for this there's there's a few reasons one is i am historically a drupal developer that's how i got my start in programming through uh learning some php to do some scripting and things and then i realized when i was trying to build a podcast this was in the year 2003 or four basically when podcasting was a new thing with the ipod i realized that i needed to do something more structured to make it easier to do the podcasting and media upload and we even did some video work and things online and so i discovered that drupal had a lot of great functionality and plugins for that so i got into drupal for that then i did news i did non-profits i did lots of different huge websites for drupal probably a number of websites that you have visited in the past couple years i have touched something on them and so i have a lot of history with drupal drupal is also an application that can be very difficult to scale especially if you don't architect your drupal website well and it's a it's a fairly common architecture for a website uh the the lamp stack that drupal is built on is used for a lot of different sites and a similar kind of stack is used for a lot of dynamic languages like like php and practically speaking i have my website jeffgearling.com which you could visit right now and see it's running on drupal as well and eventually my goal right now it's running on a single vm somewhere my goal is to eventually get that into one of my kubernetes clusters so kind of a selfish reason i've been doing a lot of stuff in kubernetes with drupal is to to prepare for making it super duper easy for me to get that into a kubernetes cluster so i don't have to do a lot of the things that i'm going to talk about today and figure out a lot of the hard problems if i can solve the hard problems on smaller sites then i can easily solve those problems on larger sites as well so anyway i have a a video that i'm going to show you that was pre-recorded unfortunately i didn't feel like it would be easy for my computer to handle all the things that i'm going to do while i'm doing the stream so i have this video and i'm going to play it back and it's going to show you how i currently the basics of how i currently set up a server and how traditionally someone would set up a server with the lamp stack i've deployed drupal in so many ways in so many places i used to do a ton of drupal development and design lots and lots of sites nowadays i only manage a few i manage my jeffgearling.com website and i manage a few other sites for other people so i i've been through many generations of how drupal is deployed but still most people probably the majority of people that deploy drupal and it's the same thing with a lot of other applications that are like it cms is like wordpress applications built on laravel a lot of these things are built on the traditional lamp stack and lamp stands for linux apache mysql and php now parts of that stack are interchangeable so we say mysql but it could be mariadb it could be postgres it could be some other database that's compatible with whatever system you're building and php can actually be deployed a few different ways nowadays and apache might be swapped out for something like nginx but in general it's the lamp stack and traditionally the way that this was set up is you would have a virtual machine running in the cloud or you would have a server running somewhere or you can even install everything on your local computer and what i'm going to do is i have a example in the kubernetes 101 repository that sets up an ubuntu 2004 machine for you to do this testing and if you want to follow this example yourself but i wanted to show this process because this is where we're coming from and moving into a cloud native uh deployment method for an application like drupal and no matter what the case if you're working in traditional servers if you're working with bare metal servers if you're working with kubernetes if you're working with docker you really have to know the underlying architecture and how things work together and the applications themselves like in this case my sequel and apache and php and the way drupal runs on them you have to know all that stuff before you can start diving right into throwing stuff into kubernetes if you throw it into kubernetes and things start having problems you're going to get so many layers deep in kubernetes and it might just be some simple configuration problem that you're running into with how mysql connects to drupal or something like that so i'm going to run through this process fairly quickly we'll see how long it takes on on my computer while it's doing this recording but i have this traditional lamp setup example in here and you can run vagrant up and this will bring up a virtual machine using virtualbox now this assumes that you have vagrant and virtualbox installed on your computer this could be work built to work with other systems too but in my case this is what i'm using mostly for familiarity's sake and the fact that it works the same across windows mac and linux although for if you're using linux there might be some other virtualization tools you might want to use that are more efficient and not encumbered by the fact that they're owned by oracle but anyway i'll wait for this to install or to set up the the vm all right now it looks like the vm is up and running so i'm going to log into the vm with vagrant ssh that just sshs me straight into the machine so i can run some commands on it i'm going to run sudo apt update just to make sure that all my caches are up to date and then i'm going to install my sql or in this case on on ubuntu and the latest versions of w and it's mariadb is kind of the drop in replacement so sudo apt install maria db server and mariadb client and once that's done i'm going to run sudo mysql secure installation to finish the setup process it's called my sql secure installation but that's part of the legacy of mariadb actually using mysql's naming and things like that for drop in compatibility so i'm going to run this and i'm going to put in a password for root extremely secure password um so yes and i'm going to go ahead and follow all the defaults for the rest of the setup process all right so now i have my sql running i'm going to log in now as as root so sudo sequel dash you root that'll get me logged in and i'm going to create a database for drupal so i'm going to grab the command out of my readme file for this create database drupal and then i'm going to create a user for drupal that has permissions on that database identified by my password you if you're going to do this on your own you should probably use a more secure password and then since i added that user i'm going to flush privileges in my sql and go ahead and quit out of the cli next up i'm going to install php so i'm going to again copy the command just because there's a lot of php packages that drupal relies on that need to be in your install otherwise drupal will not load up correctly so i'm going to grab that command and it's php cli fpm json common my sql zip gd intel internationalization mb string curl xml pair tidy soap bc math and xml rpc it's going to go ahead and install all those packages and once that's done it's time to install apache and in my case just to make things simple i'm going to install php with mod i'm going to install apache using mod php to connect php to apache this is the older traditional way to do it although if you're going to build a more modern hosting setup you might want to use fpm instead but that is something completely unrelated to kubernetes 101 because now we're getting into the nitty-gritty of how to deploy php in infrastructure and there's many ways to do that just like there is for any programming language that uses dynamic scripts like php so i'm going to say sudo apt install dash y apache 2 lib apache 2 mod php that'll get me what i need for that and it's there and then i need to configure a couple things for php so sudo nano etsy slash php 7.4 is the version that's installed apache2 php.ini and in here i need to edit a couple things so i'm going to find memory limit technically drupal could run on like 256 but i'm going to give it 512 because that's always a little safer i'm also going to change the date time zone setting um just because that makes it uh easier for drupal to use the right times and things america slash chicago even though i live in st louis chicago is apparently a much larger city that's in the same time zone and then i'm going to uh edit the i'm going to create a file that tells apache about my drupal site and you know you might be thinking at this point why are you doing all this again it's because you really need to understand the things that are going on behind the scenes and one of the things that happens is whether using apache or nginx or something like that you need to give it the right configurations to be able to know where your code base is for your website where to serve files how to serve them and all those things so i'm going to edit i'm going to create a new drupal configuration which contains a virtual host for drupal so uh sudo nano etsy apache 2 sites and they uh available i can't type today uh drupal.conf and inside here i'm gonna paste in my virtual host block and i'm not gonna deep dive into what all this is but basically it's giving us a name for the server and where that where that document root is going to be located and we haven't set this up yet but i will soon and i'm going to save this file but i'm not going to restart apache yet after you add this file into apaches configuration then you have to restart apache to pick it up but since it doesn't exist apache will fail if i try doing that so the next step is i'm going to install composer by downloading the installer from the web and running php composer setup and that downloads a composer.far file which i can use but i'm going to move that into a directory in my path so i can just run composer command so move composer dot far into slash user slash local slash bin slash composer and now i can run composer and use composer which is php's kind of package and repository project management tool and then i can use composer to create a drupal project so first i'm going to set the permissions on the www directory so that apache and intern composer and drupal can work with it so i'm going to say sudo chon dash r www data and that again var www and then i'm going to create a drupal code base and the demonstration i'm going to show you in kubernetes today actually skips over a lot of this stuff because it's all kind of magically wired up together for you but again the reason i'm showing this is because i want to show you that all of these steps are important in making sure that drupal is running securely inside of an environment on a vm and in kubernetes some of these things we worry about them in different ways some of these things we worry about in the same way in this case i want to make sure that the apache web user has the permissions to be able to work with this this file system and there are some different concerns that you might have running a vm like this in a cloud environment versus running it in a docker container especially if your container is isolated in a kubernetes cluster or isolated via podman and not running as root on a system so anyways i've created that and then i'm going to switch to the www data user so sudo su-l www data s spin bash that lets me log in as this user to run this next command which is going to create my drupal site so composer create project drupal recommended project far www html slash drupal so that's going to download all of the tools that drupal needs into its codebase directory and this is basically creating me a new drupal site and you can see it's going to download drupal 9.1 which is the latest version just released a week ago i think and it's downloading all of drupal's dependencies into that folder so it gives you all this guide and stuff here but basically what that did is if i uh ls that directory var www.html drupal that installs drupal into this directory using composer so that i can use composer which is a php dependency management tool to manage my drupal code base and how everything works together so i did that i'm going to exit out of that apache user's account now and finish setting things up so i'm going to use sudo and use apache 2s tools to enable certain modules and sites and things and then restart apache so a2 and mod rewrite and then sudo a2 and site drupal.conf so that enables that drupal configuration i added earlier sudo a2 disk site i'm going to disable the default site that comes with apache default.conf and finally sudo system control and notice it's system control not system cuddle anyway so if i say cubic can oh qb cuddle cube control whatever we'll keep coming back to that i'm sure restart apache 2 and now at this point if i go into my web browser and i go to the ip address 192.168.80.80 it should take me to the drupal site and it's not installed or anything yet but it's you can see that everything is running now so if i save this and continue i'll just choose the standard installation profile i guess i can put in the database credentials so we created a drupal drupal database with the username drupal and the password my password highly secure if we're running the database on a different server you could change that here but we're running everything in one stack and one vm the lamp stack and you can see that drupal is installing and after it finishes we should see that drupal is here and again i have to reiterate you're probably never going to do that especially if you're watching this and not if you're not a drupal user i know some of the people watching this live stream are drupal users like myself but but it's to show you that there is a complexity to running these things and even if you try to use some magical layers on top of things to manage your applications for you you have to understand how those things work together why did we change the permissions on the www data directory how does that help things because some of these things you're going to run into issues in kubernetes and it's going to misdirect you sometimes if you don't know how how drupal interacts with the database and how drupal interacts with the file system and how php and apache work together so sometimes i think people get into kubernetes and think that it's just kind of some magical varnish you put on top of infrastructure and you don't have to worry about your applications you don't have to worry about connections between them and networking and things in my opinion you have to know all that stuff even better if you want to do kubernetes well i would not be able to be productive with kubernetes if i didn't understand all those underlying layers well and so that's one thing that i caution you know if a company is like oh we can just throw kubernetes on top of our stuff and it's easy developers can deploy their applications and it all works great it's not that way you still have to understand all those things so anyways i'm gonna hand things back over to me again uh it's always a fun thing to be talking to myself so hello jeff in the future yes hello indeed um thanks so much to jay rogers and uh scott for uh the super chat donations um one interesting thing yesterday was github sponsors and sorry for going on a little tangent but yesterday was github universe and um one one thing that that reminded me of is that this year i have had a number of people sponsor my open source work on github which is basically the reason why i was able to get into this youtube channel more get my kubernetes 101 series and all that so thank you so much and if you do want to uh sponsor my work and and help to make these things happen a little more often please consider that i think there's links in the description if there's not i can add them for my github sponsors and patreon and things like that but that process that i talked about it might seem antiquated and it is antiquated it some people actually do still administer servers like that just like some people still use ftp to upload their web files and not just secure ftp but there are still people using ftp and it is an antiquated process but i haven't really done that for many years now probably 2013 2014 was the last time i remember sitting there with a run book and typing commands like that to set up a server nowadays we use tools like configuration management like ansible and somebody was mentioning the ansible this is a rare artifact i don't even remember where i got this one but i remember getting it and thinking i have to take very good care of this because there's not many like it but it's like a little squeezy toy but it's a plan where it's ant's a bowl and this is a bowl anyway so ansible itself if you if you it's just like with kubernetes if you dove into something like i i created a tool called drupal vm that lets you build basically all the stuff you saw there but way more configurable and all you have to do is change a few settings and you can have your finely tuned drupal server running locally inside a virtual machine just like i did with all those manual steps but it only takes a couple minutes to set it up and you you can either use the defaults or change a couple settings but ansible itself is kind of magic if you don't know what everything is doing underneath and i think a lot of people getting started with kubernetes see kubernetes as a layer of magic as well and and a lot of times they might start using tools because vendors are sometimes the the biggest perpetrator of this issue of saying well when you switch to kubernetes you don't have to worry about this and that and storage and all these different things but really they're trying to mask some complex things that you still have to worry about in some different ways and so i i also pre-recorded this a little bit where i'm going to tell talk about deploying drupal with helm which i think helm is home is great for a lot of use cases and a lot of different things but i think a lot of people also get into kubernetes and start using helm without understanding what it's doing and how it works and things so i wanted to show how to deploy drupal with helm and show kind of a little bit of that magic and how it can mask some some of the complexity that we just talked about on the topic of magic and kubernetes i'm going to talk a little bit about helm and we're going to install drupal using home now helm on its home page calls itself the package manager for kubernetes and i would like to say that i kind of disagree with that statement about what helm is or at least what helm should be because i see a lot of people pick up something like helm which is a great tool and a great way to create and share and distribute and update your applications in kubernetes it's one of many different options and it's it's one of the most supported and most widely known options but a lot of people pick it up and right at the beginning of their kubernetes journey they they hop into helm find a chart that they like and just start using that to deploy their applications but they never really understand what the chart is doing and so many of the helm charts especially the most popular ones for some applications that are harder to deploy do so many things underneath that you might never even understand how it all works and when things break you're going to have a lot of trouble figuring that out so i typically like to approach things from the other end of the spectrum and we'll get to that in a little bit but i want to show you how you could use helm to deploy drupal and one thing that i do like a lot about home is the fact that there are a lot of charts out there that will get something deployed in your cluster quickly and especially when you're doing development testing or trying to get a reference for how to do something in kubernetes there's a lot of helm charts out there that can help you with that so that's that's what i typically use home for is kind of kicking myself off on a new project trying to figure out how other people are doing it helm often has a great reference for how to do it and i can even pull the definitions that home generates out of my cluster to help me get started on my own definitions so this is home.sih the the home page for home has some good documentation you can get links to how to get help with home things like that but the place that i usually go to find things is the artifact hub and it's artifacthub.io and it searches through all the helm charts that are publicly available and so i i can search for something like drupal and there's actually enough well it's unofficial but it's it's a pretty popular helm chart for drupal from an organization called bitnami and to add a home chart it's pretty simple the first thing that you need to do is make sure that you have helm installed on your computer so i'm going to say brew install helm because i use brew to manage most things on my computer it's already on my computer spoilers but you can also download the binary just like many other tools in kubernetes you just download it to your computer put it in your path and you have it there or there's other ways to install it too if you go to the helm documentation but once you have helm there's only two things that you have to do you have to add add a repository so in this case bitnami maintains a repository that has the definition of this drupal helm chart and so we can do that with this command helm repo add bitnami so i'll put that in there and it looks like i already actually did this so it's on my computer but if it wasn't then it would download it to your computer and then you can create a release and in helm a release is basically like an instance of your application so i'm going to go ahead and install install an instance of drupal and i'm not going to call it my release i'm just going to call it my site so my site and i should say that i have a kubernetes cluster running in the background just with mini cube i ran mini cube start before this and i so i have my mini cube cluster running so when i do this it's going to use helm to grab that chart and put in all the default configuration that that chart has and put it into my kubernetes cluster now there's a ton more options for helm i'm not going to get a deep dive into helm today that's not what this episode is about but at the end of it it spits out some information and the first thing that we want to do is we want to be able to get to this drupal thing so it deployed stuff for me and that's where the magic is here it defines things like how to get a database running and how to get drupal and apache and php up and running it gets me a drupal code base for free all this kind of stuff so that's nice but now i need to be able to access the drupal site in kubernetes so i'm going to use the command up here uh qbcuddlegetservice and it's in the default namespace that's where we're still working right now and it's my site drupal and that's the service that i'm trying to get and right now it says that the external ip is pending and it's a load balancer so one quick thing to note and this is one one area where some people start running into issues earlier in their kubernetes career because they go into a cloud infrastructure provider like amazon and they start deploying these things and they might have 25 drupal instances and then they realize that the default is to use a load balancer and you need to know these things because load balancers on amazon use i think albs or elbs by default whatever the case they cost money and if you deploy 25 sites and those sites all costs i don't remember what the price is right now but you know let's say 20 bucks a month 20 times 25 that's starting to be a lot of money just for the fact that it's using load balancers there are other ways to deploy things and have them accessible besides load balancers that can save you a lot of money and make things easier for you but that's kind of an aside that's one of the reasons why i'm showing these different ways of deploying before we get into deploying it the way that i like to start things off there's no external ip and we can watch this forever there's not going to be one right now because i'm using mini cube and minicube doesn't have an external load balancer but minicube can actually emulate an external load balancer by kind of building a tunnel in the background so i'm going to create another terminal session because this is a foreground operation i could background it but i'm just going to open a new terminal session and run the command mini cube tunnel and when i run this it's going to set up a tunnel into mini cube for my computer that can be used for load balancing and after i do that in a few seconds let me switch back to this screen you can see oh it's asking for my pseudo password so i'll put that in and now it's getting an external ip here so now i can take this external ip address and bring it over into my browser and i'll make a new tab and visit it and i can see the drupal site so this is a quick way to get things running like you know this is drupal and you saw how complex it was to install it on a vm and you might be thinking at this point wow this is great i should just get into kubernetes and start using helm and deploy things this way but there are some concerns that i have one is this fact that it uses a load balancer by default another is the fact that there's a lot of underlying configuration that went on that you saw that i had to do manually before so how how can we just skip all that we want to understand what's going on behind the scenes before we start doing things like using public helm charts or even building our own home charts because there's a lot of complexity that's masked there so uh one one quick note is that you can't override all these things like i can change the load balancer if i go back over to the documentation for this chart and go to the values schema it shows you all these different options that you can change and you can change the service type which is default as load balancer and you can change it to a node port like we used in the previous episode or cluster ip and you can change many many different options in here but again if you don't know what these things do it's a good idea to start from the foundation and learn what these things do and then you can start using all these tools on top of kubernetes to manage them for you anyways i'm going to hand it back over to myself and we can talk about another way of deploying drupal that lets you understand the basics and how this chart actually works thank you me and thank you also to corrupt corey and scott dexter for the sponsorship and and chat uh the super chat um speaking before we get off the topic of magic and infrastructure i wanted to say that not all kinds of magic are bad magic and that reminds me of this series sponsor amazi.io they are a company that gives you the right type of magic in the right places they make using kubernetes feel like magic because of the fact that they manage your infrastructure for you and make your kubernetes clusters work well so that you can focus on just running your applications and as a side note for today they are drupal specialists and the reason that i um actually knew about them and know a lot of the people that work for amazi is because in my opinion they are amazing people who have uh have been part of the drupal community for years and um they're probably one of the best companies on the planet that knows drupal the deepest and and can run it really well in infrastructure so it's a double win if you're using drupal and you want to run it on kubernetes anyways you can check them out at amazi.io so i i want to talk about deploying drupal into a multi-node kubernetes cluster and i'm going to use a cluster that i'm running on linode so i'm going to switch back over to my screen here and you can see i have a cluster running for episode 4. it has three servers three servers each one has what through four gigs of ram or something like that 12 divided by three is four yeah and uh two cpu cores so it this is more of a real world it's a very small small cluster but this is a cluster that that gives us the opportunity to see what happens when we deploy drupal onto a real kubernetes cluster with multiple nodes and see some of the issues that we might start running into and sorry about that um the uh i i have an example in the project for this website so if if you don't know about it if you go to cube101.jeffgeorland.com i have links to all the episodes a link to the youtube channel a link to my book which is still 4.99 right now until the end of the series it's on sale this book a lot of the examples from this this series actually come out of the book even though it's ansible for kubernetes it covers a lot of generic kubernetes things because of the fact that ansible is just automating those things that you do in kubernetes that we're covering in this course but there's a link over here to the kubernetes 101 project which has examples from every episode and in here i have the example that i'm going to deploy to a cluster and we're going to show kind of behind behind the scenes of that home chart what a lot of things are doing and talk about the way that the the actual deployment is structured so there's actually a readme here that guides you through it if you want to do this after the episode the readme is probably going to have more detail than i give you just because on the fly it's hard for me to remember all of the talking points that i want to hit but i have these two manifests and when you're working in kubernetes you're going to deal with home charts which have templates which look similar to this you're going to deal with raw manifests like this you're going to be deploying operators to your cluster and things everything you deploy is going to have a definition of what the object is and 99 of the time we're going to be working in yaml so you're going to want to get yourself familiar with emel and if you have an editor like i'm using sublime text 3 but if you're using vs code or whatever editor you're using you're going to want to make sure you have yaml syntax highlighting and ideally you could integrate a linter that would also check things like making sure you're using spaces instead of tabs it's a little hard to see that on your screen probably but i have a plug-in that that lets me see if there's trailing space so i can make sure that's gone it shows me the the characters uh whether they're spaces or tabs and i can set the defaults uh for tab width to use spaces so that ammo is happy with me when i'm editing it and adding it um anyway so we're going to talk about these things as i said in the first part of this episode when you deploy drupal you're going to need the lamp stack basically and in this case we're substituting linux it's still going to be linux underneath everything unless you're going to run this in a windows kubernetes cluster but we're going to need linux on the base which is going to be our operating system running the containers and things the container is going to set things up for us but we're going to have apache and php just like we did in that first example running together in one container and then we're going to have a database and another container but to get those things to run in kubernetes you need a few more resources so for a database but what what you don't want to have happen is you deploy your database and it's running well and then you know your server restarts or something and then all sudden your database goes away because you didn't have any persistence you didn't have a storage volume where the database was stored because with containers containers are ephemeral that means that they they will go out of existence and all the data inside of them except for what's in the container image will go away so anything that we want to store and have be persistent we have to have stored somewhere and in kubernetes most of the time the way that you would store things is using a persistent volume claim or pvc and different kubernetes clusters will have different ways of setting up storage and some of them give you multiple options other ones only might give you one or two out of the box and you might have to set up different storage classes and things but we'll talk about that later for now you just need to know that if you need storage you need to claim storage from your cluster and most clusters like mini cube and linode and amazon will give you a default way of getting storage like in amazon it would be an ebs volume in lino it would be a linode block device i think or linode volume uh whatever these things are block storage um and in minicube it would be a local volume on the mini cube instance so the first thing that we need is a persistent volume claim that's going to tell us that's going to be able to be used with your database so that the database can write things to a persistent place that will not go away even if your container dies or your cluster is restarted or something like that and owing i do not stream on twitch unfortunately youtube is has been a lot easier for me to work with lately so anyway so with your persistent volume claim or pvc you want to give it a request for how much storage you need typically you want to go higher than lower i'm just doing this because this is a demo i don't need i don't even need a gig but if you're it's it's typically better to give yourself a little more room and padding because some storage providers make it more difficult to expand or move to a new persistent volume or something like that that's just a tip from years of usage there are a lot of storage providers that let you have more flexible storage uh options but that's just something that i i would always give about two to four times as much as i can imagine needing for for my persistent volumes that i use the next thing is the deployment of my sql now this right now this particular resource definition this this manifest for kubernetes is not what i would call a production grade one this is more of a demonstration one and that's how i like when i'm learning something like kubernetes i don't want to dive into show me the production grade fully secured 100 ready thing i want to see how something works first and then i'm going to start pulling things out so you might notice if you know much about kubernetes you might notice that there's no secret here like you need a secret for a database and you need this and that i'm not going to do that yet so don't you know i know some people your alarm bells are probably going off but don't worry about that we'll get to it later but we need storage and we need the definition of how to deploy the database into the cluster and we're going to use a deployment for this i'm going to talk later again alarm bells might be going off there are people watching this that should really be in like a kubernetes 502 or something you know don't worry we're going to talk about replica set soon but for now we're talking about deploying a database into the cluster we're going to have one replica and i'm going to tell you why in a little bit and uh this is this is the basic way to define a deployment in a cluster so all these different things once you start doing enough kubernetes you'll you'll understand uh what all these fields mean and things for now i would say starting out copy and paste a a definition and then tweak it but you'll you'll start to understand that the metadata defines like the name and namespace usually but there's some other things you can find in there match labels is used for kind of labeling a thing inside of kubernetes and you can use this in different powerful and flexible ways for now i'm just doing a simple a simple one for the app name equals mariadb technically you don't even need to do some of these things but it's it's a general best practice to do a lot of these things the most important part of deployments and the part that you'll be tweaking the most is the spec under spec template spec because this is how you'll define what runs for this deployment and you can actually have more i mentioned this a few episodes ago you can have more than one container in a pod so you could have i could do all of drupal's deployment with a database and drupal inside of one pod with two containers if i wanted to do that i don't want to do that because i want to manage the database separately from drupal but there's many different ways you can deploy things into kubernetes including that and i'm going to talk about a little special thing that you can do inside of here as well with the drupal deployment but in here this is where the meat of things is if you if you know much about docker you'll this is very similar this this looks in some ways a lot like a docker compose file now with resource limits and things this would be docker compose version two versus three but a lot of these things are similar because you define things like environment variables you define the ports that the container is going to use you to find the image the name for the container and volumes that it might be mounting into the container so all these different things help tell kubernetes when i run this container i want to run it using this docker image and in this case it'll come from docker hub if you're coming from a private repository or somewhere else you might need to put the server name slash namespace and all that kind of stuff that we talked about i think last episode or two episodes ago and um and and like i said this putting your my sequel password in line like this and using a password that's really simple to to hack like drupal is probably not a great idea um and we'll talk about better ways to do that soon but this is how and the way that i got all these variables was i looked in the mariadb docker image documentation and it said define these variables if you want to have a database set up for you when you run this container so we set those up in the environment and those will tell the container the mariadb container to set up a database when it starts the first time and then the most important thing for our database is to make sure that the var lib my sql and this again is documented in the mariadb container documentation i'm going to make sure that that directory is mounted as a persistent volume so that when the container loads the first time this will be loading from a block storage device or some other storage folder that's persistent and outside of the container's life cycle and it will be mounted into this path from this volume claim so again the syntax here you'll you'll start picking it up the more you do this but i started out by i would take a resource definition i knew worked copy it start tweaking things start understanding the structure of it that way some people's brains work a little differently you might want to read the documentation for things more i i tend to work better by taking something that works modifying it seeing how that changes things and then learning from there so anyway this this is the most important thing for this database we want to make sure that the the volume is used that we set up up here this persistent volume claim and it is mounted into the container in this path where mysql or in this case mariadb will be looking for that database all the files that it stores all the data for drupal in and then the last little thing here is resource limits this is something you actually don't have to define these when you're beginning with kubernetes you might not even care too much about them just because uh it's extra work to deal with them and if you if you try deploying like two copies of drupal into a cluster that has one cpu this will not work because it's trying to use half of a cpu 500 m is 500 milli or something it's half of one and so i i do this for almost everything i run in production though just because you don't want to be surprised when one of your pods starts going insane and then uh you know it takes up all the cpu on one of your servers because these things are sharing the same server for multiple pods especially if you are running lots of things in your cluster so it's a good idea to have to keep tabs on your limits and things and this is something that we might go into a little more in detail later but for now just know that you can cap resource limits for cpu memory and even some other things too but we won't get into that in this course finally we have a deployment we have mariadb running in our cluster after we deploy this we haven't done that yet and we need to be able to get to that from drupal so we need to be able to tell kubernetes set up a service and we've dealt with services before the service is just going to be kind of an internal kubernetes load balancing thing that lets you send requests to a dns name or a cluster ip address that kubernetes is going to set up for us inside the cluster and that service is going to route port 3306 on it into this container on port 3306 which we expose up here so so far we have mariadb set up and it will be able to the other thing about services that's important is to set a selector that matches the the label up here on here so that when the service is set up it routes the request to the right container so anyways this is what that service does and technically the service could work with multiple uh mariadb containers running at the same time if you set replicas to two or three but i'm going to talk about why that can be dangerous here in a minute so anyway we have this and uh one of the things you might have noticed in all these definitions is something that might be new to you um we br i briefly mentioned it last episode but namespace namespaces are something that you're going to want to make sure you always are using in kubernetes because it's a way to separate out different things that are running in your cluster into their own little isolated area so if i have a namespace drupal here which i'm going to use also in the drupal definitions that namespace allows me to set up this mysql server in the drupal namespace and it will be accessible inside the drupal namespace but it will not be accessible through the service outside of the drupal name space and there's other things that you can set up per name space for instance we talked about pull secrets and that was the first time i mentioned name spaces pull secrets are specific to a particular namespace so that lets you do things like run different workloads in different namespaces and not have them necessarily be able to tie into each other there are ways to kind of punch holes in things but by default it's it's a pretty easy way to isolate resources from each other inside a kubernetes cluster and we're not going to deep dive into security in kubernetes in this course but that's just something to keep in mind and something to research more if you are especially if you're running multiple different applications that you really don't want to see and don't want to have them be able to see each other there are some security concerns that you still need to go a little deeper on most of the clusters that i run you're running one client in a cluster or one set of applications that should be able to work together usually it's through http apis so they can be in separate name spaces namespaces but that's that's just something to keep in mind if you are going to be running like multiple clients in one cluster there's a lot more that you might need to be concerned about anyways so there is um our mariadb definition i made a change oh i i just added spaces i see this little dot up here and i'm like oh i made a change i don't know what i did um all right so that's mariadb the drupal one is similar but there's a little bit of a difference here so down here there's a persistent volume claim because drupal has a file system that it needs to be able to write to when you upload images and things like that and it has a deployment i'm going to get to that soon and a service to expose drupal to the outside in this case but drupal has one more thing that you'll be using a lot of and it's called a config map and this is this is one way to do it uh there's there's probably 10 different ways to set up a drupal site in kubernetes and configure it with the right settings for database password and all those kind of things this is one way to do it and like i mentioned here this is not the most secure way to do this this is a quick and hacky way to do it but drupal needs to know how to connect to a database it needs to know a hash salt that it uses for generating password reset links and things like that it also needs to know um right here i think this is like allow any host to access it that's not secure you should definitely configure whatever host names your site is going to operate on and it gives a directory for configuration files to be written to and this is the public file system directory and the only reason i'm doing this is to make this demo a little simpler we're going to talk a little bit more next episode about drupal a little bit beyond this and ways to make all these things a little more secure but the point is that drupal does need settings available to it that when you launch the container you have to inject those somehow this is one way of doing it this is going to write a settings.php file that tells drupal all the stuff it needs to know about how to start up and connect to a database there's other ways you can do it you could you could build your drupal site in a docker image in a way that actually pulls environment variables in and you can pass environment variables you can also write a secrets file to the file system somewhere and read those in and different applications can be diff configured in different ways drupal is not what i would call a 12 factor application and there's a website here that talks about this the idea is that your application could be portable and easy to run in any place and 12 factor apps are typically very easy to get running in kubernetes because they are built cloud first drupal is not quite like that and a lot of applications that you run are not going to be full 12 factor apps and an app an app doesn't have to be 12 factor to be a good app that's easy to run kubernetes but there are going to be some things like this where you have to write a settings file or can change your application a little bit to pull in things from from the environment or from a secrets file or something like that so anyways that is a config map and config maps are just stored in kubernetes own database as a blob so this is going to be a blob of data that contains a settings.php some text and and this is a little yaml shorthand for take all the data in here so what this what this little thing if it's folded scale not a folded scaler i don't remember exactly what this is called but what this says is basically take this block of text that's right here uh and then what it's going to do is it's going to put that into a file that looks like this and is php when it's written and so the file will look like this so it preserves the line breaks in here but it takes out any white space that would be over in this area and it this is also a fun fact for trivia night if that ever happens again and we're all in person somewhere this is called the chomping indicator so it chomps off any uh any new lines or white space at the end so if you had some white space it would chomp that off and you just end up with this so that's a config map um we'll we'll probably see a few more of these as we go along and i made a change in here there's that little dot what did i do okay the next thing is a persistent volume claim so this is going to be for drupal to write files when you upload when users upload things to their files to their drupal file system like images or media files or anything like that they will be written in here this will also allow drupal to write things like configurations and stuff like that you could you could use so i believe that the helm chart actually does it a little differently and writes the entire drupal code base i believe to the persistent volume which i don't like that's not a kubernetes way to do it i don't think it is a valid way to do it i i don't i'm i like to have my entire code base for my application as part of the docker image so i like to have all the code be in the image and then you inject a few folders that need to be shared like this files directory instead of having the code run off an external volume that's mounted into your container anyway we'll talk about that a little bit more probably in the next episode as well but we need a place to write files so we have that and in here we're going to use that down here under volume mounts drupal files and here's that drupal files pvc under volumes so we need that the other thing that we're doing with that settings file is we're using we're going to use it as a volume mount and and you can see that there's a way to do that in kubernetes you can take a config map and mount it as a volume and you can do this kind of thing with with docker as well you can take a file and mount it straight in and so we can take a config map and mount it straight into drupal's default folder where it's going to pick it up as the settings for the drupal site that we're going to run and that is mounted here so some of the stuff you're not going to make you're not going to understand it all in the first go-round but once you start building things with kubernetes you can you can start start seeing those how things connect together uh a couple other things that are going on here first of all this is not necessary this init container thing is not necessary if you're running this in mini cube but it is necessary on linode because of some volume permissions differences and that's where it comes in when i talked about earlier in this episode how it's important to understand how everything works underneath you need to know how drupal file system drupal's files directory works to make sure that you can deploy it correctly in kubernetes otherwise you start running into permissions errors users are uploading images and they're getting error messages and things what this does is before the the drupal container starts it changes the permissions on the the default files directory to be owned by the www data user that's inside of this drupal drupal docker image and the reason it does that is we don't necessarily know what what the user gid or uid is going to be before we launch the container there's there's a couple other ways we can do this as well in kubernetes there's always 10 ways to do everything but this way is simple and is also recommended in the linux documentation but if we don't do this then drupal gets an error when it tries installing because it can't actually write to the files directory because it's owned by root and doesn't have write permissions so that's that's one thing that you can do and the cool thing is with this is kubernetes will run any init containers prior to starting up the container so you can do things like prep files uh i remember with magento you had to build the themes and you couldn't you couldn't pre-build your themes for some crazy reason in my opinion it's crazy you had to have a running website connected to a database to build your theme assets which was sorry anyway total tangent there but you can use init containers to initialize files on your container or to copy things across places or to set up something you could even use it to do an automated install of drupal or something like that if you wanted to so that's that's in it containers and then we're back to our normal container definition and another difference in here it's just using the drupal image it's exposing port 80 and it has the volumes and it has resource limits but another difference here is we're using liveness and readiness probes to let kubernetes know you can define whether or not a pod is working and ready and responding to requests and that's used in kubernetes if i well i don't have it running yet but if you get deployments and it shows like zero of one is is running you can define whether or not that's going to be zero of one or one of one by saying uh is this container ready when uh you you basically say like when it when it can be connected to on port 80 it's ready and then it also can do a liveness probe which checks every uh i forget how often it checks uh by default but it will wait 30 seconds before checking the first time to check if the the container's still running and if it's not if it says it's unhealthy it's not live then it will kill that container and bring up a new one to make sure that it comes up healthy you have to be careful with these things though i don't always define them for everything i'm running but for a lot of things i do but you have to be careful if you if you make it too if you make the probes be too frequent or you don't give the right delay kubernetes can get stuck in a loop where it keeps killing your container because it takes 35 seconds for the container to start but you're doing the probe in 30 seconds and then it kills it and starts it again it takes another 30 seconds so just be careful with that um all right so that's that's our drupal our drupal deployment and we're just going to deploy one replica for now uh that's also what the helm chart that we deployed did and then finally we're going to expose it in this case we're going to use a node port just for simplicity's sake next episode we'll talk a little bit more about other options and again this is going to expose port 80 on every node in our cluster using a random port that's that's a high port and it's going to route that port 80 to port 80 on any container matching this selector app drupal which is again defined up in here and so now i want to deploy those things so finally again this is using the namespace drupal so it will be it'll be isolated inside that namespace and i know i'm running a little bit over time today but i wanted to get this done so we can get deep into things in the next episode so i have my kubernetes cluster up here and i'm going to make sure i can connect to it export cube config equals linode uh no dot cube slash linode and then cube control git nodes just make sure i can connect should see three nodes yep i do so i'm going to first create this namespace before you can deploy anything into a namespace you have to create it otherwise you'll get an error message so i'm going to say qb cuddle create namespace drupal and qb cuddle apply dash f mariadb oh i'm not in the right i need to actually get into that directory hub cd 101 episode 04 and kate's manifests all right qb cuddle apply dash f mariadb and what this does is it takes this entire file takes all the definitions inside and individually applies them so this one and then this one and then the other one and the order doesn't really matter that much because kubernetes you can deploy the service and it's just going to sit there looking for any containers with this selector for now and forever and whether or not there's a container it's going to be working and routing requests of course they fail if there's no container running so i could have deployed the service first i just like to deploy things in this way i usually deploy any persistent things that are required so config maps and secrets and volumes and things then i put the deployment and then i put the service and we'll get to ingress and things like that later so i like to do things that way just because that's how my mental map of how these things relate to each other works so anyway there's mariadb and you can see that it created these three resources and i'm also going to deploy drupal so i'm going to say drupal.yml and then i can watch with qb cuddle get deployments dash n drupal and i added dash n drupal because every command that you run with cubie cuddle or cube control needs to be run inside of whatever namespace you're running if i do this i'll do this without dash and drupal and you'll see that there is uh there's nothing so that's why it's given nothing there's nothing because it's going to work in the default namespace with no no namespace selected you can actually change the context that qb cuttle works in so this is useful if you're working with multiple clusters later down the line or if you're working in multiple namespaces and you just want to run a bunch of commands in a namespace you can use qb cuddle config set context and current equals our current ah sorry namespace equals drupal so that's going to set my context that i'm in inside this cluster to be inside the drupal namespace so now if i run that same command qb cut will get deployments it's inside the drupal namespace already so i don't have to specify dash and drupal on every command that i run and you can also drop out of that context by setting namespace back to nothing like that but i'm going to i'm gonna work outside that context just because uh sometimes you just need to do something quick in a namespace and so just passing dash n and then the namespace is useful enough so i'm gonna check if that's actually running um yeah it looks like everything's ready and again that liveness probe and readiness probe come into play when when kubernetes is determining if something is ready and available until that point is reached when port 80 is is accessible these will return 0 of 1 and 0 like they did somewhere up here yeah up here so let's see so i'm going to see how we can access this first of all if i say qb cuddle get nodes it doesn't give me their ip address and i could look them up in here i could go into the console but who wants to go in here to find an ip address that's boring and old so i can say qb cuddle get nodes dash o wide and this is a handy little trick for a lot of different resources in kubernetes if you just do get the basic resource it gives you a condensed version that's easy to read even in a narrow narrow terminal window but if you do o-wide it gives you a lot more information including the external ip address of the nodes so i'm going to grab that ip address paste it here and then i'm going to get the service for drupal so qbcuddle get what is it service dash n drupal and it's the drupal service and it's a node port exposed on this port so i should be able to go to any node in my cluster and go to that port and i should be able to hit drupal from it first time you load a drupal page it takes a little while because drupal has to build all his caches and things but here's the drupal installer i'll quickly run through it sure we can use the standard install profile and a cool thing here is since i defined in here how drupal connects to its database and stuff i didn't have to do that in here like i did in the other examples because drupal already knows and that's one thing that's important to keep in mind for your applications you want to inject as much as you can into the environment and build it as much as you can into a 12-factor application so it's easy to deploy and redeploy things so i'm going to create a site name cube101 example.com test test test highly secure passwords here and save no i don't want to save that highly secure password oh and there's an unexpected error so i have to say that this didn't happen last time and probably the easiest thing would be to just delete the thing and restart it so i'm actually going to do that cubie cuddle delete namespace drupal that's another handy thing about namespaces if you screw things up in one especially when you're testing things out you can just delete the namespace and recreate it and it takes a minute or so but now that that site is gone so this is something where uh i've actually had this problem before especially when i'm running it on raspberry pi's because they're a lot slower but drupal sometimes has issues depending on your your storage uh where it writes files if it's too slow or has some some connectivity issues drupal can actually run into weird race conditions in cloud environments and that has happened before so it didn't happen when i was testing this of course but whenever you're doing a live demo it happens so i'm going to try building this again uh oh create namespace drupal do that do drupal again get deployments watch uh oh no not drupal dot hammel always more fun doing it live because then you get to see that this is like half my life is just redoing things wait a second why are there two oh this is just when i'm watching it it's showing that it's ready now whenever you do something live you get to see that my life is not actually as perfect as my pre-recorded segments make it out to be 90 of my life is spent deleting and recreating things just like i'm doing here and there will be a new port that this is running on so now it's saying it's saying drupal's ready so let's see if that's the case i'm going to get the service again grab this port and put it in and let's see if it let's see if the install goes well this time installing drupal uh thank you very much roger for uh for your uh super chat there as well yes greg mentions that the pre-recorded can be nice sometimes so uh cube 101 let's do this again test at example.com test test test hopefully it's not uh i actually found two bugs in two different things by using the username test or calling a name of something test before so i'm not going to do that i'm going to say admin but still use the secure password of test one of them was ansible a part of ansible is name something test and then wow that's great and it uh and it blew things up because of that i wonder if it's i don't know i i could debug things here i can also go uh cubic cuddle get uh what is it keep cuddle logs dash n drupal dash f to follow dash l app equals drupal and this will follow the logs of any pods in here let's see what's happening uh core install it's showing me that it's a f where's the 404 that's weird drupal's already installed i don't believe that it's installed because i'm getting these errors anyways there's some other logs that i could get into i'm not going to do that just because we're over time today but i will show it doesn't matter that the drupal website's working and let's just say that this is this is the way drupal works sometimes because that's actually true sometimes you end up with the white screen of death they call it and but but this is working but we want to scale this deployment up because we want it to really work like this on three different replicas so i'm going to say qb cuddle edit deployment uh deployment drupal dash and drupal and you can see that also kubernetes adds in if you're editing things it adds in annotations and things that aren't in my original definition but if i reapply this it won't overwrite anything that kubernetes added in so it's one easy way to update your deployments is to maintain a file like this and then if i wanted to i could just change this and then reapply it and it would update the replicas i'm just doing it in the cluster itself i'm going to put this to three and save it and if i go back to qb cuddle get deployments and watch it it says that there's one of three and only one of the three is available right now i'm actually going to check the pods and see what's going on pods i don't watch it uh so this one is on a knit and it should be pretty darn quick i don't know what it's doing let's i'm going to check out one of these pods i'm going to say cubic cuddle describe pod dash and drupal and see what's going on let's see multi-attach air so here's where we start running into issues with real real world applications trying to use them in kubernetes this is one issue that you can't resolve that easily and and so many guides like when when i was doing a little research work for this particular episode as i wonder how all these different getting started guides are doing it and all of them do the same thing they set up a pvc and you might notice something here access modes read write read write once it's so hard to say ours especially when i haven't had a drink in a few minutes anyway so read write once means that you can attach this claim you can attach this volume that drupal is going to build on one container at a time and so many tutorials out there that almost everyone that i've ever seen does the same thing because it's easy in kubernetes you can you can set up these volumes now it's saying it's attaching these volumes you can set up these volumes but kubernetes volumes and amazon ebs and google if i forget what their volumes are called all these different volumes can only be attached to one running container at a time and so when you want to scale up you can't do it because these volumes don't scale like that so um you know you're throwing away a lot of functionality that kubernetes can get you because of the fact that you chose a storage option that most tutorials will tell you to choose that only works with one container at a time which is kind of crazy like i want to use i want to be able to scale up drupal to five replicas so that i can handle a big traffic surge for the day that you know my website goes viral on reddit or something so we're going to talk about next episode how to take this which is a very basic this is similar to a lot of uh the the tutorials out there that you'll find a lot of the simple like i know how to use kubernetes here's my thing and it's like your thing works great for like a hello world but we're in the real world we want to start using our apps and scale them so we're going to talk about scaling shared files we're going to talk about different ways of doing persistence we're going to talk about databases and how you can make them scale up and different options for that we're going to talk about dns and tls on your site so that you can have secure https access and also have a custom domain name because you don't want to tell your customers to go to an ip address and we want to talk about things like cron and how that ties in because drupal requires a cron job to run to be able to do its own cleanup work so please stay tuned for that next episode uh it'll be next wednesday as always at 11 o'clock us eastern time and until then i am jeff gearling you

Channel: Jeff Geerling

Views: 21,942

Keywords: kubernetes, devops, introduction, intro, beginners, guide, k8s, eks, geerlingguy, kube101, kube, kubectl, docker, apps, applications, images, private registry, linode, development, cluster, create, deploy, deployments, helm, pv, pvc

Id: mrxA8g3w6ic

Length: 73min 24sec (4404 seconds)

Published: Wed Dec 09 2020

Reddit Comments

[u/Mormon_Underwear]

my name is... Kubernetes

[u/wtk]

I didn't know Steve Buscemi teaches Kubernetes!

[u/mehphistopheles]

Thanks, Jeff! Looking forward to it 😁

[u/geerlingguy]

I wanted to let people know that tomorrow and in next week's episode (every Wednesday) of Kubernetes 101, I'll be covering Drupal deployment in Kubernetes, so if that's something you're interested in, please hop on into the stream (or watch the recording later!) and join the fun!

Information about the whole series is here: https://kube101.jeffgeerling.com — but episodes 4 and 5 are especially targeted towards Drupal (since I know and love it, and deploy it a lot :).