Setup Prometheus monitoring on Kubernetes | Setup Prometheus Monitoring on EKS Cluster using Helm

hello everyone welcome back what we are going to learn in this video is how to set up monitoring on kubernetes cluster using prometheus and grafana so let's quickly go over the agenda what we are going to learn is what is prometheus and we will also learn what is grafana and how to install both prometheus and grafana what are the different installation option and then why we have to go with helm in order to install both prometheus and grafana and then in order to set up these prometheus what is the prerequisites for example we need to have a cluster setup so we will talk about that and then we will get into the demo part so the way we are going to approach the demo is yes we have to create a namespace and then we will install both prometheus and grafana using helm chart in the namespace and then we will access grafana in the browser and then we will create some dashboard in order to monitor our kubernetes cluster that includes all the cluster nodes parts and memory and cpu usage all right so let's get started guys so i have put together all this information in my website coachdevops.com i will be posting the link right below the video as well so what is prometheus pramatius is an open source monitoring tool which provides out of the box capabilities for monitoring kubernetes clusters using prometheus you can also monitor database servers and servers as well but scope of this video is to monitor kubernetes cluster the way prometheus is going to collect the data is based on time series so once it collects the data it is going to store it in prometheus it is going to store it in the persistent volume where graphene and all other tools can pull the information from prometheus and what is grafana so grafana is an open source tool using grafana you can visualize the data collected uh used from prometheus but graphana also can uh pull data from other monitoring tool as well awesome so let's quickly go with the architecture diagram so this is the prometheus server which is going to collect the data from various clusters based on pull mechanism so it is going to collect the data based on time series okay and this is a prometheus alert manager which is going to send alert to third-party systems such as slack channel or microsoft teams it can also send email as well and then this is your grafana okay and then pramat is going to collect and stores the data in the persistent volume it's pretty straightforward very simple architecture awesome so let's get started so if you look at the prerequisites yes we need to have kubernetes cluster setup so if you click on this link so i have provided instruction here on how to set up e case cluster so i have used the ek ctl command in order to set up the cluster which is more efficient way of setting up cluster of course there are other ways as well so you can follow any one of the ways to set up your eks cluster in aws cloud and my eks cluster has couple of nodes so you can actually see that so i do have two worker nodes and then i also have installed few parts as well okay so this is springboard app which i have installed perfect okay and then i also have helm install as well on this machine so if you see helm version is going to tell you this is a version what i have installed helm3 okay and then i'm going to use this ec2 instance in order to access the eks cluster so this is the easy to instance which i'm going to use okay perfect so now let's go ahead and then install both prometheus and grafana using the helm chart so let me go ahead and then execute this particular command so what this is actually going to do this is going to add the helm stable charts to my local machine to this machine okay perfect the stable repo has been added okay very good and then let's go ahead and then add prometheus helm repo so so if you look at here there is a community version right for for prometheus what they have done they have created or they have combined everything in this particular helm chart all we have to do is go ahead and add this prometheus community repo and this includes everything this is going to have uh prometheus operator alert manager grafana right node exporter all the promotions as well as grafana components in order to monitor the kubernetes cluster so we just have to go ahead and then add them okay so let me go ahead and then do that perfect it has been added to our repository awesome and then if you like to search you can go ahead and then perform home search okay so let's go ahead and then say prometheus community right there you go so as you can see here let me expand the screen as you can see here there are a lot of or a poll has been available right you see that there are a lot of repo a lot of different ways you can install a prometheus and these are the different different helm chart available so the one what we are going to do we are we are going to use we are going to use cube promotion stack so this one so this is what we are actually interested in you see that okay and as you can see here this particular stack is going to have everything right so that's what it is okay so we'll be interested in using this particular repo awesome okay great so before we get started let's go ahead and then create a namespace okay let's go ahead and then create a namespace i'm going to call this as a prometheus namespace again you can give any name but that's fine so i'm going to go ahead with this name awesome so the namespace have been created and then we have to just go ahead and then install the q primathia stack in this particular namespace so this is the command so help install stable and prometheus community so this is the stack right and then i want to install under prometheus namespace okay so go ahead and then execute that command so this may take few seconds as it has to install all these components right and that includes both prometheus and grafana and node exporter alert manager and what awesome as you can see here the stack has been deployed right so everything is good and look at that this has been installed and then if you like to check if all the parts are running again they already gave the command or you can also write the symbol command so go ahead and then say get parts and then provide the name space which is nothing but promote yes okay so as you can see here everything is up and running we just uh you know set up like what like 30 seconds ago right so everything is good and we are able to see grafana as well right you're able to see grafana we are able to see node exporter we are able to see kubernetes state metrics component alert manager and then this is the actual uh you know server right so we are able to see everything perfect okay and then let's also go ahead and then look at the services as well okay so as you can see here so these are the services let me scroll a bit to my left so that more clear yeah there you go you see that so we are also able to see all these services so we are actually interested in this one as well as uh what do you call as well as this one right sta this one okay we are interested in this one these two awesome so pramathias and grafana have been set up now all we have to do is how we are going to access them in the browser right so what i'm going to do i'm going to actually edit the service of the prometheus service right so i'm going to go ahead and then edit the service because right now as you can see it is exposed as a cluster ip so in order for me to access i want to access through a load balancer so what i'm going to do i'm going to go ahead and edit the service so i'm going to give this particular name and that is actually under prometheus namespace okay so go to the end of the file here and then i'm just going to change cluster ip into load balancer okay or you can also use node port as well but i would like to prefer load balancer enter and then let's also go ahead and then edit this service as well cube ctrl edit service stable grafana and then the prometheus namespace and we are also going to expose this as a load balancer so that we can access in the ui perfect okay so now let's go ahead and make sure everything is running everything is changed right it should those two should be changed to load balancer okay there you go so as you can see here now it got changed right from cluster ip into load balancer right both of them perfect so now let's go ahead and access grafana so i'm going to copy this ip i'm going to go ahead and then access in the browser it might take a few seconds for grafana ui to come up okay so you can give few seconds and if you like to access prometheus you can also access on this particular port number which is 1990 so again this may also take few seconds to come up let's see so i have to enable my javascript there you go so this is your prometheus server right of course this may not be like a great ui when you compare with grafana but as you can see here right you can pretty much see all these alerts and all right how it is configured and up right okay you can query it can create the uh the unnecessary ui the ui may not be great but this is your prometheus prometheus ui okay but we are interested in grafana so that you can see the data more easily right okay so our grafana is actually coming up so let's log into grafana so the username is admin the password is uh this one okay let me go ahead and then say and then enter the password which are all out of the box username and password there we go so that is our actually graphana right graphana ui but what we are going to do is we want to create some kind of a dashboard in grafana so that we can monitor our kubernetes cluster right so what i'm going to do i'm going to go ahead and then create some kind of a dashboard in grafana so how do you create the dashboard so click on this plus symbol and then say import so there are various ways you can create cluster but i'm going to use some id right some some predetermined ids i'm going to use this id which is actually going to create a kubernetes monitoring dashboard so let's go ahead and then enter that number and then click on load oh i have space here okay click on load okay and then select prometheus as the default data source and then go ahead and then click on import before that we'll look at that right so this is going to create kubernetes monitoring dashboard click on import wow so look at here guys see so this is actually coming from kubernetes right so this is our cluster memory usage this is our cluster cpu usage right and then when you scroll down you can see all the parts running isn't it look at that so i have my spring boot app as well right springboard part and then these are all the default uh parts right so we are able to see that okay look at that so this pretty much has everything for you here right container cpu usage container memory usage right you can pretty much see everything using this kubernetes monitoring dashboard right and then we can also create additional dashboard as well if you like let's say you want to monitor the worker node let's go ahead and then create this another dashboard using 3119 okay so let me go here you can discard this fine let's go ahead and click on import load and then you can choose this one and then click on import wow look at that so this is like another another dashboard right yeah so this will give you part cp usage okay guys so we are literally able to see all the parts running so yeah so you can create variety of dashboard in grafana by mentioning the the number right yeah so that's it guys you know that's how you can monitor your kubernetes cluster using both prometheus and grafana thank you for watching this video

Channel: DevOps Coach

Views: 11,316

Keywords: Kubernetes Monitoring using Prometheus and Grafana

Id: C38dT0Kt3zs

Length: 15min 59sec (959 seconds)

Published: Thu May 26 2022