How to Monitor AKS Cluster using Azure Log Analytics Workspace and Grafana | Setup monitoring on AKS

welcome back in this video we are going to learn how to monitor AKs cluster using Azure log analytics workspace and graphana let's say that you have configured your AKs cluster and you have deployed some Docker containers now you are wondering how to capture CPU usage how to get memory usage how do you create those pod Matrix right that's exactly what we are going to learn in this video if you look at my website here there are many different ways you can monitor AKs cluster you can monitor AKs cluster using Azure log analytic workspace and graphana that's exactly what we are going to talk about in this video so I will come back to that in a minute or you can also monitor AKs cluster using Prometheus and grafana so you can install pramodia's Cube stack and then you can monitor your AKs cluster or you can also leverage Azure monitor as well Azure monitor has a lot of capabilities to monitor AKs cluster and then other Azure resources such as virtual machines and whatnot so you can also leverage Azure monitor as well but scope of this video is to monitor AKs cluster using Azure log analytics workspace and grafana so let's try to understand what do you mean by Azure log analytics workspace basically it's a logical storage unit that is where all the logs data metrics everything is captured for all the Azure resources let's say that you have set up your virtual machine in Azure cloud or let's say you have configured your AKs cluster so you can leverage log analytics workspace for capturing all the data and then to store the data and then you can manage the data right so that's the beauty of this Azure log analytics workspace so you can consider this as a folder which has all the monitoring data everything is captured and managed okay so if you look at this diagram so we are going to use this log analytic workspace in order to monitor AKs cluster and then whenever you set up this workspace what this will do is this will install agent on the AKs cluster so which is going to capture all the data and it is going to send it to over here you would be able to uh you know monitor the data you would be able to visualize the data right so that's exactly what we are going to talk about in this video all right so let's get started so let's quickly go over the preocco sites so of course you need to have Azure subscription okay and then you also need to have Azure CLA installed because we are going to create a case cluster and we are going to create log analytic workspace uh and then you know we are going to perform some you know some some commands using Azure CLI and then of course you also need to have help me installed as well because we are going to install grafana using help okay so these are the prerequisites okay and then how we are going to approach this particular lab exercise so we need to create a resource Group in Azure Cloud so obviously right anything you do in Azure any kind of a resources you're creating in Azure Cloud so the first thing is what you know of course you need to have Azure subscription and then you need to create a resource Group so we would be creating a resource Group for this particular demo and then we would also create log analytic workspace okay and then we are going to create a new AKs cluster with monitoring enabled and then whatever the log analytic workspace we created we are going to get that ID of that and then we are going to associate that log analytic workspace ID with aaks cluster and then we are going to configure Cube config in order to access the AKs cluster and then we'll be creating a service principle with log analytics reader road so this is very important so we need to have a service principle which is going to have all kind of a permission in order to pull the you know the logs and metrics from the from the locality workspace so we need to have the service principle created and then we are going to install graphana on the AKs cluster okay so we are going to install graphone on a Case cluster using help and then we are going to configure a graphana with the port forward so that we can access grafana locally using port number 3000 okay and then we would be accessing that uh locally we'll be creating a data source for for pulling the data from from AKs right so that is what this data source is for and then of course once the data source everything is created we are going to create a dashboard and then we are going to uh you know view those data right CPU usage memory usage all kind of report in grafana as well and then of course we will also view the Pod metrics data as well in grafana awesome so let's get started so folks this is my website coachdevops.com I will be posting the link right below the video as well so the first thing is we have to create AKs cluster with monitoring enabled so let's get started so I'm going to open up my item so this is my item so as I was saying in the prerequisites so I did have Azure CLI installed and then I also have a Helm install as well Helm 3 actually right so version three as you can see here so everything is good so now let's go ahead and create all the resources what we need so so like I said in the in the demo so like we discussed we are going to create a resource Group uh in Azure so let's go ahead and then get started so we need to First login into Azure Cloud so type AZ login and then I'm going to select my credentials and then as you can see here I'm successfully logged in into Azure perfect so now we can go ahead and then create a resource Group so this is the resource Group what I'm going to create so this is the name of the resource Group and then this is the location so let me go ahead and then copy that okay so there you go so as you can see here we were able to create the resource Group perfect and then let's go ahead and then create Azure log analytic workspace so how do you do that so as you can see here this is the command okay so we are going to create a log analytic workspace this is the name of the workspace and then we are going to create this workspace under the same Resource Group what we just created uh you know a few seconds ago okay and then if you see here this is also the output of this command also going to give us log analytic log analytics workspace ID as well so we would also get that so let me go ahead and then copy this command let's go ahead and then enter so as you can see here wow see here so this has created log analytic workspace and then we were we were able to get the log analytics workspace ID as well right so whatever the ID associated with that perfect okay very good now what we can do we can go ahead and then create a case cluster uh with the two worker node and then we would also go ahead and then enable monitoring as well so like I said uh we are going to create brand new AKs cluster but let's say you already have a case cluster setup you can still do that you can create login workspace and then you can associate log analytic workspace to that AKs cluster and then you can still capture log and metrics but but we are going to create a new cluster okay so how do you create the new AKs cluster so again same thing right so we are going to copy that okay and then and then like I said we are going to associate the workspace ID right so that is actually nothing but the output of the previous command which is nothing but this okay so go ahead and then paste that there you go make sure you give space here let me just do that right now okay like that enter so now this will go ahead and create AKs cluster with monitoring enabled and then this will also this will also go ahead and then associate workspace log analytic workspace ID with the AKs cluster as well so this might take a minute or two so I'm going to pause the video and then come back all right so as you can see here the operation is complete so this should have created AKs cluster and everything so let's go into Azure portal okay so this is my Azure portal click on resource groups there you go so this was the resource Group what we just created wow see here guys so we were able to see the AKs cluster and then this is also log analytic workspace and then of course this also created this as well so this is actually part of it perfect so let me go back to the instructions very good so now uh we are going to access the cluster using Cube CTL command so let me go ahead and then perform this particular command actually this will show the the details about the cluster okay very good and then we want to connect to the cluster so I'm going to perform this particular command which is going to update the cube config entry okay but before that let me go ahead and then remove the existing Cube config there you go let me go ahead and then execute that command awesome you see here so the cube config entry has been created over here okay so now what I can do I can go ahead and then query the cluster okay there you go see here guys so we were able to get the node information right so yeah so we created AKs cluster with two worker nodes so these are the worker node of course I have not deployed anything yet so that's the reason why it says that no resource form perfect so so far so good guys so basically what we have done so far is you're able to create the research group we created log electric workspace we created a new AKs cluster with the monitoring enabled and then we also assist Associated log analytic workspace ID with that and then we were able to uh you know create a queue config entry and then the next thing is we have to go ahead and then create service principle okay with this particular role okay or we are going to create a service principle with this particular role okay and then that particular role has to be uh you know it should have assignment under the scope of your resource Group ID right so we need to first get the resource Group ID okay Resource Group ID of this one so let me go ahead and then execute this particular command which is going to get the resource Group ID so this is nothing but this one of course you can also get the resource Group ID by going into Azure portal as well so when you click on Json view of your resource Group so this is what I'm talking about so that is your resource ID for your source group what we just created a few minutes ago okay so perfect so we got that so now we are going to go ahead and then create service principle uh with this particular role and then the scope of that would be this particular Resource Group okay so let me copy this command okay and then between Scopes we are going to go ahead and then provide this information okay all right there you go so what this will do is this will go ahead and then create a service principle and then you know with this particular role okay perfect so whenever it does it is also going to go ahead and then provide the app ID all this information right so which we will be using uh while creating the data source uh in in grafana okay so let's save this information okay this is important so you can save this uh you know in in a in a place where no one can you know access to it right all right so so let's let's move on now what I'm going to do is I'm going to go ahead and then install grafana right so we have to install graphana as well so like I said we are not going to install QQ primitia stack so we are going to install graphana alone itself because we are going to use log analytic workspace for pulling the data and then we are going to use grafana for visualizing the data okay so so we are going to go ahead and then add this so I want to First add the charts to my repo so this already exists no problem and then if you want to search you can search it okay there you go right so we are going to go ahead and then install this one okay so this is the uh grafana stack which you are going to install Okay so for that what we have to do is we need to go ahead and then create monitoring but before that what I wanted to do I want to save this information okay I want to save this information so let me save this over here okay I want to save this information over here okay so we will come back to this okay so let me go ahead and then create uh create a namespace okay so I want to create a namespace called monitoring okay very good so namespace got created and then of course you can also view it like this okay so as you can see here this name space got created okay very good so now we want to go ahead and then install grafana so like I said we are going to install this particular uh entry right so let's go ahead and then install and we are going to install under monitoring namespace there you go so this went ahead and then install so of course now you you can view it as well right so if you if you try to get the parts that is the workspace name monitoring there you go so it's going to take uh probably 15 to 30 seconds for parts to be up and running so let's give a few more seconds there you go so it is actually up and running wonderful okay so now the next step would be to to access grafana so how do you access grafana so this is where we are going to use port forwarding so I'm going to go ahead and then copy this command and then what is the part name for grafana that is a part name okay and then we are going to forward to port number 3000 there you go so as you can see here we were able to uh do that okay but I forgot to tell you one thing before we do that so we also need to get uh grafana password as well so for example if you try to access grafana so you see here right so so we we all know the default username which is admin but the password needs to be you know fetched from this uh you know I mean we need to get the password so for that this is the command which we have to enter so let's do one thing I'm going to open up another shell for that or you know otherwise you can you know Ctrl C and then exit and then you have to redo that but I don't want to do that so there you go you see here so I was able to perform this particular command so this is the password for grafana okay so let me go ahead and then login into grafana so admin is a username and this is the password go ahead and click on login wow there you go guys see you guys so I was able to login into grafana right now perfect so everything is good so far okay all we have to do is go ahead and then create a data source in grafana to pull the data from from AKs cluster right so let's let's go into grafana so click on the settings and click on the data sources okay so click on add data source and then look for Azure type Azure click on Azure monitor so we are going to add this data source wow see here guys so all we have to do is go ahead and then provide all these entries so which we have already you know captured here right so when you create a service principle this was created of course I also store that here as well no problem so let me go ahead and then fill that information from here so this is the app ID so that is nothing but this one okay and then this is the password that is actually nothing but secret over here client Secret and then this is the tenant ID let me pick up the tenant ID which I'm going to put that here perfect so let me click on load subscriptions there you go you see here guys so it was able to load the subscription and then go ahead and then click on Save and test okay so some issues insufficient access error okay this problem should not come but let me try one more time okay I don't see any anything any problem with that what we have done so far application ID okay there you go see it guys so everything is good so click on Save and test okay awesome so we were able to create this data source wonderful now all we have to do is go ahead and then create dashboard so how do you create a graphana dashboards okay so what I'm going to do uh go ahead and then click on this four squares here click on import okay and then we are going to provide you know the dashboard ID right so we have uh there are some default grafana dashboard uh which you can create so one of them is I believe one zero nine five six let me verify here one zero nine nine nine five six okay so click on load and then click on import wow see here guys now we are able to see the cluster CPU you say as well as you know cluster memory usage wow look at that right so we're able to see the data and then let's also go ahead and then uh create one more dashboard as well you can discard click on import so let's go ahead and then put 14891 there you go click on load click on Azure monitor click on import wow see here guys now we are able to see pretty much all the parts which are all deployed inside our AKs cluster look at that guys and then if you want to change the namespace and we have created monitoring namespace right so you can click on it so then it is going to show only those parts so as you can see here only one part is deployed under that namespace so that's the reason why you just see only one part right and of course you know it's going to take some time to to fill up all these data and whatnot right so I mean to to fill the Matrix so this is the system namespace Cube system namespace as you can see a lot of parts deployed as part of that and then of course you can always go back like this um and and this was the first rep first dashboard what we created foreign it's going to take some some time to to show all the data guys all right yeah so that is how guys you know that is how you can configure your AKs cluster uh to to monitor data using log analytic workspace and and grafana so before I let you all go so how to clean this up right because yes we did create this AKs cluster I especially when you're creating any resources in Azure Cloud you want to make sure that you are not being charged so much right so of course what you can do you can go ahead and then perform this command what this will do is this will clean up all the resources uh including what AKs cluster including login workspace and then any any support supporting resources under that Resource Group everything will be created so let's go ahead and then execute this in order to clean up whatever we have done so far and it's going to take some time and then last but not the least what I also have done is since we have executed a lot of uh you know commands what I have done is I have created a shell script so if you see here I have created shell script which can do some some level of automation for you right so if you see here whatever we have done like creating a resource Group creating log analytic workspace and then creating a case cluster with monitoring enable and then associate logulity workspace ID and then of course uh try to get the cubeconfig entry and then creating service principle with this role right so everything I have created a cell script all you have to do is just create a script and then you will be able to execute all of this thing so I have I have done that over here in my machine so if you see here I have created a script like this you see here right so I have created everything over here and then all you have to do is just go ahead and then execute the script right so which will do the job for you and then of course you can create uh you know you can create a graphene namespace and then you can just do few additional tasks manually right so yeah so that's it guys you know that's how you can monitor your AKs cluster using Azure log analytic workspace and grafana thank you for watching this video

Channel: DevOps Coach

Views: 4,696

Keywords: AKS Monitoring using Grafana, Monitoring Tutorial, AKS Monitoring using Log Analytics and Grafana

Id: fQzCYFmIjuE

Length: 24min 29sec (1469 seconds)

Published: Wed Apr 19 2023