Monitor Memory and Disk Usage with Cloudwatch Custom Metrics

[Music] in last video we have seen how we can use blue green deployment with the help of qualifier service in this video we are going to focus on cloudwatch and i am going to show you how can you create custom metrics for memory uses and disk space aws cloud watch by default does not provide these metrics in your cloud watch dashboard so we are going to see how we can create such vertexes [Music] i have already created one article on custom matrix on my official website and cloud watch custom matrix memory disks okay so we are going to see how we can create custom metrics and i have created one ec2 instance and this is my c2 instance let me take login at okay [Music] where's my putty [Music] right here and [Music] here we go [Music] easy to user and i'm going to make some changes like color background is black okay foreground and this looks good for me and the font size sorry 18 okay and i'm going to use my research key and it is in my dropbox [Music] i think it is downloads yeah so where is my ppk right here [Music] and here we go right now i'm inside my easy instance so what are the steps for creating custom matrix let me tell you so we need to install some pearl dependency as aws providing this monitoring script in pearl so we need to install couple of prerequisites that that are required for that script okay so you can check in my article what another step so as i am using amazon linux2 if you are using red hats and os or ubuntu you can follow further steps so let me copy this for me and okay and the dependency packets are being installed done so next step is to download the monitoring script okay so this is the command to download monitoring script so before i download it let me navigate to my it it's not required you can put the description any of your folder but i'm going to use my otp and okay if i see right here these are the scripts let me make it unzip cloudwatch script okay so now there is a folder called i download script mark aws script [Music] if i list this directory i can see how many files are there one get instance state start pl and mon put instance data dot pin so there are two scripts one is for getting data from matrix and another is porting data into metrics and there is one more file called aws cards.template but instead of using uh access key or secret key based authentication you should allow a role to your ec2 instance to perform any action on your cloud watch okay let me see what is the next step [Music] okay so you can take a look what are the these things are so what can you do with the scripts so you just need to copy your template file with aws.com copy aws cards dot aws card start sorry okay start con okay if i go into this file and i can see it requires my access key and secret key okay so let me find my excess key and check that key okay i have not created any access and secret key i'm going to create one for this demo and users [Music] sharma secret credential create access key and here we go i'm going to download this secret key i'm telling you again this is not a best practice to do but my focus in this demo is to let you know about the cloud watch customer matrix what about the how what is the best practices about using uh iam in your application so best practice is to create a role assigned to your ec2 what i am going to do this way okay and [Music] okay and this is the secret key don't try to use this uh secret key and access key at your end because i'm going to delete the secret key and access key after all okay [Music] okay [Music] yeah of course you should hide your access game secret key and these are the uses of this script you can take a look on the amazon official side or you can take a look on this mat on this article as well okay [Music] so let me tell you this is the command that you you should use [Music] before we go ahead let me tell you about the statistic about my configuration like i have i'm having 30 around 1gb ram and this is space around 10 gb so i have one evs attached with my ec2 it is of 10 gb and 1gb ram so i am going to monitor these two things with the help of this script okay so whenever i trigger this command okay let me talk about memory first so i have passed three months three parameters first is first for memory utilization this one second for memory use used and the last one is memory available similarly i have used two for disk space one is disk i will remember is for disk space used and also mentioned you have to mention the path of your volume so the volume is mounted on root so i have given the root path okay [Music] if you want to go in detail what these all options are available you can take a look right here member utilization memory used my memory used include cassette offer memory available swap it you can use a matrix for swap utilization swap used and this is for disk space okay to execute this come on let's see what happen okay successfully report matrix to the cloud watch reference id is this so let me navigate to my cloud watch and my cloud watch is right here cloud watch where it is right there okay [Music] before i go ahead let me see my instance id is like f99 matrix [Music] these are the linux systematics [Music] okay file system instance id and mount point this kind of metrics are for disk uses and instance id is for memory if you can see there are three memory matrix available we have created for this instance memory utilization memory used and memory available and let me see for disk okay so we have created two metrics one for disk space available and one for desk space used for root mount point and for the same instance if i want to create matrix and i'm going to go to dashboard create a dashboard that's what name is [Music] cloud watch demo dashboard and i want to [Music] add a memory for memory i'm going to use a stacked area chart okay and going to linux matrix instance id and i'm going to use two matrix one is for memory used and another for memory available okay create widgets as you can see i don't have any data in these matrices right now you can see only single dot point right here because we have entered that command only once so right now i'm trying to send it each minute so i'm going to put this command in chrome okay [Music] and one put okay so let me copy this part notepad so actually i'm setting up this in clone so i need absolute path okay [Music] so this is what i'm going to put in my chrome java [Music] okay you can trigger it once more time and contact hyphen e [Music] and star star star star star which denotes for running any command each minute and there is one more thing if i am using this command in chrome and i have to append option name hyphen hyphen chrome it is right here mentioned like this hyphen iphone from iphone chrome so i have to append this in the command space if an iphone from command and save it okay so this command is going to put memory and disk use usage data in that specific matrix each minute okay and let me show you this it is for three stack and okay custom for each minute okay also you can see the data is showing till 986 megabyte and very similar to [Music] this 985 so what i'm going to do i know i have 1gb ram so i can do one thing i can put from zero to one okay and or you can say zero two one zero two four okay that would be much better if you can see clearly what is the memory being used and what is the memory available and just update [Music] okay now i'm going to create one more widget for number and configure and this is for [Music] memory utilization so 30 memory is being utilized and now i'm going to create another widget for disk space okay so this is face used and available i'm going to use for both and data obviously for each minute and obviously i have 10 gb disk volume so i'm going to use medium to 0-10 [Music] okay and here we go [Music] um you can drag drop this matrix right here now you can see uh you have three widgets in your dashboard and we have used all those matrixes that we have created using that script so this is how you can create your custom metrics actually aws providing these monitoring script a free of cost you can download these scripts from the aws official website or you can follow my article on aws which is on a devops india dot in and you can follow this entire article now let me create a alarm if my disk space goes beyond beyond any limit suppose if 10 gb of a volume consumed around 6 gb or 8 gb space then i should get a notification of that okay so let's see how we can do it let me navigate to alarms create alarms okay and here custom matrix so i'm going to use for disk space and this space used okay if disk space used then disk space used [Music] i know my volume is around 10 gb if this is used more than 5 gb okay and action state is alarm send notification to me by email to my email id so just create this [Music] now it is going to check what is the state of this disk right now on the basis of it is going to set the state of that specific arm so sometimes if you have uh configure any alarm and you see that insufficient data that may be because it is not able to access your disk volume suppose your ec2 instance is in stopped state then cloudwatch will obviously show the alarm status insufficient data [Music] so it may take a couple of minutes whenever you create an alarm to get the correct state so i'm going to pause this video for a while so now you can see the alarm state is okay so i'm just going to create a large file on my instance that will consume use space on the instant so for through that we can trigger this alarm okay so creating a file i'm going to use for allocate typhoon and 5g [Music] sample.txt and this command is going to create a file name sample.txt of 5gb so this may take couple of minutes to create such a huge file [Music] oops if all of the sudden created so it doesn't work like dd so there is one more command to create a file which consume any specific amount of space you can use dd and fall allocate is similar to that and if i check my df hyphen h and you can see available is 3.5 gb and use almost 65 offend so using that statistic the alarm should be triggered we can wait a while because that alarm check all the data each minute so it is going to take around and a minute so it is going to take a minute so we can wait [Music] or you can go to your dashboard and check what is the status of your [Music] data right now it is showing similar as old let's see let it get the latest data okay now you can see matrix got changed and disk used is increasing and scalable is getting decreased okay now you can see the the state of uh alarm has been changed okay so you can see it has reached to the threshold limit that i have said like if the disk space used is more than [Music] 5 gb then it will trigger this alarm i should get alarm put that on my mail if i go to my mail okay right here you can see there is alarm for me okay right and you can see all the statistic of that alarm what is going on with your disk space okay so this is how you can create your disk related memory related alarm and matrix because uh aws is not providing memory and this space specific matrix you have to create so you have to create this way and you can monitor your ec2 instances okay this is all about this video and you can see how we created these metrics alarms if i delete this file if i delete this file [Music] sample.txt df hyphen h and only 13 percent used in a while that alarm will get to normal okay so we can watch in the matrix oh you can see now your alarm is in okay state so this is how you can monitor your disk space and memory with the aws custom matrix and so in next video i am going to show you how can you create custom metrics and alarm on your apache web server blocks how can you import your logs into metrics and create such alarms okay that's it about this video hopefully you like this video [Music] you

Channel: Cloud Guru

Views: 180

Keywords: cloudwatch metrics, cloudwatch custom metrics, aws cloudwatch metrics, cloudwatch monitoring, ec2 cloudwatch metrics, aws cloudwatch custom metrics, cloudwatch, cloudwatch tutorial, aws cloudwatch, amazon cloudwatch, cloud watch, aws cloudwatch pricing, cloudwatch in aws, aws, amazonwebservices, aws custom metrics, cloudwatch custom metric memory utilization, memory utilization metrics, aws cloudwatch tutorial, amazon cloudwatch tutorial, how to create cloudwatch custom metrics

Id: 6J0ipYuEpGQ

Length: 28min 30sec (1710 seconds)

Published: Mon Dec 06 2021