How to Install and Configure Unified CloudWatch Agent on AWS

hello folks Prasad Amla here in this video I will show you how to install and configure rom unified cloud watch our logs agent or na AWS so let's get started prior to this unified logs agent we had two separate agents want to collect detailed metrics like memory and other to ship logs to a cloud watch so AWS are recently released this unified cloud watch agent which does both the jobs so using this unified cloud watch agent we can collect system level or matrix from ec2 instances like CPU disk memory Network processor and so on I am NOT going into details of what metrics can be collected using this agent I'll concentrate on installing and configuring it we can also use this agent to collect system level metrics from on-premise servers and it supports both Windows and Linux operating systems and as far as distributions and versions are concerned the agent is supported on Amazon Linux Ubuntu 1604 and 1404 CentOS 7 and 6.5 rhel7 7.4 and 6.5 Debian 8 and all 64-bit versions of Windows Server 2016 2012 and 2008 coming to the process of installing and configuring the cloud watch agent these are the steps we need to perform first we need to create an iam roll allowing permissions to collect metrics from ec2 instances and integrate them with Systems Manager under cloud watch locks next we need to attach this role to our ec2 instances from premise servers we can create an IM user and assign this role and you can use the access keys for this IM user to configure the agent next we need to download and install the agent package we can do it either using Systems Manager or manually installing it or not ec2 instances I will show you both the methods in this video next we need to configure our raw agent we can create configuration on the first server and store it on s is some parameter store and this configuration can be used on a subsequent door service finally we will start the agent either using a system or ROC command line again I'll show you both the methods in this video first let's start with creating I am role required for our Wroclaw watch agent so I am on my I am console here so first let's have a look at the policies by default AWS saw gives us two policies for our rock cloud watch agent let's search for a cloud watch shop agent and here we have admin policy and server policy so the server policy will have permissions to write logs to cloud watch or log streams you know described by ec2 instances and get SSM parameters coming to the admin policy it will have one additional law permission which will allow write operation on SSM parameter store so let's have a quick look at these policies so if you go to server policy you can see that it has permissions to cloud watch easy to describe tags and then logs and we'll have a system get parameter permission on this particular raw parameters with prefix or Amazon Cloud Watch and if you have a look at the admin policy we'll have one additional action which is put parameter on the same ma Amazon Cloud watch are prefixed parameters typically this admin policy is used on first machine where we install on configure our wroclaw watch agent and during that process will store the configuration in a system for which we need this put parameter or app or machine on subsequent instances we just need the read access to retrieve this configuration and install the agent so let's create a role using these policies so I'll go to roles here and then create role and this role will be for our ec2 instances so I'll just select ec2 and then easy to again and then go to permissions so let's search for our cloud watch agents and then I'll select admin policy as I'll be installing my applaud watch agent on my first machine so I want to store my configure Janina system so I'll go with admin policy then review I'll call this policy as a CW agent policy or CW agent rule and then create rule apart from this CloudWatch admin policy if you want to install our agent using a system we need easy to roll for a system policy as well so let's attach another policy search for easy to roll for a system so select that and then attach policy if you are planning to use only command line to install and configure your rock cloud watch agent we don't need this easy to for a system roll now let's assign this role to our ec2 instances so I have two easy two instances here so I'll assign this role go to actions settings and then attach replace or I am role so I'll select my CW agent role here and then apply I'll do the same for my second instance as well okay now we have the roles assigned to or both our rounds ec2 instances now let's install our rock cloud watch our agent and as I said this can be done either using a system or a command line first let's use a simple to install our rom agent before that make sure Isis I'm agent is installed on our raw easy to instances and is visible under managed instances so if you scroll down to a Systems Manager and then manage instances we should be able to see both our raw instances sorry stirred with a system so let's click on run command here and for raw command document will be your selecting AWS configure AWS package and then I'll select my web zero one I'll be installing the agent on wave 0 to using a command line so for SSM I'll just select web 0 1 and then the action will be installed and the name would be Amazon CloudWatch agent and I leave the version as our latest and then click on run you can view the command status here by clicking on this command ID as you can see it's already are successful so you can go to output tab here and we can view the output and we should be able to see a message saying successfully installed on his own cloud watch agent and then the version number now let's do the installation using command line on our second instance so here I'm logged into my second ec2 instance that is Sub Zero - let's first download the agent using this URL you can find the URL and all the commands I used in this video on my blog I'll leave the link in the description I'm using W get here to download the agent okay the agent is downloaded now let me unzip it and then we can install the agent or by running installed autosave script that is dot slash installed or SH and this will install our raw agent now we have our agent installed on both the instances using SSM and command line next we need to create a configuration file and this config file can be creating using a wizard installed by default with our raw agent and this wizard basically asks a series of questions and creates a config file based on our raw responses so our agent will be installed under opt AWS and then Amazon CloudWatch agent so let's navigate there and then let's navigate to bin here we have a script called as Amazon CloudWatch agent config wizard let's execute this wizard using dot slash so first we need to select our operating system so we are running on Linux so I'll just select one and we need to specify whether we are using our ec2 or on-premises so we are using ec2 so I'll select the default choice and then we need to specify if we want to monitor any host matrix so I will say yes which is the default value and we need to specify if you want to monitor CPU matrix per core and additional cloud what charges may apply if we say yes here so I will select the default option one which is yes and then we need to specify whether we want to add ec2 dimensions that is image ID instance ID and so on I'll go with the default value which is yes next we need to specify the frequency of metric collection for this demo I'll just say for example say 30 seconds which is option 3 next is the metric category we have three categories here basic standard and advanced for each operating system so for basic we just to get memory and swap matrix and for standard we get additionally or CPU on disk matrix and for advanced we will get disk i/o under next at matrix and similarly for Windows we get memory on paging for basic and then processor physical disk and logical disk for standard and additionally TCP matrix for advanced so for this demo I will just go with the say standard so I will just select the option 2 here and this is the preview of the JSON file it creates so you can verify it and say yes to create it so let's select the first option next it will ask if we have any existing cloud watch locks agent which you want to migrate to the new unified agent so if you are using old logs agent to ship logs to cloud watch this wizard can migrate that configuration to the new agent on this instance I don't have any existing agent so I will just say no here say if we say yes the wizard will ask you the location of the existing config file and migrates it so I will select option 2 next the wizard will ask you if you want to monitor any your log files so I will say yes here and then we need to specify the log file path for this demo I will just give our log messages which is our system log or syslog and we need to provide a group name I'll just call it as my sis log you can provide any name here and then it will ask you if you want to specify any additional log files to monitor but for this demo I'll just say no now the config file is created and it is located at opt AWS Amazon CloudWatch agent and then slash bin slash config dot JSON we'll be using this config file while starting the agent we can use the local file to start it but if you have bunch of servers we need to copy this config file to each server and execute the start command and if you want to change something in the configuration we need to update on all our instances so the better way to do is to store the config file in SSM parameter store centrally and use it to start the agent on all our raw servers so when we make any changes to the config on a system and restart the agent the config will be applied on all agents so the wizard will ask you if you want to store the config file in SSM parameter store I'll say yes here it is option 1 and we need to provide a name for your parameter make sure it starts with Amazon Cloud watch as the I am role we created will allow to create parameters with this particular rough prefix I'll leave the default name which is Amazon Cloud watch - Lennox and then enter and then my region is episode is - next we need to specify the credentials so for this ec2 instance we have assigned the instance role so we are getting these credentials from the IAM role so I will select the first option so the wizard execution is now complete so let's go back to our a system parameter store and check the configuration there so if we go to parameter store and we have our Amazon CloudWatch - linux parameter created and if we check the value it is nothing but our wrong config file now we can use this parameter to start our cloud watch agent on any of the ec2 instances so let's start our raw agent on the first instance using a system so let's go to run command again and then click on run command and now we'll select Amazon Cloud watch manage agent as our raw command document and then let's select our instance I will select our web 0 1 and then now the action would be configure and then the mode is easy - because we are using easy - instances and then now we need to specify the SSM parameter name which is Amazon CloudWatch - linux make sure the name is correct and then we can optionally restart our agent so I will just select yes here and then click on wrong you can check the status of the command it's already successful so let's login to our instance and check the status so I am logged in here to my of f01 so we can check the status using servus Amazon CloudWatch agent status so you can see it's active and running so that is how we can start the agent using a system now let's log into our second instance and start the agent using a command line so I'm logging into my second ec2 instance which is web zero two so let's go to our installation directory which is opt AWS and then Amazon CloudWatch agent and then let's navigate to bin and here we'll have a command called as Amazon CloudWatch H and da - CTL so we'll use this command to start our wrong agent using a command line we can see the options for this command using a - help so I'll just say Amazon CloudWatch agent da - CTL and then - - help so we need to provide - a which is nothing but action and then we need to provide a mode which will be easy - in our case we can use - C option to specify the configuration we'll be using a system and then our parameter store name and then - S is to restart the agent so let's run the command using dot slash and then the command name Amazon CloudWatch agent - CTL and then - a fetch conflict because we need to fetch the configuration from SSM and configure it so I'll say fetch conflict and then - M the mode would be your ec2 and then now - see the configuration that would be your assistant : or parameter name which is on his own cloud watch - Linux and then - s for raw restart so this will fetch the configuration from a system parameter store and then configure our raw agent under restarts it so we can check the status using our service cloud watch agent uh status so we see its actual running so that is how we can start the agent using a system under command line now let's have a look at our cloud watch console first let's go to logs and we should be able to see our log group which is my sis log so if you go into my sis log you should be able to see the instances so these are my old instances so we see our two new instances here and if you drill down into the instances we can see our rom syslog next if we go to our metrics we can see CW agent which is nothing but our crawl watch agent under custom metrics and then if we go into a cloud watch agent we can see your different Dom metrics you let's go to say for example CPU and we can see the metrics for both our web 0 1 & weck 0 2 and we can visualize the data as required so that is the process of installing and configuring unified cloud watch agent using a system and command line I hope you found the video useful so if you did please hit that like button down below and subscribe to my youtube channel for more videos on na AWS have a good day and see you in the next one

Channel: Prasad Domala

Views: 23,498

Keywords: Cloudwatch logs, unified logs agent, send os logs to cloudwatch, system logs, aws, amazon web services, ssm, command line, parameter store, cloudwatch agent, logs agent, detailed metrics, custom metrics

Id: z_bUDHUEWUY

Length: 17min 37sec (1057 seconds)

Published: Sat Apr 28 2018