Logrotate

hi I'm Kirsten and in this video we're going to talk about log management with log rotate so when you manage a server you're going to have to deal with logs logs and Borlaug's so your operating system itself has logs so if you look in Ubuntu you typically see an off log a syslog a kern log etc etc when you install services you those typically generate their own logs as well so for example if you install a web server such as nginx it's going to generate its own log files and typically it generates an access log in an error log so the access is what access is being logged to the web server and then the error log or what errors are occurring if you're installing a database such as postcards SQL it's going to generate its own logs it typically generates one log file and it has a particular convention it uses based upon the database cluster on the server so it calls them post cards SQL - the version - the actual name of the database cluster so in this case the default is is main and your own application you're going to be generating logs from that for example in a Ruby on Rails application you typically in your production environment you typically have a log that gets created called production log and if you're using a Ruby application server like unicorn it's going to generate its own logs as well so this is where log rotate comes in it helps you manage this mass of logs so what it does is it does rotation or it rotates them it creates new files and archives the old ones it allows you to compress those old ones and eventually remove them so it helps you manage the log rotation process now it runs a daily job with cron so it runs once a day and goes through all the different logs you have configured to rotate them and you can use this to rotate your own applications logs and we'll take a look at how to configure that so I'm using Ubuntu and I'm using my desktop here I'm not using a server but there's plenty of logs to be looked at with that if you just go to VAR log that's where the majority of the logs are kept on a Ubuntu system I'm just going to do a list and you can see all these different logs that are generated so there's the auth log the syslog and the kernel log that we were talking about the OS generates as well as when you're installing packages you know aptitude has its logging dpkg has as another log file related to that so what you'll notice is that there are different versions of the log files the oldest has the highest number so this is the oldest log file and the most recent one that looks like this so let's let's look at this in more detail so this is the current log so there's no number appended it's just the auth dot log when rotation happens it then takes with this current login log is a muzet in names that dot one so this is the most recently archived if you will log now you notice that these have been gzipped so this these are compressed that's an option you can say when you want to the particular log files to be compressed so in the case here it's compressed immediately you'll see there's aptitude and then the that one does EZ is compressed immediately but there's a delay in the compression with this one so your oldest log would be the the highest number one now there's also directories in here and typically servers services sorry will place their log files in here so if you take a look there's an engine X log folder and then the post grows SQL folder so if we take a look at listing out that folder we can see okay here's the access log here's the heir log so let's take a look at how this rotation is managed so I'm going to go to another folder here my Ubuntu system et Cie /log rotate D so this is the folder containing configuration files for rotating logs so we can see nginx has a configuration file in here postcards SQL has a configuration file you'll see one for uncomplicated firewall one for apt and aptitude so let's take a look at in nginx configuration for log rotate so the first thing you'll see here is that it's specifying the directory of where the log file is so it was invar logs slash engine acts in every file with a dot log extension this configuration will apply to so for the log files in this location you're going to rotate them on a weekly basis and you can choose daily if any log files are missing don't worry about it continue doing the logger rotation rotate 52 that means how many files you want to retain so because you're rotating weekly it's going to basically be keeping 52 weeks a whole year of log files on the system now you can choose to adjust this if you want to but this is the from the default Engine X installation this says you want them to be compressed as they're rotated and the delay compress that is what we saw previously where the first or you can actually see right here the most recent archived log is not gzipped not compressed that's what that means - like the compression not if empty means don't rotate it if the log file is empty you can specify to create the log files with a particular permission set or a mode with a particular user and as a particular group now these next commands are related so prior to rotation you can run a command and then after the rotation of the files you can run a command and shared scripts just basically means you don't have to run these pre and post just this particular log rotation which you can wait till the the end of the process after all logs have been done okay let's take a look at push CRA's is configuration so this is a much shorter again that's doing a weekly log rotation it's only keeping ten weeks it has this new parameter called a copy truncate so if a particular service or application has difficulties switching to a new log file what the copy truncate does is it copies the existing log file and then it truncates the current one so rather than doing a move process to say use this new log file it always writes to the same it allows the service or the application always write to the same log file but the rotation process does a copy and then a truncate so that's what this does again it delays the compress it does compress and it delays that compression don't worry or I should say if it's empty don't do the rotation it's okay if log file is missing don't worry about it and here just happens to specify the group in the user to do the rotation ends now typically you're going to want to set up a log rotation configuration for your application logs so I tend to use Ruby on Rails so what I would do is create one and I typically name them rails underscore and whatever my app name is so that just makes it very clear which log files these pertain to so here's the particular configuration I would use so again like before the first thing you want to specify is what is the set of files that need to be rotated so I tend to put applications in oh btw w this is the name of the application just test app and that stores the logs in a shared area because I tend to use Capistrano in the shared log area and everything with a dot log will be rotated now this includes the Ruby on Rails applications log like production dot log as well as a unicorn dot log and any other log files for services you're using with your application that you've chosen to place in the rails application log area so by default I usually do weekly rotation but you can't short shorten this to daily like I have an application that has a lot of traffic and I'm switching it to daily if it's missing that's okay and I'm keeping 52 weeks again you can shorten this if your application has generated a lot of logs and you want to get them off the system somehow earlier I'm usually enable compression and delay that compression if I need to look back at the most recently archive log don't worry about doing rotation if it's empty and then I intend to use the copy truncate command so that it does a copy and then truncates the log so once this file is saved on the daily running of the log rotate process your logs will start being rotated so to find out more about the configuration of log rotate the man pages of log rotate are a great resource explaining all the different parameters that or configuration settings you can use to configure your log rotation well I hope you found this useful if you want to learn more please subscribe a tributary software.com slash newsletter thanks

Channel: Ruby Tree Software, Inc.

Views: 36,623

Keywords: Log Rotation, Ubuntu, Ruby (Programming Language), Ruby On Rails (Software)

Id: 0U2RMbzKUCY

Length: 11min 4sec (664 seconds)

Published: Thu Mar 05 2015