Graylog 6: The Best Open Source Logging Tool Got Better!

today is May 25th of 2024 and 18 days ago grey log 6 was released followed by some minor bug fixes in 6.01 and 6.02 which only came out a few days ago you'll find the full change log down below I want to talk about today why gry log is still my favorite open- Source logging tool and I continue to use it and that yes gry log 6 was a pretty pain-free transition if you're already running gry Log 5 they give it a facelift but they did not change things so much that you are in unfamiliar territory so we're going to take a look at gry log six and talk about my previous tutorial which too long didn't watch yes it works as long as you are not just following along that tutorial but following the instruction tutorial of pulling down the latest Docker composed from my GitHub because I have updated it now to not install gry Log 5 but to install gry log 6 so you'll find my previous getting started with gry log video down below and outside of the interface color changes they still work the same so let's get started in talking about gry log 6 [Music] now I want to start out and be clear that I am not sponsored by grey log or any affiliation for this particular video but I have been on their podcast and they have sent me some shirts so hey thanks team over at graylog gry log is a couple different products essentially it starts with grey log open and that's what I'll be talking about today which is the free and open source version of Grey log which is specifically for logging then they have add-ons such as your gr log Enterprise and gr log security that kind of gives you more more enhanced detections if you're interested in doing those things they also have several free tools they have a cloud management system but as I said we're being talking about the free gry log open specifically for this video and as I said gry log version six you'll find a link to this down below but this is the full change log for all of the details in here there are a lot of changes that affect very specifically some of the other gry log Enterprise features that are part of their API and Security Management tools we're going to talk about just as I said gry log open but of course this is link so you can go through and see all of the release notes all of the changes that have occurred now the first thing you'll notice when you log into gy log 6 is a much updated color scheme I think this is all for the better it adds better visibility when looking at it but it's not that much functionally different we do have a couple new buttons here at the top to offer their security overview this is a demo by the way that it displays right here CU this is a paid add-on and as I said we're going to keep this in scope of just the free system and of course they still have their Enterprise option here right there you can go right to and request if you're interest in purchasing those now going over to the changes and the ones that are a little bit bigger here in cre log six open is when we're looking at the index sets we can go over here to the different indexes that we have such as this one and if we edit the index set we have the default Legacy options of setting your rotation strategy but you also have the data retention option of just setting the days that you want mid days storage and Max days in storage before going ahead and and purging this so this is a little bit different because you aren't setting it by size like I was in some of my previous videos but your settings from gr Log 5 to gr log 6 when you do the upgrade will transfer over now something else they've added here is under alerts we're going to go ahead and create a new alert notification the notification options here are the same with slack Microsoft teams and email HTTP but also now we have custom HTTP which adds a lot of extended options so you can do a more advanced and more in depth sending including different types of Json application and text plane options this can be really handy and I do like that they have a skip TLS verification option this can allow you especially if you're trying to build web hooks for internal systems to be able to get that data over from a triggered event inside of greylog to some maybe local server that you have to get that data passed over now there have been some changes to the way information is displayed especially the stacking that they do here just looks a lot nicer and a lot cleaner did previously all the other functionality is still there so we can drill down but being able to see where these logs are coming from and drilling down the correlation data between them is I think a little bit cleaner in the way they display it here I also like the way the search works a little bit different with the undo and redo buttons and you still have your history here so I can sort things by maybe a specific IP that I'm looking for like this one here we'll go ahead and do a quick query on it and we see that this IP has been attacking me over this period of time let's go back and actually start with how I found this which was classification miscellaneous attack which is being dumped from my firewall from Sak cotta and let's actually extend this all the way to 30 days so we can see the different attacks that are happening then we can go here and drill down and one of these and if we want to look for example let's look at this particular IP and we're going to go ahead and add it to the query and we can see what days it chooses to do this attack that SRA cotta Flags then from there we can just undo cuz maybe we want to see attacks from different ones on here I really like the Simplicity of this the undo or maybe let's look at that again and hit redo it's a minor thing but it does make a pretty big difference they also have all the other features here to allow this to be added to the dashboard but they also have a simple duplicate option so if you're wanting to build out your message count or any other the visual displays to represent data you can simply duplicate it and then start modifying each one of those to create the widget so we have message count here message count here but maybe we want to modify this message count and have the other one display things in a slightly different way or maybe how we want the intervals to look on each of these so we'll update the widget and now we've created two different views here and of course we can stretch this one all the way across and stack them on top of each other just like that so now we have two different message count views this is the copy one and of course you can keep editing this and then when you're done save it now one of the things I want to talk about here is the difference between the production docker fig that I'm using versus the one in my GitHub I mentioned this in a tutorial when setting up gry log that you can store the logs elsewhere that is preferred especially when you want to keep the virtual machine or even the machine that it's on a little bit smaller this makes it a little bit simpler for management now I have this mounted just as a standard NFS Mount this is where the confusion sometimes may come and is I'm having the operating system handle the NFS Mount so you just add this to your ETFs tab there's plenty of documentation on how to get a mount set set up inside of Linux and this does mount a trass server that I have and well there's an entire playlist of trass that you'll find on my channel as well that being said let me show you the functional differences in the docker compost file now this is a Docker compost file from my GitHub you notice we have the mango data log data gry log data gry log and mango data are really small those are just the configurations for gry log but the log data obviously is going to vary greatly depending on how many logs and what your retention settings are in grade log and that can get easily into terabytes when you have a lot of different systems that are logging in my production system you'll notice that there is not this log data we just have the mango data and gry log data what you do is you go down here to open search and you look down here at volumes you see it's Mount gr logor logs user share open search data now let's compare that to the version that is in the docker config it is just log data the volume that was defined so it can be stored locally and then user share open data wherever your amount is and mine is simply Mount SL gry logor logs it just a colon and this for the volume now please note I very clearly said it was done by the operating system yes there are ways Docker can talk to NFS I did not go that route I found it simpler just to have the OS handle the mount do make sure that you have proper permissions so that the data is readable by Docker in here prior to setting this up or it'll give you a bunch of Errors now there actually are a lot more changes I didn't cover in gr log 6 but as I said in the beginning I'm keeping this narrowed and scope to gr log open the open source and free version those changes are significant to the paid enhanced add-ons that are offered from greylog now if you're interested in learning a little bit more about gry log or having the full installed tutorial as I said my tutorial that is for gry Log 5 still works for G log 6 provided you download the latest version of my Docker compos file from GitHub it's already been updated so you can still follow along the tutorial and outside of the color changes in the interface it still will work just fine I've also added a video on how to set this up in Windows you'll find that link down below along with an entire discussion on how threat detection in Windows works and how to pull all that data and event log data and the conversion of it into your gry log this is actually really cool if you want to dive deep into logging it is uh really handy to have all your systems putting all the logs in one place to figure out problems that aren't just related to one system but correlate them across all of your systems like And subscribe to see more content from this channel check out the swag store in including some of the new shirts that we have available there head over to my forums forums. laoren systems.com to have a more in-depth discussion on this and other topics and I will see everyone on whatever socials you want to connect with me on or in the next video you'll find all that over at lawen systems.com along with links to my newsletter and whatever else I got going on there all right and thanks [Music]

Channel: Lawrence Systems

Views: 29,135

Keywords: LawrenceSystems, open source logging, graylog dashboard, graylog, open source, centralised logging, server logs, graylog tutorial, syslog server, graylog windows event logs, graylog docker, open source software

Id: PoP9BTktlFc

Length: 9min 35sec (575 seconds)

Published: Sat May 25 2024