How Structured Logging With Serilog Can Make Your Life Easier

what is structured logging it's a practice where you apply the same message format to all of your application logs because your logs now follow a similar structure it allows them to be treated like data sets and they can be more easily searched and analyzed than simple text in a previous video I showed you how to solve logging as a cross-cutting concern using mediator's pipeline behavior and we implemented a login pipeline Behavior where we added information logs before and after executing our request which you can see here and we also logged an error whenever processing the request returned a failure result if you didn't watch that video then don't worry you'll be able to follow along this video without a problem but if you want a refresher you can go ahead and watch that video by following the link that's going to pop up in the top right of your screen right now I wanted to discuss structured logging in more detail in this video and I wanted to use a very popular library that I'm sure you heard of which is called serilog so I'm going to show you how to configure serilog in your.net core application and then we're going to see how we generate structured logs using serolog as our logging provider let's start off by adding the serial log library to our web API project I'm going to search for the serial log nuget package and the one that I want to install is the serolog asp.net core package so let's go ahead and install it and then we're going to see how we're going to configure server log in our application so with Sarah log installed let's go to program.cs to introduce Sarah log to your application you're going to access the Builder host property which gives you access to configure the host Builder and now you have access to the use serolog extension method which was added by installing the serolog library that we just added and we're going to use it to configure serelog we are going to provide the delegate which is going to contain the host Builder context and the logger configuration instance we're going to use this logger configuration instance to be able to configure serolog settings so if I access the configuration which is the logger configuration type so if I try to access the logger configuration I'm going to be able to configure many things related to Sherlock the settings you are most likely to configure are probably going to be for example minimum level allowing you to set what is the minimum level that is going to be applied to your logs you can see that you have methods to set the minimum level to for example debug or error or information and so on you also have access to the right tool setting which allows you to control where the logs generated by Sarah log are going to be written if you take a closer look you can see that I have extension methods here allowing me to configure that I want serilog logs to appear in the console in the debug level console and also in a file so I can actually write my application logs into a file and serolog is going to take care of writing those logs into a file there is also the concept of serolog syncs several log synths are a mechanism allowing you to persist your logs in a given sync some of the more popular cell log syncs are the mainstream databases that you will be using regularly such as SQL Server postgresql maybe even Raven DB if you are familiar with it and you can also write your logs to seek which is a structured log server allowing you to ingest your dogs and be able to filter and search through them using some Advanced querying mechanisms so let's for example say we want to write to the console and we also want to chain a call to set the minimum level for example to information and let's say we are happy with this the disadvantage of configuring serial log like this is that you are hard coding your serial log settings into your application code if you want to apply different minimum logging levels per your application environment for example in a development environment you might want to be logging at a minimum level of information but in a production environment you may want to raise the minimum level to perhaps warning to just reduce the noise in your application logs this is why I don't like to hard code my serolog configuration so I'm going to show you a different approach what you can do instead of all of this is chain a call to read from which is going to allow you to specify where serolog is going to read its configuration values from and you can tell Siri log to read all of the configuration from your application settings by calling the configuration extension method and passing in the eye configuration instance from the host Builder context so this is all that you need to do to configure server log from your application settings I'm also going to add one more feature from Sarah log which is the ability to log HTTP requests and you can do that by calling the use serolog request logging method which is available on your web application instance so we're going to call use server log request logging and this is going to allow us to log the HTTP requests coming to our asp.net core API so let me show you how to configure server log through application settings let's open up the application settings you can see that I have a logging section here in my application settings which is the default way to configure logging in an asp.net core application but if you want to configure serolog through application settings you can go ahead and replace this with something like this and let's go ahead and discuss what I just added so I added a new configuration section which is called serilog and this is how serolog knows which configuration to read from your application settings and let's see what I introduced in here the first part is the using section here which contains an array of syncs that I want to apply to my serlo configuration so I have one thing here for writing to the console and another thing for writing to a file so my logs are going to be written to the console so that I can view them while developing and the logs are also going to be written to a file which is a popular way to store your application logs if you're not using some external servers for ingesting your logs the next section configures the minimum level for application logs and you can see that it's very similar to what we had previously I set the default log level to information and I can also specify overrides Force specific name spaces in this case Microsoft and system namespaces the next section is where I want to write my logs and the first is very simple and self-explanatory I want my logs written to the console and this refers to the sync I just added over here and the next section this one here refers to the file sync which we configured previously and we also have the ability to specify some arguments to our file sync so let's see what those arguments are I can configure the path where I want to store the actual log files so in this case I'm logging to the logs folder and I name in the files log slash and then Sarah log is going to add a unique identifier here this is typically going to be a timestamp then I can configure the rolling interval which allows me to have a single log file in this case the interval I specified is a day so Sarah log is going to create a separate log file for every every day I also set the role on file size limit setting to True which is going to create a new log file for the given day when the log reaches a certain file size and the last thing I specify is the formatter that I want to use for my actual logs in this case I'm using the compact Json formatter which is going to store my logs as structure Json but it's going to use a compact format to save on some storage space and the last section here is the enrich section which allows me to Define some additional context that I want to apply to every log statement in this case I configured three sources for enriching my logs one is from the log context the second is from the machine name and the last is using the thread ID of the currently running thread so this is how you can configure serolog using your application settings I'm also going to apply a similar configuration for my development application settings so let me replace this section with something like this you'll see that it's very similar only the minimum level is debug and I'm using the regular Json formatter this is going to make it easier for me to review the application logs instead of the compact Json formatter that you will normally use I sent an API request from Postman to update a product and let's observe the logs in the output window and see what we have going on now with serolog introduced to our application so the first thing is going to be the information level log and it's going to contain just some simple information then we're going to execute our actual request and we're going to observe what is the result in this case it's a failure result so we're going to get an error log logged to our output and you can see the error log appearing here if you take a look here this is our log message and we are using a specific syntax that allows us to use structured logging and you do that by specifying the arguments in the curly braces here and you can see how they are highlighted in visual studio and also if I switch from one parameter to the other notice that it's actually highlighting what is the matching parameter in the parameters list I wanted to focus on the error parameter here which is an object containing a code and a message if you take a look at the log that is generated notice that it contains the entire code and message object in the log statement it's not just a simple to string conversion it actually contains the entire log so this is the power of structured logging and I'm going to show you how this actually looks like in a log file but first let's complete our login pipeline Behavior so the next is the information log that the request was completed and when I press continue we get a few more logs for the HTTP request such as this one that the HTTP put request responded with a 404 response that the product was not found and now let's take a look at the log files generated by certain lock what you're seeing here is the log file generated by Sarah log and if you take a look at the file name you can see that it follows the convention that we defined in our application settings so the name is log then hyphen and Then followed by the current day if you take a closer look at the logs you can see that this is actually just Json so every log statement is converted into one row in our text file which contains valid Json I'm going to focus on just one log statement here we're going to structure it in a more readable way so that you can actually see what is going on so if I start adding some indentation after every property then it's going to start making a lot more sense so the first thing you have is the timestamp then you have the log level then we have the message template which if you recall is request failure and then we have a few parameters for the request name the error and the date time and then comes the most interesting part which are the properties in our structured log so let's expand that further and you can see that the first property is there request name which matches the one in the message template then we have the error which is the second one in the message template and it's also an object and you can see here the values of our actual object so we have the code the message and then we have the last property in our message template which is the date and time and then we have some additional information what is the source context meaning where did this log come from you can see that it's coming from the login pipeline behavior and this is very useful when you are debugging then you have the request ID the request path this is coming from our HTTP request and the connection ID so this is how your entire structured log would look like you can see how this follows a very nicely configured format and this makes it easier to allow analyzing and searching through this data you would most likely look at the properties and then look for maybe a specific error code such as product not found the point is you can easily search through structured logs and figure out what actually went wrong I hope that this video about structured logging using serolog was useful and if you enjoyed it then make sure to smash that like button and subscribe to my channel so that you don't miss any of my future videos until next time stay awesome

Channel: Milan Jovanović

Views: 26,797

Keywords: structured logging, structured logging c#, structured logging best practices, structured logging serilog, structured logging nlog, structured logging vs semantic logging, structured logging example, structured logging benefits, structured logging asp.net core, structured logging .net core, structured logging .net, structured logging clean architecture, .net, .net 7, .net core, c#, structured logging .net 7, serilog, nlog, serilog configuration, structured logging configuration

Id: nVAkSBpsuTk

Length: 13min 36sec (816 seconds)

Published: Tue Feb 21 2023