packagemain #22: Logging in Go using logrus

hi this is alex and welcome to package mine the channel about go but not only [Music] login is a very essential part of large software it's hard to overstate the importance of login be it performance metrics login or error login or debug login for troubleshooting later go standard library has a log package which can print messages can panic but feels very limited when working on enterprise level software but you need better logs distribution control over login or just better structure that's why a lot of third-party login packages have been born such as logrus or k-log xerolog and others they have a lot of similarities so we are not going to review all of them we will take logos as an example which i'm currently using usually they are backward compatible with built-in log package so shouldn't be a problem to replace your locked package in your project let's talk a bit about what can be log um but actually there are so many things you can log that's sometimes easier to ask what not to look some things to lock are errors and warnings when something serious has happened and developers need to know about it and maybe get some alerts then there is debug login with contextual data which will help later with troubleshooting your code when it's deployed to production and there is performance metrics login so you can see how your api services perform and now what not to lock in general it's a good practice not to lock pii details such as email addresses credit card numbers but it may depend on your project now let's write a simple api service and see what we can log there i'll write a simple api service with a single endpoint just to check if a string can be converted to integer or not i'll use error middleware for that new and set up a single endpoint so get maybe is in now let's receive the the string from the query parameters so maybe query param a um let's now continue it later so for now just okay now it will start and set support example 5050. okay now we can use str conf package try and convert string to integer so it will be a to e a and maybe if error and can do it directly here so maybe return something simple as well just status but request for example just saying not okay okay let's just execute it to see if it works all right our server has started now let's try to call this endpoint yeah so x can be converted to integer and now let's check with some number yeah we can see okay so we haven't locked anything yet apart from ejo logging some debug information for us it's very important to understand that logs can be sent not only to sdd out but also to std error to files or somewhere else i'll be using sv out assuming that there is some aggregation software behind it that will collect our logs that's usually a case when you use for example google cloud login or cloud watch and and yeah all other tools and using logos you can configure where to send logs so even if i do sd outlets anyway uh defined for example now init function that would be lock set output and that will be os std out now i don't want to use default lock package so what i'm going to use is i'm going to use to switch right now to logos so it will be syrup some logos and now let's finally lock something i we'll be logging for example all the requests that are coming to our router we'll log the latency to process the request we will log the http method the status code and we can do this using the echo middleware so let's define our middleware so now we want to actually measure how much time did we spend for processing the request so that that would happen here so what we need to do is we need to uh measure when we started so now and then actually after we process the request let's log it and we will be using logos helper function service fields it would be lock fields which is a map and let's look for example method that would be c dot um dot request method then path see the path then then we can lock status so that we see response status and latency and and for example in nanoseconds maybe so it'll be time since start dot nano seconds and we can do it as info sample request request details right and now finally let's tell how to use our login middleware for all the requests so we can introduce in our login login middleware function all right now let's restart our service and see if we get any locks so yeah the service has been restarted now let's trigger the endpoints and yeah we got this two log messages we got the latency in nanoseconds the path and the http status code sometimes your aggregation tools require you to have your logs in specific format and it's very often that they're required to be in json which is now very popular for tools like splunk and logstash so let's tell our logos package to actually format the logs in a json format and we can do it here in init function as well so log set formatter and that would be log i believe json format right now let's restart this service and see if anything changes okay now our lock is in json format which will be very easy to parse so far we lock only info messages we don't log any debug message messages so now let's change our code a little bit to introduce some debug login as you can see our function is very simple but it could be a complex endpoint where debugging information would be very helpful but yeah let's work with what we have um let's look some debugging for here and maybe we add a field to have a contextual data and debug messaging for example bar parsing a string then sorry then we can add something in the end string is parsed and also actually if something felt we can do the same as well here and i think here maybe error or the back would work but yeah it depends i'm able to parse the string and now let's restart our service again and see if we see these messages let's try this integer and this not integer as you can see i don't see debug messages here i see my error message i see request details message but i don't see the back messages that's because the default log level for logroos is set to info which means only locks with level info or with higher severity will be only printed to change this we can accept environment variable log level and change our program to be more flexible and yeah let's do it in init function as well let's use the function log bars parse level that would be string so we will get it from from environment variable log level that would be lock level error maybe let's say in case it's not valid look level we will just set lock level to default as is info so that would be i believe lock info level and then finally let logrus know what level we want to use here's outlook all right now let's restart our service and see if we can pass environment variable there and what it will do so what we'll need to do is to yeah lock level equal debug and yeah it would be very helpful to add log level debug in your development environments but on production it could be set to lock level info or log level error even depends on your use case but let's for now run with debug and let's call the endpoints again so non-integer and integer and yeah now we have the debug logs as well which which is very nice feature because on production you probably don't want to see them or maybe you want to enable them just in specific time slot when you want to enable your debugging yeah sometimes you write the same log messages with the same contextual data so and logos has a very nice feature you can save this contextual parameters into variable and then reuse it so in our context as you can see we repeat this lock with field a a lot how we can improve this we can basically save it to variable like for example context and then don't repeat it anymore so just replace it here and here and here and it's a normal variable that you can actually pass into functions so you can have a trace of your context for example in some cases as we discussed previously locks can be sent to different outputs one more nice feature logos has is called hooks where you can set up your logs to be sent to any third-party aggregation software for example you can send all errors to century.or stackdriver and yeah you can basically get alerts immediately there are other multiple helper functions in logos package but you can check them on your own okay what do we do now with all these gigabytes of logs we collected ideally you use some tools behind it that will aggregate your logs set a proper retention policy store them etc sometimes i use logs to also build some metrics charts for example using the google login log base metrics where you can parse your lock messages and pick some specific field and actually then build a distribution chart or counter chart uh based on it for example in our code we locked latency ns field which can be then easily parsed and any aggregation function can be applied to it and we can build the charts and monitor for example the our api servers performance and latency that's it for today the login award is quite diverse there are a lot of packages and tools and ideas let me know how do you log in your projects in the comments below and hope it was interesting helpful and see you later

Channel: Alex Pliutau

Views: 3,572

Keywords: go, golang, logging, logs, logrus

Id: tmAq3rE8_c4

Length: 15min 13sec (913 seconds)

Published: Sun Apr 11 2021