Python Logging - Tutorial

[Music] now i'm sure many of you just like myself are guilty of using the standard print debugging method where all you do is you go in your program and you do something like print run or print run and then the value of a variable and you just have a bunch of print statements kind of scattered all over the place and this is really what you're using as your debugging log this is giving you information now there's nothing inherently wrong with doing this and in small programs this is completely fine once you get into larger programs or you start dealing with stuff that's a lot more complicated you definitely want to have some type of persistent log that's going to store all of the data from your program and have different levels of logs what i mean by that is you could have something that's an info or something that's a warning something that's critical or an exception or an error it's just going to be better and a more sustainable practice so in this video i'm going to give you a quick introduction to the logging module show you how simple it is to use and then you can implement this in your next projects now before we dive in i do quickly want to mention that if you want to get better at python go and programming in general then do check out my programming course programming expert dot io this is the best place to learn how to code it's interactive has over 300 practice questions has programming projects teaches fundamental programming advanced programming object-oriented programming software design and so much more if you guys want to support me check it out from the link in the description and you can use discount code tip with that said let's get into the logging all right so let's get started the first thing to do here is import logging now this is a built-in module in python you do not need to install this and it comes with a bunch of basic functionality that you can use right out of the box so first of all we have a bunch of different levels of logging now the first level here is going to be debug so let's just type that out the next level we're going to have is going to be info after this we are going to have a warning and then continuing we are going to have an error and then finally we are going to have a critical okay so these are the five levels of logging that we have and they go in this order so debug is kind of the least important info second warning third error fourth and critical fifth now by default when you use the logging module you're only going to get the output for anything that's at the warning level or above so warning error and critical will actually log to your console anything that's info and debug will not and i'll show you how we change that later on but for example if i just go in here and i say debug i say info i say warning i say error and i say critical and then i run the code you'll see in my console here that i just get three errors here warning root warning and sorry not uh errors but logs error root error and then critical root critical now root actually stands for what logger i'm using and i'm using the root logger because we haven't configured our own which i'll show you later on obviously this is the logging level and then this was the message that we actually had that we wanted to lock so right out of the box you can do this you will get a log it will go to your console but this really isn't much better than what we had previously when we're using print debugging so what i want to show you now is how we configure our logger so we can get this to go to a file and we can change the format of these messages here so we get something like the time then error curta or some other information that you might want so to configure our logger here what we can do is logging dot and then this is going to be basic config now the first thing we can do is pass a level now the level here is going to be what level we want to start outputting or logging at so if i pass debug this means we're going to log everything at the debug level and above if i were to pass say info then same thing it's going to be at info and above so we would skip debug so this is really useful because later on if you want to get a log of only say the errors and criticals you can go in and change this to be say error or critical and then you'll get a log only containing those and then in the future maybe you want to get all of the different types so you go in and again you would change it to something like debug or info for now we'll leave it at info but i want to show you how we actually get this to go to a file so to do that we're going to say file name and we're going to put the name of the file we want this log to go to so for now i'll just go with something like log.log so now that we've done the file name we just need to do the file mode and we're going to say file mode is equal to w and what this means is we are going to create this file every single time overriding a file if it already exists there's a few other modes you can put here but usually you're just going to use w again that will create the file and override it if it already exists if you wanted this to create say logs based on the time or something then you would change this name to use a variable and be dynamic so maybe the name of your log is going to be based on the current time or something along those lines anyways if i run the code here oops we got an issue it says basic config is not defined let's just put logging before that because i forgot to put that let's rerun here and now i get log.log and inside of this log file i have this information now if i do rerun this code then it's going to override this log.log file and place inside of it whatever we logged in the next kind of run hopefully that makes sense uh anyways that is the basics there that is how you set up the logger to go to a file there's a bunch of other options i'll show you shortly but really this is the fundamentals of using your locker now one thing i do quickly want to mention here is that this command can only be ran one time so as soon as you make any logs you do debug info warning any of these methods here this will no longer work and you'll get some type of exception or kind of weird error so only do this one time typically right at the beginning of your program okay now one more thing i want to show you with the basic config here is that we can change the format of our logged messages so right now we're getting the logging level we're getting the name of the logger and then we're getting the message associated with that however we can change this to be much more specific and to be really whatever we want so i'm going to put this on a new line here what i'm going to say format is equal to as another argument here and now i can put in a formatting string that's going to be used to format all of my logs so to do that i'm going to do something like percent and then i'm going to use this special property here called asc time which is essentially a human readable time i'm then going to put a hyphen i'm going to put percent and then i'm going to put inside of my parentheses here the level name i'm going to put an s and then i'm going to put hyphen i'm going to put percent inside of here i'm going to put message and then i'm going to put an s now this is just a standard formatting string in python you have percent you have your parentheses and then you have your type in this case we're putting s for string so inside of here we're going to have a bunch of different properties that are valid to place here now you may be wondering you know how do you know asc time level name message etc well there's an entire list of attributes that you can use from the python documentation so let me actually open that up here and bring it over to this page uh if i scroll down here you can see that we have args ase time created uh exe info file name funk name all this type of stuff and it shows you how you would actually uh do the formatting for this so you can have a look at this i'll leave this link in the description anyways let's have a look at what happens now when i save and run my code and i go back to log.log notice we get some better formatting here where we actually get a human readable time string we get the level and then we get the message and of course we can add a bunch of other stuff in the future if we'd like to all right so now that we've looked at that i'm going to show you a few ways to log some more stuff you may be interested in logging for example how do you log the value of a variable how do you log a stack trace and then we will get to custom loggers so let's have a look at how we log a variable first of all this is pretty straightforward you can just embed it in a string in really any way that you'd like but let's say we have a variable like x it's equal to two i can do something like the value of x is and then inside of curly braces i can simply put my variable name and i'll just put an f before this string and i'll use an f string to embed this variable inside of the string now you also could just use a concatenation you can do something like plus string x you could use a standard formatting string in python like we were doing up here really any way works but if you're using python 3.6 or above this is the simplest just put an f before your string write the string and then anything you want to embed put in curly braces and it will show up so let's save the code here go to log.log and ah sorry it's because my logging level is info this isn't showing up so let's just change this to info here let's go to log.log uh okay let's run one more time go back here and then you see the value of x is 2. perfect so that is how you log the value of a variable next if you want to log a stack trace so an exception occurred and you actually want to log that there's a few ways to do that so let's just write a try except block here i have something like try oops i didn't want all of that let's go with one over zero we know this will raise an exception i'm going to say exep accept and i'll accept the zero division error as e and then there's a few ways the first way to log this exception here would be doing something like logging.error and then inside of here i'll just do something like zero division error and then i pass this parameter here exc info is equal to true and sorry this is an argument not a parameter and this will actually log the exception that occurred as well as the zero division error message so let's save this let's run let's go to log.log and now notice we get the traceback here as well after the error that we looked so that's the first way to do this the next way to do this is to actually use this custom function here called exception uh now inside of here we don't we don't need to pass the exe info equals true instead we just pass some type of string message and when i run the code now it will automatically give me the stack trace as well as the message and the message type will be error to prove this to you let's just change this to be test so you know i'm not using the old log when i rerun this and go here we get test type error and then we get our trace back okay so that is the basics that's probably all the stuff you're really going to want to log other than that you can embed it in a string however you see fit now let's have a look at custom loggers so often times when you're working in larger projects you want to have separate log files and each log file will contain some different information right you may also want to send the logs through say http you may want to send an email with the logs you can do all of that using the logging module now i'm not going to go through http and emails that's a bit more complicated you can find that from the documentation but i will show you how we separate into different log files and use kind of custom loggers so to create a custom logger we're going to say logger is equal to logging dot and then this is going to be get logger now inside of here you simply put the name of the logger that you want and if this logger exists it will give it to you if it doesn't exist it will create it for you so this actually means if you have multiple python modules for example you don't need to pass this logger object around you can simply use the logging.getlogger and if there is a logger that already exists with this name it will give it to you otherwise it will just create a new one now the convention is to use the underscore underscore name variable as your logger and to have one logger for each python module now you don't have to do that this is just the convention and the reason you would use name is because that gives you the name of the current module and you know that's going to be unique in your python project at least if you've set it up correctly so we have logger is equal to logging.getlogger we're going to use name now this one won't already exist so it will give us a new one and now what we can do is use this custom logger to log a message so let's do this i'm going to say logger so rather than logging it's logger which is the variable we have here and then we'll go dot and we can just do something like actually info here and we'll just say test the custom logger now let's run the code let's go to log.log and then notice we get test the custom log so that's great but you notice here that it's going inside of the original log file that we set up using the logging.basicconfig it's also using this format and this file name and all of that stuff so if we want to change that and we want this to go into a separate file we need to set up something called a handler now a handler is just what allows us to configure this logger essentially so i'm going to say my handler is equal to logging dot and then this is going to be the file handler there's all kinds of other handlers related to say http email i imagine the most basic one you guys are going to use is just file ones so that's why i'm going with filehandler and then inside of here i just pass the name of the file that i want my logs to go to so handler is equal to logging.filehandler and i have test.log now that i have this i also want to set up a formatter and then i want to configure my handler with the formatter and then add my handler to my locker i know this seems a little bit confusing you set up a handler that's what's going to handle how this info is getting logged you then have a formatter and that's going to format the logs and then you add that to whatever logger you want and that kind of sets everything up and configures it for you so let's make our formatter we're going to say formatter is equal to logging.formatter and then we can use the exact same format that we had before or we could change this slightly if we want so let's actually go and add one more attribute in here that is going to be the name of the logger just so we can see this so i'm going to go name like that and that will give us the name of whatever logger it is actually logging to this output okay so now that we have that we need to set the formatter for our handler so we're going to say handler.setformatter and then we'll set the formatter and then we need to add the handler to our logger so logger dot add handler and then handler like that okay so now we have logger.info and when we do this we should see that it goes to a new log file called test.log so let's run the code and now test.log has underscore underscore main as the name that's what we're expecting we have info test the custom logger and the time as well perfect and then log.log is unchanged because uh we did not override this or do anything in this we were now using the custom logger from this phone so with that said i do think i'm going to wrap up the video here i do know that i went very fast in this video i just wanted to be concise and give you the information as quickly as i could this is a pretty useful module definitely recommend it in a larger project and this is super useful if you have some type of production code base because you can actually send the logs to yourself you can view them afterwards and again you have that persistent data as opposed to having a few print statements and having to kind of parse through the console and find the information that you're looking for with that said if you guys enjoyed please do leave a like subscribe to the channel and i will see you in another one [Music] you

Channel: Tech With Tim

Views: 152,817

Keywords: tech with tim, python logging tutorial, what is logging, how to log properly, what is logging in coding, what is logging in programming, scattered print statements, logging levels, what are the loggging levels, logging to a file, how to log to a file, custom loggers, what are custom loggers, what are logging exceptions, logging exceptions, programming expert, logging variable values, how to debug your code properly, debugging tutorial, debugging in programming, debugging

Id: urrfJgHwIJA

Length: 15min 2sec (902 seconds)

Published: Wed Apr 27 2022