Logging Tutorial in Python | DON’T use Print for logging | How to Log messages in Python

hey guys it is said that the hardest part of fixing a problem is to identify where the problem is this statement is especially true when it comes to fixing an error in your program in this video we shall understand how to use logging module in python this can really be helpful when you are trying to figure out which section of your code is actually causing an issue logging is generally used to debug your code what is debug debug is a feature that lets you track step by step execution of your program so let's say you have a program which is causing an error or you have a program whose output is not as per your expectations then you will need to debug this program in order to identify which section or which part of your code is actually causing an issue there are several tools that we can use for debugging but logging messages is arguably one of the most efficient ways to debug your code we can place different logging messages in different sections of the code and then based on the recorded log message we will be able to identify or narrow down the section of the code which might be causing an issue once we identified the section of the code which is causing an issue it's pretty easy to fix the issue let's look at some practical examples to understand this further but before i start writing the program if you like this video then please make sure to like and subscribe to the channel thank you and let's begin as you can see i have already written a program here in this program i have a function by the name name check so this function will accept one input parameter that is of a name and it's going to check if the name is valid or not so it's going to do this check based on the length of the name if the name has special characters or not etc etc so let's not worry about how this program is written uh we are only focused on how we are going to use log message here currently i'm not using any log message so i'm calling this function uh from down here and i'm passing a name as an input parameter and then whatever is getting returned from this function i'm just going to print it okay so to start with let's say i'm just passing the name as alexander and if i execute this program you can see that the program is executed and it's saying name is valid okay now let's go ahead and change this name to something else so let's say i make it alexander123 and if i execute this it's saying invalid name so now let's say i wanted to identify from which section of the code this message is actually being returned because this invalid name in my code you can see here i have it here i have it also here and i have it also here so from these three different sections this return statement would have come but in order to identify that it would not be possible just by looking at the code in this case maybe it is possible because it's a very simple program but in the real-world scenario when you're writing programs which may be hundreds or thousands of lines long then going through each line of the code to identify from where a particular section of the code was executed would almost be an impossible task and this is where we are trying to use log message so when i say log message it's basically a message that i'm going to pass in different sections of the code then based on this log message i'll be able to identify which section of the code was actually executed okay so so if you're a beginner in python then probably you would simply use a print statement to capture this kind of log but this is actually not the best way to do logging and i'm going to tell you why so first let's let's see how we can how we can use print here okay so let me just uncomment these print statements okay so i have uncommented this print now if i execute this let me just clear this and execute this you can see that it's saying invalid name and then on top of that i have this failed all checks so now if i search for this message you can see that in this program failed all checks is actually here so now you will be able to identify that okay this return statement was actually returned after executing this section of the code now this is fine so we have used print statement to kind of record a log and then based on that we are able to identify which section of the code was being executed but as i told you print is actually not recommended to be used as a log message basically we use print whenever we we want to print some output to the user something like let's say the program is executed successfully or if you want the user to enter some values then we can use print but when recording certain logs we should not be using print and the reason for that is so let's say you have written your program and in this case it's a small program but in the real world if you have written a big program and then you have used tens of hundreds of different print statements to record a log after you have finished your testing and once you know that the program is fine it's ready to be committed you will need to delete manually delete all of these print statements from your program because you would not want all of this unnecessary basically junk values to be committed into a production environment correct so in that case when you're trying to manually delete this print statement you might end up deleting certain print statements which were actually required for your program so this is an additional overhead that you will need to do when you're using print and secondly let's say you didn't want to delete the print statements you wanted to commit it as it is then the problem is your program will always print the values that are mentioned in this print statement so this may be garbage values which you just wanted for testing some piece of code and now it's committed to the production environment which actually may not be the best thing to do for your software so that is why we should not be using print and instead of print in python we have a module called as logging so in order to use this module we can just say import logging after this i just need to do one uh configuration so i just need to say logging dot basic config and here i am just going to set the level of my logging into debug so i'm going to explain you what this level is and what debug is but for now let's say i just write it like logging dot debug that's all and then what i'm going to do is i'm going to replace this print statements with my logging statement okay so i just say logging dot debug okay so not here okay so what i have basically done here is i'm just importing my logging module i have done some basic configuration where i have set the logging level equal to debug and then i have replaced the print statement and passed logging.debug okay that's all so now if i execute this program let me just clear this and if i execute this program you can see that it's printing invalid name and on top of that it's printing our message that is failed all checks but along with that there are a few other additional information as well and i'm going to talk about that in a short while now you might ask how is this different from print because when we used print also it printed some statement and when we are using logging also it's printing some statements now i'm going to give you two two usages of using logging over print so the number one is that okay here i have printed the output or the logged messages into my screen but let's say if i didn't want this to happen and instead of this i wanted all the log messages to be captured into a file then what i can do is i can configure a file name here so i say file name equal to give some name so let's say i'm going to give the name like demo dot log okay i'm going to save it and now let me execute it but before that let me just show you what i have in this folder so in this folder i have nothing like demo.log file so when i execute this you can see that it's just saying invalid name so the logged messages that was being printed here is now no longer showing in my output but then there is a new file that is demo.log that has been created if i go into this file you can see that it's basically having my log message so this is one advantage of using logging over print the second thing is i told you once you finished all your testing and debugging if before you commit your code into production you may have to delete all the print statements but when it comes to logging you will not have to delete all this debug or logging statements the reason for that is you have the option of disabling logging how you can do that is just by passing a statement like logging dot disable that's all so i can also pass logging levels here but for now i'm just going to keep it like this so when i do this and if i execute this program okay before that i'm just going to delete this log just so it becomes clear to you and if i execute this program now you can see that it's saying invalid name no log message has been shown here if i go to my log file the log file is created but there is nothing printed here there is nothing written here and the reason for that is i have disabled the logging so this is very useful so you might have hundreds and thousands of lines of program and you have written let's say hundreds of different logging statements when you are trying to commit it into production environment then you don't need to really remove all these debug statements you can just do a disable so if you didn't want to print your logged messages then you can just disable it and let's say in future if someone else wanted to use the same program to do some debugging they just will take this code and then comment out this particular piece of line and then all the debug statements can be reused this is the advantage of using logging over print statements now if i execute this program again just to show you the log file so if i see the log file you can see that this is a message that i mentioned but then you also have something like debug here this is basically the logging level now in python we have five different logging levels as you can see here so we have debug we have info warning error and critical and these are all in hierarchical order so meaning that debug is the lowest level whereas critical is the highest level so it each of these levels will have a value something like 10 20 30 40 50. so i'm going to tell you how it's going to impact the debug statements or the logging statement that we are going to use in our program now to explain this further what i'm going to do is let me just comment out this line of code and then i'm just going to have another block of code here so what what i'm doing here is i have all the file statements for each of the logging levels so i have logging.debug info warning error critical and i have different messages for each of these statements okay so as i told you the default logging level will be warning if i did not mention a level here now if i execute this program so let me just clear this and before that before i show you this there's one more argument that i can use and that is file mode so uh so let's say by default the file mode for the file that is uh three mentioned here the file mode by default will be append so every time i execute the the log messages will keep on appending but i can change that to w this means right so every time i execute the log file will be recreated and all the previous data will be erased so for now i want that to happen so i'm just using file mode equal to w and now let's say if i execute this program okay so i'll just run this and if i go into my demo.log you can see that warning error and critical messages are being displayed the reason for that is by default the logging level here is warning so all the statements that is after what including warning and after warning will be printed this is based on the hierarchy of logging level since warning is 30 everything after 30 will be considered so let's say instead of that if i just mentioned here like level equal to logging dot error then what happens is everything after error will be considered that is 40 and 50 that is error and critical so if i execute this program and if i go here you can see that only error and critical is shown here so in the same way let's say if i instead of error if i just put info and now if i execute this program you can see that everything after info is being printed so info warning error and critical okay so this is the significance of different logging levels now you might ask why do we have five different levels of logging when basically all of this logging statements are basically going to print a particular statement into the log file right now the reason for that is depending on where you want to print a message you may use different types of logging level for example if you wanted to message if you wanted to print a message related to an error in your program then you could use logging.error now this generally we would use it in inside an exception block so it's like your program has thrown an exception so one particular section of your code has gone into exception but it actually not stopping the whole of your program in this case we can just record that message using the error but let's say there is an exception that occurred in your program which basically is making your whole program to crash so in that case we can just say it's like a critical message critical error so i'm just going to record such messages or statements into the logging.critical statement and let's say if you have if in your program there are some issues which is currently not causing any errors but in future it might cause some errors so this can be a warning and this kind of warning messages can be recorded by using the logging.warning statement and we generally use info whenever we want to record some information something like let's say a start of a program or end of a program or something like that and debug is generally used for almost all the simple details so any debugging of any piece of code and we generally would use debug so depending on the different levels that we are going to use in our message it would be very easy for any uh third person looking at our log file to identify this uh log message indicates that there is an error this log message indicates that there is a warning etc etc so that is why we have different types of levels in logging and it's up to you how you are going to use or configure your log messages in your program so now we have seen what this logging levels are there is one more thing here that is root i'll come to that in a while but before that if you see here this message that is being printed this is in the default format in when i configured this logger i did not mention any format for the message but actually i can do that so what i can do is so in order to specify a format in which i want the messages to be displayed i can just say format equal to and then provide a format now to provide a format i can just refer to the python.org website i'll leave a link to this in the description below so here i have all the different formats that i can use in my when i'm trying to log a message so for now let's say if i wanted to record the time when that particular statement was executed i can just say this one that is percentage asc time and yes okay so after that let's say i'm going to just leave a dash and then i want to record the level name so the level name i can just copy and paste this one and then finally i also want to record the message so i just copied this and then i paste it here okay so now if i just execute this program and what i'm going to do is i don't want the mode to be w now i'll just remove this so let the mode uh be the file mode be append again so if i execute this and if i look at my log file you can see that the previous messages are still there so since i changed it from w i removed that file mode now the default file mode will be append so all the log messages will be appended to the same file and you can see the new messages that have been printed here is basically in a different format so first i have the time so this is the date and time and this after comma is a milliseconds and then i have my logging level and then finally i have my message so i can specify any uh format for the message that i want to print and i can use all the keywords that are mentioned in the python.org website so i hope this was clear and what we have done is we have imported logging we have done some basic configuration and then we have seen different levels of logging and seen based on what the logging level that is set uh what kind of messages will be displayed there's one last thing that i wanted to show you is that i told you we have this root mentioned here this root is basically the default logger name logger is basically an object that is used to log your messages so by default the logger is going to be root and that is why it's printing root here now we can specify our own logger and i'm going to talk about that in a while so if you're trying to use uh logging into a single program or just some sample program then it's okay if you would like to use this basic config way of configuring your root logger but this is actually not the recommended way there is a drawback of using this root logger in your program and i'm going to explain you what that drawback is so for example i have written two other programs here so the first one is if i just close this the first one is save to file dot py so inside this program i'm already using logger logging and this logging is exactly the same way that i have done here so what basically this program is doing is i have two functions here first is name check again this is basically to check the validity of the name and then i have another function here which is basically save data so this saved data is i'll enter the details like name age and email and whatever i enter it's going to store that details into a file that is data.txt okay so we let's not worry about what how i have written this program and what this exactly is doing we are more focused on how we are going to use log messages here so along with these two functions in this program and also what i'm doing is i have imported logging module and then i have done some basic configuration for my root logger then i have another program here which is called as employee.py inside this program what i'm doing is i'm actually importing my own program that is saved to file so the program that i written here i'm actually importing that so i'm just saying save to file and then i'm passing an alias as stf and then i'm again using logging to configure the root logger and passing some debug statements and then i have some three variables where i'm capturing name age and email and then what i'm doing is i'm doing a check here so i'm calling the function that is created in this save to file program and i'm passing this name that i initialized here so if that name is a valid name it's going to return true and if it is true then i want to save that name along with the other details agent email into a file okay that one i'm doing by calling this function save data and let's say if the given name is an invalid name then i want it to record and critical message in my log file saying that employee check failed when i execute this program what i plan what i want is all the log messages that is present in the employee dot py should be captured under my log file employee dot log and then all the log messages that is present in this program that is saved to file dot py i want it to be captured under my log file saved to file dot log okay this is what i want to do now i'm going to execute this from the employee dot py so if i execute this you can see that it is executed it's telling details here successfully and if i see on the left hand side there is this data.txt file created and this is basically storing the details that i passed so i passed here as name age and email and that name agent email is being saved into this file this is fine but if i look closely here i have a log file created that is saved to file dot log and this file is basically created from my program save to file dot py and this is fine but in my employee.py file i was expecting there should have been another log file created that is employ.log but you see here there is nowhere employ.log file and this is the problem of using root logger to record our logging messages what actually happened here is that when i executed the employ.py file if you if you are not aware the import statement whenever you try to import any program it will straightaway execute that program so the first thing that would execute in this program is import logging so uh after that it would it would execute this program that i have created myself save to file and inside this program if you see i'm actually already config configuring an root logger so inside this i'm passing the file name to be saved to file.log i'm passing the level to be debug and i'm passing the format of the message and then i'm recording some log messages once this import program is executed the next thing it will do is it will try to run this basic configuration but the the problem with root logger or the functionality of root logger is during the execution of the program once the root logger is configured it's not going to overwrite this configuration meaning that when i executed this program it called to save to file it went here and then here it already configured my root logger with all the details that i mentioned here so the file name is this level is this and format is this and after this when it comes to this line to execute this basic config it it already knows that the root logger is already configured so basically this line will never be executed it will just ignore this line and that is why employee.log file was never created so all the log messages that were used here this logging.debug start of employee program end of employee program will actually get captured to the root logger that was already created here that is the save to file.log my intention was to create two log files one log file for each of this program but since i'm using the root logger it's only creating one log file that is saved to file and all the log message from both these programs are getting saved into this file so to avoid that what we can do is instead of using root logger we can actually configure our own logger for each of our program it's actually pretty easy to create our own logger in each of our program so to do that what i'm going to do is i'm just going to create a variable called as logger and then i'm just going to say logging dot get logger and here i can just pass any logger name so by default the logger name was root that was configured from here here i can just pass any name that i want but it is recommended the name to be used is underscore underscore name underscore underscore now this is a special variable in python what this is what value this variable is going to have is if i execute this program from here then this would have the value like underscore underscore main underscore underscore but let's say if i executed this program from some other program that is what i'm going to do now that is from the employee program this save to file will get executed in this case the underscore underscore name underscore underscore variable would get the name of this program that is saved to file so i have created my logger the next thing is to set the logger level so what i'm basically going to do is all this configuration so all the basic configuration that i did here that is i set a file name for my log i set the logging level and then i gave a format for the log messages this this same configurations i'm trying to do with my own logger here okay so i have created my logger next to set the logger level i'm just going to say logger dot set level and i'm going to pass like logging dot debug okay so my logger is created i have set the logger level to debug the next thing is let's say the format of the message in order to format the message i'm just going to say a variable ef and i'm going to say like logging dot formatter and here i'm just going to pass the same format so i'll just copy the same thing okay and i'm just going to paste it here so i'll follow the same format and then once my format is said the next thing is i need to uh set this file so in order to set this log file i need to create a file handler so i'm just going to name a variable like fh and i'm going to say logging dot file handler and inside this i'm going to pass the same file name so i'll just copy this and i'm going to pass it here so my file handler has been set pointing to this file and then what i'm going to do is i've created this file now this file the messages that are going to get printed into this file needs to follow this format so to do that what i'm going to say is fh dot set formatter and just pass the variable name which is having the message format that is f okay so what i have done is i have created the logger i have set the level of the logger to debug i have created a formatter and and then i have created a file handler pointing to this file and that file handler i have set the formatter to the format that i created here and then finally what i need to do is the file handler that i created here i need to add it into my logger so to do that i can just say logger dot add handler and then i just pass fh okay so that's all so this steps might seem a little confusing in the beginning but it's pretty straightforward and this is just one time configuration so you just configure it once and you're good to go now there's one additional thing that you need to do since i have now changed my route logger into this logger that i'm creating here and i'm assigning this logger into this variable logger i need to use logger instead of logging everywhere okay this statement i can just comment out because i'm no longer using the root logger and then everywhere else wherever wherever i'm using the logging statement i need to replace logging with logger so let me do that okay so i have done that i have now replaced all the logging statements with the logger as you can see here so everything is like logger dot debug logger dot error etc okay so that's all and now i'm just going to do the same in my other program that is employee.py so here i'll just copy paste that and what i'm going to change here is i want the log file name to be employed.log and the message format i'll just copy this message format that i have here so if i just paste it here and that's all so everything else stays the same so let me just comment out this root logger configuration and then everywhere wherever i am using logging i need to replace it with logger so i'll just do that and let me execute this program now so i executed this program it's selling details saved successfully and now you can see that i have two log files created so save to file dot log if i open that this is all the log messages that was captured from my save to file dot log so all the log messages that were mentioned here all these messages would get saved into this file that is saved to file and then i have another log message log file that is employee.log and these are all the log messages that were created from this particular program that is employed.py okay so i hope this was clear this is how we have configured our own logger and then specified each logger to each of our program so i hope this was clear if you have any doubts then please make sure to leave a comment below i'll try to answer as much as possible thank you and have a nice day

Channel: techTFQ

Views: 42,658

Keywords: Logging in Python, DON’T use Print, DON’T use Print for logging, Python Logging for Beginners, python tutorial, logging python, python logging, python for beginners, logging module, how to log messages in python, logging levels python, disable logging, disable logging in python, root logger, python loggers, python logging advanced tutorial, python logging tutorial, python logging module, python logging beginner, Logging Tutorial in Python, How to Log messages in Python

Id: gsa1oFn9n0M

Length: 28min 55sec (1735 seconds)

Published: Fri Jun 04 2021