Python QueueHandler & QueueListener - Offload Log Processing to Separate Thread | FastAPI demo

in this video we're going to look at the Q Handler and Q listener objects in the python loging package and these objects allow you to offload your log processing to a separate thread and that's a practice that can be beneficial for performance in certain circumstances now an example of this is when you have a critical error in your application you want to log that and perhaps send an email to administrator admins can then get that email and they can respond to the issue very quickly but the problem with this is that the operation of sending emails can be very slow in a web service such as a jangle or fast API application you will often want to offload this into a separate thread and that will avoid your response times becoming impacted by this latency and any operation where you're sending logs over the network it can be beneficial to offload that work using Python's Q Handler and Q listener and that's what we're going to demonstrate in this video so let's get started now let's start with an overview of the handlers that we're going to use in this demo I have the python documentation open and this is the login. handlers module and this module defines some useful handlers that you're going to use typically in a python application now you have the common ones such as the stream Handler and the file Handler these are actually defined in the top level logging module but they're available here in this documentation as well now we're going to look at a few different ones here the first one I'm going to look at is the SMTP Handler and this is a class that supports sending your logs to an email address via the SMTP protocol so let's imagine a CRI iCal error occurs in your application you might want to set up an SMTP Handler in order to inform senior developers or administrators or devops professionals about this error and that allows them to get the email they see the problem very quickly and they're able to then go in and resolve that problem now of course something to note about the SMTP Handler you don't want an email to be sent for every single log entry that's generated by your application that would result in a huge volume of emails and that would be extremely annoying for your ad administrators and for developers you want to limit this to only critical instances when something goes wrong in the app and someone has to respond very quickly now we're going to see how to set up a python Handler in order to forward critical errors to the SMTP Handler and thereby send those emails now as well as the SMTP Handler later in the video we're going to see a couple of extra features of this login. handlers module we're going to look at the Q Handler and we're also going to look at the Q listener I'm going to leave the explanation of these until a little bit later in the video and what we're going to do to get started is we're going to create an SMTP Handler so let's go back to that page and what we're going to do in our application is we're going to send some dummy emails to a service and it's this one here which we'll leave a link to below the video this service provides a free SMTP server for testing and you can send emails to this particular host and it's going to make those emails available in this service and we can inspect the logs that are being generated and sent to these emails through this service so let's get started I'm going to to go to VSS code and I have a python file open here at the top of this file we can bring some imports in we're going to import Python's logging module and we're also going to import the SMTP Handler and I'm going to import the time module as well from Python's standard library now we need to Define an SM SMTP host in order to send our emails to a particular service now what we're going to do is copy the email or rather the host that we had here so let's copy this and bring it in here and that's going to be the server we're going to send these emails to and by doing that we're going to see an example of the latency introduced by sending an email over a network and that kind of latency is what can affect performance if you have a web service for example and that's why later in the video we're going to offload that work using the Q Handler now what we need to do now is create a logger object we're going to use the login. getet logger function and we can pass a name into that so I'm going to use theore name property in Python that's going to refer to the name of this file once we've done that we can then set the level on this logger so we're going to use the the logger do set level function and we can pass a level into this function and only outputs of a severity equal to that level or more severe than that level are going to be sent to our handlers so what I'm going to do is set this to logging do critical and this is the most severe log level that's defined in the python logging module and that means that for this particular logger that we've just created only critical log entries are going to be sent to the Handler that we're going to Define which is going to be the SMTP Handler now as well as the logger I'm going to quickly add a formatter here and we're going to Output the time of the log entry as well as the level and the message when we create these logs now let's create our SMTP Handler now so I'm going to create a variable called SMTP Handler and we're going to instantiate that object that we imported from the loging package now the SMTP Handler it requires that host that we're going to send the emails to and we're going to set that equal to the service online here we also need the source email address and we're going to set that to no reply at company.com so so that goes in the keyword argument from address and we also need to determine who are we actually sending this email to we can do that with the two addresses and that can be set to a string if it's just a single email or if you want to send the email to multiple people you can set that to a list of all of the users you want to send the email to so for example if you have a list of admin users you could pass that list into this argument and it's going to send the email to all of those users and I'm also going to pass in another argument and that's the subject of the email so let's say have the subject of a critical error in the application that's going to be the subject for the recipient of the email they're going to see that and finally I'm going to pass secure equals none and that's because we don't need to use a secure protocol here we don't need to use TLS we're just going to send this message directly to the Handler or rather to the host defined in this Handler so that is our SMTP Handler what we can do now is set the formatter on that Handler and we can do that with the formatter property we're going to set that equal to the formatter that we created on line 10 here and then finally we can take our loger and we can call the ad Handler function and we can pass this SMTP Handler into this function and that means when we create log entries for example by using methods such as logger doino it's going to send the logs to the Handler and that's going to then process those logs now because we set the level to critical that means that we can only log out critical errors here so for example Python's logging module the loggers have a critical log level so we can create logs for critical events and I'm just going to create one here called a test and then what we're going to do is we're going to run this application so at the bottom on the terminal I'm going to run Python and then it's going to be app. Pi so let's run that just now and you can see that that's running at the bottom and that's been completed now now I imported the time module at the top here so I'm going to use the time module and we're going to time the execution of this log event so we can create a start time here by calling the time.time function and then just below the logger do critical output we can print the duration of the script using this statement here so it's an F string we're getting the time when this F string is created and we're subtracting the start time that will give us a Delta for the time it's taken to execute this script so let's run python app.py again and we're going to see the time here and it's 1.9 seconds so 1.9 seconds just to run this very simple script and all of that time is coming from this loger do critical call and that's because that is going to use our SMTP Handler and it's going to try and send an email to this SMTP host at the top over the network and that is going to introduce some latency to this application now what we're going to do is we're going to take the address that we're sending this email to and that's admin company.com and I'm going to go to the browser and we're going to search for the emails that have been sent to this SMTP server so I'm going to paste that email address in here and you can access the inbox for that user and you can see the outputs or the emails that have been sent here it's got that subject critical error in the application and if you click the email you can see what kind of message has been sent so it's coming from no reply company.com and that of course matches what we passed as the from address and if you look at the content of the email we can look at the text for example and we can see the log output and the log output here is matching what we had in this formatter with the time the level and the message that's what we see here so the emails are being received by this SMTP host that's all working but the problem that we have and what we want to solve here is the latency 2 seconds nearly to send that email so imagine that kind of log output occurring in your fast API Handler function or a jangle view that's going to really slow down the response that's being sent to the client we need to fix that one way of fixing it is to use the Q Handler and the Q listener so at this point I'm going to go back to the python documentation on the logging module and we're going to look at the Q Handler now this is a class that supports sending your logging messages to a que and those cues can be the ones implemented in Python's q and multi-processing mod modules and the Q Handler class in conceptt with the Q listener class they let your handlers do the work on a separate thread from the one that's actually doing the logging and as it says here it's important in web applications and that's because when you're returning responses to clients that needs to happen as quickly as possible so what you want to do is move any potentially slow operations such as as it says here sending the email with the SMTP Handler move them to a separate thread so I'm going to change the code up here let's go back to the application I'm going to change the address that we're sending the email to so that we can see these coming into a new inbox and if we go down here I'm going to comment this line of code out we no longer want to add the SMTP Handler directly we're going to use the Q Handler instead now I need to import these at the top so as well is the SMTP Handler we're going to import the Q Handler and also the Q listener and as it says here in vs code the Q listener class that implements an internal threaded listener and that watch is for log records being added to a Q and and it removes them from that queue and then passes them to a list of handlers for processing so by using the Q listener we don't need to worry about creating the thread or anything like that as we're about to see now let's bring another Import in at the top here I'm actually going to import the python Q module we're going to use that to actually create the que that we're going to pass to our Q Handler so let's scroll down just below the SMTP Handler and below this line of code that we've commented out I'm going to paste in a new comment what we're going to do here is create the Q object and the Q Handler so let's start by creating a log Q we're going to use the Q module and we're going to instantiate a q object once we've done that we can create the Handler here so let's create a variable here and we're going to instantiate the Q Handler class and we pass into that a q like object so we're passing the log Q into that and that's the instance of this Q object and the final thing we can do is we can copy this logger doad Handler call and we're going to pass the Q Handler into this now instead of directly passing the smt P Handler so what we've done here is we've created a q Handler and we've passed a q into that and added that Handler to the logger object now when we perform these logging operations these level methods such as critical it's actually going to send that content to the que and what we now need is a q listener that's going to pull those log entries off the queue and process them in a separate thread now I'm going to go back to the documentation again here and let's go down to the Q listener if we go to the signature of this you can see that the Q listener class when we instantiate that we send the Q into that and we also pass any number of handlers into the Q listener and these handlers are what's going to handle the log entries that are placed on the queue in the main thread of your application so let's now go back to vs code and underneath the Q Handler code I'm going to add another comment what we're going to do is create a q listener and then start that listener by calling its start method so let's create the variable listener now and we're going to instantiate the Q listener and we're passing the log Q into that and what we need to pass now is any number of handlers that we want to process the logs that come into the que now we have one Handler and that was the SMTP Handler so let's copy that and go back down here we're going to pass that into the Q listener and that's the only Handler that we need to pass in here so we can now call the listener. start method and after we perform the logging we're going to go to the bottom of this script and I'm going to call listener. stop so let's now test this out I'm going to run the application and this time you can see the duration was Z seconds basically instantaneously this line of code here printing the duration ran in this script and that's because this line of code was no longer a blocking operation we passed this output this log entry into the Q and The Q listener had this Handler set up that pulled the message off the queue and processed it in a separate thread so this was not a blocking operation and you can imagine this being very useful if you're performing these logs to an SMTP Handler or any other source over a network you don't want that block your response so this will greatly speed things up if you need to perform these kind of operations so the key point is that when we call logger do critical the email is sent on a listener thread and that's very useful in a web application and that could be D jangle flask or fast API it allows you to return the response to the client without being blocked by the processing of the log entry now just to demonstrate this in an application I want to finish with an example of how we could do this with fast API now what I'm going to do to start with here is I'm going to rename this file and it's going to be called logger dop this is going to contain the setup for the logger in this application and then what we can do is we can remove all of this code that's performing any kind of log we're going to move that into a fast API application now the code for the Q I'm going to comment that out and we're going to start by just looking at the SMTP Handler itself and I'm going to create a new file here called main.py and that's going to host the fast API application so at the top let's import fast API and we're going to create an app object object here by instantiating that and in the application we want to import this logger object that we've created so at the top let's import that from the logger module and I'm also going to import the time module at the top here so we've created our fast API object here what we're going to do now is we're going to create an endpoint with The app.get Decorator so let's just give this a path of Slash orders and we're going to write a function called get orders that's going to handle a request to this endpoint now when the request comes in I want to create that start variable Again by calling time.time and as an example I'm just going to paste a logger do critical call in here and let's give it a sample message of databased configuration options not found now at this point if you have a critical log level you typically want to exit the application because a critical level is associated with a failure in the application where it cannot recover from that failure it cannot proceed so you'll probably want to exit the application at that point we're not going to do that here we're going to return a response here that contains a single key key called duration and this is going to time the duration of this request including the call to logger do critical and to get the duration we can call time. time and again we're going to subtract that start time once we've written that endpoint at the bottom I'm going to start the uvicorn server and the file is called main.py so we're going to look at Main and the name of the app object is app we separate them with a colon and I'm also going to pass the D- reload flag in here and this is going to start the server on Local Host 8000 and if we go to the browser and look at this particular endpoint you can see the response here the duration it took 2.27 seconds in order to process the request and return the response and again that latency is introduced by the call to loger do critical and that means whenever a user goes to this endpoint and sends the request you can see that being processed it takes quite a bit of time in order to return that response because this is blocked by the latency introduced by sending that email to the SMTP host so again let's show how we can offload this work to a q Handler let's go back to the application and I'm going to go to logger dopy and I'm going to uncomment these objects that we had here and I'm going to remove the call to listener. start so we're going to create the listener by passing The Log q and the SMTP Handler into the listener but we're not going to start this until the fast API application has been started so let's go back to main.py and actually before I do that let's go back here I need to comment this out because we are no longer adding the SMTP Handler directly and instead we're adding the Q Handler so let's go back to main.py and at the top from the logger module as well as the logger I'm going to import this listener and right before we create the fast API application what I'm going to do is call listener. start so we're starting the listener as soon as fast API is loaded and remember we need to call listener. stop as well so what I'm going to do in order to do that is create a fast API lifespan event and the purpose of that is to stop the listener when the uicorn server that we can see on the terminal here whenever that's shut down we want to stop the listener so we can do that in what's called a lifespan event I'm going to go to the fast API documentation so I have this page open on lifespan events and you can Define logic that's executed before your application starts up or after it's shut down and that's what we're going to do now but if we scroll down we can see some example code on how to define one of these we have a decorator called async context manager and then we Define a function that takes the fast API object so I'm going to copy that line of code and we're going to go back to VSS code here and right above this listener. start call I'm going to define the context manager and we need to import this object and that's coming from a package in Python called context lib so we can import that and then within the object we can yield because we don't need to perform any actions when it starts up and any code that we write after yield that's code that will be executed when the fast API application is shutting down so at that point all we need to do is call listener. stop and that's going to stop the listener and it's going to ask the thread to terminate and as it says here if you don't call the stop function before your application terminates there may still be some records that are left on the Queue and they will not be processed so it's quite important to call this before the application or the server is terminated now one last thing to do we've got a function here called lifespan and that defines this Liv span event we need to pass that into the fast API object when we instantiate that we can do that with the lifespan keyword argument so we just pass the reference to the function and that's going to then perform this action when the application is shutting down so that code will reorganize the application a little bit in the logger module we create the Q Handler and we add it to the logger and we also create the listener which is an instance of Q listener that takes the Q and the SMTP Handler and then in this very small fast API application we import that listener and we Define a lifespan event that's going to stop the listener when the application is killed and we pass that event to the fast API object via the lifespan parameter and the effect of all of this code is that the logger do critical call that's going to send the log to a Q and then the listener is going to pick up that log message from the Q and it's going to process that with all of the handlers that are defined on that listener and in this case we only have one Handler so let's go back to the browser now and we're going to test this out if we go back to the page for this fast API endpoint when we refresh this page you can see the latency is very much less now so instead of taking nearly 2 seconds this one is taking a minuscule amount of time basically no latency introduced by the logger call now I'm going to go back to loger dopy and I'm going to change the email here let's change it to admin 55 just to test out that this is going to work in the web UI so I'm going to save this and let's go back to this page and refresh the page the duration now says zero so tiny amount of latency if we go back to this site here and go back to this this page I'm going to check if the email was received after being processed by that Q Handler so let's access this and you can see that the emails are coming in and it's the one at the top that's been sent just now I think I use the same email address below here for testing this out but if I go back to the logger dop let's change this to something I've never used admin 885 and go back here again if we look at the inbox for that user we see that we don't have any emails but if we go back to this page and refresh and then go back here when we refresh this we going to get this email coming into the inbox but it's now being sent on the separate thread and it's been sent by that Q Handler so that's all for this video in this video we've seen how to use an SMTP Handler in order for critical log entries to be sent to administrative users and that's done with an email but we also saw that using this SMTP Handler directly can introduce some latency into your application which is not convenient if you have a web service using something like jangle or fast API so an alternative to doing this directly with the SMTP Handler is to create a q Handler and offload the work of sending that email to a separate thread using the Q listener and we demonstrated the use of this in a very simple fast API application where we have a call to logger do critical and rather than sending the email in the middle of our Handler function and taking time to process and causing the client to be delayed in getting the response instead of that what we're doing is we're offloading the call to critical to a separate thread so the processing of the email is done on that thread and we can return the response very quickly to the client if you've enjoyed this video and learned anything please consider subscribing to the better stack YouTube channel we're going to have many more of these technical guides and if you have any suggestions please let us know in the comments and if you've not already done so give the video a thumbs up and we'll see you in the next video

Channel: Better Stack

Views: 1,838

Keywords:

Id: _UgXBArTWOA

Length: 21min 50sec (1310 seconds)

Published: Thu Jan 04 2024