Python Async | Asynchronous IO Introduction

hello my name is xander welcome to the python beginners plus course in this tutorial we're going to explore the concepts of asynchronous programming to give us a high level view of this programming paradigm in python let's just start with a an example to give us some context here and just to build up a picture right so if i was to ask you to send two emails to two separate people probably what you wouldn't do is compose an email send it to user a and then wait for reply that could take however long that might take an hour a day or week and then you would respond so once that's completed you would then start the second task which is composing the second email to send to user b and then the same process again wait for reply and respond so you can see straight away that's a complete waste of your time waiting for a reply before you even do anything else although you might be making lots of money waiting and doing absolutely nothing ultimately you're not going to get much done so you're probably going to tackle the task like this you're going to compose an email you're going to send it to user a and then you're just going to wait for reply and then in the meantime you're going to compose the second email and then send that to user b and then wait and reply whenever you get the reply and then start another task and so on so that seems like a much more efficient way of managing your time let's put this in a context of a program so let's put this idea in the context of a program so far in the python beginners course we've been developing small applications we could say that we've been utilizing a synchronous method in that for example we create a task to can be completed maybe an if statement that completes and then maybe another if consent statement starts or maybe we call a function that function completes and then another task takes over so let's take a look at this example here where we're just simulating the idea of sending and returning email you might be able to see what's going to happen we're going to start this program run the first function we're going to send our first email we'll just print out send first email and then we're going to get a reply but it's not until we get the reply that we can then perform another action in this case this new function and perform some more code here and then send the second email and then get a reply and then so on so here we're working synchronous in a synchronous manner so we're waiting for this function to complete before we can then start this next function so asynchronous or working in an asynchronous way is going to allow us to send the first email and then wait for a reply send the second one while we're waiting and then send the third email while we're waiting and then as and when in whatever order we get a reply from any of these email requests we can then deal with them inside the program so now i've transformed that example into asynchronous code now i'm not going to explain this just yet we're just going to run this and then we're going to work through this step by step working up to this point but you can see here what's happening is that we are printing out or we are sending our first email but now what i'm doing utilizing a synchronous tools i'm now telling python um that okay um i've sent my email and now you can go and do something else so we start another task concurrently in this case we are fire off this next task which will then also send an email and then we can do the same thing again okay now we can send off this email for example now what we're doing here is we're just simulating like it says here that the email takes for example two seconds before we get a reply so we can turn that into any any second so let's do this let's put this as five seconds this is two seconds and this is going to take one second okay so that's how long it's going to be until we get the email back now as soon as we get the email back or the timer goes down to zero it means that we've received an email so we're just simulating this and then the next task is going to happen so we're going to print out first email reply so let me just show this in action so we see here that first of all synchronous using synchronous code we sent and received sent and received so here using asynchronous code let's go ahead and run this you can see straight away that we've sent our first second and third email and notice that you saw that the third email that only took because that took less type that happened first so we received that and then we received the second email and then we received the first email so we can change these numbers around we're just simulating the idea that this has taken a number of seconds in order for the action to take place so here for example um task one is going to complete first uh and then let's do task three is gonna happen at the same time and then task two is then going to finish so let's do it again so we're sending off all our emails and now the first and the third are completed and then we wait a couple more seconds and then the third email is completed so you might be asking yourself well when would i actually use this in a real application well your application needs data that data is going to come from a database for example now that database might be on a network somewhere it might be that there's some processing that needs to take place so you're going to send some data to a server a database server a file server there may be some sort of calculations that need to take place maybe you're doing some machine learning so you're sending some data off and then you expect there to be maybe a little delay so whilst there isn't while there is a debt delay and then use in you sending data to the server for some sort of processing um you can then perform some other actions while you wait for that data to return so this isn't the only type of action where potentially you're just waiting for data to return or waiting for some sort of processing we can also use asynchronous programming for a slightly different context so which we'll move through as we look at some more examples so just before we move on to some examples building up some examples let's just take a look here and notice some differences between an asynchronous function which is commonly called a co routine and a normal function which we're used to and you can see the differences here is that we have this async word and something different here we have this await keyword so these are two kind of key aspects of asynchronous programming in python that we're going to learn a little bit more about let's start by understanding that typically this function here what we might describe as a function normally we now describe this as a co routine so sometimes asynchronous programming in python is called python co-routines so this slide pretty much outlines what i've already said asynchronous function in python is typically called a co routine coroutine is declared with the async await syntax we've seen that and keratins are special functions that return co-routine objects when called so we'll explore that in a short while so first of all let's just introduce async right so the async package this is the package that we're going to utilize or inside of this package all the tools that we're going to need to program asynchronously with python so we're going to bring in different tools and i will introduce some of the functionality that it provides as we move through the examples so we've already seen a number of times now utilizing the async here before the defining our function and as we previously learned now what we've created is generally called a co-routine so what we have here now is our asynchronous co routine now this wasn't always the case uh you may see for example this presented in a slightly different manner with older versions of python so for example from python 3.3 this was generally the way of creating your co-routine bringing in asynco.co routine here and then creating your function as per normal so typically that isn't utilized anymore and it's not considered um the best approach and i believe it's going to be phased out in python 3.10 so currently as of record in this tutorial this is the preferred way of creating your code routines so let's go ahead and call this so send email so let's see if we can run this here so i've just changed the code in slightly now this is a function apologies okay so this is our code routine here um it's called send email so we're going to try and load that up so i'm going to start this and see what happens so notice here that unlike when we create a function for example and run that we print out hello this time that's not the case what we created here like i mentioned before was the codings are special functions that return a co-routine object when called so let's just um let's just say that when we put the async in front of our function here we've created a coroutine object now this object needs to be run we need to tell python to run this object so what we need to do is bring in some tools from asynco right and then we're going to need to utilize run and we just need to then define what we want to run so that's going to be the send email okay routine okay so now we're actually going to run this k routine so it should now print out hello and there we go so like checking for data types we can also check to see um if our function is a co-routine um by utilizing print and then bringing some more tools in from our asynco and then here we're going to utilize is co routine function and then we can just name our function send email so that's going to tell us whether it is a code routine or not sorry yep okay everything will not say true so it is so if i were to change this for example to just a normal function give that a go we now have false so so far we haven't really run any asynchronous code at all and we've just really printed out hello so the second tool that we need to become familiar with probably the most important is a weight so this word here kind of describes what we're doing we're now going to say well wait for this task to complete a wait and then perform an action so the awake command here is not proceeds or starts a task that potentially may take a couple of seconds to complete or may take time to complete and then once it does complete the result of that task can then be returned and then we can perform some sort of action now typically what you'll find in the examples is uh to simulate that we use a sinker's sleep for example so sleep is basically going to provide us a way of simulating some sort of delay so we can then perform some actions and then just showcase some of the features and how to work with this asynchronous code so here we're just going to put it to sleep for two seconds so let's just um put something underneath here uh away let's awake now for example so what we're going to do is print out hello we're then going to simulate the idea that we perform some sort of task that took two seconds and then after we get a return once that task is completed we're then going to perform some other action in this case we're just going to print out awake now so give that a go hello okay a couple of seconds passes and now you can see it returns and it says awake now so just moving away from this idea of getting data from a database or another resource on a network uh and waiting a little bit of time because that may take a bit of time to process etc how else we might want to utilize synchronous code is to make our program um flow in a certain way maybe we want to utilize this to perform certain actions in a certain order for example so let's just take this example here imagine we had a number of different tasks so i'd like to say t1 so we'll just start off here i'll just make a few tasks t1 t2 for task 2 and then task 3 for example so we've broken our down broken down our application into smaller components and we know that potentially we want to perform an action in some of these tasks and then hand over to another action so we can control the flow of our our application right so let's go ahead now and run await okay so and we're going to wait for example in this case t2 all right so what's going to happen here is that we're going to start off by running t1 task 1 and then once task one fires off we'll then start task two and then what we can do is we can then in task two say well actually you can now run task three for example right so let's go ahead and just uh print out task one that let's print out task two here and let's print out task three so let's just give this a go and see what happens here so you can see that we returned task three first then task two then task one so you can clearly see this idea of using a way to hand off to different tasks and this task here would then hand off to this one we then print out this now this is completed so the wait here now um responds because this task has now been completed so we are waiting for this task to complete it's completed so therefore we run the print now we know that this task is now completed so we're waiting for it here remember waiting for that to finish so now we print off t1 so you can see that is the case by the order in which we printed out t3 t2 and t1 so you might be wondering well how the hell would i actually put this into my application when would i actually still use it so it can be a little bit abstract at the beginning utilizing asynchronous code and there is a obviously a lot to this this is a very difficult topic as you move further into this topic but these are the basic principles and hopefully that we've established that hopefully i haven't over belished let's try and establish a more concrete example here um let's go ahead and create an async function here called email reply again right so this could be an email reply let's just let's call this a data reply okay so let's await so um we're going to use for example the sleep again okay so we're going to sleep for four seconds and then we're going to return something so we're going to return how are you doing uh let's let's return some data in actual fact shall we um let's go ahead and return a dictionary okay so data 100 there we go so this here let's just pretend this is somewhere on the on the net on the web and we're now going to send a request to it so that we can perform some sort of action right so that's our server somewhere we're going to send a request to that server it's then going to process now we're just simulating it's going to take four seconds to process and then it's going to return our data that we requested although we're not really requesting it in here of course right so now let's build our task right so um let's call this task task two and let's go ahead and run a a request apologies so we're just going to print first of all um let's print waiting for data okay so we're going to wait for data cool and now what we're going to do is we're going to await so we're going to send our task off to the server which is going to process our data so that's our data reply all right so we send our request off now when the request is finished we're going to find out and then we're going to return this data here into our x variable and then we can print it out right so let's print it out there we go so you can probably already see what's happening here uh let's just run our task it's then going to print out waiting for data we're then going to send a request for our data make some sort of data request we're pretending this is somewhere maybe online so it's a server that processes our data it's going to take four seconds let's just make it two seconds so we'd have to wait so long and then it's going to return our data so let's give that a go waiting for data there we go now we've got our data we can perform some sort of action okay so in the meantime although this setup here doesn't hasn't provided that in the meantime what we could do is we could tell while we're waiting we could tell another task or we could have another task running so let's give that a go let's create a another resource uh let's call this a file reply well so we're going to get a file from the server that's going to say say four seconds and so we're going to return uh let's just return for example oh file file returned there we go right so we've got some sort of service here that we're going to request so let's now go ahead and let's just make task one let's make this into task one and then let's create another task two so task two is going to be waiting for file and then we're going to request our file let's see file reply bring this down okay so we're going to fire off that it's going to take four seconds to respond and then we're going to print that out potentially right so those are our two main functions again we're just pretending that's maybe somewhere on a server somewhere else and we're just pretending this is a function that's actually going to fire up a request for a file or some data so let's now build a a controlling function a function that's going to um set up everything and fire off all of our other co routines so we're going to call this main so let's go ahead now and create this so an async function we're going to call this main okay so now what we're going to do here is use some other tools right so we've got two tools that we could utilize here so we can start both tasks potentially at the at the same time so we could try and run these tasks uh concurrently so let's have a look at this so let's go for x x equals and then we're going to run from a sinco let's bring in the create underscore task right so that's going to allow us to run this task we're going to call this task to run that's task one and what we can now do for example is if we remember it's a e okay so now we can now undo is where we can do the same thing for task two so let's start task two and let's get anything that's returned um back in y so that's now all set up nicely okay so that's going to essentially call these tasks and get them running now what we need to do here is await them because what we want to do is we want to potentially return something from them so let's await them see what they return said um so there's x and then also we can await y okay so let's give this a go and see what happens so we're waiting for the file data waiting for the file you can see that we've returned the data and then we've gone ahead and returned the file okay so you can see how potentially that is working we've fired off or we've started our two tasks um utilizing create task and we've awaited them and then what's happened is our tasks have started they've obviously fired off um our functions here which have a sleep timer just simulating the idea that they take a few seconds to be processed and then when they're ready we then return some information so they the await here is expecting a return and it's then saved into x and then printed out in the terminal so you may be wondering well how does this actually work what's going on behind the scenes when we await so what's actually controlling what's actually managing the weight and actually then responding when there is a response for example from the service so to kind of move into that area there's a concept or the idea of an event loop so as soon as we have a weight what we're doing is we are managing and controlling that task in the event loop so just i like to think about the event loop as a very efficient task manager so we know that we create a co routine that creates an object and then we use a weight to initiate that object so the event loop is we can say is managing that await is managing the for example the request and when there's a response it can then deal with that response and then return it back to our program say in order to perform other actions so it runs asynchronous tasks and callbacks and coordinates the tasks so put it into a picture our application it sends a request or it starts at a co routine the event loop is going to manage that code routine send a request to the server and then we get the reply and then the event loop will deal with that reply and that would go for example back into our application so we can perform some actions so just exploring this code here you may be wondering why do i need to wait because for example i briefly mentioned asyncrate task that's going to initiate a ok routine here so if i remove this code here let's just explore what happens so for example here i'm running and you can see what all this returning is a wait waiting for data and waiting for file so you can clearly see that the create task is firing off our code routine in print but um because we're not waiting because we've taken the await out here we're not waiting for a reply at all so moving that back in and using a weight that ensures that we're waiting for a reply and we get the reply in this case the data obviously we're just printing it out here but it does mean that it then deals with that response how we've set it out here in this program so let's put our new knowledge into action by completing some code challenges so the first code challenge here we're going to create three co-routines routines named t1 t2 and t3 so each co routine should print out the name of the co routine so we're going to call or run the first co routine and using a weight have t2 print out first t1 print out second and then t3 print out last so by means go ahead and have a go and then we'll have a look at the solution so here we have our three coroutines we're going to start off by calling t3 that's going to wait t1 and then that's going to wait t2 and that just runs back in order t2 t1 t3 in terms of the printout here we go okay so this is a little bit long-winded let's build a co routine called fetch data which simulates the collection of data from a network database so similar to a last example that we completed let's assume that um it takes a few seconds it should return a dictionary data 100 and then next build a new co routine which counts down from 10 to 1 using range so we're going to use a weight uh have each number printed to the screen every two seconds okay then finally build a co routine called main and run fetch data and the countdown co routine concurrently so we're going to print the data that is returned from fetch data while also counting down from 10. so we're going to have two co-routines one's going to just count down from 10 every two seconds and then that's going to appear in the terminal obviously 10 9876 so on and then in the meantime we're also collecting data from a data source in this case a network database for example so this is just going to give you an established idea that we're kind of performing activities whilst also performing another activity in the background right then so let's go ahead apologies if that wasn't very understandable uh if you didn't understand that uh that text so we're going to create a new uh code in here called fetch data we're going to print fetch data or fetching data and then we're going to wait so we're just going to simulate the fact it's taken two seconds to process the data and then the date is going to return data returned and then we return some data so we print data returned and then actually send back some data and then that's our fetch data co-routine right so now we're going to create a new current team called task 2 and then we're going to create arrange it so we're going to have it count down and then utilizing a for loop here we're just going to print and down we're going to use a for loop sorry to count down from 10 and we're going to use a weight here right so we're going to put it inside of our for loop so what's happening here is that we're going to print and then we're going to use a weight and of course we can use sleep here which is going to kind of pause that for two seconds isn't it so in this case so then we're gonna then run through the for loop again ten two seconds nine two seconds and so on so that's going to give us that type of action so let's build our main and let's utilize great tasks to fire off our two tasks and then we're going to await x that's going to return some data remember so we're returning from the server some data so this time we're going to um create a a variable and a weight and move the data into a new variable and print it out and then we're going to wait y as well and that's going to count down right so once we've done that let's fire this all off using utilizing async run so fire off our main that's going to then start our tasks um via the awaits here so we're going to wait for them to finish and then deal with anything that is returned so there we go so with that in mind uh let's have a look to see what this looks like so we're counting down now you imagine after two seconds you can see that we've finished this set of tasks here we've returned data and you can see that is being returned after a different amount of time and if we were for example to move that to six obviously it would come deeper inside of this list here so zero so once we get to i think the third number it's going to return some information that would be six seconds and there we go so that was just really trying to simulate and just to establish even further the idea of running two tasks concurrently and producing data from tasks or fetching data from external sources etc so there we have an overview if you like a high-level view of asynchronous programming with python if you are still a little bit confused don't worry maybe have another go the general idea here was for you to understand the basic concepts of or the the fact that we can run multiple tasks at the same time and i think i probably established that too much in this tutorial so i will apologize if i've over embellished that as for normal i hope this tutorial was useful and hopefully i'll see you in the next tutorial if you do have any suggestions or any comments um on how i can improve things or any topics that you would like me to cover please let me know and i'll try and get back to you and fulfill those requests thank you very much and i hope to see you in the next tutorial

Channel: Very Academy

Views: 5,194

Keywords: async python, asyncio, python asyncio, python async await, python await, async await python, await python, python async requests, asyncio wait, python asyncio tutorial, python requests async, python async function, async def python, async in python, asynchronous python, asyncio python tutorial, asyncio tutorial, python asyncio example

Id: 7LU1npoPmcg

Length: 31min 10sec (1870 seconds)

Published: Fri May 28 2021