import asyncio: Learn Python's AsyncIO #1 - The Async Ecosystem

[Music] hi this is Lucas from HDB and I'd like to invite you to a series of short videos where we present some parts of our tech stack in this particular series were introducing async i/o a Python framework enabling asynchronous single threaded concurrent code using cortines we are contributors to Python and it's asynchronous stack in particular as well as use async i/o extensively in the HDB server code base this series is intended as an introduction to async i/o so if you haven't really used it yet that's perfect some knowledge of Python is assumed but don't be afraid to try it out so how how are you gonna do this first we will look into making sure that you understand why we would even want to use async i/o in the first place it is a very interesting technology and it has a very long history in the making so we are gonna use this as a pretext to talk about the history of asynchronous programming in Python and in general so later on we're gonna move to the event loop which is the main building block on of any asynchronous framework or library in any language actually so that episode is going to be about the event loop from the perspective of a user as well as its workings under the hood later on we're gonna move into looking at coroutines which are how you're supposed to use async i/o in the high-level cortines feel kind of you know almost right but not really so we're gonna focus this episode purely on high-level examples from the user perspective so we have enough to go on to understand why this construct is so natural to use however we wouldn't like you to have this magical thinking of how do curtains do their thing so the next episode is going to be about how coroutines actually are built in how they work under the hood this is very important to see in practice so that you actually have understanding of the inner workings of a sink IO that makes it sure that you don't think of it as a magic thing that just happens to work but understand it so you're ready for anything that async IO will throw at you later on I think are actually ships with quite a lot of stuff already included there's ability to write servers and clients and deal with sub processes are all sorts of synchronization primitives and so on and so on so we're gonna go through all of this in a special episode just to let you know about all the things that you don't have to write yourself but come included with async IO but I'm most excited about the next episode which is an example web application we're gonna use this to show you how easy it is to create a production level application essentially from scratch we're gonna be using async IO on starlett which is a very fun web framework to use with async IO and we're going to be using a gdb as the database in the backend so if you are curious about how HTTP looks in practice that episode should answer some of the questions we're gonna be working on a microblog which should be at the same time rather basic enough for a single episode but also to solve an actual problem and have some semblance of complex queries and complex things that you might actually encounter in production applications next we're going to move to a problem that many asynchronous frameworks have which is if everything that we are supposed to call is non-blocking what do you do with all the code that is on pi PI that is built with blocking in mind well we wouldn't like to reinvent the wheel and essentially rewrite the world just to be able to use async IO so there are good ways to enter with the traditional blocking libraries and frameworks with async IO in mind so we're gonna be doing just that but leaving you in this place would be a little dishonest on our end because it's never as easy as it looks on a video on a video we have pre-recorded pre-prepared projects that go from path a to b with little hurdles right but what we want to show you is how you should approach error handling in your code and how you should test your async IL code so you avoid many of the pitfalls that asynchronous as a new concept for you can actually hide and when neither of the tests and error handling techniques that you're gonna use help you avoid bugs we're going to also look into how to debug applications written in async io in production so I hope this sounds interesting to you this is sort of a full-sized 101 course on async IO right there today though we're gonna focus on the very beginning right so the ASIC ecosystem why would you even want to use this to answer this question we're going to be looking at the latency problem so we're gonna be talking about this first and then to actually talk about the difference between concurrency and parallelism how they are related but not quite the same we're going to be talking about traditional ways to do many things at once so threading and in about threading in Python in particular later we're gonna do a sweeping course over at the async i/o and related technologies history both in Python and in general so that you understand that this is not something that was created in a vacuum and finally we're gonna cover some of the other contemporary asynchronous frameworks that you might encounter you know in the wild even if you are looking for a Sakai or related problems you know on Stack Overflow or otherwise on the web so that's the plan this is what we're gonna be working on right now so first latency one of the interesting pieces of insight I've heard in my time at Facebook was that it's the latency that drives the experience not the time it takes to complete an operation that's why Facebook measures and optimizes time to interact which is a metric meaning the time from user action through the appearance of the first interactive subset of the UI that doesn't have to be the entire UI yet just a subset that you can already interact with minimizing this time requires the website to be split into parts that can be loaded independently the most fundamental part for the user to start playing with gets loaded first and then the rest is loaded in the background if you wanted everything to be rendered perfectly and available from the start there would be many more database queries to be made and much more data to send over to you it would take much longer for you to be able to see anything and some experiences like the infinite scroll would be downright impossible so in the screenshot here you can see Facebook with two placeholder posts because the network could not keep up with my scrolling speed I actually did this on purpose just to show you those two placeholder items those are not interactive but they are great suggestions that something is coming up so the user is more likely to wait usually you won't see them because new posts are downloaded in the background long before you reach the current bottom of the page while scrolling but if whatever happens in the background is not fast enough you'll reach the end of the infinite scroll and you will have to wait so now the browser and the script code running on the website must fetch additional content in the background and glue it to the bottom of the page before the user manages to scroll all the way down but this needs to happen skilfully without blocking the user from interacting with the things that are already there if the script code did block it would lead to a very poor user experience you would click on things and they would not react you know you might even see a dialog like the one on the slide so this is a general rule for all user interface frameworks don't block the foreground the user is interacting with it right only do long-running operations in the background this allows people to scroll to cancel to see progress indicators etc while they wait for the long-running operation to complete this is the basic difference between synchronous and asynchronous execution synchronous execution means that code running now blocks the program from running anything else until that piece of code is done if you want to run something else you have to wait the blocking call has to finish first so the entire program like a highway lane is only as fast as the slowest car in it and if the car actually stops and blocks the lane because the driver is busy writing down an address somebody is giving them over the phone then it doesn't matter if you're driving a supercharged v8 Hellcat you have to wait in programs with user interfaces you can get away with doing some work in the foreground but if that works takes too long then the users will say the UI is laggy it does not react to their actions fast enough and when it stops reacting whatsoever that's when you get a beach ball on Mac OS so we should rather do long running tasks asynchronously in other words in the background but what is the background I keep using that term coming back to our highway analogy the easiest way to unblock the flow is to add more lanes those lanes are called execution threads you want to do many things at once in a single program you spawn some more threads the original thread that your program started with is usually called the main thread the other threads are often called secondary threads background or worker threads threads are not free they take up space so they take up space in the operating system schedule and in memory and very often you need a way to synchronize them what what what is synchronization between threads what I mean by this is that threads are not really independent programs they share data within your program so if multiple threads change the same piece of data at the same time they could corrupt it this could crash your program or silently break consistency of your data and in time that would lead to a corrupted data base and sometimes that would have dire consequences let me give you one example let's make a custom thread in Python and run it it won't do much when the thread is created we will give it a mapping then we will start the thread it will iterate over the mapping it got and sleep for the amount of seconds specified in the mappings values don't worry if you haven't used threading in python yet this code reads almost like English right there's a class that inherits from threading thread it has a regular dunder init method where we create a map attribute and now we create a run method that as promised has a for loop that iterates over elements of the dictionary that we assigned from the user on creating on the of the object so let's create a dictionary like this three elements let's create the thread let's start it but you know what it would be cool if the mapping had more elements so let's add one more oops well we see here that the background thread raised an exception while it was iterating over the dictionary it's changed size and iterating over it is no longer safe if you look closely you will notice that the error message was produced by the background thread you can see this because the prompt is broken right it is not actually where the input is now accepted but it's still back there the main thread does not know anything when wrong until it looks at the thread object to check worse yet sometimes when you run your program the timing between the two events is going to be just a bit different and there won't be any error those kinds of errors are called a race conditions and they're very hard to find and hard to debug so how do we avoid threading issues well there are different synchronization primitives just for that purpose let call-sign afford mutex is even object queues and so on the most basic one is a lock so what does a lock do it allows one thread to acquire exclusive access to our resource until it's done then the thread has to release the lock so other threads can acquire it if you try to acquire a lock that is currently held by a different thread you have to wait in our previous example we could put a lock on any access to our shared dictionary and in that way ensure the exception we've seen won't happen again so it's all good are the locks the Silver Bullet sadly no let's imagine a bunch of threads that are running happily ever after let's say there's four of them with nice colors they're never blocked on anything they have 100 CPU usage 100% CPU usage so you are getting all that you paid for in hardware you are very happy but as soon as you have a shared resource that you need to synchronize well things get a little less optimistic and they get a little less easy let's just start with seeing how the situation works is just on one thread so having just a single thread that wants access to the shared resource when that thread is interested in access to it it can probably easily acquire it and as soon as it's done with the processing that it's doing on the shared resource it can now release the lock do something else and live happily ever after until it wants the access again then it can acquire the lock at the same time right do its processing and release it right so that looks rather easy there maybe the only surprising part is the very dark background part well not every time that you want a lock you're gonna receive it right away right there might be another thread that is already doing something with the lock it is already holding it at that particular time so the thread needs to wait which thread is it well let's see an example with two threads just so that we can see how that works then let's assume both of those threads are actually interested in that same resource at the same time the first one was lucky it got the resource and when it's finally done it can release the lock and now finally the second thread can acquire the lock do its processing and when it's done release it in the meantime the first thread tried to acquire the lock waited a bit finally got it and when it's done released it so this is exactly how locks are supposed to work so always good right well yes technically that works but is it perfect is it perfect you know in the sense of usability and resource usage well the more threads you add the less clear that situation gets if you have four threads sudden you will see that even though the first thread happily acquired the lock all the other threads now have to wait and even if it released the lock only one other thread got access to the shared resource now and when that got um released the first thread still acquired the lock and only after it was done the third thread could actually do anything interesting with the shared resource what about the fourth thread it never actually even got to use the resource at all well bad luck for thread so this single example shows two problems that can arise when using just one lock lock contention which happens when there is a shared resource that is wanted very often by many threads most threads will waste time waiting for access to this resource instead of doing other interesting work the other problem is lock starvation in this example we saw that locks are not always fair the first thread got another go at the lock even though threads 3 & 4 were waiting for a longer the last thread never got access to the lock wasting a lot of time and resources quite surprising how complicated the situation gets with just one lock imagine if there's more there's more opportunities for contention there's more opportunities for starvation and don't forget that the locking mechanisms that we use have a price they're not free so the more locks the slower your program gets and the more memory it uses but there's an even more fascinating example of a lock problem when there's more than one the deadlock and while there's many technical ways to explain what a dead log is I guess the simplest one would probably be this you go to a store to get safety gloves but they won't let you in because you're not wearing gloves well I guess that explains it we have many locks all over the place in our real time and real world systems Python in particular has a one very special lock that is you know the subject of many discussions that's the global interpreter lock this is the single lock that the Python interpreter is using to protect crucial shared data structures from corruptions when many threads are used that lock by design allows only one thread to execute Python instructions at a time this is a limitation in the modern world where devices have many CPUs with many cores so a single Python process is unable to utilize those course unless threads use C code which frees the Gil while doing non python computation that's tricky and rather limited but contrary to the popular belief the Gil is what makes Python fast there were many attempts to remove it and replace it with finer grained locks to allow multiple threads executing Python instructions in parallel one serious attempt was made by Larry Hastings around 2016 and 2017 that project called the Galactic has demonstrated that removing the Gil is not very hard but the resulting performance is much worse and deeper changes in Python would be necessary to alleviate this so while I just said that the Gil is what makes Python fast it is not without problems the limitation of running Python instructions only in a single thread at a time is initial look at this example it's a simplified timeline from a real application running and production at Facebook that I debug a few years back the observable problem here is very weird we have an application with a main thread blue another simple worker thread green we know this after running it for a while the tasks in both those threads are running increasingly rarely it's like we're dealing with air quotes CPU leaks if those were a thing log still has nothing unit tests don't show us this behavior even when stress testing so what is going on turns out there was a third thread whose purpose was ironically gathering performance metrics that third thread was running Python code that was recently changed to do some pre-processing to minimize Network data usage unfortunately the algorithm chosen for the pre-processing was naive and in effect the pre-processing took increasingly more time each run leaving less and less time for the actual business-critical code to run so don't get me wrong the Gil is a problem since running only a single thread at the time with Python code severely made the situation worse than it would have been if all of those threads could happily run their Python code but also those kinds of issues are rather tricky to find and debug sometimes they're like actually trivial you know after you've seen them but finding those in the first place it's not easy at all so this is the world that we live in programming with threads is hard to get right but even if you were willing to risk it Python itself has to lock access to its internal data structures and by doing so causes threads to be underutilized but you know what else makes threads underutilized waiting for external resources also known as input/output or i/o for short so here you can see Firefox loading the front page of HD be calm there's quite a few files to download and you can see that they are requested in batches the green part is the actual data transfer the blue is waiting while the orange is the initial dominating resolution server connection TLS handshake but there's so much waiting what if python could use that time to run more python code while the external resources are not ready yet that's the idea behind async io so the goal of async io is to maximize the usage of a single thread by handling io asynchronously and by enabling concurrent code using coroutines we're going to gradually learn how all of this works but let's try to understand it on a very high level right now so I think IO enables single threaded programs to be more productive by filling the gaps that would otherwise be wasted on waiting on IO to do this efficiently async IO avoids blocking functions remember the slow car clogging the highway lane and instead uses co-routines which can be executed in small chunks concurrently for example concurrency here is dealing with many things at the same time while being able to do only one thing at a time instead of waiting for the expensive database fetch to complete let's do another one in the meantime an update the activity log this shortens the TTI remember better yet since most of the fetch from database two and update activity log is waiting anyway we can handle many customer requests at the same time it that one simple thread so the graph shouldn't look like this it should really look more like this on a very high level you can pretty much see that this one thread is suddenly doing way more things and on very i/o heavy workloads asynchronous i/o can be orders of magnitude faster than blocking i/o this is why so many important web technologies are built using it like nginx Redis node.js browsers if this graph looks confusing to you there's a popular analogy that I guess explains it well one bartender can prepare one beverage at a time but he can look after several customers at the same time that's concurrency without parallelism for parallelism you would need multiple bartenders so can you achieve parallelism in Python yes of course you can use multiple processes you can either spawn them manually in your operating system use a Python library like multi-processing or use a dedicated process manager like system D supervisor circuits and so on or finally you can even deploy a full-blown service with multiple load balanced tasks using kubernetes or Amazon for gate and so on so there's many ways to deal with multiple cores using Python today and this is actually an old and battle-tested idea async i/o wasn't created in a vacuum far from it in fact the history of asynchronous i/o goes all the way back to eight in 1983 that's when the 4.2 BSD release added an early implementation of the tcp/ip stack at the time there were no powerful inter process communication mechanisms yet in BSD so if a networking program wanted to share server stack between multiple clients accepting a connection and forking was not an option so the Select cisco was introduced to allow multi applications to multiplex their io we're not go through the man page of it even though we could we could look at you know some C code and try to replicate it and whatnot but I don't really think this is productive now but what I would like to show you instead is a good basic example of why select is needed and this is our log in a remote login application something like SSH but very basic so what does it do it's a program that allows you to interact with a remote server just as if you were working with a local terminal so it has to deal with a bunch of activities that it's typically does right you will want to read the user's keyboard input then write that input to the remote server wait for data to come from the server when it comes read it and then finally write that server data to the user screen so they can see it this all sounds basic and it is but there's one problem traditional api's would block waiting for data so if a program wanted to read key presses from the keyboard they would call the read function and that function would wait until the user actually typed something the program couldn't do anything else in that time similarly for reading data from the remote server connection calling the read function would block until the server had anything to send us so the program couldn't do anything else in that time the simplest solution for this would be to put timeouts on the blocking calls so the application can continue to try the other thing but this would cause the entire interaction to be very laggy making the remote logging application very frustrating to use instead non-blocking i/o was introduced making calls to read returned immediately when there was no data but this was not enough when you check for new data coming from the keyboard and coming from the remote server you need to write a loop that will make the non-blocking read calls but since the non-working calls don't work at all on no data the program would run hot with 100 CPU usage when nothing is happening well like the donkey in Shrek it would just keep asking are we there yet are we there yet are we there yet millions of times that is a tremendous waste of CPU power again a knife solution could be to make the program sleep between each loop iteration but this just puts us back at square one either you waste CPU time or you introduce lag so instead the Select Siskel was introduced it allows the program to list a bunch of sockets at the same time and ask select which ones have new data waiting on them if no sockets have data the Select call would block with a timeout that timeout means that when you are unknown you know unblocked after a Cisco even if there's no sockets waiting for you you can still do some non-essential bookkeeping in the meantime in the end in the programs architecture there still a loop waiting for socket events an event loop that is exactly what an event loop is we will be talking in detail about the async i/o event loop in the next episode now let's quickly go through a few pivotal moments in the history of Python related to a subpoenas IO that'll help us understand which Giants provide shoulders for the async i/o library to stand on Python provided access to the Select Cisco and its equivalents in non BSD operating systems from its earliest days however recreating an event loop in your application from scratch every time well that's both cumbersome and error-prone and there were many typical problems that found patterns of solutions and you would like to reuse them in the next server in the next kind your writing so in the mid-1990s Samuel rushing created a framework called Medusa which pioneered event-driven asynchronous networking in Python over the years among many others this framework was used to power the initial version of the Google web server the Google web crawler the zope web server as well as Yahoo's SMTP engine most interestingly Sam carved the core asynchronous bits from Medusa as a separate library and offered it for free on comp Lang Python News Group under the name Aysen core guido van rossum included it in the standard library with the second beta of Python 152 in February 1999 it's still there in Python 3.8 however its deprecated so don't use it for new things it's there for backwards compatibility but you're welcome to just have a look at how it works so that's one approach to asynchronous i/o around that same time in 2000 lyft started the twisted project that was a different spin on the same event club idea it was organized around callbacks known as the ferns in the twisted projects terminology you need to understand that this was before pep 8 this was before guido van rossum moved to the u.s. python didn't even have lexically nested scopes yet there was no unit test in Python twisted over the years gathered an immense following and a collection of included batteries with numerous non-blocking asynchronous Network client and network server implementations usually the FTP there it was you needed to connect to a database there it was you needed to write a web app he was there so some of the biggest Internet companies on the planet including Apple Google and Facebook used twisted based services for high volume applications like DNS clients like load balancing like calendar servers and so on unfortunately the requirement for all API stouby non-blocking and organized around the third cause that project and its community to grow separately from the general Python the general pattern world kept writing libraries and frameworks using blocking api's instead around the same time christian tessmer started the stackless python project so the original idea involves running micro threads also known as green threads or user threads to achieve cooperative multi asking in the same operating system thread that was a powerful idea more so than straightforward event loop based networking because it allowed multiplexing general computation as well your programs didn't only handle multiple networking clients all at the same time they could also execute many long-running background tasks in Python all in the same thread now every real operating system thread allocates a full sized call stack its overhead so you cannot run hundreds let alone hundreds of thousands of them so stackless python makes Python function calls happen without an operating system level call stack overcoming this limitation it also allowed one of the most mind-bending constructs I have ever seen in my life micro threads could be pickled so you could literally save the state of a green thread and put it in a database and retrieve it you know awhile later to resume the work as if never happened as if the you know our pickling never happened so that is pretty cool and it still kind of boggles the mind of the possibilities you could you could do it this and in fact I know of two major adopters of testiclees Python the Icelandic massive online multiplayer game company CCP creators of EVE Online and ironport an internet hardware appliance company using it for their email and firewall devices EVE Online in particular used green threads in massive numbers to model real-time events in their anonymous universe so that's pretty cool does that mean that stackless python god merged in mainline Python sadly no the implementation of stakus python required deep cutting changes in the interpreter in my conversations with christian and guido I gathered that ultimately that was the reason why it was never merged into mainline Python parts required hand written assembly code which was not portable the design also made most C extensions incompatible and Python was increasingly used in 10 with c4 interacting with operating systems databases imaging in these days scientific libraries so it's kind of bittersweet to see this pap as forever deferred there's a companion pep to 20 with motivation around co-routines and generators which was quickly rejected by Guido which is kind of hilarious for you know if you think about the things to come stackless python never received large adoption but its legacy lives on the green threads implementation was extracted and enhanced by Armen rego in the form of the green light library and that was the basis of the G event library which combined green lets with ela VV a sea based event loop so G event promised a Vantage driven non-blocking networking that still looked like regular synchronous Python programs how did they achieve this well there was a bunch of monkey patches that you could put two replaced and library blocking calls with non blocking equivalent so that idea gained some traction but ultimately it also fate failed to gain mainstream adoption because of tricky debugging as one long-running blocking thing could invisibly slow the entire application down and also the monkey patch that it was doing wasn't really playing well with some of the third-party Python libraries which relied on the behavior of the blocking standard library calls so G even still actively maintained the massively popular G unicorn HTTP server for Python started as a G event focused project another synchronous framework that was created around that time is tornado open sourced in 2009 by Facebook as part of their acquisition of FriendFeed that introduced cooperative multitasking originally through well careful use of special non-working methods you were supposed to on subclass some tornado class and you could run call those methods looked like code that you read from top to bottom but in fact those methods were kind of exiting early and then tornado did all of its work for you in the background you could also keep the request handler optionally alive through use of a magic decorator so that was an improvement over the typical callback based approaches of the day but still nothing close to what we were using today but today tornado is still actively maintained and can use native carotenes so that integrates with a single rather well and it's still a rather lively project so how did we get a sync io in python well through a bunch of peps and very hard work let's just look at this from the very beginning back in the early 2000s even the widow was initially reluctant to the idea of cooperative multitasking remember the fate of pet 220 well even though he didn't like generators and continuations over the years a number of pipes got written and accepted that ultimately led to the creation of async io let's go on a journey through some of those important types while we do it take note of how much thought effort communication and cross-pollination of ideas have to occur for async IO to get to where it is today and some people change their minds while doing it so that is also pretty cool to see we're starting in the iterator spam which made for loops in Python much more powerful by allowing arbitrary objects to provide the next value to a loop basing on that we have simple generators which introduced a simple form of the yield statement at the time and Tim Peters as he does already noticed right from the start that hey since a generator frame is kept alive as long as the generator is being consumed it is a good foundation for quarantine support via mechanism called a trampoline if you just got confused and you like what's a trampoline don't worry we'll discuss this closer in the fourth episode Tim was right but generators as described in pep 255 had to be extended to allow for sharing data with generators and a bit more so there were a few attempts at enhancing generators one by Raymond heading her another one by Samuel Pedroni finally Guido and Phillip GEB introduced enhanced generators which added support for sending information into a running generator you could also inject an exception to be thrown inside a running generator as well as yield became an expression which made it way more usable and also you could use it within try blocks which wasn't possible before so a good few years has passed but between a few inspiring show-and-tell conference talks it seems like nobody actually made use of those curtains in a successful framework as far as I can tell why not Greg Ewing demonstrated that part of the problem was that when a generator consumes another generator to immediately yield values from it the correct handling of that is not as easy as a two line four loop even that two line four loop would be very cumbersome and ugly to write and look at if you just needed to use it all over the place to achieve cooperative multitasking so he wrote pep 380 and that introduced the yield from expression which renewed the interest in co-routines built on generators in fact greg himself had an idea how to do this with a proposal introducing the koko keyword that was ultimately rejected at the same time Laurens van houten worked on bringing a better base a synchronous i/o framework to the stand library as he noticed people who wanted to write a synchronous code in Python at the time either were faced with a sink or or they had to roll out something handwritten most likely based on the select module from 1983 write or use a third-party library like twisted orgy event and those weren't really compatible with each other or the stunning library to a large extent so nobody was happy so interestingly in the pep Lorenz rejected the idea of using generator based coroutines but ultimately the pep was not finished its reboot though by Guido van Rossum with the help of twisted glyph and tornadoes Ben Darnell and several others that actually succeeded it designed the async i/o module which used the yield from expression for pure Python coroutines it also included an extensive array of built-in batteries that was from day one so was very easy to start building real-world applications with it we'll be discussing some of them in a future episode in fact just you know in the initial Python 3.4 release of it like you know we could already be using this you know to a large extent at Facebook and there was an application that orchestrated cash invalidations or running on all my sequel databases at Facebook successfully for two and a half years written with that version of a scenario to start with so even the bleeding edge early adopter version could pull off some rather impressive things later on the more we wrote async i/o the more we noticed that using yield from is rather error-prone you could maybe forget the magic decorator that you needed to put over each core Edina or maybe you wrote yield instead of yield from or maybe you missed yield from all together and all of this you know was a rather bad experience and if you actually wanted to use a regular generator you could by mistake put it in a sink i/o to great havoc so Yuri Sullivan of the co-founder of HDB wrote pet for 9 - which you know got inspired you know from other languages like dotnet c-sharp to introduce coroutines as a standalone C interpreter level concept as well as async await keywords for easier operation and as well to differentiate between the coroutine and the generator but having already all this and having new keywords we could also have asynchronous iterators now and asynchronous context managers all in all this was a massive productivity boost and a massive usability boost for xnk all right then right there however isn't kayo and async/await were still rapidly evolving Yuri Sullivan have added a synchronous generators as one of the juiciest examples of how Python sometimes goes like full circle initially async ioco routines were actually just regular generators but now that they were native constructs in the language they could also function as generators in the asynchronous context so that was pretty cool to see you know that kind kind of the language grows in both weights in both you know kind of in all sorts of dimensions cool so having this you could already think now that we have generators and now that we have a sync iteration it would be cool to have a synchronous comprehensions so now our weight expressions are allowed in lists set dict comprehensions and generator expressions that was pretty cool but again more real-world experience with async IO demonstrated a problem that problem was this sometimes programs require local storage for computation like that local storage shouldn't be shared across workers because for example it's just local to a single web request so when each worker back in the day used to just live in its own process and only process to one item at a time oh that's easy every global variable is effectively only access to while processing one item at a time so there is no opportunity for to data corruption so all is good but when each worker lives in its own operating system thread you can no longer use global variables this is why many operating systems support a mechanism called thread-local storage which essentially is global variables that are only seen by the current thread so you could keep data around which is specifically not shared with other threads however with asynchronous i/o the entire point is to multiplex and process multiple items at the same time then one operating system thread so global variables are dangerous but thread-local storage is also dangerous so what to do the typical answer back in the day of early a sink I always yeah it's cumbersome but it's safe to always pass a context dictionary down to every color teen that's a weed and on so this makes calling coroutines ugly and it's easy to forget to do it sometimes or to pass the wrong context and sometimes you're using third-party libraries which don't know about your you know kind of convention so now you actually lost the track you you cannot pass the context that you wanted so the execution context pep was an attempt to solve this problem by introducing generic non-local state for purposes of out of order execution and that was authored by both of the cofounders of HDB in the end the complexity was overwhelming so that was withdrawn in favor of context variables which are a streamlined simplified version of context introduced into at 5.50 async IO is still being improved and future versions of Python are for sure gonna include new features and refined api's some internals of HDB are in fact a playground for this kind of research and development but there's more research and development happening in the community as well let me give you two examples curio was created by David Beasley an experienced and respected Python developer it rethinks concurrent python systems programming using the current state of the art native coroutines a synchronous generators context variables and so on so it's the answer to the question what could a sync i/o look like if it were created from scratch with Python 3.7 now that is actually very interesting interesting enough that Nathaniel G Smith authored trio a similar concept in fact directly inspired by curio so Nathaniel is mostly focused on the usability aspect and also on the correctness aspect so one innovation of trio is the concept of nurseries or task groups this is yet to be integrated with a sink I own so that's it for now like you know that was the main kind of world view of the async ecosystem in Python with some history for you and some fundamental knowledge of why this is a cool piece of technology I hope I convinced you to actually look at this so soon enough you're gonna have more videos on our youtube channel and the very next one is going to be about the event loop as I already mentioned we will be looking at it from the high-level user perspective but we are not going to be afraid to look under the hood to see how this particular thing is in fact implemented so thanks for your attention see you on the channel [Music]

Channel: EdgeDB

Views: 35,575

Rating: 4.9593024 out of 5

Keywords: python, asyncio, edgedb

Id: Xbl7XjFYsN4

Length: 49min 32sec (2972 seconds)

Published: Tue Apr 14 2020