Crust of Rust: async/await

hi everyone welcome back it's been uh quite some time until since the the last stream and uh it's because i've been on vacation which has been really nice um it was a an odd vacation but but that's a story for a different time a q a or something um so uh this is this is going to be another crust of rust um so it's going to be one of the shorter streams um and in particular what i wanted to tackle this time around was async and weight um async await is something that like has i've been asked about it a lot uh i have a stream on async await i'll link it like up here in the in the final video um but but that goes a lot more into the technical detail of like how does the machinery work and how do the traits work and the the like bits underneath but it doesn't really talk about just like how should you think about async await how should you use async await what does it use look like for you as an end user uh or or as a developer rather um and so i wanted to take a crust of rus sort of a step back in complexity and just look at the the mechanism the but from like a usability standpoint and just talk about more sort of the intuition and the mental model you should have more so than the the nitty-gritty details of how it actually functions for that you can see the other video in case you were unaware i'm john i have a twitter account if you want to know about upcoming videos you can go follow me there um i'm also writing a book um and async is actually one of the reasons why i'm writing this book which is the original rust book the rust programming language which is fantastic uh doesn't cover async like there is no chapter on async and i think they are going to add one um but one of the challenges has been the async ecosystem has been so much in flux it's been hard to actually write one um i do cover async in my book um there i cover it more in terms of like the lower level details because this book really is for those who want to take it take sort of a step beyond the basics and understand how stuff actually works in rust at sort of a deeper level um but go check that out if you're interested in you know rust which uh you probably are given that you're watching this um also read the rust book if you haven't read the rost book it's fantastic um now there have been some efforts at sort of explaining acing programming and rust um there is the rust async book which still has a lot of to do's um it was written a little while ago it explains things in at least to me a little bit of an odd order i think there are some efforts underway to sort of re-jig it um to make it easier to follow and and sort of go through a more maybe pragmatic approach of how should you think about async await before diving into the details of how it works um uh it might be that by the time you watch this video like much later this book is the place to go um but i thought for now a video might be a good way to sort of um fill out some of the things that it doesn't really talk about i also highly recommend you take a look at mini redis so mini redis is a little project put out by the tokyo project which is a re-implementation of a redis client and server it's incomplete it's sort of a toy example but but it it tries really hard to show you how to structure an asynchronous application client-side and server-side and and it's really well documented sort of written to be a piece of code that you can read to understand what's going on and to understand some of the design principles and thinking that goes into it um i highly recommend you give that a read um if you get a chance um now i will also say that here let me pull this up for a sec um i will say also that while there are a lot of different executors out there for asynchronous code i will generally be focusing on tokyo just because it's the most widely used one it's the most sort of um [Music] most actively maintained one um that now most of what i'll be talking about will be independent of the executor i i'm not going to go into sort of tokyo specifics here because it's not really relevant to how you think about async code i just want to sort of lead with that as a sort of a a flag that that i'll generally be having that in mind um let's see oh no it's got the wrong video title that's all right we'll we'll survive um actually i can fix that right now let me fix that right now give me one second that's annoying but i can fix it update titles update now this should not say that it should say okay let's see in theory now the title should be uh should be something semi-reasonable um okay so let's start out with something that um is code right like i i like code we like code uh we're going to call this uh patience patience and we're making a let's make it a bin patience right because you need to await it get it it's pretty pretty clever pretty clever um so let's go in here and uh look at your main function all right so we'll leave the main function as it is um let's say that i write async even all right so the sort of simplest async function you can write all right that's let's allow dead code that's fine um like so so we have an async fnfu what does that mean right well what what is what is this async keyword what does it do well first and foremost the async decorator on functions uh just really means this um output equals basic this these are the same now i guess use standard future future so if i have an async fn through one let's make this food too these two are equivalent they do the same thing in fact this basically gets turned into this um so the async keyword on a function is not special it's just a sort of transformation directive uh to the compiler um that there are some differences but we'll get to those in a second now notice that this um this output variable for the future is the same as the return type of the function so if this was a u size and it returned say zero um then this would be output u size and async 0. um and what this syntax means um so we're returning some type we're not going to name it but some type that implements the future trait and the future trait means uh it sort of signifies a value that's not ready yet but it will eventually be a u size so in this case um the for the async block we just return zero immediately um but this thing that we give back is sort of from the javascript terminology i promise that i will eventually give you a use size i'm not going to tell you when it's just at some point in the future this will resolve into a u-size and in fact if i hear say let x is for one what happens now if i don't actually care about this let's go with unused variables too so if i now create a variable x that is the result of calling foo one or fou two they're equivalent right then x now is not a u size right this would not compile and in fact the compiler is pretty helpful here right it it tells me consider awaiting on the future and what it's trying to tell you is that foo one is not a you size what fu-1 returns is the thing that will turn into a u-size eventually and it's telling me i need to await and what a weight means is don't run the following lists of of instructions until this actually resolved into its output type that's all a weight really means um what might surprise you though is if i hear let's say at a print line foo right which which would be equivalent to doing this uh print line foo these are the same then this will never actually print foo and i can show you that um patience cargo run so prince hello world but it does not print foo and this might seem counterintuitive right like we called foo one the first thing in through one is printfu so why doesn't it print foo right like this is a future sure but this code is right here there's no await in here um the reason for this is because a future does nothing until it is awaited um and in particular here um the future just describes a series of steps that will be executed at some point in the future and we'll get back to exactly how those get executed so here the first time this future is awaited then it will print line foo and then it will immediately return 0 and resolve now there might be other cases here so for example imagine that in here we called food 1.08 right and then we print lighten see food 1 food 2. right so here you can imagine that if we here do food 2 we await the future that's returned it's going to start running because we're awaiting it uh then it's going to print one then it's going to call foo one which is going to produce some other future and then it's going to wait foo one and it could be in this case foo one will return immediately because it resolves to zero it doesn't do any like network operations or anything that might have to wait um but imagine that it did imagine that this was like read to string of some file right then that reading of that file might actually take a while it might not be ready to yield the bytes or whatever that it read or the string that i read immediately as what will happen is that the program will sort of wait here because we told it to right we told it to wait until the result of this future is is ready and so it will print through one and then it won't print anything for a while because it won't print until this has resolved if on the other hand we did this right then it would print future straight away because this would just return a future but we're not telling it to await anything so no actual work happens and therefore the the way that you can think about async blocks is really that they execute in sort of chunks um so let's say let's just sort of copy paste this a bunch um and say let's do something like this and this is gonna be file one file two file three file four right um you can sort of think of this as there's sort of a first time it executes until here wait here and then second time and when i say time here what i mean is um when this has completed so it'll start executing here it'll run all the way until here and then it won't do anything it'll sort of yield back you can think of it as sort of doing a um a standard thread yield now sort of in a loop so it'll do something like in fact let's try to write out an example of what this does so fute is going to be this while not feud is ready yield now and then food dot try complete you can sort of read it like this that uh it's gonna check whether that future is done and if it's done then i guess here like a result is few dot take result or something right like it um so while it's not ready it's just gonna sort of spin and yield in practice is not what happens but you can have that as sort of the mental model that when while it's not ready it lets other things run and then every now and again um it's going to check whether the future has made progress towards completing and making progress for a future sort of means getting to the next await point so when a future does get to run it'll continue executing from the last point it yielded until the next point where it might fail to make progress so so um let's say that let me return this to what it was um which was this so let's say that this uh future has sort of gotten to run a few times uh and right now it's sort of uh waiting on this right that's how far it got last time and now let's say that the read of file 3 finishes so now that the read of file 3 finishes this await resolves right so you can think of like we exit the while loop that's yielding to other people and then we get to execute and let's say here that we had like a expensive function right um in fact let x and then we're gonna pass x to expensive function so because there's no awaiting here once this await resolves we get to run and we get to run sort of imagining that there's no async we just execute instructions which include potentially calling all these expensive functions we might call we might do all sorts of things and we're not going to get interrupted other things aren't going to run in our place although at this point we're in the standard threading model right of our thread might get interrupted and some other thread might get to run but apart from that there's no there's no magic related to async at this point we're just going to keep executing as if we were a normal function all the way until we create another future and then await that future at that point we're going to sort of yield again and wait on that to complete so once this awaited we got to execute this chunk of code which contained no awaits and we execute all the way until the next wait point all right does this basic sort of async lets you chunk computation does that the rough idea or this rough structure here makes sense before we continue um there are definitely differences between javascript promises and this i'm not saying they're the same i'm more saying that in terms of the the naming it's a useful uh comparison that a future can also be thought of as a promise not in terms of the actual javascript type promise but in terms of the word promise all right so you can think of an async function as or or any async block in fact anything that is a future as executing in chunks it runs until it has to wait basically until it cannot make progress anymore and then it yields which is effectively what this await turns into so so i gave you one de-sugaring of a weight another de-sugaring of weight is something like uh let few let's say we write let x is a read to string file 0.08 you can sort of think of that as being rejiggered into into creating the future while it's not ready um we're gonna do a um or maybe even while trying to find the a nice rustic way to sort of say that this works uh so we're gonna loop and if we're gonna say if let's some result is few dot try check completed then break with result otherwise we're going to feud dot try make progress and then yield where yield you can sort of think of as a as a thread yield now but but it has this additional property that you actually end up returning all the way up to where this future was first awaited like think of it as there's sort of a stack of things that are awaiting each other right you have um uh right so here main waits for food too future waits for read to string let's say read to string waits on foo one so you can sort of think of this like a call chain of things that are awaiting each other um and whenever you yield you actually return all the way to the top of that call stack and sort of return to there but the next time something calls like try check completed or try to make progress it continues from the yield point of sort of the bottom-most thing that previously called yield so this is sort of the way to think about a weight that that it um it is a loop that's yielding whenever it can't make progress does that roughly make sense so you might wonder okay why do we do this like why is this interesting um so the reason this is interesting is because imagine that you have many of these right you have a bunch of futures so you might for example have um one future for reading from the terminal right like you're waiting on the user to to write something into the terminal um and you're also waiting on let's say um a new connection coming in over the network so you have two futures and you don't really control when either of them will resolve right you don't control when the user types something into the terminal and you don't control when some external program is going to connect so you sort of want to wait on both of them and you don't really care which one happens first well if you were in a threading system what you would have to do is you would sort of do let's say how would we do this so we're going to have a read from terminal and we would like spawn a thread um which would like lock um io standard in lock um and then we would like uh for line in x dot lines and then we would do something on user input and then we would separately have a thread that's like read from network uh which would do thread spawn um and it would say mutex is standard net tcp listener bind and it would do you know bind on some port and then it will while let stream is x dot accept this has to unwrap um and then do something on stream and now this isn't too bad why does this require an argument standard standard in um yeah yeah i don't actually care about the compiler errors here so so this isn't too bad right we have one thread for handling terminal reads and one for accepting connections but it gets worse if we have to have one thread for every operation that we have to do if only not even thinking about performance but it just gets annoying to wrap your head around that now okay let's say that for each stream we need to call like handle connection right so we want to handle connection of the stream but now we have a single thread that's managing all of our connections and imagine like a bunch of users are connecting to download some large files or something you'd really like to be able to use many threads or at least if like if one connection is full you want to be able to write on the other and that gets really weird if there's one thread because at some point like handle connection has to call like stream.write and imagine one of the clients is really slow so you're not going to be sort of stuck writing to the slowest client even though all these fast clients you're able to write to so so we maybe then need to like spawn a thread for each stream right they're gonna handle the connection for that stream and now we get like a thread handle back we need to do something with that so that we remember to wait on the thread if we're trying to shut down the server now we have all of these threads running and like it's possible to write programs this way and and many big programs are written this way but but it feels weird somehow right um what i'm going to propose to you with the async model is you can actually make this simpler the async model lets you model this in a way that matches your thinking better so let's say that there's a read from network not not these um not these variables i declared up here but some function right so i'm going to call that network and then there's going to be a read from terminal right and let's say these are both futures then now what i can do is i can say i can do select and we'll talk about what select means and then i'm going to say stream is and there are lots of different syntaxes for here i'm just going to sort of make one up network dot await and um line is terminal 0.08 so this the the reason why do something with line here the the select uh macro which exists in a bunch of different libraries like there's one in tokyo there's one in the futures library what it does is it waits on multiple futures and it tells you whichever one finished first so under the hood what really happens here is that it tr um i'm not gonna define what it is yet but the code um the select macro if you will is going to try sort of remember how we wrote the de-sugaring down here of try check completed and try to make progress right it's going to try to make progress on network and if it does make progress on network then it's going to give you a stream and call the code that's in here if it doesn't make progress on network then it's going to try to make progress on the terminal and if it makes progress on the terminal like if if the terminal now has new lines then it's going to run this code with the line that it got if neither of them make progress then it yields and then at some point in the future it's going to retry right and and try to check progress again and in practice it's actually a lot smarter than this what really happens under the hood here is that when you yield you don't just say yield you say yield until this thing happens so for a network socket for example what you yield is yield until something happens on this network socket on this file descriptor on the terminal you say yield until something happens on this um this input channel or this in this case standard in right and behind the scenes whatever is running the future and sort of has this loop of um of try check completed and try to make progress and such what it's actually going to do is use some operating system primitive under the hood to be um smarter about when it tries when it retries so it's not actually doing like a spinny loop or something it just goes i'm going to try this i'm going to try this if neither on the succeed i'm going to sort of go to sleep and then retry once the operating system has notified me that it's worth retrying and it can be even smarter than this right it can even realize that oh nothing has changed about this network socket so i don't even have to try to make progress on this but the terminal state did change so i'm going to retry this operation okay so so that's one of the reasons why futures is handy is because you have this you have this nice way of trying multiple things at once because a future is not code that executes all at once it is this chunked operation that also sort of describes how to try again later which makes it really nice for doing any kind of programming where you have primarily io whether that's networking or disk um or even timers like anywhere where it's not just like spinning on computation there async doesn't really add that much but if you ever have reason for some part of your code to wait for something else to happen then what async allows you to do is sort of give up the thread's time like if you have a thing that needs to read from read from disk while it's waiting on the disk async allows something else to run instead right and you can see that here while the network is not does not have anything for us to do we can read from the terminal instead if neither of them have anything to do then we can sort of go to sleep we don't have to do anything and that's what async sort of allows you to um to express very nicely the other way to think about async await is sort of as a big state machine right of you're in the current state of the future and there are multiple paths forward one path forward is that something becomes available on the network socket another path forward is something becomes available on the terminal socket and then you sort of follow the appropriate edge in the state machine and then you run the code in the next one now in the case here right where we don't really have a branching so select you can think of the select operation as allowing you to branch the execution here there's no real branching right here there's just we're just saying if we can't do anything here we just await um and if so we just yield right and then we're going to continue running from here whenever it's possible for us to make progress so it's really the state machine in this case is we're either in the state of being here before let's make before reading file 1 or we're in the state of before reading file 2 or when the state of reading before file 3 and the edges between these are file 1 has been read file 2 has been read file 3 has been read and these compose really nicely right so here um let me see if i can explain let's let's add another thing here uh let me tidy this up a little because there's a lot of it's a lot of annoying errors in here [Music] and then i'm gonna do async then read to string like so just to make it stop yelling at me expensive function um takes nothing great it's yelling at me less now so let me get rid of these as well um and say these are fine i can leave those in so let's say that we also here said um foo which is fu2.08 so now when food to when we call future right we get we get something that doesn't execute straight away it doesn't execute until we await it and now imagine that the network has nothing to do the terminal has nothing to do so future gets awaited okay so it gets awaited which means that it's gonna run until it can't run anymore so it runs through one then it runs read to string one of file one which is a disk operation so it has to wait so it yields so at this point there's another opportunity for the select macro to run either of these other two branches if there's still nothing on the network and still nothing on the terminal then it might be that the read of file 1 is completed in which case it's going to continue from the previous yield point of foo 2 which is this await so it runs this and then it reads file 2 and then yields again and when it yields control flows back up the stack in this case to the select and the select gets to retry these again right so so this this um await sort of being a return is what enables this whole mechanism to work is that every await is a opportunity for whoever is above you to choose to do something else and it could be that they don't decide to do anything else and they let you run instead it could be that they let some other things happen and then continue to let you run right so so here it could be that um well this is maybe a bad example but let's do uh foo is food too so we're going to store the variable out here food.weight let me make this a loop right so imagine now that uh foo gets run for a bit it reads file one it reads file two it's reading file three but it yields and at that point the network has a new stream so we execute this so whatever happens in here happens gets run and we don't even check on foo again foo doesn't get to do anything more because it's not being pulled it doesn't get to run in the background it's not its own thread it's cooperatively scheduled it relies on the the sort of parent of it which is the select to actually continue awaiting it right so food doesn't get to run on its own it's whoever has the foo is responsible for actually awaiting it again so in this case if a new stream comes back and we eventually finish running this block of code at that point we exit the select we go around the loop again and then select happens again are there any new streams now let's say no there are no lines from the terminal so now we continue awaiting foo so now maybe file three finished reading and foo awaits again but this time i'm reading file four so it gets to make progress right and then maybe there's a line from the terminal we go around the loop again and and so on and so on so so the way to think about this is really this this idea of cooperative scheduling where if i don't run i'm going to let whoever is above me decide who runs next and it might not be me right and so this brings us into the topic of cancellation if you don't want the future to keep doing what it's doing you can just drop it right the the trick is if you don't await it it doesn't get to do anything awaiting it is what drives its progress now you don't really have a way in in sort of straight line code like this to cancel right because once you call dot await you don't have any control of execution anymore what you've told rust is i don't want to keep running until you have the promised value from this future and when that happens run the next line so you don't have really have a way to cancel here but if you wanted cancellation of this operation what you would do is something like select um let's say done is this um or cancel is going to be let's say uh what's a good example of this let's say there's a cancellation here there's a cancellation channel which is like a tokyo sync mpsc receiver so cancel is going to be cancel.08 so here we're just going to sort of continue or fall through to print line below if we get a if so you can another name that was proposed for select uh by without boats i think um is race and that is a good way to think about this we're sort of racing these two against each other and whichever completes first whichever await can produce its result first gets to run and the other one does not it just sort of gets cut short at this point unless you sort of unless you're looping and trying more so in this case if we get a weight then we're going to return or something in fact you could sort of imagine that this is like a result instead right but in this case let's just say that in that case we just return zero and we don't execute any of the remainder so this would be the way that you do cancellation is you describe the circumstances under which you should cancel the operation does that make sense that you get to basically async isn't magical it just describes the mechanisms for um changing or for cooperatively scheduling a bunch of computation by describing when under what circumstances code can make progress and under what circumstances code can yield so this might then make you wonder well is it just turtles all the way down like at some point you need to get to a point where like uh like here right where i say i said read from network and then i'm gonna await that well ultimately that does like a system call it does a read system call from the operating system saying give me these bytes and the operating system goes i don't have the bytes what do you do well this is where the notion of an executor comes in and we're not going to talk in huge depth about executors because there's a lot of mechanisms going on there the basic premise of an executor is that you're only allowed to await in async functions and async blocks so this wouldn't actually compile it would only compile if this was an async fn but then you see the compiler complaints main function is not allowed to be async because ultimately at some point in your program you're going to have this giant future that holds the entire execution of your program but it's a future so nothing runs yet right so you have this sort of top-level future that describes the entire control flow flow of your application something has to like run that in a loop right something has to have the the loop that we expanded earlier that's like try to see if it completes and if it doesn't complete then what it can't yield because there's nothing above main right like something if it can't make progress what does it do because it can't yield you can imagine it just spins on a loop but in practice that's not really something we want to do and so an executor a sort of primitive executor is one that just pulls futures in a loop and does nothing else it just keeps retrying aggressively instead in practice what happens is that the the executor crate so tokyo would be an example of an executor crate provides both the lowest level resources like network sockets and timers and also the executor loop at the top and the two are sort of wired up together behind the scenes so imagine that you're doing like a read from the network and you call dot await what's going to actually happen is tokyo's tcp stream is going to do a read from the network realize that i can't read yet and then it's going to sort of store its file descriptor store the the socket id if you will into the executor's sort of internal state and say the next time you go to sleep watch for this file descriptor changing its state right like tell the operating system that it should wake you up if anything changes about this file descriptor and then at some point when when we've yielded all the way back up to main back up to the main executor the main executor instead of just like spinning in the loop what it's going to do is it's going to take all of those resources that that it knows it needs to wait for and it's going to send them to the operating system and say wake me up like put me to sleep but wake me up if any of these change if the state of any of these change wake me up because i need then i have more work to do and this is sort of the epoll loop for those of you familiar with linux system calls but it doesn't have to be epoll right on windows it's using um uh the name escapes me i forget on mac os is you in case using kqs um but but basically there's sort of a different implementation of the outer executor loop on different operating systems to best make use of the the underlying mechanisms so in practice what you do so with tokyo for example you and you do something like this and that allows you to write async fn main but what that really is is just a procedural macro that rewires your code a little bit turns it into fn main and then does the following it does like tokyo let runtime is tokyo runtime runtime new and then runtime.block on and then async of and then the remainder of the function that's the transformation that it makes so your main is not async right even though you wrote async if n the procedural macro turns it back into a non-async fn main because like when linux executes your binary it needs to just have a function to call a regular function it doesn't know anything about rust async and then what the thing it doesn't mean is create a runtime which is this executor this this sort of big loop that tries to pull the future that's passed a block on and then if it can't make progress on that go to the operating system say wake me up if any of these things change and then loops that until the future resolves until the future has no more tasks to run um so the example here would be this is an infinite loop right nothing this this async will never finish it never resolves into its final value but you can imagine that the moment we get something from terminal for example we're going to break which means that the moment someone writes something on the terminal this loop exits we're going to drop the network future and do no more work on it we're gonna drop the foo future and do no more work on it um and at that point we're at the end of the async block so this async block resolves into just unit the empty tuple that point block on finishes because it the future has resolved the future it was given has resolved uh and at that point we're at this this end of execution and then main exits so that's sort of the the higher level flow of how these futures get executed in the first place um all right again let's let's pause for a second and see whether all of these little bits and pieces uh that i've explained roughly makes sense together there's a lot more to talk about here and sort of how do you use this in practice what are some of the pitfalls um but let's see that that everyone's uh following along and and please do ask questions like this is pretty hairy stuff it's like weird and convoluted and there are lots of moving pieces so if you have a question chances are other people have questions so please ask them um i'm pretty new to this i only know about how the the javascript event loop works where would event loop fit into this like a comparison you can sort of think um of the executor loop the outer executor loop as being the event loop right its job is to keep running the the sort of available futures until there are no more futures to run and at that point it's done and the program is finished now the the difference between the event loop and javascript and here is that you can choose your own event loop you don't have to use tokyo in fact tokyo has multiple runtimes or multiple variations of runtimes that you can choose to use and arguably you can also write your own event loop this this loop select right here is an event loop it's sort of your own event loop within the context of the larger event loop you talked about futures yielding how does this yielding look like in code in general you don't have to worry about the yielding yourself whenever you do dot await implicitly the think of 0.08 as being sugar for doing this yielding you can't actually actually use the yielding machinery itself directly in your own code if you try to implement a future yourself like manually by implementing the trait you don't have access to the yield keyword instead you have to basically manually implement the state machine which is what the futures ecosystem and rust used to be before async await landed and trust me this is a lot nicer in general though these days it's very rare that you need to write your own future implementation um uh best way to pass data to tokyo spawn uh we're gonna talk about tokyo spawn um why is tokyo main a macro and not a simple function so i'm not sure i follow so the idea here is that this is very easy to write it's like a very simple setup instruction you could imagine tokyo having like a tokyo colon colon main that you passed in an async block to but it's it reads a little more weirdly i think there might actually be a tokyo main like this too um there's no reason it couldn't be one or the other um so does tokyo re-implement kq or libyuv it does not re-implement kq tokyo use it well tokyo uses mayo and mayo is a crate that abstracts over kq and e pole and whatever the windows thing is which still escapes me um but it basically gives you access to sort of something like libeuv basically an a an operating system event loop or not even event loop but just event register right you can say i want to go to sleep until any of these events occur and then you just go to sleep and then the operating system through whatever mechanism myo chose will wake you up when one of those events happened and you can make progress now you might wonder well what if um so down here we waited on cancel which is a um a channel receiver so if you await on a channel receive there's no file descriptor you can give to the operating system the operating system doesn't know anything about something like a receiver in those cases there's a little bit more going on under the hood here in practice you should just trust that the executor knows how to wake its own futures up so in this case if you're using the the tokyo runtime and you're using a tokyo channel um it it knows how to work that out and in fact the the pieces that are being used here are fairly run time independent so for example um you can use so there's a crate called futures um which has also a lot of just utility mechanisms for futures and i think one of the things they have is a a channel as well and you can use the futures channel with tokyo because they use the same underlying mechanisms that the rust language provides so so the executor does have to include mechanisms for dealing with non-operating system-based events what happens if network dota weight runs but terminal weight and foo await don't then on the next loop of a select does network the weight get run again yes so the way a select works is it selects among all the given options it doesn't remember anything about it having been run in the past in practice the way this actually works if you want to sort of redo things is you would do this uh the and the the borrow checker will complain about this like if i had actually imported the select macro it would have told me that i have to do this um because otherwise the first time through the loop network ownership of network is sort of transferred to a weight and so the next time around the loop the bar check would be like well network has been moved and you can't re use it again so in practice you would do something like this which allows it to be reused across multiple iterations of the loop um but yes it will consider all of the cases it doesn't have memory about past attempts um sort of a lot of what i'm saying has a little bit of hand waving right like we're not going into the the real mechanism details and you should consult the documentation if you rely on any particular corner case behavior here but the general idea is that select selects over all branches every time um okay so here's another good uh question which i was gonna get to but but a question came up which is um what happens if an abandoned select arm has side effects um so here let's say okay let me see if i can come up with a good example here let's say the one of the operations that we want to do is a file copy so we're going to have f1 which is going to be like a tokyo fs file open foo and f2 which is going to be file create bar and then we're going to do copy is tokyo bio copy mute f1 to mute f2 and then one of the streams here is going to be copy. let's just remove the loop for a second so uh first of all you'll notice that i'm using tokyo fs file instead of standard fs file um that's because if you do operations on a standard fs file there's no await there's no async functions on it because the standard library doesn't define async functions um in general um because they rely on this integration with the executor um that that you need in order to get the cooperative scheduling and the sort of smart wake up that we talked about um so you do actually need to use the asynchronous version of i o resources and then here what i'm going to do is create a future that's going to it basically takes a thing that implements read and i think that implements write or more precisely a thing that implements async read and i think that implements async right and it's going to read from one and write to the other great so that's one of my select arms now as i mentioned the way that select works is that it tries all the branches until one succeeds so imagine that it tries this nothing happened it tries this nothing happened it twice has nothing happened it tries this and it writes like a megabyte and then it has to wait on the disk like the disk is saturated and so it can't make progress but it hasn't finished copied the file yet and then we go back to the stream and let's say that the stream completes so now we exit the select and then down here we're now in a state where some bytes have been copied from food to bar but not all you can imagine that we copy dot await here to sort of complete the copy but it's very easy to forget to do that and there isn't really a good solution to this problem this is something that you need to identify has happened is that whenever you use something like select um you're now in a world where you might have partially completed a future right it's hit some yield point in the middle of its execution but that means that the sort of trailing end of the future hasn't gotten to run yet and at that point if you drop the future it doesn't get to finish it doesn't get to do the rest of its work it just gets terminated at that point it gets cancelled and so now you need to reason about the fact that your your program might be an intermediate state this is a particular i want to call a problem but this is a this only really affects selects because if you think about something like here that can't really happen right this either gets completed well then it sort of has to get completed there's no way for you to cancel the operation midway through in fact the only way that this operation or indeed any operation that you call dot await on gets does not get completed gets interrupted is if there's a select somewhere up the chain so in general when you write select you have to be careful about what happens if one branch runs and then another branch completes so the first branch didn't run to completion but the second branch did where does that leave you so this is a an error case that you need to be concerned about when you're using select um uh as you presented the executor's making the assumption that the futures are not greedy how do you avoid a future not taking all the compute time and grind the all async to a halt this is also a great observation and i'm i'm very glad you made it because it means that people are understanding sort of the mental model here which is this is cooperatively scheduled which means other things get to run as long as the thing that's running occasionally yields if you have a future that just is just a busy spinning loop or uses say standard io file and then calls like read on a giant thing or on a network socket that just never gives any bytes that thread is blocked if it used the the async tcp stream then the read would yield if it couldn't complete but if you use a standard i o tcp stream which doesn't know anything about async and you do a read its implementation is to block the thread do nothing more so certainly not yield just block until the read completes which might be never there might never be a byte coming from that particular tcp channel ever again um and and then you're sort of in the dog house like this is a really bad situation because now none of your other futures get to make any progress they're never polled again because the other one doesn't get to run because nothing the thing that is running the thing that's holding up the executor doesn't yield and so it is very important that that you use this that you lean into this cooperatively scheduled world and this is also why you have to be very careful about using blocking operations that is operations that aren't aware of async yielding or very compute intensive operations when you're in async context um there are some mechanisms that exist for for improving this situation so for example in toky in tokyo this is a function called spawn blocking this was the one called block in place and these are if you're in an asynchronous context and you need to run something that you know might block for a long time whether that's because of compute or because it's doing a syscall or something you can use these methods to sort of tell the executor hey i'm about to block make sure that other tasks get to run the way they do that we won't get entirely into today it's a little bit too technical but i would i would recommend that you read the documentation for these methods to understand the differences between them and what the trade-offs are but there are ways to sort of signal that um that you're about to block and and that the the executor needs to do sort of take the appropriate steps uh to make that be less of a problem um [Music] uh i am intentionally not mentioning future poll [Music] does that mean that select with a huge number of cases will potentially be slowed down by trying all options out every time you might think so and the answer is it depends on the implementation of select um in general if you have a select with like a million cases that seems like a problem but given that select actually forces you to write them out that seems unlikely in the first place um now it is possible for you to have such a large select for example you could imagine you have a code generation pipeline or something that generates it and in that case i think most select implementations are optimized for few cases rather than many cases but it is possible for select to be smart uh smart in the sense of only poll the ones that might be able to make progress i'm not going to go into exactly how they do that but basically there's a way for select to integrate with the rust machinery for dealing with futures and wakeups to sort of when if when a when a future becomes runnable through whatever mechanism like a file descriptor was ready or a send happened on a on a an in-memory channel there's a way for the select macro to sort of inject itself into that notification that this future is ready and sort of get a signal to update its own state when that happens so you can imagine that the select keeps almost like a bit mask across all of its branches and when that notification comes in it flips the bit for the appropriate branch and then the next time the select comes in or the next time the select is awaited it'll only check the ones where the bit is set in practice i think in general selects don't do this because it's a bunch of machinery and most selects have few branches you could imagine having a writing a select macro that only did this trick if you've got many branches um i don't think any of them do that now this is a good time actually i'm going to mention join in a second but let me see if there are more questions about selects is the select macro fair as in can it happen that only one branch will run forever so it depends on the implementation for example if you look at the futures crate so the futures crate has a select and a select biased and the select one is uh if multiple futures are ready one will be pseudo randomly selected at runtime so that one sort of tries to be fair and practice might not be entirely fair select biased is a variant of select that always runs if multiple or ready it runs the first one so that would not be fair so it depends on which select you use um uh when you use fuse with select i'm not really going to talk about fuse um so fuse the point of fuse is uh let me talk a little bit about fuse um because you might actually run into something that requires it fuse is a way to say that it's safe to pull a future it's safe to await a future even if that future has already completed in the past right so so let's take the case where i have a loop over this right so the way we had it and we go through the select and let's say both network and terminal are both ready so the select sort of uh checks on both of them and both of them say i'm ready here's the value but then or yeah let's say they both say i'm ready here's the value what does this select do well realistically it's not going to ask both of them it's just going to ask one and then the next one right so so it checks with network network says i'm done here's the value and it goes great thank you i'm going to run this branch we go through the loop again and now the select still includes this branch even though this future has completed but the select doesn't remember that it's completed and that it doesn't need to check this branch again because the select just knows about being called once the loop is not a part of the select and so this future needs to be safe to pull again even though it has already yielded its value and that's what fuse describes let's see how would you continue a half completed than abandoned arm you can await the same future after the select it depends on how you select on it but if you select like this then this is awaiting a mutable borrow of this future so after the loop you can still await that that value this is one of the reasons why in general you will want to select on mutable references to futures rather than the future itself because if you did this that would only work if this wasn't a loop um then the network would be moved into the select into this await and you wouldn't be able to await it later it would just be dropped at the end of the select so but whereas if you just await a mutable reference to it then that mutable reference sort of ends when the loop ends and so you would get to await it again later um how bad of a performance hit is it to use async when you don't call any async stuff down the line um there isn't really any overhead to like async doesn't add any overhead for executing code right like if you just have a let's say you do like a matrix multiplication in the middle like you just have you have a an fn matrix multiply which which currently implements a matrix multiply you just add the async keyword this doesn't make it any more expensive right it just the same code ends up running it's just that it gets wrapped in the future type and you have to wait it to get the result but the await doesn't do anything it doesn't it doesn't change the generated code in any meaningful way so there's no overhead to marking something as async i think maybe the the analogous question you're after is what's the overhead of doing an asynchronous i o read versus a synchronous i o read and there is a little bit of overhead there because now you need to you need to tie into the executor machinery there's a little bit more work there but in general the additional system calls that happen get amortized across all your futures or acro across all of your resources so it doesn't really add much and usually the benefit you get from the benefit you get from not needing to spawn like thousands of threads or hundreds of thousands of threads more realistically it usually ends up actually being faster to just have the fewer threads that are run by the executor and then have them cooperatively scheduled and there are a couple of reasons for this one common one is that you don't need to cross the operating system boundaries often if you have lots of threads like actual real operating system threads then if one has to block on a read then the read like you do a system call and the operating system has to context switch to a different thread which is not free and then runs a thread instead with async if a read fails the operating system returns to the same thread and says i can't make any progress it yields the executor continues running on the same thread so there's no context switch and then just pulls a different future and so in general that ends up being a little bit more efficient in practice it's hard to say which one is like objectively faster because it really depends on your use case but i would say on balance if you're doing things that are i o bound like web servers for example async is probably going to lead to more efficient use of your resources and also i think um easier to read code maybe not necessarily easier to reason about but easier to read um are these protothreads yeah they're user space threads is one way to think about it uh let's see uh what would select return so select is a future uh this returns well sort of so select generates a future for you and then calls a weight on that future um in general you can construct a select manually too um it's usually a lot more hassle but you can construct the select future for a block um yourself it's not quite true select expands into basically a loop with a um with a match or a bunch of ifs um it doesn't it doesn't really generate a future but you could make it a future by wrapping it in like an async block and then assigning that to a variable um and then it awaits it like select doesn't actually expand to a type it expands to like rust code that does the appropriate stuff to make this work but you could make it a future if you wanted it wouldn't really have a meaningful return value apart from a future though great now there are a couple different paths we can go from here the i want to start with talking about join so we talked about how select is sort of branching the control flow of saying do this or do this whichever happens first the other operation you can do is a join which is saying wait until all of these futures complete an example of this is let's say that you wanted to read 10 files right so you're going to do like files is actually let me go down here let files is 0 to 10 map i um nope tokyo fs file open format all right so here i have an iterator of futures right and i want to wait for all of them to complete before i continue with my program imagine that i'm like concatenating them or i'm computing the hash across all of the bytes or something so i really need to wait for all of the bytes to complete so in that case uh there is the the join operation and there are a couple of different joins um so the in the futures crate there is a join macro which looks a lot like the um the select macro except that you don't really specify the branches you just give you just list the things you want to join on so here let's say i collected this into a vector so i could say uh join and usually just you assign this to like so this is going to be a little bit janky let me let me make this three instead file one file two file three is join of uh file zero files one files two so this is saying uh actually this would be something like read to string so this is gonna run all those three reads in parallel and notice the reason i need to do this right let's say that i wrote let file one is is file0.08 file2 is file1.08 uh and file three is 2.08 so compare this to this so in this first instance what's happening is i'm first reading file one from start to finish or file zero from start to finish then i'm reading file one from start to finish then i'm reading file two from start to finish and then i get to complete without my program that works it'll give me the right result but the downside of this approach is that it's sequential this means that um rather than give the operating system all the read operations and have it like read the disk in the most efficient way or access the file system in the most efficient way or let's say these were network sockets and i wanted to read all the bytes from multiple network streams it might be that why one stream has no more bytes yet but another stream is ready i wanted to keep reading those like use the cycles to read those bytes while it's waiting for the first one in this first case it won't get to do that because i've said don't execute the next line until this file has been read so it doesn't even get to this line it doesn't get to start the next operation until the previous one is completed what join lets you do is say run all of these operations concurrently and then once they're all completed give me all of the outputs from the three futures so this one is more efficient might not be the right word but it allows the operations to continue concurrently which is a big big benefit generally because it lets you overlap compute and i o so that's that's sort of the join macro and it can be a little bit annoying to use it this way because you need to explicitly enumerate all the things you're joining like imagine if this was not 3 but 100 you clearly don't want to like list out 100 things here and then assign it to a tuple with a hundred elements but this join is really convenient if you just have a few things generally all of these provide you with um with multiple ways to do things so you see the tokyo one is similar but generally there's also let me see i forget what it's called here it's not i'm surprised well uh i think there's one here yes you see there's like there are multiple functions like join which joins two things joint three joint four joint five et cetera um but then there's also try join all so try join all takes an iterator over things that are futures ignore try future for a second um or actually a try future's a future whose output is a result so this is going to try to join all the things in the given iterator and the reason why the join space is a little weird is because in general you probably care about the uh you care about the fact that the output result order you can map back to the input order so i want to know that these are the bytes for file zero right so if i do the the what was the try join all right of files file bytes i want to make sure that file bytes zero is equal to files zero right and try join all will do that it will make sure that the result sort of output is in the same order as the input even if they completed out of order right it might be that the read here completed before the read here but then try join all will sort of reorder them at the end so that the output matches what you expect that reordering is not free there are ways to opt out of it so there's a type called futures unordered for example which as its name implied implies um gives you the results out of order if you don't care about the order this is generally more efficient so here you say you create a new one you can push futures onto it and then after you've pushed the future onto it you can then well it implements stream which we're not going to talk too much about but it implements iterator where the outputs of the iterator are the results of the future is completing so the idea here is that you stuff all your futures in there and then you loop over you basically await the futures in order multiple times and each time you await it you get one more thing one more output from one of the futures you stuck in but you don't know which one so this might be helpful if for example um the result contains all the information you care about and you don't necessarily care about the input because the output describes which input it was from so that can be more efficient uh so multiple multiple way does cascading yeah so another way to write this would be filezero dot then files one dot then files to then sort of pseudo syntax whereas this one is do all of them at once and join us like promise all me you can think about it that way um uh yeah there's probably also join all um which does the same thing or you can manually construct one of these those futures unordered and those futures ordered um and these so i mentioned how select isn't necessarily smart about making sure to only check the futures that might be ready but join is because it knows that there might be there might be a lot of futures you're joining over like imagine you're downloading like all the dependencies of a cargo project well there might be thousands of them and you want to download them all in parallel or at least some subset of them in parallel and then there are many branches and you want to make sure you don't like have to check all of the futures every time you only want to check the ones that have actually made progress and so in general the join operations like futures unordered will implement this little like hook into the runtime system to um make sure that it only checks the ones that have had an operating system event sort of readiness event happen to them um yeah and join all and try join all use futures on futures ordered under the hood um not futures unordered um great uh so that's join um and join is great uh select is great but all they do is allow things to run concurrently that does not mean that they get to run in parallel and this is a big important distinction that is often missed when people deal with futures uh let's see where to start with this okay um the tokyo runtime actually starts up multiple threads uh and each thread is able to sort of await a future but remember that awaiting a future that's not really what i want to say i'm going to say it differently let's imagine for a second that the runtime only had one thread so when you call block on you give it a future it doesn't know that that future contains a bunch of other futures inside the the the one like this function this is a like this is a method on a type in tokyo right it just gets one future and it can await that future it doesn't know about this await it doesn't get to look at the code inside it doesn't know that there's a join or a futures order or anything it just gets one future so the only thing it can do is run that future like try to see whether that future can complete and if it can't then go to the operating system go to sleep and wait for an event to happen and then try making progress on that future in return when it does that future internally checks its inner future and so on down the stack or if it's a select it checks all of its contained futures but ultimately at sort of the top there's only one entry point into execution and that's kind of unfortunate right because it means that only because there's only one future there's no advantage to having multiple threads because there's nothing for those other threads to do there are no other futures there is only one and therefore if you had 10 threads because say you had 10 cpu cores there's still only one future so only one thread can await that one future at any one time because awaiting a future requires immutable reference to it an exclusive reference and so multiple threads just can't do it and even if they could it wouldn't make sense like what would they do there's only one piece of code you don't want to execute the same code multiple times and this means that even if you do something like a join and so you're doing all of these operations concurrently they're happening concurrently on one thread which means that that's probably not actually what you wanted right imagine that you're writing a web server and you have like a loop over accepting tcp connections and for each tcp connection you get a future for handling its connection you stick it into like a futures unordered and then use the weight on the futures unordered right to do like um in fact let's let's try to write this out so here's what i'm going to do let's make up a tcp server so we're going to have accept is going to be a tokyo net tcp listener bind 2 0 0 0 0 8080 notice that i haven't actually added tokyo as a dependency i don't get completion or anything like it that's fine stop yelling at me great um and then i'm going to do something like loop i'm going to select on i'm going to have a let mute connections is going to be a futures stream future maybe um futures unordered new um and i'm gonna select over this and say either i get a new stream in which case i'm going to call i'm going to have like an async fn handle connection which takes tcp stream and then does who knows what with it it does things right like there's a let's just say there's a to do in there or something you know uh so in this case i'm going to say connections.push handle connection of stream and down here i'm going to say nothing is going to be connections.oh wait this won't compile for a number of reasons i and it's not really important i just want to demonstrate the the the higher level problem um the reason we need to have this branch down here is because something needs to be awaiting all of the the futures remember a future doesn't run unless it's awaited so if we just had this there's nothing is awaiting the futures unordered which means that nothing is awaiting the futures are inside of there which means nothing is awaiting any of the client connections which means none of the client connections are being served so we do need to await on connections but we also want to sort of see if there are new connections coming in so we need to wait on this accepting as well which is why we need select and you can think of these futures in order this is basically a join right i want all these to execute concurrently this of course won't compile because i have a mutable reference here and immutable reference it here and they're being used concurrently not okay but i'm going to ignore that for a second there's a different more fundamental problem i want to get at which is this is still just one top level future which means there's still even if the runtime had as many threads as you of course only one of them gets to run at a time it will get to multiplex across all the different connections but imagine that there are a hundred thousand connections that thread is going to be completely busy dealing with all those connections it's not wasting any time like it's not asleep or anything it's doing work it's just there's more work than it's able to handle on its own but all the other threads can't help out because there is only one future so the way that you can help this problem and introduce parallelism not just concurrency is um instead of having this this connections we're going to get rid of that uh and in fact we can get rid of this too which means that we can get rid of this too which means this can become while let so let's say i have this um there is a a function that's provided by basically every executor called spawn um in this case let's say tokyo spawn and what spawn does is it's sort of a hook into the executor whatever that executor might be that you give it a future and it gives that whole future it moves it to the executor so it it sort of is as if you gave it to the runtime so now the runtime has two futures it has this future which you passed a block on and it has this future which means there are two separate futures which means that two threads can run them at the same time so with this spawn if one thread is busy doing this work so accepting connections another thread can handle the future for a particular connection notice that this is not a thread spawn right the threads are spawned by the runtime and there's a fixed number of them fixed but we're giving additional futures sort of sticking them on the job queue for that pool of threads this is why spawn generally requires that the future you pass in a send is because otherwise it couldn't be sent to another thread to work on there's like spawn local but we won't really talk about that right here in general spawn also requires that the future you pass in is static because it could be that inside let's say that inside of here we also do a tokyo spawn right inside of this with some other thing that does you know whatever it needs to be static because it doesn't know the lifetime of the run time and in fact the handle connection async function might complete right this outer async thing might complete but this spawned async future still needs to be running and therefore it needs to be tick static it can't be associated with the lifetime of handle connection um imagine that this had like a i don't know x is a vector and this tried to use at x then if handle connection returned but this future still tries to run x would be dropped but this has a reference to x so that's not legal so that's why spawn requires static so this is the way that you introduce parallelism into asynchronous programs is you need to communicate the futures that can run in parallel to the executor now it could be that the executor doesn't have that many threads right like with uh with the tokyo runtime you can set like worker threads to like one in which case there's only one thread so it doesn't really matter whether these are can run in parallel because there's only one thread so there's no parallelism but in general you want to use this pattern so that these futures can run not just concurrently on one thread but in parallel on multiple threads all right is that the does the reason why we need spawning makes sense okay yeah so this is why it's important to remember to spawn and often why when people who aren't very familiar with async await start writing async await they find that their program performance drops a lot and it's because they're not spawning anything so their entire application is running on one thread and when your entire application is running on thread of course it's slower than if you had multiple threads because nothing gets to run in parallel what's the best way to pass data to tokyo spawn um [Music] what are the best practices to handle uh errors in async when we call spawn so um spawn is a little bit weird because just like thread spawn you don't really get to communicate with the other thing ex sort of implicitly right it's it's just running somewhere else and you have no control over it so you need to apply the same kind of techniques that you would use in a multi-threaded program which is if you want to say share data between this and that or let's say that i have two things i want to spawn and i want them to share access to some vector i would have to do like arc new mutex new and then this can lock and then do whatever and this can lock and do whatever um in practice i would have to do like x1 is our clone x x2 is our clone x this this gets x1 and it's an async move and it's an async move so they both get their own arc um and i need semicolons um so they each get their own arc and they both have a mutex regards the underlying value and everything is happy or you can like have them communicate over a channel uh you can have them uh communicate over like if they're if it's read-only over static memory like you have all the same techniques available to you as you do in a multi-threaded program and you really should think about it in the same kind of way now there is one exception to this which is if you have a if you spawn something and you want to communicate the result of that back to the the thing that spawned it um at least in tokyo spawn what you get back is a a sort of join handle similar to if you do um a thread spawn where um let's say that this ends with zero right then if you do you could sort of assert equal that join handle dot await is going to be zero if you don't await the join handle and just drop it it's the same as if you drop a thread handle it just does nothing like it doesn't terminate the thread or anything it just you don't get to learn its result value so this is one way to communicate the the outcome of a spawned operation back to the caller now if if one thing to keep in mind is that if you spawn just like if you spawn a thread if you spawn a future um and let's say that like i don't know it calls definitely errors right and x now let's say this like returned a result right and it's the error case of the result what do you do with the error you don't have anywhere to communicate it necessarily um because you don't have a way to communicate with your caller i mean you could return it but there's no guarantee that they're awaiting a join handle and you can't really like you don't have anywhere to do it but the question is the same as if you spawn a thread or if indeed if an error happens in main like what do you do with that error you need to have either you could just print it a standard out uh you could log it to a file you could use um some kind of logging framework like tracing to sort of emit an error an error event that gets handled somewhere else in general that's the kind of approach i favor where if you have an error that you can't propagate any further um you use an event distribution tool like tracing um to to decouple the production of events and the subscription to events um so that'll be the way to go [Music] is there any benefit on calling tokyo spawn and immediately awaiting on it so there can be it's a little it's it's fairly uncommon to do the advantage of doing that is that you get to let other things on the current thread keep running while something while that operation is running elsewhere if that makes sense so um imagine that you have an operation that has to do like d serialization so it's somewhat cpu heavy it's still io so you probably want to do it in async context and not blocking context but you could do like uh d serialize over here and then join handle 0.08 um and now that await is going to immediately yield because this spawn hasn't returned because it's spinning due to deserialization so this future is going to yield and imagine that it's in a select or a joint or something other futures on other tasks on the current thread like on the thread that's running handle connection they get to run and then this deserialization operation gets to run on a different thread and gets to do the cpu intensive operation so it gets to happen concurrently and in parallel with these other tasks continuing to make progress so that might be one case where it could make sense to spawn and then immediately await it's rare that you actually want to do this um how is the tokyo spawn connected to the runtime instance created above uh there are sort of magical thread locals that um that are used basically uh so runtime new just creates a normal value there's nothing special about it block on will set some special thread locals inside of the executor so that when you call tokyo spawn uh it checks those thread locals to find the current runtime and then spawn on there and then similarly when the runtime eventually runs that pass in future it sets the same thread local so when that future calls tokyo spawn it can find the executor and so on it's not a singleton right so so you could have multiple run times and if you call tokyo spawn in the context of one runtime it will spawn on that runtime not on the runtime and this can be valuable um so for example there are there are some services where you might care about you might have prioritized traffic for example like imagine you have control plane traffic and data plane traffic and you want to make sure that control plane messages are always handled so one thing you could do is for example have but there's relatively less control plane traffic but you do need it to be handled you can imagine dedicating say two cores to um control plane traffic and have everything else for data traffic what you do is you create two runtimes one with a thread count of two one with a thread count of however many course you have left you spawn all of your control plane operations on the the run time with the two threads you spawn all the data plane operations on the other run time and then you both run times get to continue running right like they're both active at the same time but you know that there are two cores are reserved for cold control plane traffic um if you had a singleton runtime you wouldn't have this operation you can imagine that the the executor itself supported like priorities and stuff that gets somewhat complicated because it also needs to integrate with the operating systems runtime controls and and priority controls it's nice to be able to do this explicitly um so there are advantages to that although it's a little harder to discover it's true [Music] um what should i do if there's an expensive function like hashing a password that i don't want to block async execution of a thread that's when you use something like spawn blocking or block in place uh [Music] what happens if you toke your spawn before creating any run time it panics it says there is no runtime um so rust rust futures do not depend on thread locals um this is an important distinction there's nothing in the the async support in the rust language or the standard library that requires thread locals um tokyo uses thread locals in order to make the and like some other executors do this as well in order to make the interface slightly nicer like otherwise uh imagine that you didn't have thread locals spawn couldn't be a free function so you would have to do something like runtime.spawn but that means you now need to thread the handle to the runtime throughout your entire application in order for anywhere to be able to spawn deep down in the stack you could make it explicit like this um but and in fact tokyo lets you do this like there's a runtime.handle i think and then you can pass the handle around you can do handle.spawn so you can do it explicitly without the thread locals um but in general the observation is that this is so common that it it's done with red locals because that way the interface is a lot more ergonomic um the downside of course is that this means that tokyo doesn't really work well on like an embedded context where you might not have thread locals but there's nothing in the rust async sort of primitives or language support that requires it let's see um when you create a runtime it doesn't really allocate a lot of memory there's like some controlled um controlled data structures but they're not generally very large there's not a lot of overhead to the runtime itself it does need to keep sort of actually i'm about to talk about that so i'll save that for a little bit of a second um okay so one thing we haven't really talked about much is um what a future actually is um we we talked a little bit about it right like before we added all the spawn stuff um in fact let me go no let me write a new one um so down here so i'm gonna have an async fn foo because i like foos and here's what i'm going to do i'm going to have an x which is going to be a bite array uh and then i'm going to do tokyo fs uh [Music] let's say that there was a read into um and i want to read file dot dat into x let me just ignore error handling for now so n is the number of bytes read and then i want to print line all of the red bytes so this is a fine new size i don't actually give the type signatures um so let's think about what actually happens here uh we talked about how an async function or any async block is really just a chunked computation right so there's one chunk that starts here and ends like this all right like this is one chunk uh and then between every chunk is in a weight right so this is sort of the the operation that happens in between the chunks um and then chunk two is gonna be uh let n let's get rid of the annotation um n is equal to few dot let's say output right because at this point it should have resolved because the await sort of finished um and it does this you can sort of think of it as this is how it's uh oops this is how it gets divided up right this is when we talked about a state machine right this is state one this is the this is state two and this is the edge between state one and state two is future completing or food completing here um and what's a little bit weird here is let's try to think about where x is right so x is a 1024 byte long byte array where is that stored you might normally say like it's on the stack right it's a it's a local variable in a function so it's on the stack and that's why this works out but if when you look at it here like this is really a yield which is really a return right so let's say yield which is really return but when you return that x goes away right like the stack frame goes away but the future has a reference to x because it needs to it needs to have a reference deck so they can write into it so it's not okay for x to go away so where is x it's not on the stack because the stack goes away when this yield happens in practice what actually happens is that when you write async event or when you write an async block the compiler actually generates a sort of that that state machine we've talked about chunk one chunk two it really generates something sort of like an enum where each chunk contains the um the state for that part of the state machine and the state here me really means any local variable that needs to be kept across a wait points right so if i here said like let z is vec but i never actually gave out a reference to vec like z is never used in any later chunks it doesn't need to be saved anywhere it can just be on the stack when foo is invoked or when foo is continued it can be put on the stack and then it can just be dropped at the end it doesn't need to be saved anywhere but x does so really x ends up being here as a uh u8 1024 in practice it's a little more convoluted to make sure the references stay the same and whatnot but you can sort of think of it this way the same thing with the future itself it needs to be stored somewhere so this also stores a feud which is like a tokyo fs read into future which actually has a sort of lifetime of tick x right because it holds a reference into x so this is sort of self-referential uh which is another reason why you can't actually write this yourself in in any meaningful way and then in chunk two chunk two doesn't actually have any state right because it defines n uh but n is a local variable it's not kept across the weight points um so the n can just be dropped so there's no state in chunk two um it does continue to use the state from chunk one though so you could arguably say that like this sort of gets transitioned into here but there's not actually two copies of it this is you can think of this more as sort of a union maybe um where some of the state actually stays in the same place so that references continue to be valid it's all kind of convoluted but realistically what happens is that the what this actually returns when we say it returns an impul future right that impul future really is a type impul trait means a type of that has a name but i'm not going to tell you what the name is and in reality what this type is is this statement this generated state machine type that like every time we try to await this future every time we continue it what we're really doing is continuing within the state machine type we're invoking a method on the state machine type that gets an exclusive reference immutable reference to that type so that it can access the future so they can continue to wait that access and internal local variables and the like and this this conversion from asic fn really ends up like rewriting a bunch of these so that instead of being feud this is like self dotfute and self.x they get rewritten to be re-referenced be references into that state machine that's a struct that we end up passing around so in main or in this case let me just pretend that it's an async main for a second when i say let x is fu the value of x is of type state machine right so when i do x dot await what i'm really doing is sort of i'm sort of calling the await method on this particular state machine that's sort of what i end up doing in practice that's not really the de-sugaring you can think of it as i'm really continuously awaiting this state machine the reason this distinction is important is because this state machine contains a decent amount of stuff right it contains all of the values that are kept across a weight points and in this case that's say uh 1024 bytes which is decently large right so if i for example do bar and pass it x i actually have to move a fair number of bytes this is like a pretty decent mem copy imagine that i was like reading a bunch more bytes right then now suddenly calling bar where bar let's say that a bar this takes an impul future and i pass it x which is a future then what i'm actually passing it is this entire state machine which includes a lot of bytes that have to be mem copied and so futures can actually end up getting really large and the other reason for this is let's imagine remember here it needs to store the future that it's currently waiting on so let's say that down here this called uh my other future or some library if you uh execute await whatever this future is we don't know how large it is it's controlled by some other library that future needs to be stored inside of our state machine if we have a select we need to store all of the futures we have a join we need to star all of the futures so futures end up containing all of the futures that they in turn await which means if you just can get really really really large one way you can see this is um imagine that you're doing profiling on your application would you what you'll see in in some asynchronous code bases and you'll see that mem copy shows up a lot and it's usually because you end up with really large futures that you end up passing around your program even just like returning a future right like this needs to return a future so this is sort of a mem copy into this variable and that can keep happening up the stack as you return futures or pass them around stick them into vectors whatever um and so this is something to to watch out for of course the way that you can solve this problem is by boxing your features so if you place them on the heap um this problem sort of goes away you could either have the this this allocation be on the heap or here we could say this is going to be box new of foo in practice we need to use box pin for reasons we're not going to get into but at this point x is still now a future but when i pass x to bar i'm just passing a pointer to a heap allocation that bar can then treat as a future rather than the entirety of the state machine that i constructed this is another reason to use something like tokyo spawn is because spawn is going to stick that future into the executor and then just keep up a pointer to that future this is why it's actually useful to have the the spawn handle is because you can store that and then like here if i instead of doing this did tokyo spawn of this 0.08 this now doesn't have to store this future whatever size it might be it just stores the pointer to it and then can await it and so we end up doing uh fewer of these mem copies we don't end up with the the future's just growing by growing the onion um that makes sense yeah it ends up being a union with the size of the largest chunk state is the right way to think about it um why can futures assigned on the stack be moved don't they need to be pinned we're not going to talk too much about pin um but basically there's nothing that prevents you from moving a future in fact there's nothing in the rust language that prevents you from moving any value but once you have started awaiting a future you can't move it again unless it's on pin but there's nothing inherent about a future that means you can't move it can you have an async function that internally creates a vec of futures from itself recursively i know recursion requires indirections in other words just putting in a vec count as a direction um putting it in a vec is not enough but if you stick it in something like a uh join handle i think you can um or yeah if you stick it in a join it depends i don't think join is enough actually because a heap allocated like a vector is really a heap allocated array um and an array needs to know the size of its elements uh and the size of its and it can be no it and it can compute the size of its elements because the size of the pointer is known so i think that's fine um although you would have to use a join not a vec in order to actually um select or await the set of futures you should be able to do that okay now that we've talked about this uh there is a little bit more i want to talk about sort of here at the at the tail end uh you may have heard about async trait so one thing that the people are sort of uh missing in um one thing that is missing in the implementation of async of weight as it stands today is the ability to write the following i'm going to go with service because it's a trait that i know decently well i want to be able to write async if n call request i'm gonna just no i'm gonna make it simple i'm gonna have a struct request instruct response i want to be able to write this right i want to be able to define a trait that has an asynchronous function in it currently this doesn't work functions and traits cannot be declared async async trade functions are currently not supported consider using the async trade crate and the reason why this is not supported is because remember this is really sugar for impul future output equals this and here we get a different error infiltrate is not allowed outside of function and method return types the reason for this is let's say that i write fn fnfu and i take an impulse service and i say let x is uh x dot call request uh let's make that few how large is few here or let's say i take a din service oh of course this needs to let's say this is mute self um this is muted how large is few tier what depends right let's say that i have uh struct x and i impulse service for x and i implement um async call async fn call mute self request i have to return response and inside here i say here i just return a response immediately so the size of this is very small and for y what i do here is i do let z is 0 1024 then i do tokyo time delay or it's called sleep now i forget sleep for 100 dot of weight and then i drop z so this is used to cross in a wait point and then i return response the size of feud depends on what stack variables are there in the async block that was used for the future and so few doesn't have a size it doesn't have a known size and so the compiler doesn't know what code to generate here because it doesn't know the size of feud you can imagine that if this was like impulse service then maybe it could figure it out but this is now a type that you can't name so let's imagine that i wanted to return a foo call and i want to write struct foo call and i here i'm going to do foo call of fute so it's going to hold the future right what's the name of this future it's sort of like service calling like type of service colon call and call maybe that's easier if we say like s-service and this is an s and this is s-call but this doesn't really work either you can do this with like existential types but it gets weird so really there isn't that there isn't really a good way to deal with async fn call like this in what we've described so far because the the type of the thing that it produces isn't known anywhere it's not written anywhere um so there are there are two ways around this uh one is the async trait crate so it lets you annotate uh like this and what that does is it basically rewrites all of your async events um you need to also place it on any implementers of the trade and it rewrites all your async events into pin box din future output equals response and does the same down here rewrites that into box uh so it rewrites the signature and then also rewrites your body into async move of this and that works if i remove async trait now well i'm not importing pin but now it won't complain because this is a type with a well-known size it's a heap allocated dynamically dispatched future which rust already knows how to reason about so the size in um i removed the block that i actually called this but if i if i do if i have a service and i call dot call the thing i get back i know the size of it i know how to sort of await it i have all of that information readily available to me because it's just a din future the problem of course is that now you're heap allocating all of your futures which means you get dynamic dispatch you don't get sort of monomorphizations and some of the nice rust optimizations also you're doing a lot you're invoking a lot more um memory allocator pressure uh you also have indirection for all of your futures so imagine that this was not service but this was uh async read and this was a read now every time you do a read you do also do a heap allocation and an extra pointer in direction so that might actually get fairly expensive this is why async trade works really well for sort of higher level things which traits often are it doesn't work so well if you have to use it at the bottom of your stack or rather it might not work so well always measure first and then see whether it's a problem the other approach you can take here is to declare a read if we go back to service here is declare a an associated type call future which has to implement future output equals response and then instead of having this be async if n call you say that fncall has to return self-call future the reason this works is that here i can i say type call future is equal to well this is also inconvenient to use i'm going to get rid of this because it's annoying um type call future is equal to this this is going to be self call future so i can use i can do the same thing here right boxed in future output equals response so i can make this be the same thing as what it was for um for async trait but i can also not do that if i want to choose to do so like this could just be a response it could just be an async response the problem i now have is how do i name this type so the reason this works is because now the associated type like russ knows how to communicate associate types to callers so they can now know how large the the actual type is you have a way to name the return type so that you can use it in other structures and stuff the problem of course is when you implement this trade it's not entirely clear how you name any type that isn't pin box didn't there is a feature called type alias infiltrate which would let you write this which sort of feels like we're almost at uh async defense in traits there are some other corner cases to to fix out but but this is why the async trade crate is often needed and why it's hard to get async traits um without doing something like boxing all right does that does that make sense why why we have async trade what it's used for um um yeah so so one of the observations right is that the compiler could in theory generate all this itself right if you write this um certainly you could imagine that the rust compiler could like behind the scenes rewrite it into this right and also automatically rewrite any call to it into the appropriate thing here there's a little there's a lot of magic going on there like suddenly what i write is this but it gets turned into like an async move and an associated type that gets auto-named for me the name of that type is not always entirely obvious it's a little bit maybe i now need to also say like where self is sized because dynamic dispatch wouldn't work with associated types it's not impossible and this is why i think we will get async functions and traits eventually in rust but there's a lot of design decisions to be made for how this should actually be exposed how it should work behind the scenes should you be able to customize what this um additional function is called um there's just like a lot of design decisions that have to be made um and there's a fair amount of just complexity in the type system needs to be worked out too um if you're interested in this i highly recommend like go look at all the discussion that's happened about this uh it is something that like is clearly wanted it's just getting it right is hard um great so let's see are there more things i wanted to talk about so we talked about join we talked about futures unordered we talked about spawn talked about number of worker threads we talked about async trait we talked about spawn blocking um we talked about send and static futures pin and box futures um we talked about how blocking code is problematic because it doesn't let other tasks run um yeah there's one more thing i want to touch on and this is something that is i don't want to say controversial because it it isn't really but let's say that i have an async well let's see that i have an async fnfu uh actually i don't even need these i can write this here so let's say that i want two futures to share state so i'm gonna say x is arc new our canoe mutex new of zero uh and then i'm gonna do tokyo spawn actually i'm gonna do let x1 is our clone x spawn uh async move bear with me for a second while i type out some code this is going to be x2 all right so i really just wanted to okay we're gonna do mod tokyo we're gonna do fn spawn uh we're gonna say it takes impul future and does nothing with it you stop yelling at me now async pub effect yeah pub basing kevin great stop yelling at me um so here i have uh two asynchronous threads and both of them are accessing the same arc and they're using a mutex and in this case they're using the mutex from the standard library so there's also tokyo sync mutex gonna move that down here just so that i can make it not yell at me anymore uh oops pub pubmod sync pub struct x i'm gonna t mutex okay so there's an argument on going about should you use the standard library mutex or should you use the tokyo mutex or like an asynchronously enabled mutex and the answer and the reason why there's a discussion about it is let's imagine that i had this do let x is x1.lock and then i did tokyo fs read to string file dot await and then i did x plus equals one this is clearly a very stupid um loop but let's say that's what i wrote so now i'm in a position where i take this lock and then i go await now at this point let's say in fact let me make this a select that's what i'm going to do that'll illustrate the problem better so i'm going to move this up here select this um uh fine i'll leave it a spawn that's fine that's fine it's fine it's fine okay uh let's imagine that i have a run time with only one thread now imagine that this gets to run first so it takes the lock and then it goes to read a string and then it calls dot await and now it goes to run this uh like this yields because it can't read anymore from the file so now it goes to run this other future instead it's been spawned right so the the thread sort of drops this one which is yielded or it doesn't drop it but it just sticks it on i'm gonna run you later and then it tries to run this future and then it tries to grab the lock but the lock is held by this future and so therefore this blocks and because it's a standard library mutex it just blocks the thread it doesn't know anything about asynchrony it just blocks the current thread but that means that the executor's one thread is blocked which means that it doesn't get to continue reading from the string because that would require continuing executing that future which means that this future never ends up dropping its lock guard which means that the lock is never released which means that this lock never completes and so we have a deadlock this is why it's problematic to have standard library mutexes right it's because you can end up in these deadlock situations now this would still be weird if you have an asynchronous mutex right if we use the the like a tokyo mutex or some other asynchronously enabled mutex what would happen is this would lock this would try to reach a string it would yield this would try to lock but because it's a sort of async aware lock when it fails to take the lock it would yield rather than just block the threat it would yield which lets this future run again which lets this com complete eventually which lets it drop the mutex card which lets this continue and eventually succeed so that's why you need async async-enabled mutexes now that said one downside of asynchronated mutexes is that they are a decent amount slower and that's because they need a lot more machinery in order to be able to do the sort of yielding on demand and know when to wake each thing up it's fairly complicated what they have to do internally and so there's this advice that in general you actually want to use a standard library mutex as long as your critical section is short and does not contain any await points any yield points so in this case our critical section that is the section under which we hold the lock contains an await which is just ripe for deadlock potential like we just saw and so here it's not okay to use a standard library a standard library mutex but in the case we had before of it just does this here the critical section is very short um and it doesn't have any weight points so there isn't really a risk of deadlock here the reason i say the critical section has to be short is if the critical section was like do a giant matrix multiplication then sure there's no wait point but you're still holding up that thread and not letting other futures work so it's sort of similar to any other operation that is like a long-term operation but it's a little worse because you're also holding the mutex which might block other futures on other threads in the same way right like they may be unable to make progress imagine that um okay you imagine you have a run time an executor with two threads and there's a future on like this future runs on one thread and this future runs on the other thread and this is doing like a matrix m this is doing let's say here matrix multiply which is let's say super slow so this thread is running the matrix multiply while holding the lock this other executor thread is trying to run the second future tries to take the lock and blocks right because this is a standard library mutex so it blocks and at this point because it doesn't yield any other futures on that other executor thread also don't get to run as you're holding up a lot more work so that would be another instance where you would want it to be an async aware mutex so that this other executor thread its future would yield when it fails to take the lock and let other futures run in this thread at the same time while this matrix multiply is finishing but in the case where the critical section is short and there are no await points it's totally fine to use a standard library mutex and often you might want to because they can be significantly faster to acquire and release which could be really important if you have um if you have operations and you need to do a lot or where there's potentially a lot of contention where a mutex might do a lot better than an async aware mutex [Music] okay so at the tail end here um there's a question of like how do you detect these kind of problems if there's like blocking or something is running for a long time or you might have a cancellation you didn't expect there aren't currently any good tools um i know there's some work on like a tokyo console which is basically it sort of hooks into the executor and and figures out how long has it been since a given future yielded how long has it been since the last time a future was retried um so there's like some of that kind of monitoring that that could point out these problems um it's not complete yet it's not something that's really tied to tokyo either i mean this one is written for tokyo but you could imagine instrumenting any executor in the same way of sort of uh noticing when particular patterns show up like a future hasn't yielded for say more than a second that seems like a problem then you could highlight that to the user oh yeah and it looks like clippy has some lints for this too uh can you elaborate on the conceptual difference between a thread spawn and a tokyo spawn a tokyo spawn gives the future it is passed to the executor for the executor to run whenever it wishes concurrently with other futures a thread spawn spawns a new operating system thread that will run in parallel with everything else in your program and is outside of the executor's control um a thread spawn also does not take a future it takes a closure so if you wanted to await a future inside of a spawned thread you would need to create your own executor inside of there um conversely if you tokyo spawn something you're not guaranteed to have your own thread and so you do need to be co-op like you have to cooperatively schedule if you use tokyo spawn you have to have yield points because otherwise you might block the executor if you do a thread spawn that's not a problem because the operating system is able to preemptively disrupt you basically think of it as threads are not cooperatively scheduled and so they can do blocking operations whereas tokyo spawn or just like futures tasks that you spawn are um are cooperatively scheduled and therefore need to yield in order to let all the futures in the system continue running um okay i think those are all the things i wanted to touch on for async and await um and you'll notice that we haven't talked much about the lower levels right like we haven't really looked at the future trait um any of like pin or context or waker uh all of the bits and pieces that executors go through um those are valuable things to know about but they're not really that relevant to trying to understand just how do i use async so hopefully this gave you a good survey if there are questions now at the tail end of like the the again these higher level bits of what what should my mental model be for async await um what what are the intuitions what are the sort of higher level techniques um now's a good time to ask them and i'll spend some time just answering some questions here at the at the end um when i await inside a future well the current future gets scheduled on the same thread when it resumes to start progress or can it get scheduled on another thread when a future yields it it just yields to whatever awaited it right so uh imagine that you just wrote something that was like uh it takes a impul future so imagine that i did this [Music] if inside of f you await and then you end up yielding you just go back up to b and it's up to b what happens at this point right b in this case is chosen to await and so it's just gonna keep it's gonna because you can't make progress it can't make progress it's gonna yield and eventually it's gonna yield all the way up to the executor at that point the executor just takes that future and sticks it back onto the the sort of job queue of futures which is handled in general by all of the worker threads at least in a multi-threaded executor like tokyo's default one but but this depends on your executor their executors are single threaded and if you have a single threaded executor it would execute on that same thread because there are no other threads in tokyo you're not guaranteed that it's the same thread you could imagine that there are or there are ways to say only run me on one thread in different executors like in tokyo there's like a spawn local which gives you some of these guarantees um and you could also imagine that you had like a um instead of b doing an f.08 it could do like a loop f dot poll i said i wasn't going to mention poll but i will um poll is the way to sort of check progress on a future so you could imagine that b doesn't actually use the await keyword at all so it never yields it just loops you in a busy loop if it does this then f will get to run again immediately and it will be on the same thread because there's no yield here so there's no opportunity for the executor to reschedule you on a different thread but in general this won't be the case so in general all the way up the stack it's going to be awaits all the way up to the executor which does have the option of rescheduling you on a different thread in general the at least i know tokyo tries to keep you on keep a future on the same thread because it generally helps with uh cash locality and such but it doesn't guarantee it if you really wanted a future to not be sent across threads you would just not implement send for it of course that makes it a lot harder to work with that future but that would be the way that you would enforce that statically um what would be your favorite method to get an async stack trace uh that is to show the async call graph up to a certain point um so i don't have a great way to do this it is true that if you if you print just a regular stack trace inside of an async inside of a future you will get the the call trace going up to the executor and usually that can help the problem really stems from spawning right if i spawn this future then and then let's say i panicked right here then this panic would not show main this is maybe a bad example um uh let me have a async fnfu and it does a tokyo spawn of an async that panics to illustrate the problem this panic will not include foo because this is a future that gets put onto the executor's job queue as sort of a top-level future to to await and then the executor one of the executor worker threads is going to pick up the future and await it and at that point foo was no longer involved all foo did was like put the future on the queue but when that executor thread awaits this future and it panics the back trace for that panic will only say the executor pulled this future so it'll point you at this future but that trace won't include foo um i don't have a great way to solve this problem i know that with um with something like tracing so there's a if you haven't looked at tracing it's great so the tracing library is a sort of logging system that lets you emit events and then have subscribers to pick up those events um and one really neat thing that exists is uh tracing futures no that's not what i meant to do and also sorry that it's bright so tracing futures uh let me see if there's a good example of this is um you can take a future import this trait uh call dot instrument and then give it a a sort of signifier that indicates the surrounding scope so in this case what we would do is something like spawn async and then dot instrument like in foo it's not quite what you would write but but you get the idea so this produces a future instrument takes a future and returns the future so this is still a future and then that whole future gets spawned and now even though the panic isn't what you would do here if you omitted like a tracing uh tracing error for example that said oops um this tracing error would include this in its sort of event path um and so that gives you a way to sort of trace it back to where that future was originally spawned um and so that's one way to to get at this that that i've had some luck with but i don't have a good sort of general purpose non-instrumented solution sadly um best practices called async code from synchronous code try not to do it it's really hard to do right you can use something like futures block on the the downside of block on is that you don't really get this cooperative scheduling um you can use like you might end up with your program well you run into some of the problems like a particular future might be using i o resources from tokyo which the futures executor doesn't know how to execute and so you get a runtime panic that's one case you can get into if you just like blindly try to block on futures um basically and the other problem is you don't give the caller control over or you're you're the user of your library control over execution um you're in basically you're enforcing an asynchronous runtime on them rather than letting them choose the runtime which tends to make people sad because whatever like imagine that um maybe one good way to get at it is imagine that i'm writing an asynchronous application and my asynchronous application calls your synchronous library and i use like spawn blocking or something and then your synchronous library creates an asynchronous runtime in order to block on a future internally so now i have a nested asynchronous runtime that causes all sorts of problems um sometimes it's runtime panics uh sometimes the two are incompatible as you get up with like other runtime panics sometimes it's just a performance problem where now you spawn twice the number of threads that don't really cooperatively schedule with one another um it's just it's just a nightmare uh i would say that if you have asynchronous operations internally expose them as asynchronous and then leave it to the user to choose how to make them synchronous so basically don't if you can avoid it um uh are there use cases where you want to use threads instead of futures there are so um anything that's very compute heavy asynchrony doesn't really add very much and and tends to sort of get in the way because all of your operations have to be marked as blocking and then it just gets more annoying to write them because you need to pass them around as closures um so there i would use something like rayon instead if it's very compute heavy if you don't really have io if you have a program that yeah if you have programs that don't really do i o there's no real reason to use async it is true that in general writing non-async code tends to make at least simple code or or streamline code or single execution code easier to read it gives you better compiler errors the borrow checker generally works better if you're not in async context the back traces work better so if you're writing just sort of straight line code like you're writing a simple command line tool or like a converter or something like i probably wouldn't bother with async because it does make your task a little harder if you're writing something where you think that it might be i o heavy in the future or you might want to do like handle multiple different types of events like this is where select is really useful and emulating the same pattern and threaded code is a little bit of a pain um then i would lean away from threading but but if you really just don't need the mechanisms that async provides and then use normal threading um tokyo tokyo by default doesn't bring in any runtime and then there's there are feature flags for enabling io frame enabling or like file i o enabling networking enabling timers enabling the single threaded runtime enabling the multi-threaded runtime so in general you you opt into the different features that you want if you call tokyo spawn and you're not on the tokyo runtime you get a panic all right i think that's where we're gonna end the stream two and a half hours that seems about right for async wait um i hope that was useful i hope you feel like you have a better intuition for for what's going on under the hood and some of the pitfalls um and if if you're really curious about how this is actually accomplished um i recommend you go uh you can look at my older video about sort of how async really works um and in the video on how pin works uh some of that is going to be a little bit more technical but but it is maybe helpful knowledge uh if you really end up digging into the details here but hopefully this has given you the the high level intuition and knowledge you need in order to be productive but they think wait at least that's my hope great thank you all and i will see you next time which will hopefully be another hazard pointer stream so long everyone

Channel: Jon Gjengset

Views: 49,584

Keywords: rust, live-coding, async, await, concurrency, asynchrony

Id: ThjvMReOXYM

Length: 154min 1sec (9241 seconds)

Published: Tue Aug 31 2021

Reddit Comments

[u/Poliorcetyks]

My next train travel is 2h45minutes long. I know what I’ll be doing !

[u/Dygear]

This is such a useful video! Thank you Jonhoo for making it. I missed the live stream by minutes but I watched all of it after. Giving that I’m working on an application that needs to maintain a TCP socket, a UDP socket, and a WebSocket this will be super helpful!

[u/sanity]

This is great, I love Jon's videos, well organized, good pacing keeps them interesting, and explanations are always clear.

I was wondering, in the past I've experimented with portable continuations in other languages. These allow you to suspend some code, move its serialized state elsewhere - perhaps over a network, and resume where it left off. I was able to do this with Scala's (since deprecated) delimited continuations compiler plugin (which roughly provided async/await functionality).

To motivate an example, imaging being able to suspend some execution state on a web server and have it resume in the browser. It would be a true implementation of "move the computation, not the data".

I even built a Scala prototype years ago, but it never got beyond experimentation.

Is something similar possible in Rust?

[u/nik123true]

Sorry to bother (and being offtopic) but mind sharing dotfiles? I'd like to get into rust, but just can't bare the thought of using anything other than vim/neovim. Currently got the usual lua lsp-config/lsp-install setup running, but I've got nowhere near the level of "useful" messages as whats seen on this video.

[u/4rlen]

I recently ve been looking for some tutorials how to properly use async in Rust. I decided to just use threads instead. And here you are!