Rust Linz, September 2021 - Yoshua Wuyts - Futures Concurrency in the Future, Maybe

hey everyone uh welcome to my talk futures concurrency in the future wait big screen i can see myself okay that's gone um futures concurrency in the future maybe kind of uh rustling hey uh so yeah i'm josh uh i work as a rust developer advocate at microsoft as my day job so you know in case you don't know what microsoft is uh microsoft um yeah so um i've got cats uh that's very important this is chashu uh she was eating from my hand up until like just now there's a treat here but she's like no longer interested so maybe she'll make an appearance later and this is nori uh sitting in a box um yeah and this was them together in our last apartment uh doing what they do best which is asking for food that they're not allowed to have and they look very cute so yeah that's oh oh oh double slide oh what else did i do yes uh maybe you know me like what might you know me from uh i helped create async stud once upon a time uh i am currently part of the async foundation's working group it's a rust working group uh responsible for making async await things happen in the standard lib and the language um so yeah um why this talk like what what what are we talking about today what what's the broader theme here so um we are talking about futures and async and rust because it started in 2015 or maybe 2008 kind of depends on where you where you how you start counting um but stood future today in the standard lib kind of looks like this uh there's like a couple traits and like three functions and like barely a description so it's not much we're trying to expand on that um you know like the future trade itself only has like a single method and you know what we really want to do is we want to expand on on these trades we want to build them out we want to make it a richer experience move past an mvp stage so in this talk uh what we'll be covering is like um how like possible designs and considerations around those designs for async concurrency adapters that we could potentially add to a standard lib so that's that's the broader theme and we'll dive into details um more specifically for that so um what we'll be covering in this talk is first we'll be looking at terminology then we'll be looking at the various modes of concurrency um establish a little bit of like theoretical background stuff there i guess uh api design comes next um then we'll be looking at uh merging streams which is another kind of concurrency not quite futures but a little bit different and finally looking ahead so starting off terminology hey we all love terminology so this this future is intended to be like uh i don't expect you to like be on the land team or anything so i'm trying to like start from the start um so if you already know these things then yeah just bear with me uh so parallelism and concurrency like they're different um and like but that what um what parallelism is you you might like sometimes like hear them like used interchangeably but they're they're separate things really like the way i like to think about parallelism is as a resource like any computer will have a set amount of parallelism that it can like give right like usually people refer to it as like the the hardware threads in your computer but sometimes it's like locked down through software like container or whatever they have like fewer threads available less parallelism available than like what what the underlying hardware exposes so it's just a resource and it's it's usually like a max like a minimum of one maximum of however much your computer are going to expose right so using multiple cores on machine is like a typical example of parallelism in action multiple cores work at the same time and do stuff at the same time right the the way you can measure parallelism using rust is you use the num cpu scrape and you like call it and gives you back like a number and that that's that's the resource usually now um we we won't be talking about that today so concurrency not parallelism so all the older multi-core stuff not really in the picture for today um instead we'll be talking about concurrency which is a system structuring mechanism i i lifted this from literature by the way um so you know in node.js uh is famously single thread like single threaded javascript runtime multiple threads behind it but but the actual thing you execute on a single threaded but still async um that is because async programming does not require multiple threads it's because concurrency does not require multiple threads you can structure a program in such a way that you just use one thread but you still have stuff going on um executing concurrently right so they're they're not the same thing uh yeah so let's give a practical example or well not quite so what's the future a future is an asynchronous value um the way you can think about it it is a value that you know will only become a value after you called a weight on it there's some magic on the under the hood going on i i don't want to dig too deep into that but here's like a practical example say we have like let num equals 10 we're defining a variable here of name num uh which has a value of 10 right here's the async version of that where we have a future that will resolve to the value 10 and when we await it it resolves to that value and then we can assign that to a variable binding right this is the same thing of writing like what we had here instead of like creating an anonymous future using the async keyword we can manually create it using like ready this is also the same the same exact result um but yeah whenever you see this you can just think that you know instead of like resolving to a number we could be doing something more interesting like reading a file uh asynchronously whoops i swapped the question mark can you wait there but you know you you get the idea hopefully um yes so yeah let numb async 10 it's a placeholder for doing more interesting things like reading files or doing like network requests or stuff like that all right so yeah well a future resolves we can do other stuff concurrently is the main takeaway of like what a future is so let's take a look at um the hello world of uh concurrency so like a quick little join here right so we uh have like one um so starting off like let's let's do some stuff like sequentially let's define like a future a which resolves to ten uh we delay it for like one second the delay is part of async stood i don't know if other runtimes have it right we say like hey this resolves to number 10 after waiting for one second right b the number 11 also wait one second and then we await a which you know results in 10 then we await b which results to 11 and that took two seconds right one second for one another second for the second total time elapsed two seconds right so what if we did that concurrently you know we could do a dot join b and then await that and then we get a tuple back and now what we're doing is while a is not doing anything right while it's like sleeping and we get back to it later we can kick off like the the execution of future b which also waits for a while and then once both are resolved we get the result of both back and now how long this took is just one second right one second one second but they happen they're waiting at the same time so we don't wait like in real time this actually reduces the real time even though not multiple cores are involved it's still concurrent so it reduces real-time execution and we're we're not waiting around for stuff uh instead right yay concurrency okay so hopefully that that sort of sets you in the mindset of like how how concurrency is kind of used all right first act uh modes of concurrency here here we get a little bit techy um first off infallible concurrency so what's infallible uh basically a future which does not return a result that's what we mean by infallible here right so we wait for all inputs uh the two modes of like infallible like concurrency is like wait for all inputs or wait for the first input right future join was wait for all inputs we give it like a set of inputs then we wait for all of the inputs to resolve and get a set of outputs back right of resolved futures um and then there's like way for first output so the way that happens is here's our like join example which is like one delay right now of four seconds right instead using a race we can rewrite like this and we say a dot raise b dot weight and that results to 11. so what happens here is we have two futures of the same type and we try and get uh the few or we get the future which resolves first so uh future a takes four seconds to resolve future b resolves instantly which means we get the value of future b which in this case is 11 and we discard future a um this is useful when say you have um a file system call like that you're like getting something from cache but maybe you also have a request going on you're like oh just give me whichever one like resolves first um i believe firefox used to do this or still does and sometimes like networks faster but sometimes you know on very fast uh hard drives uh uh hard drive is faster so it kind of does both and says give me give me the first result um that's what bracing does so yeah wait for first output now um if we introduce uh results like fallability into the mix uh things become a bit more interesting so fallibility is when or like something's fallible when it returns a result so in this case a future of a result uh these are like effect systems you need to like figure it out or like no not too much about it but you know it creates like a fun little matrix so fallible we we still have the first two which is like wait for all outputs wait for first output that's like one axis and then there is also um do we continue on error or do we return early on error right so we can plot them out like this a little table and then you know our join example we can fill in there which is we wait for all outputs and then we continue on there so if an error occurs we we just keep on going now here we have a race right um here uh we can put that in future race uh which way it's for the first output um and if an error occurs uh we actually do return early it's kind of like counterintuitive that like sits on the bottom right corner but it's not aware of like what result does so an error occurs it says oh cool i got the first value if it wasn't there it's an error right so we return early now what if we oh here yeah what if we like started using different ones right what if we wanted to target the other one so in async instead we have a method called tri-join uh which does that oh the alarm's going off outside um right which can do a dot try join b uh both like okay and then here we uh get back 10 11 because both values are okay um but if we switch it up and we get like a okay and an error uh future right then this will resolve and will like return early uh it will short circuit and resolve to the error value right so we can put that there this returns early on error but we'll still try and wait for all outputs like otherwise right so if the values are all okay you get all the values if one of them errors it drops all the futures and it just gives you back the error now the bot in the top right corner a.race we can replace that with like a.tri-race right we await that and here you know in in the case of okay we just get back okay but if we convert this to error then we get back yeah uh we still ah you know the the first feature to resolve is future b but we get back the value of future a it looks at future b it says oh wait that is an error uh let me not return early quite yet um let me try and like wait for future a to come back and then it says like okay then the name tri-race is kind of like not great here and we need to work on that but you know um yeah we can fill that into into the chart like this so if we rewrite this and we both have errors then you know the output of this is it's an error right it will try and keep going and try and get like a single value like race values up until it has no more values to like resolve uh and if there's an error it will like retry um that's what that does uh yay that's good okay so uh yeah yeah that's our little overview um other languages have concurrency methods as well um subscript or node.js uh you can plot the the promise methods out kind of like this problem is all settled goes there promise raise goes there promise all there i promise any in the top right corner and and these do the same thing uh these are all like added or like stabilized late last years when they when they all became available so now we have like a sort of like framework of like the various concurrency modes um we can dig into api design like i briefly mentioned uh some of these things are already like available in libraries in async stud for example both have futures rs and i believe tokyo probably also has something you know but but you know we're trying to like figure out um ways to like stabilize this into standard lib and recently this talk is i i think we can do better with api design so let's let's dig into into the exact apis here for a sec right so we have a two-way join i'll use join for as the example but kind of applies to all the all the combinators here so two-way join looks like this future a b we join it we get a tuple back right so the return type of that is join uh t1 t2 for like a tuple because tuples can contain like uh different values values of different types right so it you need to represent that like in the tuple so a three-way join um looks like this so we start off with our two-way join uh then we create oh no very fancy we split it off into like its own future it doesn't matter right but uh we create a third future resolves to number 12 we join that and then this becomes uh a nested tuple so we get um open brack a tuple of 10 11 close and then like on the right hand side number 12. you would probably expect like 10 comma 11 comma 12 but becomes like nested right that's not good it's kind of like annoying to work with so what yeah that's the return type it's because we're like nesting join um futures right so a a four-way join you can probably expect it now but you know uh we add a future d and we get back you know a a three level nested like tuple that's you know you you can imagine as we like start to like nest more and more things become worse and worse right so the the way this is um solved in many of the libraries is by having a join macro which uses an async block and this cogen based on the input to then return a flat tubable of of the output types and the return type of this is generated by the by the compiler at compile time essentially using the async block okay so um anyway drawing right a question here that we need to consider is like is n const or cons means does the compiler know the length at compile time can quite fit into a bullet point so yeah but yeah do we know the length of um the type or the size of the type at compile time right so here uh if we don't like what we can do is we can like create two futures put them inside of a vec and keep adding futures to avec effects can grow as long as you like as long as you don't like run out of memory you can just put them in there you don't need to know up front how long a vec will be that's it's nice quality right and then we can call something to join it and then we await that and we get it back back and the method in futures uh for that non-basically we don't have this uh is join all so you know that's a different import you need to be aware of it so you need to use futures future join all for this whereas um for the joint macro use it you for example use async stud future join these are all like slightly like different ways to get there and you like need to remember them which is not great and like they're also like different calling conventions oh whoa yeah like one uses uh like you know there are different import paths one uses like is like a macro invocation lives in the macro name space the other one's not but you know depending on like a variety of factors you need to think of like okay which one should i use do i want to use types like race all dry erase all tri-join all race bang join you know if you look at the whole matrix it becomes like a lot so um yeah it applies to like all the other concurrency adapters too so i set out to be like well can we unify this i've been thinking about this for a couple years now i've written a couple blog posts um i came up with an experiment uh it's called futures concurrency it's a library uh you can you can use it today it's it's not super it has like eight commits so don't expect too much yeah but um i i just want to show like the direction that we could be taking things or you know i'm i'm currently taking things so two-way join um with you know we saw it like this a dot join b but using uh future concurrency and using the traits defined there we can uh use a comma b dot join and if you join that then the output type uh becomes 10 11. if you want to have a three-way join all you do is you define a new future a comma b comma c comma like dot join and then you get back 10 11 12. and you can sort of like keep expanding on this right so you know if we have 15 futures we get to like you know just join all of them like manually if we want to and and that will just work um not just tuples also also arrays by the way uh it's kind of nice so for like um non-cons types for like types um whose size we don't know like during com compile time so like using vex of futures for example rather than doing join all rewrite it to um then just be like avec vec ap and we just call dot join on the vec and we get a new back of the resolved futures which which is quite nice so um yeah that that's that's sort of like where we're at in terms of the design um you can use this with tuples you can use this with vex you can use this with um regular arrays as well um and you know they'll they'll just work the the nice thing here is you have a container type you put a little features in there and then you suffix it with your um your mode of concurrency whether it's join race join all etc right that's the idea you have your futures you join it or you you use method of concurrency and you get your values back out in the same shape that they went in so the translation hopefully becomes like very fluid um so that's the theory uh implementation yay okay so it's very short so what do we have today right futures concurrency uh the joint trade looks like this it is a single method that returns a future there's not much going on not much magic and it's implemented manually for a whole bunch of like tuples for vec and for array um so that's not much done we don't have any of the other methods quite implemented yet right and you know like some some stuff that we still like need to figure out is like uh is this the right trait uh how can we like move this right into standard lib um there's some i have a working theory that we can like move it in there today but we need to seal it and like you know lock it down so we are like future compatible but uh yeah and there's like legit questions also like how do we introduce try join there's some like fun considerations of like try join should be on the same method as like join as a joint rate but we can't express that today so how do we do that you know there's stuff to figure out so experiments left to do but if you want to use this mode of concurrency today it is fully implemented like an async stud so you know although try joy and try race all these things they they do exist albeit like flagged um yeah so uh act four uh merging streams and this is new content um what i mean by that is i published a blog post last week covering most of the stuff up until now um but this is a unreleased blog post and essentially all the stuff that i had to cut from the previous blog post that has become its own blog post so um if you didn't read that then this will likely be new anyway that that was a lot of words to like me have a sip of water as well um so um all right uh merging streams so oh oh interesting i forgot what example this is so i'll just talk it through what we're doing here is we define a future of one we give it like of a value one again you can replace it with like do a do a file system call or like get something from a channel stuff like that where it officially added a delay of one second so i'll resolve after one second right second future also with a delay a few seconds third future three seconds so we know a results before b resolved before c right so here we create a little array we call dot race on it and then we call await and then then we get one back so the problem that we're like trying to the resolve or the the problem that we're trying to cover here is like um what if we want to do two things right like get uh futures back as soon as they're available right so we want to have the value of one like available to us as soon as it like resolves but rather than like discarding uh future b and c right which we currently do race does that um we eventually also want to get the values of b and c because we care about those but we want to start acting as soon as a is ready right so race is not the right way to do that and so um yeah well how how can we do this so looking at the past um the futures arrest library uh says that we can use the select macro for this so um the way this works is and i i took this example from the async book which is officially part of the rust org right it says like hey this is a hello world of how to do stuff with like select so you know not trying to create any any specific examples but i i think this is like pretty representative right so this is the the basic example which creates a counter that sums up a bunch of like things uh numbers like going through it line by line we import futures stream and stream x we import the select macro from the futures lib um we create an iterator that gives us or like a stream right from an iterator which yields the numbers one two and three then we create a oh and we call fuse on it which is important like what fuse does is um it says that once this stream has uh yielded none once it will forever continue to yield none rather than say panic which is valid behavior so we fuse it and then we have stream b and we like make it return like a vec of like four five six we fuse that and then we initialize our counter we initialize it to zero right create a big loop because we want to loop over the streams within it we now start to use the select macro uh the select macro will return like an item of a type um so here um we say we assign item to a.next so whenever a dot next resolves something we create a variable called item and we return that from the block same for like b you know that also returns item and here's like a special select keyword called complete and says when complete when all the streams have been exhausted um break so that breaks the outer loop right and then we match on item and we say hey if this is sum get the number contained within like the option and add it to our total and finally we assert their totals 21. so again putting it all together we we have like the various parts and that's hello world for using select um yeah so um async stud stream merge that is a different way of doing something very similar and i'll show you how here so um you import these into prelude you import async stream and then you know we're going to create the two streams again right stream a and stream b however we don't need to fuse them that that's like one very nice like aspect of this right no fusing going on you just have two streams then we initialize our counter like we did and we call a dot merge b which creates a stream of like a shared stream from two streams it says as soon as a value is available from like either you know uh give me that value so in this case and it'll do some like random stuff in there but essentially it merges multiple streams into a single stream not like zip which creates tuples i believe or like chain which exhausts one and then the other now it says like whenever there's data i'll take that data it's all async so stuff resolves at different times right so aid up merge b creates one one one stream here and then we say for each number uh we add that number to total we await that and then we assert our total is 21 and then we can you know because we're doing stuff with numbers here you probably you you may notice from like the iterator uh async that stream like follows all the iterator adapters and stuff we can instead call aid up merge b dot sum so we don't like initialize a total number but we just say like sum all the numbers and then we get back a total and then you know we can even in mind that and then we truly have like reduced that whole like logic into a one-liner that's not unreasonable i'd say right so um one thing that um like the select blocks like says it's like really good for or like attempts to also like do is like not just like handle streams but also handle uh resulting futures so you can put like a stream in there but you can also just directly give the futures and though when those resolve you can like select over those so [Music] let's take a look at how that looks using stream merge oh yeah so the there there will be like roughly three steps um first we convert our futures into streams so um here we have three futures or are like uh sorry a a is an array of values which will become a stream b is an array of strings stars like static stars uh convert that into a stream because multiple values and then c is the future which is also a type like stir right so stream from itter stream from error and stream once stream ones takes um a single future and converts into a stream right now we have a b and c and now all three are streams we normalize the futures into streams first second step is map the streams to an enum so what we want to do is we want to convert all of the different types into a single type and the way rust does like what you call them set types is using enums we define an enum so here we say in a message which either has a num or text right and then here in our stream we say hey all the ones that are like numbers uh dot map them into numbers all the ones which are text dot map them into text right now they're all of the same type and now that they're all the same type we can call merge on them right so here we go adap merge b dot merge c this now works which gives us back our stream and then we iterate over the stream we get a message out because it's an e m and we want to get the values out we match on it then we say num uh we print it then we say text and we print that and now we have like it all together we have like a variety of input types and we merge mole into the same stream variety of like you know it's straight it's streams combined with futures combined with like different types and they all like get flat into like a single loop and rather than having like a select block with like custom syntax and semantics and stuff like continue or what was it uh on break yeah for the i forgot what the keyword was but it just becomes back into into a match statement which is kind of nice right so um shiny future this is the lasso part um it's the last little part it's all it's what oh we're almost there we're almost there all right so sort of like taking putting our little like hats on and being like oh what could stuff look like in the future if we had like a bunch of like language features right so uh warning this is all made up none of this lives in rfcs all of this lives in my brain so what you're seeing here is not real don't expect it to become real but it's fun to think about and i think it's also like fun to sort of like set a dot on the horizon of like where could we go right like what would be better how could things look like because that helps us like design things as well right so this is my grandiose design um there's a lot going on here let me break it down so uh first off uh whoa yeah yeah we have our we have our three part oh wait hold on oh did i add the wrong yeah it was wrong sorry there's like three parts about this right we have the enum message part where we define our enum then we have our center i added those arrows wrong yeah then we have like our center little part where we're like mapping values into like stuff and then we have our final part where we're like looping over things so first off async iteration right so async iteration currently doesn't exist with like while at some await loops uh instead one could conceive we could have like 408 message in sorry like four uh x in y loops we would have four or weight x and y loops right so that's something that we could get into the language most likely async overloading is a fun one for the blog post on this maybe um the idea is that just like swift async has async overloading built in um maybe we could have it so rather than like having separate trades for like um either an async hitter right or ether and stream the way it is like today today um we could detect like uh is this running like an async block are you calling this an async block if so convert this into an async method right so here we see um our thing we call intuitor and that just works because it's like an async version of that trait um finally the merge trait what if uh stream merge had its own like traits and it was like part of the top level like imports and scope right we could do the same thing that we were doing earlier which is be like oh you just have like um an array of um separate like streams and you call that merge on them or vec or a uh yeah maybe not tuple but the other two yes uh right then async io in standard lib like what if print line you know actually didn't like block but would like be async requires executor runtime all this stuff right inline print args i believe it's coming in russ 2021 which is a couple weeks out but you'll just be able to do this so you don't need to like have separate comma and then param but you can just inline the param which is nice uh match shorthand is another fun one what if we could like rather than like indenting the match statement in a separate block we could lift it out to the top like so it becomes like a one-liner with the 408 maybe that's nice right so yeah that's all like very shiny future maybe maybe we can make these things happen i i think it could be fun to like get some of this stuff in right so act five looking ahead uh yeah so what's next um for this whole proposal i'm sure it was like quite quite a bit to take in um but yeah i'm gonna release a third blog post in a series of asynchron currency blog posts or futures concurrency blog posts um i'll be covering what we covered in the second part which is the select blocks and stream merge and stuff right i need to put it into writing so i can share it with like the rest of the working group um so yeah that's in progress right now uh we need to implement try join i think i mentioned that earlier as well there's like we don't have to try joint methods and tri-race um in in like the the futures concurrency library yet um then that will have some fun design complications implications that we may want to address at a language level as well so we'll be fourth blog post and after that we'll probably need some time to digest all of this and figure it out talk to folks see how people feel like stuff works and if all that goes well we can start writing rfcs to actually starting these things with standard lib yeah anyway that's the timeline there that's it also for my talk yeah thank you so much you

Channel: Rust

Views: 4,146

Keywords: rust-lang, rust, rustlang

Id: QlPDI9IsSXU

Length: 37min 5sec (2225 seconds)

Published: Fri Oct 01 2021