CppCon 2016: James McNellis “Introduction to C++ Coroutines"

so good morning everyone can everyone hear me I hope excellent can everyone see the slides okay so my name is James McNellis I'm a member of the visual C++ team at Microsoft and I'm going to give an introduction to the C++ Cortines feature today just a few notes before we begin so this feature is still in development it's not yet standardized I will not be part of C++ 17 but it is going to be part of a technical specification that should be done soon I'm here to describe the current proposal not to argue about you know one particular design versus another though I'd be happy to do that outside you know after the talk so with that out of the way just one brief thing so our team the visual C++ team is very interested in your feedback in you know learning a bit about how you use C++ and so if you would like an opportunity to win an Xbox 1s we've got a survey here and I'll have the link at the end of the talk as well I've been told that I am not allowed to win it so I want all of you to try and you know have that opportunity so to start off here what's the motivation for this feature so why add co-routines at all you know a lot of effort has gone into adding this to the language a lot of design work I don't know if it's quite as much design work as went into the digit separators but it has been it has been a substantial lot of work so what do we gain from adding co-routines to the C++ language and to demonstrate that I'm going to use an example that I've lifted from a presentation gore gave here last year from his talk so here we have a little synchronous function in this function we have let's say a TCP library and it establishes a connection to a TCP server and then in a loop it reads for K chunks from that connection until either it's read the number of bytes that were requested or it reaches the end of the stream and then it returns how many bytes were remaining that it did not read so this is a synchronous program the call to TCP Connect is going to block until it establishes the connection and then each of those calls to read inside the loop is going to block until the read completes and note that in reality within that loop we would actually be doing some data processing which we've just emitted here for brevity so since these operations may take a long time we don't really want to block write we want to take we want to tell the Machine hey go establish a TCP connection for us and when it's ready let us know so that we can continue with the program and then hey go read four kilobytes of data from that stream and when you have them let us know so that we can use them and continue on in order to do that we're going to have to turn this into an asynchronous program and to do that we're going to have to make a few changes so the first change is to the signature of the function instead of returning an integer we're going to have to return a future of the integer because we're not going to have the value when our function returns next we're going to have to wrap up all of our state into a little structure so that we can pass it around between the parts of the asynchronous operation and because that that state is going to outlive the frame of the function we're going to have to dynamically allocate that on the heat then we're going to establish the connection and we'll assume that our off the author of our TCP library is very helpfully giving us a nice asynchronous API to replace the synchronous one we were using before and when the connection is established we'll say okay call this continuation and we'll use it so then inside of that continuation we're going to enter a loop the same read loop that we had before within there we're going to actually go and do the reads so each iteration in the loop we're going to read from the connection and then each time the the TCP library has data for us it's going to call us back and let us know the state that we have there's still a little bit of magic here this do-while function is not part of the standard library but we can implement it rather simply so how many people think this is just as simple and good as the synchronous program how many people would like to write code like this how many people are stuck having to write code like this today yeah okay there we go so there's actually a bug in this slide can anyone can anyone see it yeah Gore can see it see I can't call on Gore last year when he gave this talk I raised my hand because I'm an idiot so if we look again at our synchronous program we've forgotten to return the result so we just need to add one little line of code here to return the result anybody like to guess we're not you've gar well we just need one more continuation and when we're done with this entire loop we will return the results in a ready future so simple yeah this is great why are we doing this co-routines thing not really that's pretty horrible so our asynchronous code ended up being a lot like regular expressions you may be able to write the code that does the right thing though not entirely sure about that and even if you do it's going to be hard to maintain and modify and even worse to debug so what if this is our synchronous program again with a little bit of whitespace added to it what if we could write code that looks just like this but actually has the behavior or similar behavior to that asynchronous program and the answer is with co-routines we can and so what do we need to do to make this program asynchronous here we go three changes if you didn't see them here they are in red so we've had to change the return type of a function to a stood future of integer because again we're not going to have the result when we return immediately then we've had to add this code weight keyword and we're going to spend most of the rest of this talk explaining what that keyword does and how it works so this program now has effectively the same behavior with respect to the asynchronous API that we had in the other asynchronous program it's just an awful lot simpler so how many people think this is a little better than the than the other example yeah okay that's basically everybody in the room so let's start with some basics what is a KO routine so a KO routine is a generalization of a subroutine well okay so what is a subroutine a subroutine has two properties so it's a thing that can be invoked by its caller and it can return control back to its caller so hopefully everyone is familiar with a particular type of subroutine in C++ it's called a function so you can call a function and then when the function is done doing whatever it was doing it's able to return control back to you so that you can continue doing other work a cur routine also has these properties but also has two other properties it can suspend execution and return control back to you and then your caller can resume you and you will pick up where you left off so with the C++ code routine specification both subroutines and Co routines are functions and the function can be either a subroutine or a Co routine so I put this little helpful table here and you'll see we have the four different operations and four subroutines we only have two of those we have in order to invoke a subroutine we make the function call in order to return from the subroutine we either use a return statement or we just fall off the end of us of this function for a KO routine we invoke it exactly the same way we just make the function call we return using this special ko return statement which we'll look at and then we suspend execution using KO await and we'll look a little bit later at how we how we resume what makes a function occur routine so here we have a function declaration how many people here thinks that think that this function is a KO routine okay how many think this is not a KO routine okay so it may be or it may not be whether a function is a KO routine is an implementation detail of the function so if you have a function today and you want to turn it into a KO routine you can do so without breaking any of your callers it has no effect on the function declaration at all there may be some return types that only make sense to use with co-routines but stood future is not one of those so for for example you can implement a function that returns to the future either as a KO routine or as a subroutine so what makes a function of Co routine well a cur routine function a function is a Co routine if it contains a return statement a code weight expression a KO yield expression or a range based for you loop that uses KO await or in short a curve a function is occur routine if it does care routine things or if it if it's capable of suspending so here is that compute value function I have two different definitions of it the one on the left is just a normal function you can write this today in C++ 11 you can see that it calls stood async to go and run you know this very expensive task on a background thread and then return you a future that refers to it and on the right we have the same function implemented as a as a KO routine and it uses this KO await which to get the result and then ko return to return the result so what does this code weight actually do well the compiler takes the KO weight expression and it transforms it into a slightly different code and this is the code that it ends up generating so first it take that expression and it stashes it in a you know a fake little variable said it can refer to it and then it asks the thing well are you already ready and it does this by calling the await ready function and if the if the thing is already ready if it already has the result that it needs then it's not going to go through the expense of suspending if it's not yet ready then it's going to call this await suspend function which allows you to customize the but whatever happens right before you suspend and then when that returns you're going to reach a suspend point and it's going to suspend execution of function and return controller back to the caller later if and when the caller decides to resume you you will pick up right where you left off right there and you'll continue on with the program and get to the next line it's going to call them this a weight resume function which allows you to do two things first it allows you to custom custom action when you've resumed and second it actually returns the value that is the result of that expression so the result here is going to be the result of that call to await resume so from this we can see that in order to await on something it has to provide three different functions basically it has to provide await ready await suspend and await resume so let's look at a really complex type that implements these so this one is called suspend always and it's part of the library support for co-routines it's very simple all it does is when you await this it will suspend the function so you can see here the await ready returns false saying I don't have the value so you need to suspend and then we don't have to do anything on the await suspend and a wait resume and here is a KO routine that uses this so if you call my Co routine it will print out my ko routine is about to suspend it will then co await on this suspend always and this will suspend the Co routine it will return control back to the caller if and when the caller decides it wants to resume the KO routine then the ko routine will pick off where it left off and it will then print my cover team was resumed there's another simple away table type called suspend never and this one just does not suspend so you can see instead of returning false from wait ready we return true and we can look at a program that uses this one too so here my ko routine if you call it it will print out my Co routine before the no Appa wait it will then await on the suspend never which will not suspend and the KO routine will just continue execution and then it will print out my ko routine after the no Appa wait so here the caller does not get the opportunity to resume the KO routine because it never actually suspends so that's the first half when a KO routine is executing it uses this KO await expression to suspend itself potentially and return control to the caller so the other half is how does the caller resume a cover teen and the answer that we have to look at how co-routines actually work under the covers so what happens when you invoke a function well when you invoke a function the compiler has to construct a stack frame on the stack and this includes space for all the arguments local variables the return value temporary storage for any registers if it needs to spill registers onto the stack but all of this happens behind the scenes you don't need to worry about this similarly when you invoke a KO routine the compiler has to construct a KO routine frame that contains space for all the formal parameters the local variables selected temporaries it also needs to have space though for the execution state for when it suspends when you suspend a Co routine it has to store any state that it's going to need to restore when you resume the Co routine and it's also going to have to store the promise which is the object that is used to basically communicate between the Co routine and the caller in general the Co routine frame must be dynamically allocated so the co routine loses control of the stack when it is suspended so it returns to its caller and the caller could call some other function and use any stack space that the cover team might have wanted to use operator new is used by default but this can be overloaded for specific co-routines to allow customization additionally the compiler may be able to eliminate the dynamic allocation if it's able to determine well the Co routine is not going to escape from the callers frame and so I can allocate it on the stack so the compiler is free to optimize there and it will do that and if you go to Gore's talk this afternoon you will learn that it will do that aggressively finally creation of this curve routine frame occurs before the co-routine starts running so it's just like creation of a stack frame not something you have to worry about it just happens there in the background and what the compiler will do is it will return and I put in quotes because it doesn't actually return it in the sense of using a return statement it will return a handle to this co-routine frame to the caller of the co routine so what does this handle look like well there's two types or it's one type that's specialized so the first is a cur routine handle of void and this has all of the basic curtain handle functionality and it's used for Co routines that do not pass objects back to their collars so kind of like a function that returns void and then there's the Co routine handle for all of the other types of things that you might need to return and this is derived from the curtain handle of void this is to provide simple Interop but it's also because it needs all of those operations and it'll provide a few more things so here's what the curtain handle of void looks like first it's possible to have an empty co-routine handle that refers to no ko routine and so the default constructor will give you one of those you can construct it from an put er you can assign and I'll put our to a co-routine handle and you know then it has an empty state and there's an a conversion to pool that lets you test whether or not the co-routine handle is empty second you can convert a care routine handle to of white star and then convert a void star back into a care routine handle so this is to enable interrupt with for example capi so there are many thread api's that allow you to pass in a function pointer that will be called and then a context pointer and so this allows you to convert to a void star that you can pass as that context pointer and then on the other side take that context pointer convert it back to a a KO routine handle so you can invoke it third the co routine handle provides the ability to resume execution of the of the cut routine and so you can do this either by calling resume or by invoking this function call operator they do the same thing fourth the care routine handle provides the ability to destroy the co routine explicitly before finishes executing so this will call this will cause all of its local variables to be destroyed as if a return statement was evaluated in the co-routine and this happens at the point that the cover team was last suspended and we'll see some more detailed examples of that a little bit later and finally the curtain handle provides the ability to test whether the co routine has completed execution it's just called done so in the other so then the other the specialization of care routine handle for all the other types just adds two extra pieces of functionality first it gives you the ability to get the promise back and again the promise is the thing that allows you to communicate between the co routine and the caller and second given the promise it allows you to get the co routine handle for the co routine from which that promise to which that promise belongs since the promise is part of the co routine frame that was allocated the compiler knows how to generate this such that it gives you the correct co-routine handle address okay that's enough of that so let's build a really simple chi routine so here is a co routine and it's got this resumable thing return type and we're going to implement that and they tickley what this function does is it'll let you know hello I was called and then within a loop it's going to suspend each time through the loop and each time it's resumed it's just going to print out to tell you how many times the care routine has been resumed and here's a main function that uses it so we can see here if we run this program first the main function will print while I'm going to call this counter function and then we call it and the counter will let you know okay I was called and the main will let you know okay I'm about to resume the co routine it resumes it once and then twice and then the main function will print out and just let you know okay I've reached the end I'm done so what do we have to do to implement this resumable thing so the first thing we're going to need is we're going to need to define a promise type for it and this is going to be that type that is used under the covers to communicate between the co-routine and the caller within the resumable thing we're going to need to store the handle to the co-routine so that we can resume it its constructor is actually just going to take that and we'll see how we construct that in a moment and then in the destructor we're going to we're going to call destroy because again in that function we will never actually reach the end so we're going to need to destroy it somewhere we have a bit of boilerplate that we need to do so here you see we have the default default constructor and then we do not allow copying but we do allow moving from one instance to another the co-routine handle type doesn't provide any sort of lifetime management it's just like a raw pointer that's exactly what you want so that you can build bigger abstractions on top of that like this type here so then we need to implement the promise type and for this we need to implement a few functions the first function is get returned object and the compiler is going to generate code that calls this in order to convert the promise type into the resumable thing that we're actually going to return to our callers and so here you can see we construct a resumable thing and we just call that co-routine handle from promise we know I'm the promise for the co-routine so get the handle that's equivalent to that and we use that to construct the resumable thing and then we return that second we need to implement these initial suspend and final suspend functions so these and we'll see in a more detailed example in a moment these allow you to determine whether or not the care routine suspends before it starts executing and at the very end of the Coe routine when it's ready to destroy itself after returning and finally we don't actually return anything from this curve routine we don't need to return a value back to our callers so we have to implement this return void function that just does nothing so back to this example so this is the Co routine the compiler is going to go and generate a contact structure for this cow routine and so here you can see it has it contains the promise it also contains our local variable I and it's going to contain an instruction pointer or something that it that it can use to when the Coe routine is resumed go and start resume execution where it left off additionally it may need to have stored four particular registers depending on how the compiler optimize the function it may have other temporary variables that's introduced and then the compiler is going to inject some code into our counter ko routine so at the beginning of the function or the ko routine it's going to construct a new instance of that Co routine context using operator new it's then going to get the return object the resumable thing and it's going to do that by call and get return object and it's going to store it wherever the caller expects it to be and then it's going to call that initial suspend function and it's going to Co await on that so if you told it for example if we used suspend always that would cause us to suspend immediately if we use suspend never then that means we'll just continue execution at that point then we'll enter the body of the KO routine and we'll execute as normal and then at the end of the function it's going to inject this additional code so here it has a final suspend label and we'll see where that's used in a little bit it's then going to Co await on that final suspend so again this gives you the opportunity to suspend execution of the co-routine before it destroys itself and then it's going to delete the context to clean up the resources so this is not the only thing that we can use this resumable thinkoh routine for we could also create another co-routine that uses that very same type that prints out a name along with the counter so here for example we construct two of these counters named a and B and then we resume them in an interleaved fashion in the output of this function you can see it's going to construct the Eiko routine it's then going to construct the B color team and then it's going to resume each of them and you can see that these are totally in parallel there's no interaction between the two of them they're independent we can resume or destroy one of them without affecting the other one so now we can fill in that last row of this table and we can say that in order to resume a KO routine we call the resume function on the care routine handle there are any questions excellent yes so the question is can you pass the resume to a different threat or could you resume on a different thread yes absolutely and if you come kenny kerr and i are giving a talk this afternoon and we're going to have a lot of examples that show how how we do that and a lot of the examples even fit on a single slide so for example if you wanted to resume a KO routine on a totally different thread yes yep so the question is inside of the co-routine frame the compiler captures a lot of things that basically you don't you don't explicitly specify like with a lambda what you wanted to capture so it's going to capture basically all of the function state and it's actually not going to need to move any of that because it's just going to construct it inside of the co-routine frame to begin with if there's any parameters they will need they may need to be moved into the co-routine frame yes but like all of the local variables it's not going to first construct them on the stack and then construct the Carine from those local variables it's actually just going to put the local variables in the carotene frame to begin with so for example you could have a non copyable non movable local variable and that would just work yes can a class member function via co-routine yes yes it would capture this and you would be incumbent upon you to then manage the lifetime of the class instance also yes I don't actually know if lambdas can be co-routines yes absolutely that can be cover teens I just hadn't written one like that okay one more yep yes so the question is given that you can destroy a co-routine at any time once it is at any suspension point does that mean that every KOA weight means you could have the stack online at that point and the answer is yes and we'll see an example of that so that's a KO routine that doesn't actually return anything but really we'd like to be able to return things from our KO routine so here is our compute value from earlier and this actually needs to return a value it needs to return an integer through the future so there's two interesting things about this this example the first is why is it ko return instead of return and the second and this kind of explains it is well we're returning an INT and the return type is future event so you can't actually construct a future of int from an INT and we're going to see how this works so we've already seen that our promised type has to have a few things it has to have this get return object it has to provide an initial suspend and final suspend functions and has to have a 4/4 co-routines that do not return values to their callers it has to have a return void for co-routines that do return values to their callers you have to implement instead of return void a return value so instead of looking at our compute value we're actually just going to look at something a little simpler so here is a care routine called get value it's going to print something out it's not going to suspend and then when you resume it it's just going to return the value 30 and here is a main function that calls that and then inside of get value of course the compiler is going to generate that same boilerplate at the beginning and end that it did before what so what we really want to look at is what does this KO return do well given that we've implemented that return value function on the promise the compiler is going to transform that into a call to return value to set the return value that you've given and then it's going to go to the final suspend label which will ask is it should i suspend or not and then if you are going to if you're not going to suspend it'll delete the context so we're going to make a few small changes to our resumable thing in order to support returning of the value so this is our resumable thing from before the first thing we're going to do is we're going to actually have to store the value inside of the promise we're going to have to implement that return value function which is actually just going to set the value and then since we want to be able to call get from that main function in order to get the value back we're going to actually you know from the co-routine handle that we have get the promise and then return the value prop member of it so I do want to note these co-routines are all very low level because I'm trying to show the examples obviously in like real library code we want much higher level abstractions and we'll see we'll actually implement the care routine support for future at the end of this talk so those are the modifications that we need to make in order to support this and so here if we execute our program we're in the main foot program and we say we're going to call get value get value is called it says I've been called and then it suspends the main function will then resume get value get value will then say I've been resumed it will Co return the 30 and the main will print that the value was seven million fifty nine thousand five hundred and sixty so there we go any questions okay so what happened here well we have to look at the co-routine lifetime so a KO routine comes into existence when it's called this is when the compiler creates the care routine context as we've seen and then it's destroyed when either the final suspend is resumed or when you call destroy on the care routine handle whichever happens first so if we look at our get value KO routine you can see that in the final suspend label we call final suspend and then when that resumes we're just going to delete the context well the final suspend of our routine type returns to spend never so it's not going to suspend it's just going to continue on the context is going to be deleted and now we're accessing freed memory and so we don't get that 30 that we so nicely stored in our promise so we have to make one small change here in order to make this work does anybody want to take a guess at what we have to do yes so we just have to change the final suspend to say suspend always and by doing that when we're actually executing through the gate value co-routine will reach the final suspend label Wilco await on this final suspend it will say suspend always return control of the caller and we will not delete the context and so when we do that it will print out the correct result unless you like the other result better in which case well you can do that so when does the the resumable thing actually or when does the co-routine actually get destroyed then have we just leaked it and the answer is no because in the destructor that we wrote a while ago we actually check and say if the care routine was valid then we're going to call destroy on it explicitly and how does co-routine destruction work someone already alluded to this before so when you destroy a KO routine it's basically as if you had returned from the last suspension point so here for example if we imagine that we have a type that just prints out a message and it's destructor inside of this resumable thing we construct two of them and we have a suspension point between them and them in the main function we construct this resumable thing we resume it and then we end up destroying the co-routine frame when the resumable thing is destroyed and you can see here that first b gets destroyed and then a gets destroyed just as it would be if you were returning from the function now what happens if we don't get to the end of the curtain or we don't co return from it well if we just resign that if we eliminate the code that resumes the cover teen then we also no longer construct B so we never have to we never actually destroy it so just to recap the KO routine is destroyed when the final suspend is resumed or when you explicitly destroy it via destroy whichever happens first and when the KO routine is destroyed it cleans up its local variables but only those that were initialized prior to the last suspension point so with any questions on that yes yes so the question is how do you handle the case where the co routine is like sitting in the kernel waiting on something some kind of event and you try to destroy it so then I would have to say that you've got of a poorly designed care routine type right you haven't properly encapsulated it if you've given your caller the ability to explicitly resume or destroy you while you're actually you know doing work perhaps on another thread then you just don't have a high enough level of abstraction and so you wouldn't want to like you can certainly write live a library that would get you into that state but you wouldn't want to do that so again these Co routines that I'm showing they're just they're just very basic in order to show that like how things work under the hood if you want to see some real examples are talk Kenny Kurtz talked my talk this afternoon will show a lot of examples gourmet go into some details yes one more yes the question is how do function arguments get stored in the co-routine frame they're stored basically like member variables and I believe they are moved into the yes so if you pass a an argument into a function it will be moved into the end of the argument or copied if it's not moveable one more yep what happens if a co-routine throws an exception I'll actually save that for when we implement the co-routines for for stood future because we actually show what happens there all right so just a recap this is the same slide we saw before so let's implement something that's actually useful these care routine types are not particularly useful again they're just for exposition purposes so let's look at that stood future compute value again and let's actually make this work so let's pretend we've got a standard library that doesn't have care routine support and we want to implement this so we have to do two things first we have to make future occur routine types that we can actually have a Co routine that returns a future event the second thing we have to do is we have to make future a way because that async is going to return a future and we need to await on it and be able to unwrap the value to get the the integer result so let's look at that first part making future a KO routine type so we'll have to open up stood future and add a promise type to it or if we don't want to do that because perhaps we don't want to modify our standard library headers there is a class template called Co routine traits that allows you to specialize types without you know invasively modifying them it's the primary template it basically just gets that promise type from the type but you can specialize it for your own types if you want to so here for example we're going to write a specialization of this for future of T and define the promise type in here inside of this promise type we're going to store the actual promise which is the the sender side of the stead future if you will our get returned object function is just going to call get future on that which gets the future associated with that promise our initial suspend and final suspend are both going to be suspend never we don't actually need to keep the the co-routine alive in this case because the stead future instead promise share state that's not part of the care routine frame it's going to be some third object in memory we have to implement return value and so for this we actually just set the value on the promise and then finally if an exception happens in the care routine we want to actually capture that exception into an exception footer and set it on the promise and so in order to do that we implement this set exception function on the promise and when the when the compiler sees that you have one of these it's basically going to wrap the co-routine inside of a try/catch and it's going to store the exception put ER and pass it to this set exception function so this allows you to marshal exceptions you know to the appropriate context so hopefully that answers the question that somebody had a few minutes ago so that gives us everything we need in order to have a function that returns a KO routine that returns a future of int now we need to actually make the future a way to bolt so in order to do that we're going to have to open up future again and add those three await ready await suspend and await resume functions or if we don't want to you know intrusive ly modify future we can implement a separate type that will provide that functionality and we can over overload operator Co await to take in the future of T and instead convert it into a future a waiter of tea and the compiler will await on that instead so you can see here we just store a reference to the future of tea inside of our waiter and then our operator Co wait just wraps up or stores constructs a an instance of the future or waiter from that to implement a wait ready all we have to do is call F dot is ready when we suspend what we're going to do is we're kinda going to call then dot then on the future in order to schedule the continuation so when the future is ready this continuation will be run and it's at that point that we're actually going to run the coding resume the KO routine in that context so what this will do is when the future is actually ready it will call the result it will resume the KO routine at that point this may be on a different thread it may be on the same thread and then finally we need to implement a wait resume that gives you the actual value back and here we just call dot get on the future there's a couple problems with this slide the first is that stood future doesn't have a dot is ready and the second is it also doesn't have a dot then so this is an actually stood feature this is stood experimental future which has these two functions on them so stood future is not particularly good in its current form to use with co-routines stead experimental future is also not particularly good to use with care routines and if you want to know more about that though you'll have to come to Kenny's in my talk this afternoon where we implement a replacement and show why stead future isn't so great so that's all we need in order to implement stood in order to support co-routines withstood future are there any questions about that yes yep right so the co-routine handle is the the handle that represents the co-routine so it you know if actually refers to that carotene frame and so when you call when you call resume on it it's just like using the function call operator and what it does is it just resumes execution of the carotene at the last point that it was suspended so wherever the last Co weight was it'll resume on the next line yep yes so the operator Co weight here basically so stood future does not have this await ready await suspend in a wait resume so the operator Co weight basically just lets you transform stood future into something that does have those and provides the functionality that you need so in this case we didn't want to modify stood future we wanted to provide this separate type future a waiter instead and so the operator Co way basically just provides basically just transforms the future into something that does that can be Co awaited on anymore all right so I'm going to show one more feature of co-routines and that's yielding so what if we have a Co routine and we want to be able to return values to our caller so we don't want to return just one value we want to be able to return a value and then allow the caller to resume you and you can return another value so why might this be useful so here is a little generator function called integers and it takes in a first and a last integer and it's going to return all of the integers in that range one at a time each time it's resumed so here's a function that might use it here we have a function with a range based for loop and it just has int you know the variable X and we run this program it's going to print out one two three four five now there's no there's no magic here we could actually write this just as easily with an on range for loop so we create we can start the generator of int and then we're just getting in a iterator from it which will show some of the implementation of and we just call begin and end on it and we iterate over it so what that cold does is it takes the promise and it calls yield value on it and then awaits on whatever it returns so in our promise type we're going to store a pointer to whatever the current element is we could also store the current element but we could imagine building generators of you know some type other than int where we not want to make a copy of it or move it our get returned object is implemented the exact same way it's been in the past our initial and final suspend are going to suspend always and the reason for this is is we actually want to be explicitly resumed each time we want to get a value out of the generator and then our yield value is implemented like so we just take the integer in we store it a pointer to it in our current local our current member variable and then we return suspend always so that each time that we yield value we suspend execution and we return control back to the caller so then we're going to have to have an iterator type for this generator we're gonna have to implement begin and end functions so in the begin we're going to check and see do we have a valid co-routine and if we do then we're going to resume it to get the first value and then if we're done if we've reached the end of the KO routine at that point then we'll return the end iterator the actual iterator itself just has to be a studied or a an input iterator because obviously we can't iterate multiple times with this single ko routine because once we've moved on to the next value there's no way for us to get back to that previous state inside of the increment operator we're just going to resume the KO routine and then again check if we're finished and then when we dereference the iterator we're actually just going to get that current value from the promise and so that is that so too just look at this table again so we've looked at the co return statement the co await and Co yield expressions we've looked at how we resume Co routines using the care routine handle we've looked at the co-routine control flow and how the compiler basically just transforms these Co statements and expressions into something slightly different and the boilerplate that we generate these are some of the design principles that were laid out in one of the original care routine papers that I believe Gore worked on I'm sure Gore worked on so the first was that we wanted these to be scalable to billions of concurrent care routines so one of the if you can imagine there's two different classes of care routines this these are called stack Lascaux routines so you always have to suspend from within the care routine itself you can't call a function and then suspend from within that that Colie the other time would be stack full co-routines where you can actually suspend from within Akali so the advantage of stack Lesko routines is that you don't have to actually allocate a whole stack or allocate a lot of memory for each of these you really only need to allocate just whatever you need for that one co-routine frame this means that you can allocate a whole lot of these so if you wanted to create you know a million Co routines and submit them all to a thread pool for execution you could do that and you're not going to run out of memory that doesn't mean that there aren't you know valid and great use cases for sackful co-routines that's just this is one of the benefits of this design they're all so efficient so suspend and resume operations are comparable in cost to function call overhead so additionally because the compiler is generating all of the boilerplate for you it actually has much more visibility into for example what is exactly going on so that it can make better inlining decisions potentially inline co-routines potentially elide the heap allocation for the co-routine frame they're open-ended so we've shown here how we were able to pretty easily bind stood future to the co-routine functionality and then we'll show later this afternoon some other types some other library types that we've been able to implement very easily with co-routines so there's seamless interaction with existing and finally it's also usable in environments where exceptions are forbidden or not available so for example if you're a kernel developer and you want to use co-routines that's also possible we didn't look into how that works but there's some other functionality that you can take advantage of to enable that to work there's just a couple references you might want to look at the first is a paper just describing the plan and then the second is the wording for co-routines the current specification document so there are three talks on co-routines today the first one you are sitting in right now so thank you for coming then we're going to have a suspension point we're all going to go and watch the keynote and then hopefully we'll all resume and you'll go to Gore's talk this afternoon which is also I believe in this room C++ care routines under the covers and then after that Kenny Kerr and I are going to talk about how we have put the kuru teens feature to work with the Windows runtime and while our examples are going to be window specific our goal is to show you know largely how flexible the co-routine feature is and how we've been able to use it in order to make our library both more beautiful easier to use and also more efficient and then finally anyone who wanted that link to the survey it's there with that I've reached the end I can't advance any further so are there any questions yes how how granular should you put a weight in your code it yeah so you certainly like if we're talking about asynchronous code you don't ever want to you want to avoid blocking wherever you can and so instead you want to await on on those things that would block and then schedule a continuation so that you're you know when it's ready instead of blocking when it's ready you'll be called yeah so the question is there is a cost to the await so it it depends and so it would depend on the relative cost of well how long do I think I'm going to block for right like if I if I think I'm only going to block for a few nanoseconds and maybe it's not worth you know going through the whole process of scheduling a continuation but in general you would just want to you know I guess you know in most like application-level asynchronous code you'd probably want to just a wait on every asynchronous operation to avoid the blocking and again the it like so the overhead if you're not you know if you're not doing a context switch right you're not scheduling something to run on another thread on the thread pool the overhead is equivalent to a function call so for example if you need to wait on a mutex and the mutex you can just acquire it you don't need to mutex is a bad example if you need to wait on an event for example and the event is already in the signal State then you may not even need to switch to another context it may just continue executing in which case it may be more efficient yes what happens if you issue an ordinary return statement from within a KO routine I believe you get a compiler error I don't think that's valid like you either have to have a subroutine like functionality or you have to have ko routine things in the KO routine yeah yes so couldn't we have made an ordinary return statement behave like a KO return I actually don't know the reason for that I know that in the original implementation that I used like a year and a half ago now it was that way and then the community decided to make it you know instead of just a wait and return ko await and ko return I thought it was because they just like you know funny phrasing but gourmet have more details nope he's nope he's got no more details there we go so the answer was that they thought that just saying plane return would be confusing and so they thought the co would help yes uh we will not actually cover that in our later talk it is described in the spec and it's not particularly a difficult read it's actually quite short yes so the question was are we going to discuss how you would overload operator new and the answer is we're not going to talk about in the next session it is described in the spec and I would be happy to provide an example if you're interested in one it's not particularly difficult yeah how would return type deduction work with co-routines I expect it does not just because you actually have to tell it what type you know so that it knows what type of promise to construct on the in the frame yep yes is there any support for a kind of placement new no because again you don't actually construct it yourself it's always the compiler it's converts generating code that's going to construct the frame so all you ever do is call a function NIT yes so you can one of the things I didn't go in here into here is when you specialized the co-routine traits you specialize it with a set of arguments and so you could for example use a custom allocator that would allocate it in a particular way yes we will we will be discussing this is Kenny Kerr who I'm giving the next talk with this afternoon we will be discussing how we optimize allocations and specifically in the stead future case with the state future example we showed you actually end up with two different allocations first because the co-routine frame has to be allocated on the heap and then because the shared state between the promise and the future has to be allocated on the heap separately and so we have several examples that show how we actually can allocate those together to eliminate some allocations in that case the Coe routine is actually more efficient yes yeah so the question is because these co-routines are stackless is it a this is a compiler error to make a function call from the KO routine and the answer that is no and so the co-routines we actually use to see out and you know with the the stream operators and so those would be function calls so what happens is is when you resume a KO routine all of its state is still in the care routine frame but once it's resumed it regains control of the stack and so it can call whatever functions it likes and it you know can use the stack for doing that you just can't what what it means by them being stackless is it means you can't resume from within one of those collies so for example if you've got a cover team named F and it calls a function G you can't suspend from within G G has to return and then F can suspend so a stack full co-routine would let you potentially suspend from within G yes yeah so the question is how common is the optimization going to be where the compiler is able to elide the construction of the co-routine frame so there are certain cases where it can't do that and so for example asynchronous code where it has no idea what the lifetime of the frame needs to be is always going to require allocation on the heap but other cases so for example this generator type that we have here the compiler should be able to see well the the co-routine never escapes from its callers scope right like it knows that that main function uses the cover teen and the co-routine is always destroyed before the main function returns so I could just allocate it on the stack there as for how common one design pattern is versus the other I mean it would just depend on the patterns used but Gore will be going into a lot of details on that or some details on that yes yep yeah so Gore's answer was if the co-routine type implements R III and I suspect if the object does not escape out of the caller scope then in that case the compiler should be able to perform the optimization additionally I forgot what the second point was but anyway yes that was the key so yes yes yes so a KO routine can call another ko routine and we'll have some examples of this afternoon in that case so when I said that their stack list so each Co routine can always suspend itself what you can't do is if you have a KO routine F that calls a care routine G like G cannot suspend f-g can suspend itself it can't suspend its collar yes yeah and again that goes back to when you see it when you look at a function declaration there's nothing that says it's a KO routine it's fully in implementation detail yep yeah so the question is when we ship this will we have tools that enable developers to you know not make common errors so for example resuming a KO routine twice you know from different scopes so I guess to answer both of those questions the first is this is already shipping so if you have Visual Studio 2015 the latest update is fully usable all of the examples from here barring any you know errors I introduced in PowerPoint should compile also we're working on implementing it in clang with respect to like helping programmers like maintain best practices our expectation is that most programmers should not need to write these Co routine types and that you know a few library developers effectively are going to build these these Co routine adaptors for the libraries and will do so in a way that you know with proper and caps proper abstractions such that it's safe to use so for example the stood future stood promise example here you know the the way that those are designed is it's it's hard to use it incorrectly in a way where you would end up with a corrupt state like you described mm-hmm yeah I think we have time for one more pardon when will decline implementation be ready that I don't know you might want to ask Richard Smith that all right thank you all for coming please do come back for the other two talks

Channel: CppCon

Views: 37,087

Keywords: James McNellis, CppCon 2016, Computer Science (Field), + C (Programming Language), Bash Films, conference video recording services, conference recording services, nationwide conference recording services, conference videography services, conference video recording, conference filming services, conference services, conference recording, conference live streaming, event videographers, capture presentation slides, record presentation slides, event video recording, video services

Id: ZTqHjjm86Bw

Length: 58min 2sec (3482 seconds)

Published: Sun Oct 02 2016