Understanding how to use Task and ValueTask

tune in to the on.net show to hear Steve and Taub talk about tasks and valued tasks and learn about the difference between them welcome to another episode of the on net show today we've got Steve and Taub with us and we're going to be talking about the whys what's when's and anything else of value tasks so Steven you're the inventor I think of a value task yeah lots of people had a hand in it yeah I'm sure so what is this thing how does it relate to tasks why do we need it short level blood just read the blog post yeah the dotnet framework for snow so we've had tasks for a long time now I think we started working on it around 2008 or so maybe shifted around 2010 what ever done in favor report came out and we weren't on the same team back that we weren't and since then you know it's sort of taken on a life of its own it's blossomed it's there api's all throughout net and many many many consuming libraries that are built around tasks and while there are some things that I would go back and do differently if I could for the most part for its core purpose it's it's been doing very well for itself but a lot of people us included have been starting to look at dotnet as a much more performance focused environment trying to use it to serve really high throughput workloads in asp net and places where we actually care about eking out every last ounce of performance we possibly can and low memory and low memories is a different different take on it yeah and once you start doing that you start paying a lot of tension to allocations and tasks is a class it's an object that needs to be allocated on the heap on the heap and that adds up meaning that if you have a method that you're going to be calling a lot on um you know a very hot path maybe those allocations start to add up and there's something you want to try and eliminate and often you know we have api's and dotnet and have since around done at 4.45 on stream like stream vijay singh stream dot write async and you really if these are being used on hot paths especially if most of the time they're actually not a sink they're async methods you will weight them but they often complete synchronously if you about allocations you want to try and drive down the number of tasks that are being allocated as much as possible right and when you say complete synchronously I think what you mean is the data that you want is already there so the the mechanism of async was not particularly useful in that case right the mechanism for async is largely about the it's often about the one percent case in these situations it's I'm gonna go out and I'm gonna go across the wire and make a network call or go to disk or something and grab a bunch of data and then once that data comes back and this is this is an operation that benefits from being asynchronous because I don't want to tie up my thread while I'm doing this I oh intensive operation yeah but then once the data comes back I'm gonna query that data a thousand times a million times without needing to go back across the network or hit the disk again and if every single you know the first time I did this operation and I gathered all that data it was asynchronous actually asynchronous but when I call this same async API the next 999 or 900 900 you know 9099 times it completes synchronously because the data is already there so when I say complete synchronously I make the function call and when the function returns to me it's already done all of its work versus I make the function call and when the function returns to me it hands back this promise this task that will eventually be completed when the work is completed in the future in the future it is a promise for things being done in the future yeah exactly I've done some reading on this yeah and and so that's really where value tasks the original reason that value task of T was introduced we introduced it in around the dotnet core to Oh time frame with a separate NuGet package that also allowed it to be used undaunted framework and it basically just contained this one type value task of T with the idea that if you had an API where it would 99% of the time complete synchronously but you had to hand back some data you could use value tasks of T instead of tasks of T and all it was was a basically a discriminative Union a wrapper with two fields a T and a task of T mm-hmm and it could only be one of those two things and so it didn't take very long to build it did not take very long to build but as with any of these things that goes to you I remember that discussion on this it seemed actually incredibly painful at the time well and the idea is you you can then hand back one of these things and if the data is already available if you already have the T by the time you're returning it back to your caller you just construct the value task with the T and your task of T field is no if on the contrary you don't yet have the T well then as you otherwise would have you create the task the task of T and hand that back under the value task and eventually the method will complete asynchronously and complete that test right so it's basically a variable wrapper and it's it's one side of the other exactly never removed yeah and you could the idea was you could just await it just like you could await a task of T and if you were waited and it just contained a T well it's gonna is it is complete is gonna say yeah true and then you're gonna call get result in is just gonna copy the T back to you if you if it was wrap around a task of T then it would just be like a leading a task of T right so one scenario that I thought of for this is imagine you have this code that 99% of the time it's going to the local file system to open a small file that literally has you know a single character in it and you would want to return back some int and then like one percent of the time you're reading over a network to drive let's assume on the same the same network but it is a network operation and so it seems like you would want to make that code async so it's non blocking but value tasks would be useful because it kind of fits the pattern of what you were just describing right and a lot of these scenarios especially if you care about scalability you're building your you know highly scalable service if you're if your API is ever potentially going to do i oh you generally want to make it async which means gonna return tasks tasks of T value tasks of T and so on and then the question is which of those are you going to do the default answer even now and we haven't talked about all the variations here but it's still tasks tasks of T and then you get into well am I in a situation where the allocations are actually potentially prohibitive and should I consider using one of these alternatives right so here you're saying like even if I'm you know a great developer some people even think of me as a performance architect so you know I care about these things deeply I should still start with tasks yep for everything yep and really only move to value tasks if I can prove to myself that it's valuable in basically all or that it should be valuable in all cases or do no harm yeah and the reason for this is while again you know there are things I might change if I had the opportunity about tasks for the most part it is sort of the more usable API it is it's it's useful in a much wider variety of circumstances for example it's pretty common to have some lazy initializing async API where you want to go out across the network grab some state come back to your program and then you're gonna use that data over and over and over and over for the lifetime of your application so you might make a call that returns a task of T store that into a field and then any number of threads at any point in time throughout the program are just gonna wait that thing right and they and they they don't have to worry about any thread safety around that they just a weight of the weighted path support next any number of a waiter's whether or not it's completed or not whether or not it's already been awaited which are all things that aren't true for value tasks its value task is really focused on basically a synchronous replacement instead of saying method async since instead of saying method I say a weight method async right so I think what you're saying is you should never never never stash a value task into some static field for use later that is true never never never never never never okay yeah and and the reason for this it gets into sort of another aspect of value test which we should talk about so when we originally released value tasks in around nine core - oh it was just value task of tea there was no non generic value task and the reason for that was very simple if you're a p-value tests only benefit at that time was handling the synchronously completing case if it was asynchronously completing it would always wrap a task of tea there'd be zero benefit to returning a value task of tea instead of a task of tea and in fact there's a little bit if you're really micro optimizing there's a little bit of additional cost involved in returning the struct wrapper around for sure you know a task of tea and it's a little bit bigger so if it's being stored into a state machine then it's making a state machine a little bigger and so on but if you were returning because it was only about the synchronous completing case if you had an async task method so there's no tea well the runtime caches an already completed task there's only one possible one because there's no difference in values there's no tea so if you have if it's synchronously completing an async task method we'll just hand back that already cached instance right so this is similar to the string empty string exactly have cached and the product that people are quite familiar with using and so in fact that instance is publicly exposed as tasks completed tasks so we didn't have a value task because other than you know the symmetry with the value tasks of tea there was no benefit only downside however we found cases where even though you know one percent case where you were completing asynchronously or maybe it was slightly higher than that for some api's you actually cared about allocation in those cases as well and so in dotnet core to one and again available via a nougat package as well we added the ability for value tasks to not only wrap a tea a saree value task of teeth not only rapid tea not only wrap a task of tea but it could also wrap a new interface that we exposed called a I value test sorts of tea written is this similar to completion source that's already on task it's very similar in terms of its purpose okay so you know with it with the tea the data is already available there's nothing to complete with a task of tea whoever's creating the task maybe a task completion source is responsible for completing it with an AI value test source of tea it's basically just sort of a glorified interface representation of what the await pattern is so they can wait arbitrary things and we allow them a value task of tea to wrap any one of these arbitrary implementations whoever creates that implementation is responsible completing it via whatever mechanism they want the benefit of this I value test source of T is you can create your own implementation that's reusable so you could create it complete it reset it complete it reset it complete in each one of these times returning another value task wrapped around that sounds interesting it is interesting and what it means is if in these very common cases like socket read async where you're calling this thing in extremely high speeds sometimes it's completing synchronously sometimes it's completing asynchronously we can have a single implementation of I value test source and back a value task wrapper around it every single time and there's no additional allocations so in.net core two one we added these send async receive async overloads to socket we added them to network stream added them to stream in general and in many of these implementations that are not only allocation free for the synchronously completing case they're also amortized allocation free for the asynchronously completing case makes sense as a result we also introduced a value task a non-generic value task the thing you said wasn't used the thing that I said wasn't useful it was only a person chronicity completing because now that we also have it for a synchronously completing now having the non generic variant has use not for the synchronously completing case but for the asynchronously completing case so in dotnet core to one we introduced I value task source of T we introduced value tasks and we introduced I value test source okay and the and then we also in donna core two one added a bunch api's that take advantage of this so we added overloads to stream we a sink that returns a value task of int and a dot right async that returns a value desc okay and now you have methods you have types like memory stream where the data is always available synchronously because it's right there in memory I just go read it and return it and so it's always going to benefit from this value task of int returning read async because it can just return a value task around the the in Thea the number of bytes that metadata we same unbuffered stream where 99% of the time the data is already buffered and it can return it and in the asynchronous case it may end up allocating a network stream it takes advantage of these new api's we had at the socket and it's always going to be able to if it completes synchronously you'll get back a synchronously completed value test no allocation and if it completes asynchronously unless you're doing something really weird you're going to get back around an already a cached object and it makes certain situation scenarios much better performing so let me see let me see if I can test if I understood so you said that value task is a more much more minimal implementation and really API than tasks yes so I'm guessing that value task doesn't offer cancellation does it well yes and no task itself doesn't really offer cancellation you can't walk up to a task and say right because it's a separate thing that's the cancellation token so but so with tasks you might walk up to a task ants a task completion source and say can set cancelled in that way value task supports cancellation exactly as ty see because it because it is a separate concept right okay write that exam that's okay now you're on the right track yeah on the right track a better example of things that you can do with tasks and you can't do with value tasks with tasks as we already mentioned you can stash away because you can await at any number of times with value tasks because it might be wrapped around one of these reusable objects if you tried to use it after it was already returned and potentially being used by someone else that could be very bad and we try and have some very low overhead mechanisms to prevent that kind of misuse but they can't catch every possible misuse so but basically once you've awaited a value task never touch it again whereas with tasks you might want to store it in a dictionary and access it in a wait at many many days writes by multiple callers similarly with value tasks don't store it somewhere where multiple people could have waited at the same time because it's only meant to be awaited once and never mind concurrently right basically you should only use them in this one method frame right and and be done with it then be done with you other things you can use a method like a weight task when any to a weight for anyone of any number of tests complete in which everyone completes first you wake up and you say this is one that completed well again falling out of the hole don't a weight of value task more than once if I take four value tasks and I pass them to a mythical ballot on their own then they're all they're all done yeah see you have no way of reasoning about them at that point exactly so you can't see you generally can't kind of use combinators to operate over multiple value tasks at the same time it's really meant for this instead of having the synchronous method I have the await method async yeah I got anymore yeah nothing um and then there's a bunch of additional methods on task for just doing a wide variety of programming styles that just don't show up on value tasks right so you've talked a bunch about how we've taken advantage of value tasks and our own implementation do you know if these are being used by regular developers because code you know I guess it would be one thing if we got a ton of value ourselves and that was the only place it was being used and maybe that would be good enough but presumably you got aspirations are more useful yeah so definitely for the synchronously completing case there's a decent amount of usage out there of value tasks of T where it's just wrapping a tea or a task of tea and in fact we've made that pretty easy because you can say instead of just async tasks and your method signature or async task of tea you can say async value tasks of tea for example and the compiler will happily generate an async method that returns a value task of tea that is really only optimized for the synchronously completing case it doesn't do anything special right now for that async completing case there's very few implementations out there that take advantage of the async thing mainly because it's it's hard and the value is no pun intended limited it's only for these kind of semi niche high-performance scenarios where you really care about getting that last ounce of Perth out that you also want to optimize that asynchronously completing case often an async complete in case there's a whole lot of other costs involved that can do Wharf the the cost of the the task allocation it makes sense certainly we have a bunch of those high performance situations in the platform itself and there are other places that I haven't mentioned for example the new system Isle pipeline's API is takes advantage of this so that it can reuse one of its existing objects over and over and over rather than allocating new tasks every time you call an async method the system threading channels library that we introduced into one similarly does this sockets network stream you know all these things that are in Donna core and we've done more in Rio as well in fact and dotnet core 3 oh we've and in c-sharp 8 in fact we take even more advantage of this so you're familiar with the ia sinc innumerable support I love I love it excellent the c-sharp compiler when you ask it to provide implement an async iterator where you say you know public async async' enumerable whatever and then you have yields and awaits and that method it will generate an implementation of Ising canoe mirabile that has you know it'll have a get a sink a numerator method that gives you back an I take a numerator that has on it a move next method and move the X a sink rather that returns a value task a bool and the reason it you know it returns a value to ask a bool is if it was just about handling the synchronous completing case well it could return a task a bool because the runtime actually caches a task of true in a task of false but it actually uses its existing object the same object it generates that acts as the enumerable that also acts as the numerator also acts as the I value test source that backs I understand and so you can basically have your entire async sequence no matter how many times you yield how many times you await things there's basically only the one allocation for that one object that's reused over and over and over and over to super useful and so from that perspective we're actually going to see a lot of third-party libraries that have these value tests because it's baked right in to this async numerable sequence and the compilers doing a lot of the heavy lifting for you the compiler in turn depend on a new type that we're adding in.net choreo which has a really long name manual reset value tests or score it's very deep in a complicated name space because it's a complicated type but basically it is a mutable struct that you can put on to an existing type that has all of the logic necessary to implement an I value test source interface so you on your object you add the struct you then input the interface and you delegate all the interface calls to this method and the and it implements all the logic around a waiting and all the synchronization that is a matter like I'm sorry short I don't know what I said you said related ok delegate all the interface calls to this type or methods on that time yeah yeah yeah exactly yeah and this is what the compiler does the compiler generates it's it's object that implements I say enumerable i sync a numerate or I value tests or such bool and it's implementation of I value tests or subpool calls methods on that that's right so to segue the conversation just one other place in them we should probably finish is I was trying out a sink enumerable one of the things I incorrectly tried to use it is yield return spans mmm so as you can imagine that didn't work that was part of my learning on both of those concepts super super useful but we're that's a segue to is I've been noticing over let's say the last five years that we've been developing a lot more Struck's in the product and I buy a lot more I mean more than zero yes and can you tell us a little bit what that change in direction is about and also maybe recap what they are for folks that yeah what I'm talking so I mean this is about classes versus trucks classes or reference types in dotnet are allocated on the heap struck soar value types in net are just data that lives either on the stack or as part of some heap allocated object that's historically been focused on productivity classes are sort of the default answer which registers so remains the right choice we made the right choice however in the last few years we've been focused on expanding dotnet into more high performance environments high performance scenarios and as part of that it's not like we're going back and placing every class with structs but more of the new things that we've introduced as part of this high performance effort have been involved decisions around should be allocated for this or not and generally we've for certain cases we've said all right well we we don't want to allocate here because we expect these things are going to be created a lot we don't want to add pressure to the GC and so we end up defining something as a struct and maybe paying a little around usability as a result that's even more the case where we introduce this notion of refs trucks in dot NIC or to one I started to use those yeah and they are even more limited or more constrained it was a good word with what you can do but you get additional benefit from those constraints yeah you you pay a little you get a little one of those new types is span of T so span of T is one of these refs trucks which is incredibly valuable and in fact the constraints that it comes with in some situations are actually incredibly beneficial from even from a usability perspective there are some downsides but you can reason about whether someone's going to be able to stash one of your spans onto the heap the answer is no yeah and that's basically the problem I ran into with my example where I was running into that fact exactly and so on the one hand it's a bummer that you can't on the other hand I know you can't and I can I can reason about that very well and take advantage of that limitation exactly it's a for example let's say you're a method that accepts a span as an argument I know that if I allocate an array and pass that to you as a span you can't cash it you can't store it in some static field somewhere which means I can reuse that array yeah I orator you you on the lifetime I own the lifetime there's nothing that you can do to store it somewhere and potentially fear with my subsequent use though because once I I don't know why once I return back to you I'm totally gone yeah like there's there's nothing that possibly could happen right which means I could call you again or I could call someone else and pass that same array and not have to worry about it being returned to an array pool or something like that right so this whole topic could be a whole show so but do you do you imagine we're kind of done with our struct effort or do you believe that this is a pattern that's gonna kind of continue for the next kind of n years I think it's a pattern that will continue still the minority case focused on situations where we expect that certain scenarios would want to avoid the allocations it's a for example there's this whole push in net core three around introducing a bunch of new JSON support and most of that support shows up as classes because it's the easier thing to use it's easier to reason about but at the lowest level we've got these utf-8 JSON readers and writers where for certain scenarios people are going directly to them whether they care about at most in performance and they don't want to pay an allocation just to read a few bytes of JSON data out of some in-memory data structure they already have and so those aren't just structs they're refs trucks and that allows us to for example store a span inside them and still get those lifetime guarantees that we talked about earlier while also avoiding allocations right so when you use those basically zero compromises on performance right okay well this was a great I think we will this was pretty in-depth yes for sure so we should probably remind people again if you want the the fireside version of Steven then this is probably the blog post to go to we'll definitely link to it in in the show notes but it's a pretty lengthy blog post and goes over a lot of this same material it does and more so and probably we talked about some things that aren't here and this also certainly talks about things that we didn't discuss for sure okay well thanks for being on the show thank you for having me yeah and this has been another episode of the on.net show thanks for watching [Music] [Applause] [Music]

Channel: Microsoft Developer

Views: 16,897

Rating: 4.8959813 out of 5

Keywords: Microsoft, Developer, async, Task, ValueTask, .NET, .NET Core, Threading, asynchronous, C#, IAsyncEnumerable, Programming, Software Developer

Id: fj-LVS8hqIE

Length: 26min 59sec (1619 seconds)

Published: Tue Jul 02 2019