Revisiting Async - Kyle Simpson

we're going to talk tonight about asynchronous programming and this is an overview or revisiting of some things that I have in a workshop titled rethinking async but it's also at the end kind of a look forward to the stuff that's emerged in asynchronous programming just most recently for JavaScript the reason for this discussion is that the most complex and difficult thing that we have to manage in our applications is to manage time time is the most complex state in our applications we are really good at creating complexity cyclomatic complexity branching and all that stuff even when the code is completely synchronous you can take a couple of functions and string them together with some if statements and you've got an exponential explosion in the possible various states that your app could be in at any given instant but when you factor in the component of time which is that the state is changing in some way shape or form over a period of time because quite frankly if the state doesn't change there's no purpose to your app state needs to change and state is going to change over time and modeling that is particularly difficult and this has nothing really to do with JavaScript itself the criticisms that I could make about our inability to model state changes over time is really a criticism that could be leveled at all of programming if you think about threaded programming for example threaded programming I'll make the claim which may sound a little strange I'll make the claim that threaded programming is ridiculously easy to do it is stupidly easy it takes like one and a half lines of code in something like C++ or Java to spin up a separate thread and do stuff in parallel that's not the hard part the hard part is coordinating that stuff if they have some sort of shared resource and here's a hint almost everything is a shared resource even something like a register in your CPU is a shared resource so two threads need some way to coordinate with each other and by coordination we really are talking about the dumbest possible coordination that you can imagine which is simply hey don't step on my toes it's not like let's cooperate together and break up this task and work together and all that there's no primitives whatsoever for any of that in any of programming we literally stopped in language design once we got things like mutexes where it's can simply just say hey don't do anything right now cuz I'm doing some stuff and I don't want you to mess it up right like that's the dumbest possible way for us to express in our program that we have these things happening at different times and in different intervals and they need to somehow cooperate with each other and so those criticisms are I believe universal I don't think there's any programming language that has somehow uniquely figured out the way to solve the expression of state management over time state machines don't cut it state machines are great but they don't cut it because they don't talk about what happens when all of these other pieces can decide they intertwine with each other to decide what the next input is that changes your state that's the question it's not about whether I can store state in an object it's not about the storage of state or the persistence of state but the decision-making on I'm in this state and all of these 15 things are happening at this exact moment to determine what my next state is and we somehow barely managed to get applications to where it kind of blows my mind that any software works at all and I'm not meaning that as a criticism against your software I can't believe that any of my software works how am I able to somehow with these ridiculously primitive tools communicate in some way to the computer that all of these thousands or millions of different states that are somehow intertwined intertwine with each other should fit together in exactly this way and this is exactly what should happen and the answer is I'm not the answer is that my application and your application are not fully expressing those things and to the extent that you do not have full and complete combinatorial explosion of testing cases your apps have bugs in them there are states in your app that nobody has ever tested nobody has ever created the scenario that got your app into that particular combination of states because you can't even envision that particular combination of states and there's no test case for it and that's why when you're using your phone every once in a while an app just crashes or the phone reboots itself or your device locks up and gives you the beach ball or whatever the reason those things happen is because somebody tried to think about all of these states and of the billions and trillions of states they got a few million of them and they didn't get any of the others and then you just accidentally randomly end up in one of those completely rare cases that nobody thought about that's the state of affairs in programming that we can't adequately express this thing depends on this thing which depends on this thing in this particular relationship such that the computer can manage those constraints for us so what we do is we fire off this thing we fire off a timer that takes a second and we fire off another timer that takes ten seconds and we think in the back of our mind the one that takes ten seconds will never ever ever under any circumstances happen before the one that's supposed to take one second and so we don't even think about that particular case and I'm saying timer here but it's not timer it's like well it's impossible for this thing to have returned results before this thing return results so I don't even consider the case where those results come back in a different order and it's not right then that that's ever gonna happen it's gonna be six to twelve months from now when nobody remembers having written that code and there's a high profile bug everything's a disaster in production there's no test cases and then it lands on your desk and somebody says go fix it and fix it now so you're gonna be looking at that code trying to figure out how does it even get into that case where this thing happened before that one that shouldn't even be possible I don't know if any of you have had that experience but I on so many occasions countless occasions have look at some console log statements and B comes before a and I'm like that's not logically possible how is B happening before a that makes no sense and then somehow weeks later I figure out how B happened before a but my brain can't even conceive of those different states so this is what I mean when I say that time is the most complex part of our application and we're dealing with the most primitive caveman types of tools to express these incredibly complex problems and it was it's with that mindset that I want to approach talking about improving the patterns that we use to express concurrency to express a synchrony in our programs I'm going to talk to you about a variety of little pieces of the puzzle no one of which is the complete solution but the combination of all of them starts to get us a little bit closer to being able to express our ideas now for the last 10 minutes I've been talking to you about we can't make the computer do what we want and that is certainly a problem but even when we are successful at getting the computer to do what we want where we utterly fail completely and totally is communicating those ideas to other human beings so even if you are an amazing programmer and you have written that Unicorn program that is completely bug free that has all possible states taken care of and test cases to ensure it there's a virtual guarantee that somebody else when reading that program is not gonna have any idea what you've written and I don't mean this in a way to like criticize you you make you feel bad we're just human beings and we're not good at what we're trying to do it's amazing that somebody gives us a paycheck at all to do this and we're using the most primitive and ridiculous of tools to do it let me illustrate to you what I mean by this idea of all I can make it work versus I can communicate well have any of you ever written an if statement before okay we're all among friends we've all written if statements right how many of you have written an if statement that did not have an else clause attached to it anyone okay most of us we've written if statements without else clauses now how did that happen how did you arrive at the conclusion that she didn't need an else clause well what she did was went through all the possible states of data that can come in to that particular branch of code he said this thing can come in as this number and this can come into this is string you went through all of those states in your head and I want you to visualize this like creating this giant cloud around your head of mental context all of those states you ran through you mentally executed the code and you convinced yourself that all the possible states that can occur in all those cases the else Clause is empty so I don't need it and because I don't need it I'm gonna leave it off because if I left it here my linter would complain at me about having an else clause that was empty so my question to you is is it enough that your program is correct is it enough that your program does what it's supposed to do and I would suggest no it's not enough it's not even nearly remotely enough because of all of that mental context that you had to decide that the else Clause did not need to get put into the program how much of that did you communicate in the program here's a hint zero all of that mental context about why that else Clause is not needed and none of it was communicated in the program so what is going to happen is six months from now there's going to be a high profile bug everything's on fire you or somebody else doesn't matter because neither one of you remember writing the code you're going to show up with this bug in hand and the pressure to fix it and you're going to show up at that if statement that has no else clause and what's the very first question that's going to get asked where's the else clause is the missing else clause the source of this bug and how are you going to answer that question you are going to have to recreate that entire mental context you're gonna have to re-execute all of those thoughts all of those and you are doing so in the worst possible conditions you're not doing so when you have like the greenfield development you can do whatever you want take your time this is high pressure the boss's hair is on fire and you have to get it perfect so you reconstruct all of that mental context and then convince yourself oh okay there's no need for an else clause and then you move on to the next thing and the next thing and the next thing and everything that you find in that program that was built based upon a set of mental context floating around the developer's head none of which got put into the program is directly responsible for why you cannot do your job at that moment which is to fix the bug so I go back to my earlier question is that enough that the program does what it's supposed to do and I say it's not nearly enough because actually our jobs are not about making the computer do something actually that's not our main job that's not why we collect a paycheck our main job is not to make the computer do something our main job is to communicate our ideas with other human beings through source code to the extent that you were on unable to communicate your ideas through source code you have failed at your primary task as a developer that might sound harsh but let me tell you why you failed because I guarantee every one of you in this room has overheard or participated in this conversation you've had a bug and it was estimated that it would take four hours to diagnose and fix it and you're on our eight and people are like hey what's gonna take when is gonna be done what's taking so long now if you knew the answer to that you'd know what the bug was and you'd already have it fixed so you cannot answer that question and at some point the pressure builds up so much that you say it would be faster if I just rewrote it right every one of yous nodding and smiling because you know that's exactly what happens is that when a piece of code is so inscrutable that we cannot figure out why the else Clause is not needed and we're wondering or we're suspect or we're worried that that's the source of the bug guess what we read it what we do we rewrite it and to me that is the primary metric that tells me you've failed as your job failed at your job as a developer because you wrote a piece of code that did not survive the next time it needed to be read I don't care how many times it executed ones and zeros in between when it arrived at the moment when it needed to communicate the ideas and it was most important it failed and somebody ripped it out and rewrote it do me thousands and thousands and thousands of times in your app every single line of your code has been rewritten and rewritten and rewritten and rewritten not to say that there's no such thing as refactoring to make code better of course there is but that's not what I'm talking about I'm talking about the code that nobody can figure out they don't improve it they just rip it out and rewrite it now sometimes we do that for ego sometimes we rewrite code because well it makes me feel better if I own this code I'm gonna own it so I'm gonna rewrite it I'm gonna do it in my style sometimes it's about ego but far more often I just can't figure out what the hell it's doing and I wrote it three weeks ago and I can't figure it out so guess what I'm gonna do I'm gonna rewrite it it's not just when you rewrite somebody else's if when you rewrite your own code the 15th time it tells me that the previous times that you wrote that code you failed at your primary task of communicating those ideas to other human beings including your future self the computer takes your code as a suggestion of what to do anyway it's not a set of instructions it looks at your for loop it says thanks but I'm not gonna do a for loop at all it's just a suggestion so why would you make a suggestion that the computer doesn't care about and that poorly communicates to you and everybody else what you were really thinking that's what I mean by saying there's a there's a higher bar here and it's the real reason why I spend all this time going all over the world trying to teach this stuff it's because it's so hard for us to get it to work but how much harder is it for us once it works to communicate our ideas well so that somebody else understands why it works code that you don't understand is code that you can't trust and code they can't trust is code that's gonna get rewritten there's a hundred percent guarantee so the bar that I'm trying to set so we need to understand these patterns much more deeply and intimately than we often do we substitute oh well my framework takes care of that my framework takes care of all that stuff and all I need to do is call this method and it does all of that stuff and that's just not good enough it's not good enough to not know how your tools work an artist would not tell you oh yeah I I have no idea how paint works you know I know the paintbrush but you know somebody else mixes the colors for me that wouldn't be how an artist looked at it no profession would tell you seriously I have no idea how my tools work and yet we as software developers do that all day every day we tell people with a straight face I get paid hundreds of thousands of dollars to use tools that I do not understand how they work and we do that uh neuron eclis as if that makes complete sense and I just think if we want to do our jobs better we need to understand our tools better I don't know maybe I'm the only one but that's the reason I'm going to talk to you about any synchronous programming tonight we're going to start with promises and actually I'm gonna start before promises because you need to understand a little at least a little bit of why promises are important and here's a hint the promises are not important because of their API it actually sucks as an API promises are important for a much deeper set of reasons let's start back with our friend to call back okay how many of you've heard the term call back hell before we've all heard that term and we've all heard that it has something to do with the call back being nested inside of another call back which isn't so nested inside of another call back this is just a quick little sketch of some code that I threw together and it's going to be the sort of example that runs throughout all the rest of the code that I'm going to show you the idea is all I need to first bill fetch a user in some way and then once I have the user object at the same time I need to fetch their archived orders and their current orders and then once both of those have been retrieved then I need to call this on orders utility and print out the results so I need to do one step and then two steps concurrently and when both of those finished do a third step you follow me that's the concurrent pattern and you're going to see that repeated over and over throughout all of these examples this is just I was trying to distill down what we typically do in our applications to some kind of non fubar example okay now there's a problem with this code there's a problem with this code that is very hard to spot but there's a bug in this code there is a race condition that exists in this code and you can't see that race condition I could spend five or ten minutes walking through the various cases with you to talk to you about it but the problem is that you can't spot it at a glance this code has already failed at its primary task it's already failed at its primary task to convince you that we definitely know that if I have received all of the both of these values then I can call the on orders function but there's a case where that on orders function can get called without the proper results so there's a bug in here that you're probably not going to find in your testing it's not going to happen until it starts intermittently occurring in production with full-scale data that's what I mean when I talk about callback hell there are some fundamental in inadequacies of the callback and it really has nothing to do with whether one is nested inside of another it has to do with the callback itself is not able to express the concept of state over time and without an ability to express and have a memory of state over time and articulate the relationship between two pieces of data over time it's too primitive of a tool so we write code like this and it seems to work and then we get frustrated when there's a bug and instead of fixing the problem we just patch it with another if statement because it seems to fix it and we put more and more if statements around things because that seems to fix the problem and when I say we I really mean we because I do this all day every day in my own code because I can't figure out what I'm doing I can't figure out why that bug is happening if I knew why the bug is happening I wouldn't have written the bug in the first place by definition I can't figure it out by definition my brain is not working the way the computer is working so I am least qualified to articulate a solution to the problem I'm the one who wrote the code and I'm least qualified to fix it because my brain is clearly broken or I wouldn't have written the bug in the first place you following where I'm coming from what I need is a way to write this code that more clearly explains it so that somebody else smarter than me is going to spot oh I see the bug I see the problem here it is but I'm using primitive tools and I'm a fallible developer with imperfect thinking and the only way for me to make any progress here is to understand my tools better it's my only hope so a tool that we could use to address this particular problem which is I need to express this idea that these two callbacks are going to happen in some kind of order but I'm not sure exactly what order which means one of them is going to happen too early which means I need somewhere to store its data until I'm ready for that's the problem that this code suffers from one of these callbacks runs too early and I need somewhere to store its data and callbacks alone don't have any answer to that problem so what do we do we patch it with global variables and if statements and who knows what else until it seems to work all we've done is make the problem much much worse because we've obscured the real issue with all this other stuff that's how code baggage happens not in the grand scale of making really bad macro level decisions it's in the micro scale of not quite sure why it's working or not working I'll just wrap an if statement around it that'll fix it and one if statement at a time and one for loop at a time and one global variable at a time that's how software degrades itself until at some point somebody says it'd be better if I just rewrote it and then we start the cycle all over again we don't fix it we just started over so we didn't understand our tools better when you understand our problems better we need to think about the problems more deeply we need to analyze these problems I'm gonna make a statement that sometimes when I say this it ruffles some feathers so I'm gonna warn you ahead of time it's not intended to be an insult but I'm gonna tell you that I think there's a difference between the developer mindset and the engineer mindset there's lots of differences but one clear difference that I think bears on what I'm saying tonight a developer seeks first to solve a problem and maybe later understand it an engineer seeks first to understand a problem and maybe later solve it do you understand where I'm coming from that the key detail is not whether we solve the problem the key detail is whether we understand the problem how many of you had a piece of code break and you weren't sure why it broke great we're all among friends how many of you ever had a piece of code work and you didn't know why it worked look I didn't change anything and it worked what this has happened to me I've taken out a console dot log statement and code started working and that's that makes no sense at all how could that possibly be I just deleted a console log statement how is that possible right still don't know the answer to that question so we need to understand the problem better we need to think more deeply about the problem I analyzed that problem and articulated in a way that most people would never go to that level of detail most people would never say what's wrong with this code they would stay on the surface and they would talk about code style and they'd say it's ugly don't use inline function expressions but they wouldn't talk to you about the real problem which is that there's a state problem there's a managing state overtime problem and there's no clear articulation in this code that that is happening or that we have a solution to it none of that's communicated in this code so let's use a tool that actually kind of does help with state over time I know that sounds shocking to you but let's use a tool that's actually built for the problem at hand I've analyzed the problem in hand what's the tool that solves it well without any kind of fancy newfangled es6 javascript since 1999 we've been able to do but never did a pattern called thunks thunk is a funny word don't hear talked about much in JavaScript and the places that you hear it talked about it sort of like this gray area like nobody has a real clear defined definition for it the best definition I can give you is that it's a thunk is a function that has all that it needs to do to compute its tasks you just need to call it you didn't need to provide it any inputs it already knows everything that it needs to do to compute its work so it's a deferral of future competition of work wrapped up in a function that's what a thunk is it represents a future computation of some value or values that may or may not have happened yet but the way I access it is to execute this function that's what a thunk is what's important about that is that funk is a clear and direct way of expressing future stuff aka time-independent stuff I have this value I need it at some later time somebody hold on to it for me somebody compute it and hold on to it until I'm ready that's what we call a future value and that's the clear problem that we have here is that we we don't have any way to articulate what do we do with these two values that are racing against each other how do we describe that they have this codependent relationship on each other a thunk gives us a way of doing that it may not be pretty but it's a better solution than what we have written here so let's use a thunk now I'm going to show you I'm not going to show you how a thunk is constructed that parts not important because you wouldn't make your own thunks anyway a utility would be providing it but I want you to understand the concept of calling a function like fetch current user fetch current user isn't the func it's the thunk making thing it gives us back a thing which I have labeled here th one th one is a function it's a function that represents the future value user okay the computation to get that user happens to be searching in a database or making an ajax call or whatever it doesn't really matter what that computation is but it's work that needs to be done and it might be happening now or it might be deferred but it doesn't really matter to us because we have one single way of unwrapping that future value and that is to call the function th one and we provided a callback to receive the answer now at this moment it may not be obvious to you but at this moment on line three there is a race condition there is a race condition because when we call the th one function on line three one of two things is true either the user is already done we've already retrieved it and we have it ready to go or we don't it was the only two possible cases we either have it or we don't does that sound like time state management to you it sounds like it to me that's the clear distinction of the problem we don't want to manage time time is too hard so what we want is abstractions that take care of time for us this thunk which is just a function that I could have written back in 1999 there's no fancy frameworks involved it's just a function that's using closure to remember state and it says hey it doesn't matter whether I have it or not just call me give me a call back and I'll give you the answer when it's ready either now or later but you'll get the answer when it's ready it's a time independent representation of the user value you follow me and furthermore once I have the user value I can now make two other thunks these two things are happening concurrently I'm requesting their current orders and their archived orders those are not dependent on each other there's a codependent relationship that they both need to finish before we can move on you follow me so they can happen independently people say in parallel the proper term is they can happen concurrently which is what we're going to do we're going to fire off both of those and they are now going and computing their work concurrently and then on line seven I say you know what I need those values how do I get the value out of a thunk I call it and I pass it a callback and that callback will be given the comp the computed value whenever it's ready on line seven there's a race condition I don't know if it's value is ready or not I don't know if archived orders exists or not but I don't care I call th2 exactly the same whether archived orders as ready or not and on line eight there's even more of an obvious race condition which is on line eight I don't know if current orders is ready or not but I don't care I just called th three and at the moment that this function is executed what I do know is that both of them have finished I do not what order they finished in but I dunno at that point they have definitely both finished and now I can call on orders because I have both of those responses did you follow how I did that how I reasoned about that I unwrapped time-independent values sequentially and that's how I know that I have everything that I'm dependent on I don't call on orders until I know the two things that I'm dependent upon are definitely done and the figuring out of how do I know the race conditions that's abstracted away by the funk so to summarize a thunk is a representation of a future value in a time independent fashion follow me so what's all this got to do with modern programming because you would look totally uncool if you showed up on Monday morning and told everybody hey we're gonna use thunks they're the new hip thing nobody wants to talk about thunks what's any of this got to do with modern JavaScript well I talked to you about funks because it turns out that promises are a time-independent representation of a future value they just have a fancier API and it took us 20 more years to get them I am embarrassed literally deeply embarrassed by the uncountably thousands hundreds of thousands of lines of terrible shitty code that I have written since 1999 that I could have been writing better if I had just taken the time to learn my tool better but we all waited around until somebody solved it by designing some new feature in the language instead of learning the tools that we already had and using them better unembarrassed that I didn't go and learn closure I didn't go and learn functions I didn't go and learn how to use those two core basic concepts to represent time in a better way that's what I by learning our tools better the solution is not somebody needs to go invent some new amazing framework the solution to your problems in your app is not the next framework that gets invented the solution to the problems in your app is you understanding your current tools better I got 25 years of this and it's that experience that says the problem in your app whatever it is is solved by you understanding your tools better so thunks are better promises some people say they're even better than that because they're like folks but they have this pretty API where we can call things and chain methods together and most importantly apparently we don't have to nest things because you saw the thunks were nested to express to express sequentiality with callbacks we have to nest to express sequentiality with promises we just chained up then dot the end up then right so supposedly this is so much better code because it's vertically oriented rather than nested not only do I not think promises are that much better than thunks I actually think in some respects they're worse because in some respects they created this nice shiny toy that people got attached emotionally to and distracted from the real point the real purpose it's not a particularly well-designed API in my opinion I'm glad they exist it's better that they exist then that they don't exist but it's not doing us any service for us to have arguments about whether it should be called dot then or whether it should have taken this kind of function or whatever and that's what all the blog posts are about they're all arguing about the surface stuff that doesn't actually matter so talking about the substance okay so when I talk about a promise and why a promise is so important is because it's a clean simple expression of time independent representations of futura values it actually comes with some additional guarantees that are important some additional guarantees that thunks by themselves don't general give us so it does move the ball forward in some respects it's a good thing that it gives us some of those guarantees for example a promise can only be resolved once so you cannot have the case where somebody accidentally calls a callback multiple times the promise can only be resolved once that's a good a good solution to a difficult problem but I don't want us to get so wrapped up in this API because this API actually has some deep problems okay so some examples of the problems first off the polymorphic nature of the then method and I'm speaking from the perspective of somebody who teaches this for a living the polymorphic nature of the then method is perpetually a stumbling block for new learners by polymorphic what I mean is we pass different things into it or there's different conditions under which it's called and it has different behavior most clear example of that is if you pass a function to the then method and that function returns a promise the machinery does something different than if that function returns an immediate value that's polymorphic behavior it's nice that we only have to learn one method name but it creates a stumbling block when somebody looks at our then and they can't immediately tell is it going to do this thing or this thing and they have to execute the code in their head to figure it out polymorphic code is developer convenience at the expense of developer readability it's this perpetual problem I see us doing or we focus on how easy it is to write code instead of how easy it is to read code you know it's been studied what we spend our time on as developers I'm not saying about the time you spend in meetings because we all hate those but when you're actually at your desk banging away at the keyboard it's been studied that when you are doing this thing we call programming you actually are spending more than 70% of your time reading code 70% of your day is spent reading code and reconstructing that mental context to figure out what the hell you meant yesterday what was i thinking yesterday literally what was I thinking I can't remember I can't figure it out it's not surprising to me that we can't write very much code because we spend so much of our time so much more of our time reading code why is it then that our entire industry is obsessed with inventing new ways for us to write code better why is that why are we optimizing the thing that we spend the minority of our time on instead of optimizing the thing that we spend the majority of our time on if you spent the same ten minutes making code more readable or the same ten minutes making code more easy or quick or simple to click and write the time spent on readability is going to pay off more because tomorrow you're going to spend 70% of your day reading and only 30% of your day writing to me this is simple math why shouldn't we spend more of our time optimizing the thing we spend more of our time on so things like polymorphic design of methods are nice and convenient because we only have to write dot then dot then dot then dot then but it's harder later when somebody has to figure out which one is doing which it's harder to read that's one of the problems with this design there are other quirks that I'm not going to get into too much but there are other quirks of the then method its behaviors one thing that I find particularly troubling is that the then methods because the uses this fluent chaining API style it encourages this idea of function expressions in line arrow functions or little one and two liner function expressions repeat it over and over and over again down a then Handler and none of those functions share a common scope which means if this function computes some value that this function down here needs where's the only place you can communicate that in an outer scope like the global scope which is what most people do which is horrible or you do something even worse which is you say I'm gonna take the resolution value of this promise and I'm gonna package up multiple values I'm gonna wrap it up in an object or an array and pass it along and then unwrap it at some point later and the later reader of that code is like what what where what because these two things are completely separate but now there's a dependency between them that nobody knows except you we need better tools we need to stop trying to you know carve a statue with some very primitive caveman type of knife we need better purpose design tools and I'm sorry but the promise API is just not it is not designed with those kinds of problems in mind I was around for the evolution of the promise API and I can tell you it is the average it is the mixing of dozens of user land libraries I don't know if you know much about how consensus-based design works and committees and things like that but essentially you had a whole bunch of different people who had different ideas about what a promise API should look like and they all came together and said no it should do this and know what you do this and know it should do this and some of them were completely incompatible with each other and the way consensus based design works is whoever has allowed enough voice to convince people that they should not veto it not that we all agree not that we all say hey this is a great idea just I don't disagree strongly enough to exercise my veto so sure great go for it that's how consensus based design works and that's what happened here is that we came up with this ridiculous ugly compromise that nobody in the room liked but nobody hated it enough to veto it I think the biggest problem with the promise API was the pit of failure design around error handling around unhandled rejection if you've ever put a dot catch mindlessly at the end of your promise chain and you didn't even know why because somebody just told you just always put a dot catch it's this off by one error handling that happens with promise chains you register an error handler in this step and you think oh well if an error happens in this step then I'll catch it in this step but it turns out the error handle in this step only catches errors from everything above it if an error happens in this step guess where it has to be caught in the next step so we've developed this best practice that said instead of having all those rejections accidentally lost just mindlessly put a dot catch at the end of every promise chain that's called pit of failure design because if you fail to do something if you omit something then the system fails instead of the system fall instead of you falling into a pit of success when you accidentally do something wrong you fall into failure and it's the worst kind of failure because it's a failure you don't even know it's happened it's an error that's just sitting there waiting for somebody to look at it and you don't even know the error exists countless thousands of times I've been bitten by exactly that problem is I forgot to put a catch somewhere and I'm like why is it not working and why is it stopping in the middle oh it's probably because there's an error happening that I'm not seeing right that's the worst kind of design of systems is errors that happen that we don't even see and we we coded that in because some user land library had done it that way and they had enough sway among people to convince him Oh it'll be fine this was a terrible design decision what all I'm getting at is that the promises as a concept are really important the promises is an API are terrible and you shouldn't use them you should not use the promise API if you can avoid it you should use promises the concept not the API and I'm going to talk to you about how we do that to build up our understanding of patterns that can use promises but not being swallowed up with all these problems of the promise API we're first going to look at an interim step which is generators generators were added at the same time that promises were added to JavaScript which was es6 they've now been around for good three or four years so this is pretty well tread most people at least have heard of them so I'm gonna try to go through this a little quickly but I want to talk to you about this concept of a generator it's a function that can cooperatively communicate with another part of the code there's a term that's often used for this in programming called co-routines Co being like cooperative a generator can cooperate with another part of the program - step by step get its job done co-op by cooperation I mean message passing so instead of simply calling a function and it doing all of its work and then giving you one answer this is a function that when you call it you get to cooperate with it as it steps through its work that's what a generator is there's lots of use cases for them I'm going to use one particular use case to apply to asynchronous programming so here's an example of a generator you know it's a generator because it has that star symbol in the function declaration and generators are unique in that one you call them they don't actually run when you call them they construct what's called an iterator an iterator is a way to step through a data source one value at a time it's nothing more complicated than that so when I construct this iterator line 8 and I want to get some values out of it I simply call dot next multiple times and every time I call dot next I get an iterator result with the value that was yielded out yield 1 yell to yield 3 this is a function that can produce values as it is running instead of only producing a single value at the end that's what a generator is it generates values ok and not only can I yield them but I can also return them although I'll tell you you almost always want to exclusively yield values so an iterator is a way to step through a data source and one kind of data source that we can step through is a generator now that yield mechanism being able to pass values out is interesting and useful but it's even more useful when we find out that it's actually a two-way message passing system not only can we pass values out we can pass values in here's a generator that's going to yield out values that it just generates on the fly in fact that could have a while true loop in here a wild true loop with a yield inside of it is a generator that's designed to run forever and that's entirely okay most people say you never want to write a while true loop right seems like a while true loop would be a really bad idea in programming when it's done inside of a generator it's actually very powerful because it's a generator that will generate values for as long as you want it to keep asking for values that will keep giving you values so this is just programmatically doing so and this is a form of iteration we have two different forms of iteration over iterators these are built-in operators in JavaScript one of them's the dot dot dot spread operator it consumes iterators and receives all their values which is why we end up with an array of 0 2 4 6 8 all of the values that this generator produces that iterated wardrobe malfunction here sorry okay so we can iterate over an iterator with dot next calls we can iterate over it with a dot dot dot or with this new looping style called the for of loop a for of loop creates an iterator and consumes it until completion you don't want to use a dot dot dot or a for of loop on a while true generator because then it's going to run forever okay all right so yield is not only an a one-way passing system it's also a two-way passing system there's gonna be a lot of arrows on the slides to just stick with me all right so here I am passing in a value 10 I'm passing in a value 10 which replaces the yield expression yield for so I yielded four out I don't know why that arrow must have got deleted I yielded four out and then the neck and that was on this call line 10 on line 11 I passed a value in to the dot next and it replaced that waiting yield expression so X gets assigned to the value 11 1 plus 10 and then I yield out 5 and I pass back in the value 20 to replace that expression one way of thinking about this let me take the arrows away one way of thinking about this is that a yield expression is like a placeholder inside of the generator it's almost as if the generator is a template for a function it says hey I need a value here and I don't know what to use I'm going to pause while some other part of the program figures out what value to give me and then I'll resume so I'm going to yield out the value 4 and say hey give me a value back tell me what to use here we yield the value 4 out and later line 11 we answer that question with you should use the value 10 so we asked the question 4 and we answered it with the value 10 then we asked the value 5 and we answered it with the value 20 then we asked the value 6 and we answer it with the value 30 so I now have X 11 Y 22 z 33 at all 3 of those up and do a final return or yield and out comes the value 66 this is two-way cooperation computing of a value I have some values you have some values we cooperate together and we figure out the final answer that's what we mean by co-routine this is cooperative cooperative concurrency what's any of this got to do with asynchronous programming here's what it's got to do with asynchronous programming when I call this Ajax function let's assume the Ajax function returns us a promise if I yield out that promise the generator is going to pause that's what it does when you yield things it pauses locally inside of itself it's going to pause and I'm saying here hey whenever you resume with the answer I'll assign it to the value V notice that we have an equal sign here on the other side of what would otherwise look like an asynchronous call you normally can't do equals with things that are asynchronous because equals is the synchronous form of assignment so it doesn't usually match with a synchrony but through the magic of the yield keyword this generator locally pauses on its inside and when we yield out that promise we yield down out here and you receive this promise out here you say you know what I'm gonna listen for that promise to resolve and when that promise resolves with a value the response from the Ajax response I'm going to call dot next and pass that value back in replacing the yield expression with the Ajax response do you follow that so from the perspective of this code it looks synchronous doesn't it we don't have callbacks and thunks and promise chains and any other stuff we just have synchronous code in other words we have code that does not concern itself with time right we have abstracted the time component into a library somebody outside of our business logic is handling the race condition time stuff and our business logic is nice and cleanly synchronous so in other words we have a representation of a future value an AJAX response in a time independent way now this mechanism that I've constructed here is highly annual right so we have code that knows that expects a promise and it's got a return it back right but this can be mechanized this loop of yielding a promise out and resuming the generator with its return value and yielding another problem that can be mechanized automated and so we take a generator like this one that's going to fetch a you current user and then fetch the promised thought all of those two happening concurrently and these yields are yielding out promises and this guy runner is the one who handles all that mechanical automation he says I know what to do with promises I'll call dot then on them and whenever they resolve I'll call dot next with their resolution value and every time you yield me a promise I'll call dot next and every time you owe me promise I called up next that pattern is called a Jen runner a generator runner well known well established every library out there that you use has a generator in it Bluebird calls it co-routine queue calls it spawn the co library did every one of them they all have a generator in them I've written like five of them generators are pretty straightforward but you should not have to go reinvent that wheel yourself you should use some library to do it for you but this code what I would argue is starting to move the ball forward towards communicating more clearly what where are lines of concurrency are it eliminates from our concern many of the time issues that were causing those problems in the earlier versions of this snippet it allows us to reason about ok I need to first get the user I know the yield is going to go get me the user and then once I have the user this pattern if I recognize what promise thought all is this pattern says go do these two things at the same time I don't care what order they finish in let me know when all of them are finished that's what promise thought all is so it says hey I'm going to wait for all of them to finish and then I move on to the next step so it's more clearly articulating do one thing then do two things concurrently and when all of those are finished do the next thing that's the flow that we need in our app and now we're starting to not only just do that but communicate it more clearly so through the magic of generators in this yield mechanism that I've now uncovered how it works you know underneath now we can write what is called synchronous looking asynchronous code synchronous code is easier for our brains to reason about it's easier than vertical promise chains it's far easier than nested thunks and it's even far easier than whatever the hell it is we do with callback hell okay same code same outcome cleaner articulation of the ideas and if the code art articulates its ideas better my argument my assertion is that code is doing its job better because there's a better chance that this code will survive the next time somebody needs to read it and figure out what it's doing if it communicates its ideas better it survives if it communicates its ideas better it's far less likely to have bugs in the first place so the sync async pattern with generators is a big win but on the other hand having to use libraries to do that kind of sucks or at least for some people they feel like having to use this run or library thing to run their code is unfortunate I've even heard some people say the star is just so ugly I just can't abide by it so you can come up with whatever your complaints are but there are people that say yeah that's all well and good but it's not good enough so almost as soon as generators landed in JavaScript we immediately started talking about what if we could do that but do it nicer without a library we do the same pattern but let's make it nice let's give it some syntax that's the solution to everything invent new syntax okay those that's more stuff to learn just invent new syntax it'll look better and people will like it better so we invented the async/await which is the exact same thing as a Jenner but it's syntax instead of a library literally the v8 engine trans piles your async/await into a generator and pumps it into a generator literally that's what it does so these are one in the same sort of this is a syntactic sugar that many people are excited about and certainly agree that there are some attractive parts to it but in just a moment you'll see that there are some trade-offs so whereas we would have written function star main we now write async function main and whereas we would have done something like yield we now do a weight and whereas we would have past main into a runner utility we now just call it but other than that under the covers it is identical it is awaiting a promise which locally pauses and then resumes with the promise value exactly the same as we yield it a promise and resumed with dot next so in other words when we take the async function style we can still represent the exact same style of code but now we're using a weight instead of yield an async instead of an asterisk but this code as otherwise identical and this is the code that all the blog posts that you've read over the last few years are all gushing over look how amazing this is and it is nice it is certainly better than what we were doing three and five and ten years ago but know your tools because the convenience of syntax comes at a price okay async await is certainly a nice thing to have and I would say in my own code probably at least 50 to 60 percent of the time this is what I'm using but I'd only use that 100 percent of the time because it turns out that some of the time what we actually still want is a generator so I want to talk to you about some of the limitations that come when we decide to offload that gen run or pattern to the JavaScript engine in favor of nicer syntax okay let's talk about some of those problems first one problem that we have and this is true of both async and generators by the way but one problem that people often run across whenever they throw an async function around some code is that they love to do things like for each anybody in here just absolutely love using that for each instead of for loops come on admit it you all use y'all love for each instead of your for loops I don't know why I don't like for each at all I don't know why everybody's so obsessed with it but people love to write for each instead of a for loop it's like using a for loop is like saying to somebody that you're sick with some kind of crazy disease and you won't want to stay away from that developer so we all get obsessed with the for each method but there's a problem here because the await keyword or the yield keyword there what are called shallow continuations they don't work across function boundaries so as soon as you wrap a function a regular function around that code you can't use the yield or the await keyword anymore the notion of being able to use a yield or an await keyword inside of an inner function how the con has a name because some languages support it it's called a deep continuation if you want to do some google searching you'll find Dave Herman's article from several years back about why the concept of deep continuations are fundamentally incompatible with javascript in the web the end answer is we're not going to get deep continuations it doesn't work and he's a really smart dude with a PhD he knows what he's talking about okay so we can't have deep continuations in our language which means that when you start using these functional iterating styles like that now all of a sudden you're nice pretty away yield stuff doesn't work anymore but that's not even really actually the problem that's just the way that we figure out there's a problem because we do it and we get a syntax error that's not really the problem really the problem is for this to work for this to work the for each utility would have to understand the concept of a synchronous iteration before each utility and the map and the filter and all the others they're all synchronous iterators which means they operate over a list that's fixed in length where all the values are current available right now and they just run through it eagerly all right away this would require lazy iteration which is I'm going to do the first one and then wait and then do the next one and wait and then do the next one and wait you actually need an entirely different utility you don't just need deep continuation you need an asynchronous iterator that's what I meant when I was talking about earlier figure out what the real problem is is the real problem that JavaScript doesn't have deve continuations no that's the symptom the real problem is that what we actually want here is a synchronous iteration and this thing most definitely is not an asynchronous iterator was designed back in 1998 when we didn't really do a synchronous iteration okay so it's not an asynchronous iterator so how do we fix that problem we invent asynchronous iterators I wrote one and it's called Phase II it's a library that gives you a synchronous iterators like for each map filter reduce all those iterators that you love using but it's asynchronous iterators which means that you can pass an async function to it and you can await inside of it and it knows what to do with that and by the way Phase II allows you to do either serial concurrency or parallel concurrency so you can pick serial or concurrent either way whichever one you call for tiny little 2k library but it gives you a synchronous iterators if that's what you need look for the actual problem and solve it with the actual correct tool and maybe you don't like my iterators there's probably dozens others out there I don't care but the point is you need an asynchronous iterator to go find one that's the solution to the problem make sense okay more about asynchronous iteration in a little bit so I mentioned that there are some problems with async functions those are problems with async but quite frankly there are also problems with generators so what are problems specifically with async functions one of the problems is that the await keyword only knows what to do with actual genuine the noble promises let may not seem like a huge issue but there are pre-existing two promises a variety of common representations for future asynchronous work most notably funks you may not have used them a lot but they have actually been around for a while there are some libraries that have used them folks were prior to promises a somewhat standardized within their own subset of the world a somewhat standardized way of representing future work in a single value if you have a generator and utility that knows what to do if you yield out values you can make that generator genuine or smart enough to know what to do with promises and thunks and any of those other representations of future values when we added the await keyword they said no no no just only worked with promises so we're limited in our ability to express a synchrony we have to use promises if we want to do that with the async await that may not be a huge deal for many of you but it is a limitation on our code and I've run into places where I have this thing that vens me a thunk and that's not something I can await so what am I supposed to do I need a promise but what I have is a thunk so I'm stuck it'd be nice if I had a mechanism that would allow me to extensively listen to other sorts of asynchrony besides promises that's one thing but it's kind of a minor point this one is I think a giant problem that basically everybody's just sticking their fingers in their ears and pretending it's not a problem if you've ever done any sort of formal study in parallel programming you'll know that the concept of scheduling starvation is a huge problem it's a huge problem that has a whole area of open research people researching the concept of how to create scheduling algorithms that are not susceptible to this problem called starvation but simply starvation means one part of your system accidentally or maliciously can zap up all the available processing resources and prevent any other part of the system from having a chance to run that's what starvation means now it's not common that we worry about malicious starvation although there are denial of service attacks that use malicious starvation most of the time starvation comes accidentally you build some part of the system and then you build this other part of the system and it's not clear that this part of the system has this susceptibility that it can accidentally get into essentially an infinite loop and starve out the processor and this part of the system never gets to run one of the most common solutions to the starvation problem is what we call round-robin scheduling round-robin scheduling says everybody gets one shot you get one shot than you than you than you now you again you so we spread out the resources evenly put that way it would seem like round-robin scheduling would be the most reasonable scheduling algorithm for them to decide to use for this new kind of asynchrony but they didn't think that was going to be a problem so they didn't specify round-robin scheduling instead they specified this greedy scheduling which is the micro task queue if you've done any reading about how promises work that essentially leads us to greedy scheduling such that in less than ten lines of code I can write for you a promise loop that will completely starve out every other micro task in the system whether I do that on purpose or accidentally is not really relevant the fact is that what's built into JavaScript in its core is a poorly designed naive scheduling algorithm now I've brought this point up many many times with people on the tc39 committee and I've pointed out use cases where it actually happens not in contrived code but in real code a real problem that I actually ran into with my real code pointed this out on many occasions and they've basically said if this was a problem more people would be complaining about it so they've ignored me and just stuck their fingers in their ears and said this isn't a problem I think that is to our detriment because there are some really smart people that have said this is a real problem in all other domains besides JavaScript for us to uniquely claim that it's not a problem for us seems rather misguided but there's a starvation problem with the scheduling algorithm in the core JavaScript engine gen runners that understand this problem like the ones that I've written aren't susceptible to that because they use round-robin scheduling the third problem is I think the big deal breaker if you don't care about those other two points listen to this point it is extremely common that we fire off tasks that we cannot predict how long they're going to take you fire off an ajax request and you think it's probably going to take like 500 milliseconds maybe a min you know maybe a second or two what would happen if it took five minutes how would your app respond more importantly than how would your app respond how are the users on your app going to respond when it's been 30 seconds and that spinner is still spinning they're probably not even going to wait 30 seconds what are they going to do when that spinner is still spinning at the 15-second mark 100% of the time they're going to click the refresh button or they're just going to close it and go away one of those two things are going to happen in other words your system failed them because it did not communicate to them that they could expect for their to ever be a resolution to the problem once it's taken 10 seconds it might as well take 10 minutes or 10 hours or Never that's how users think what I'm getting at here is that users variance requires us to design with limits on the length of our asynchronous actions I don't see this being talked about nearly enough in UI and UX design I don't know why but I don't see people talking about I need a timeout on almost everything because there's almost nothing that we do asynchronously that we should allow to run unbounded ly take as long as you want almost everything that we do asynchronously should have some way for us the sanity check hey five seconds that's too long if it's taken more than five seconds we might as well start over because it's not gonna happen right I don't know what the time that is maybe it's five seconds for one thing and two seconds for another and thirty seconds for an it doesn't matter the point is every asynchronous action should have an ability to be canceled if it's taking too long I can't come up with very many examples where we don't need some mechanism for cancellation so async functions we call an async function and it starts doing its fancy away stuff on all these Ajax calls it's a complete black box from the outside we have no idea what it's doing under the covers and more importantly than that we have absolutely no control over it you cannot tell the async function hey stop running you've taken 30 seconds stop stop using up network resources I've moved on to other stuff the user unloaded the page stop doing that you can't communicate any of that to an async function it's a complete black box now this is not a point that went undiscussed I was very vocal in the spec process leading up to async functions landing saying hey this is a major problem don't design a whole new syntactic feature in the language that fails at this most basic of use cases which is I need some way to cancel it it was very vocal for a couple of years on several different threads and mailing lists and what they did was shift a single weight without any external cancellation so that's what we ended up with so we have all these people writing all of this asynchrony with no way of canceling it and it's like we've just said and that's not an important user experience part of what we do and I feel like I'm crazy because I feel like I'm the only one that cares about this I don't hear anybody else worry there's nobody else writing blog posts about it nobody's talking about it how are you handling cancellation are you how are you doing it with a single weight because I'd like to know because the async/await function does not provide you any facility for doing some to me this is a major drawback in the design more importantly to our discussion if you use a generator driven by a generator guess what you're generating your generator can decide to not call dot next and therefore it can abort right in the middle of some process so your generator has the ability to abort a generator but the async function can't be aborted to me that means that generators are still superior to the async await function because they have the ability to be canceled externally okay so it's this external cancellation problem that I'm concerned with now I understand why people like the ergonomics of the async/await function I understand why they like being able to call a function and get a promise back and let it do its thing so what I did was I decided to try to build a tool to help us bridge the gap between the two it's called CAF you take a generator and you wrap it in CAF and what you get is a function that looks like an async function you can call it all by itself it gives you a promise back it seems to be like an async function but under the covers you pass in a cancellation token when you call it right here you pass in a cancellation token and it knows when you pass in a cancellation token I better listen to that cancellation token and if at some later point you decide it's taken too long I want to cancel the token the jhen runner inside of coughs is up you're done and it cuts it off immediately so it tries to give us the ergonomics of in Kuwait functions but with cancel ability that's what CAF stands for cancelable async flows okay now this idea of timeout based cancelation is so core to the use case that instead of making you write your own set timeouts this is just directly in the API you can make a timeout based cancellation token it will automatically cancel itself after the specified amount of time and optionally cancel with a particular message so you take those timeout tokens you make them you pass them in when you call these calf wrap functions and they will automatically cancel themselves if they're taking too long and by the way the cancellation token that's being passed along you're getting it by way of the signal whether you're aware of this or not but the most common form of a synchrony that we do at least in our browsers is Ajax requests most of you are probably using the fetch method by now you may not be aware that recently like over the last year such developed the ability to cancel itself with a cancellation token so now not only can your async function cancel itself but the actual underlying Ajax call will stop itself at the bite level and stop consuming network resources and the signals that calf passes along are exactly the same cancellation tokens that fetch knows how to use it's a good standard and we didn't even invent it it was invented in other languages we're just saying hey that's a really good idea let's just do cancellation tokens okay I don't know if this is the future of cancellation this may end up in the long run being a terrible idea but the fact that nobody's thinking about cancellation of asynchronous functions except me is troublesome we need to be thinking about it and inventing better ways to cancel the asynchrony okay so we've moved from callbacks communicate very poorly they don't store things they don't store values over time well so let's use thunks thunks do Express that there a good representation of value over time but they have other trust ability issues and that's what promises give us but promises have a really crappy API so let's put generators around it but generators require this runner utility so let's put syntax around it you notice a repeating pattern we like we see a problem we invent a solution that creates another problem so then we wrap that in a solution that creates another problem and wrap down in a solution that creates another problem the takeaway that I want you to get from that is not that I'm just trying to all over everything that everybody designs I understand how hard it is and I would not want to be on tc39 let me just say that for the record I do not envy them in any way shape or form they have an almost impossible task designing a language of so many constituents so I'm not here to actually criticize and say they've done a poor job I'm saying that we're doing a poor job of learning our tools and our poor job of learning our tools is poorly informing the design process of the future of our language they're getting poor data from us because we're not doing our jobs to learn our tools better and figure out what our tools actually need to be doing we don't need to fix more syntactic sugar that's not our problem we have deeper core problems that we should be focused on there is one remaining gap in if you were to plot out what we are capable of doing with asynchrony and JavaScript up to this point there's one remaining gap it doesn't sound obvious because you haven't maybe to spent the time to think about this in such a formal fashion but what we essentially have if you plotted this out on a grid one of the four quadrants of the grid has gone unanswered up to this point because what we've had is the ability to do pull of single value and push of single value and pull of multi value but we've had no way to do push of multi value it's this totally open-ended unsolved use case and as a matter of fact two that in user land we invented this thing called observables you might have heard about them observables are many value push okay well that's a missing feature of the language and for a while we thought observables themselves were gonna land in JavaScript the tc39 committee decided with this year's javascript to land a feature that is not an observable but is sort of the core principle underneath an observable that sort of you could kind of implement an observable on top of and as things currently stand I'm not really sure that observables are ever going to land natively in JavaScript because at this point this nos last feature that I'm about to talk to you about kind of addresses at least some of the use cases in a decent way it's not really a full and complete solution but it's probably good enough for most cases so that probably means it relieves them from having to add it to the language and we'll just keep using observables as a userland solution I don't know observables may land someday but they've been stalled out at stage 2 for like a year and a half so I'm not sure if it's looking particularly likely so what is that last and final feature well I've talked to you today about generators generators are a push interface but they can push a single value they can yield a value out and the iterator can grab you know receive the value as its return right so generators are a push interface and then we talked about async functions and thought about in the proper if you twist your brain just write an async function is a pull interface I'm saying a wait a wait some value I want to pull some value a single value from a source so if I want push and pull and I want multi value what I really need is an async generator I need the love child of async functions and generators stuck together so that I have both push and pull semantics in the same function and what async generators are it's gonna look a little strange but you guys just stick with me all right so here's a this is not an async generator this is a regular async function but I just want to show you the idea here what I'm doing say I've got a set of URLs that I want to go grab all the responses to one way of doing that is to make the fetch receive the result push it in and then make the next fetch so I'm going to do it one at a time this is sequential calls that's super inefficient right so what if I wanted to make all of those calls at the same time and then as those calls were coming in I wanted to consume them and do something with them so in other words I want to request many values I want to pull many values but I also want to push many values I want to do the same both of those things in the same utility instead of doing them all at once up front either sequentially or concurrently let me pull one and push one and then pull another and push another and pull another and push another that's the most efficient way for us to have that many to many asynchronous relationship so let's think about a generator how does it generator do it a generator does allow us to push values out we don't have to collect all of the values and return a giant array we can yield out a single value every time see I'm yielding out the response text so an iterator could get one result at a time if I was going through a thousand URLs it would request one and give me one and request one and give me one so we actually do have the push-pull here but it's kind of a problem because what we are actually doing is we're overloading the yield keyword to do both push and pull even if we had a utility that understood how to make yield intelligently do both push and pull like for example when I yield out a promise it says okay that's a pull but when I yield out a non promise it says okay that's a push even if we had that which that isn't actually a utility but even if somebody invented something like that this does a poor job of communicating the diff forints enrolls between those two usages of the yield keyword you follow me one of them is pulling values and the other one is pushing values and that's not obvious by looking at the code that's why we need an asynchronous generator we need an await keyword to pull values and we need a yield keyword to push values so we now have async generators both the async function and the asterisk it seems like we just keep inventing new function types every couple of years so I guess we're just going to gotta get used to having lots of different function types all right so we have async generators now here's what's key this async generator is going to fetch one it's going to pull one within a wait and then when it has a value it's going to yield it now I know that doesn't seem like a big deal to you that we now have different keywords doing different things but this much more clearly communicates the delineation between push/pull semantics and when taught properly this code will actually I think be much clearer in its communication style okay but more importantly than that is what's going to happen on the other side of the equation which is how do we consume this stuff okay so I want you to think about if I had an asynchronous generator it's a generator so the way I'm gonna get its results is one at a time iterating through with dot next calls right but let's think about how that iteration is actually gonna happen what are those doc next calls actually gonna look like let's imagine that my fetch URLs was this asynchronous generator right and I said let me just do a four of because I know I can do four of loops over iterators the reason the problem is this is not going to work and here's why because when I call a dot next call on a synchronous generator what I get back is an iterator result it has a value in it and it has a done property that tells me true or false I have that result immediately now now you itself the value in the property value that maybe I promise for some future value but importantly I have an iterator result right now I know that I have a value and I know if I'm done or not you follow me if I call dot next on an asynchronous generator it doesn't know yet whether it's complete because it's still pulling its value so the return from the dot next call of an asynchronous generator is not an iterator result with maybe a promise in it it's actually a promise for an iterator result we don't have the results yet but we've got a promise for an iterator result so when that promise resolves when the pull has finished and now we have a value to give to you that promise will be resolved with the full iterator result the for of loop doesn't know how to handle that because it expects to get an iterator result back not a promise for an iterator result so we do what all good language committees do we invent a new style of iteration here's how you could do this manually or how you might think about trying to do this manually you would say all right if I call IT dot next I'm not going to have this res right this is not a result it's going to be a promise so this code is not going to work because I'm trying to check a done property on a promise for a thing not the thing so you say to yourself all right well that seems pretty simple why don't I just await the IT dot next and then once I have it undone so I will await the iterator result and then I can check if I'm done and then I have my value and I can move on so we could do a while true like loop like I'm showing you here and you could effectively iterate asynchronously through in a synchronous generators pushed values nobody however wants to write a loop like that we want nice dedicated syntax so we went invented the 408 loop the 408 loop does exactly that which is that it iterates through asynchronous iterators and it awaits their results automatically before assigning off the result so now we have a synchronous generators and we have syntactic asynchronous iteration remember earlier tonight I talked to you about that Phase II library which provides you a synchronous iterators in functional form this is a synchronous iterators in syntactic form they are isomorphic meaning that you can express asynchronous iteration in either way so eight synchronous generators and asynchronous iteration fill in that final missing quadrant so now we can do both push and pull of single and many value we can model all the types of concurrency that we're going to run across including in a very primitive way something like observables that's probably why observables are not going to land so that was a lot to throw at you I'm sure your brain at this point is like where's my beer I'm certainly there with you I understand just as a review of what we've talked about tonight the bar that I first set was can I make the code work can I get it to do what I want quite frankly you all are experts at that you're really good at that that's why you get paid so much money that's why you have seen your architect in your job title or whatever it's because you're really good at making the computer do the right thing getting the right sequences of ones and zeros at least enough of the time that most of the time there aren't bugs in production you're pretty good at that where I think we can really stand to improve is on learning our tools better so that we can communicate these ideas better with ourselves and others that's what tonight's talk was about not trying to show you new shiny toys and say oh well you know the balls the game is always changing and there's always something new to learn that's true I get people asking me that all the time like how do you keep up with it and my answer is I don't I don't keep up with it it's too much to learn but what I do think is reasonable and useful is for whatever area I'm focused on for me to be intensely curious about how that thing actually works so that I can understand the problems that I'm solving better and use my tools more effectively I'm not going and looking for the next shiny framework or library to fix my problems I'm looking at what can I do to learn this system better some people say that would make me theoretically a less effective employee where I to join some hypothetical development team because we supposedly think that the most effective employees are those that effectively go out and use all those tools and churn out more code I have a different belief the employees that are really good at going out and getting the new hot framework and the new library and putting all these things together and shipping more code they're not the employees that actually move the bar forward as much as the employees who understand the problems and solve the problems with the best tools available so what I aspire to and what I hope I've maybe challenged or inspired you to try to reach for is that desire to be more curious and dig further into the tools why do I use this thing and why is this thing better than this thing you've heard that old adage like you know use a hammer for a nail and a screw for a screwdriver don't use the wrong tool for the wrong job there's also a deeper meaning to that if I have a nail that I want to drive in the wall and I have a hammer in my hand that's the right tool for the right job and if I swing it like that I'm probably approximately going to drive the nail in the wall but if I take the same hammer the right tool for the right job and I put it in my mouth and I start swinging like that well right - all right job right you got to use the right tool in the right way and the only way you can effectively do that is to understand your tools better that's what I hope I've inspired you with tonight thanks for listening [Applause]

Channel: UtahJS

Views: 5,393

Rating: 4.9178081 out of 5

Keywords: JavaScript, Generators, Promises, Async Functions, Async Await, Async Generators

Id: XcOJzPyXpJg

Length: 93min 43sec (5623 seconds)

Published: Thu Oct 04 2018