Netflix JavaScript Talks - RxJS + Redux + React = Amazing!

we go so managing state it's hard right if you've written any anywhere near a complex application you've quickly figured out that it is really hard to manage your state and some of the react community even the angular2 community some of the ember community have started to use a library called redux why because it makes managing your state a lot easier to reason it makes it simple now it doesn't necessarily make it easy which there is a difference between simple and easy but it does it does make it simple and much more easy excuse me much easier to reason but there's one problem with redux it doesn't actually do anything to help you with your async code and that's a problem because async is arguably actually harder it's a harder problem to solve than then state management particularly particularly if the problem you're trying to solve is just inherently complex there's just nothing you can do about it it's just a very tough problem doing things like parallelism or multiplex WebSockets they're they're inherently complex there's nothing you can do about it so this talk is going to be about using a library called rxjs to at least make it manageable so we there's nothing you can do it's going to be complex but let's make it manageable so at least you can understand what's going on and be able to dive your way out so Who am I I'm Jay Phelps obviously that is not the same shirt that I'm wearing right now even though it looks like it is I just happened to wear another blue shirt but that's my Twitter picture it's this in case anyone hadn't recognized me obviously I'm a software engineer here at Netflix and you can follow me at underscore Jay Phelps the regular Jay Phelps is unfortunately a 16 year old guy who likes to retweet football pictures and I always bring that up just because you'd be surprised how often people do tweet him when they need to tweet me and so and and still Twitter will not give me the the cool there I'd badge so that people you know don't accidentally tweet him so anyway let's just dive it what is Redux now this talk is not going to teach you all about Redux or even all about rxjs because both of those things are mega talks on their own numerous talks but i will give you a quick crash course just so if you have no idea what it is you at least have a high-level idea so redux provides predictable state management and it does so by using things called actions and reducers so what is this action it's going to describe something that has or should happen but it doesn't say how then what I mean by that is if we look at an example of a simple action this create to-do action you've got the content you you know you you you got the intent you know you want to create it to do and you've got the content for that to do but it doesn't say how you're going to do that it doesn't say go to the server and and store this in a database or store this in local storage has nothing about that there's no async async' stuff no none of that no side effects it's completely serializable as an intent so keep that in mind what is the reducer oil reducers are pure functions which just means that the input that you provide it every time you provided inputs the same input decisions using the same output will always be the same it's deterministic it does not have any side effects or any viewable side effects more importantly and it what it does is it takes you it takes the previous state it's provided the previous state and in action the current action being dispatched and then it's expected to return some new state now that new state could actually be the exact same state like let's say that there's absolutely no changes that need to be made it just returns the existing state basically in OA lot of times you may need to do compute some new State so let's say in this in this very trivial reducer example that you always want to take the action payload and append it to the existing state that is another example of a reducer in the redux use cases like this this counter so we basically switch over the action types and whenever I receive an increment I go ahead and actually increment the state and when I receive a decrement I decrement it this is fairly basic hopefully everyone understands this but I'm gonna just breeze past it if you don't so reducers they have your state transitions but they must be done synchronously and that therein lies the kicker of all this so that means when I receive that increment I have to increment it right then in there I can't debounce this I can't send something to the server and ask can I actually increment this or anything like that it's just I received that action I have to do it right then in there it's all synchronous so what type of async stuff do we commonly do why is that a problem there's a bunch of things right there's user interactions keyboard mouse mouse movements and stuff like that Ajax is probably like the bread and butter of what you all know you know we all make Ajax is right timers and WebSockets and all sorts of stuff and this is not an exhaustive list and some of these things can actually be handled synchronously even though they are async in nature the way reactive redux currently are architected you could handle them synchronously and what I mean by that is taking a look at this react example you'll notice that your packet passing a callback to the on click which most of you are probably familiar with and so when that click happens you go ahead and just synchronously dispatch that that increment and and the reducer will pick that up and we'll do the increment and always happy right technically speaking this is async but you're handling it in a synchronous way why because react is actually abstracting and handling the asynchronous e4u now it is a problem though because what if you wanna get what if you wanted to bounce this what if every time someone clicks that button you didn't want it to actually increment right then and there you want it to wait a period of time and then if they haven't clicked again then increment it you know or throttle there's different there's different different ways of doing it so sometimes you need more control as my point and the some of the things you need more control on is Ajax cancellation or composition composition meaning like as an example if you you make one Ajax request and you need to make another Ajax request using the response of the first one and so on and so forth doing denouncing or buffering and those type of things basically manipulating time drag and drop in WebSockets so in the Redux world commonly or excuse me most commonly people use they don't wear for this and a piece of middleware basically sits between your application and the redux store so any of the the actions that you dispatch will go through and the middleware either before or after they reach the reducer it's completely up to the middleware to decide in which which order they happen and a lot of these existing middleware use callbacks and promises to to give you that ability to do more complex async stuff so let's let's take a look at these two options these these two most popular ones callbacks the almost everyone should be familiar with them because they're the most primitive way in Java scripts to to handle asynchronous e and it looks something like this you know you you call a function passing it a callback and when that data comes back you're going to receive the data or there and then boom you can do your work now there's lots of problems with callback but I'll point out one in particular and that's callback hell right especially node programmers if there's any node people in the room you've all experienced this and and that's that you know when you want to chain you want to do something async and then when that's done do something else a sync whether it uses the next one or not you want to sequential e do them or even more complex with callbacks what if you wanted to do them in parallel and then when they're together like when they all finish then do something else like it things become very complicated right and my colleague Ben Lesh likes to call this the the Flying V because it's you know it's sweeps out more horizontally just like the Mighty Ducks so there's a solution that was created called promises to try and resolve some of those issues and with a promise there's a neat trick that some people don't actually know that's when you provide your call back to your to your promise if you return yet another promise you can then change them horizontally so here we fetched the data for the for the given ID and then we fetched some more data with the parent ID we're turning that new promise which that that's what lets us do the horizontal style of composition and so once you understand that pattern this is a lot easier to read it's pretty good I mean so this is a really good candidate to look at so let's let's let's dive a little closer to promises they've got a guaranteed future which means once you make that promise and it's going to go do what it's going to do there's nothing you can do to stop it it will do it no matter what so it will either succeed and call the success callback or it will fail and call the fail callback there is no cancellation so they are immutable they represent a single value and they have inherent caching you know so if you if you create the promise and then then then listen to it multiple times they'll get the same value each time instead of making multiple requests with the same promise so there's in real rural applications we quickly discover that there's two problems with some of these things that the guaranteed future and the single value so let's take a look at the first one that guaranteed future and again remember promises cannot be cancelled why would I ever want to cancel a promise all right why would I make a request why what do I cancel it well there's there's lots of reasons but let's let's look at a couple the most common changing routes or views imagine that you you mount some component you let's say you're using react router or something you mount not your component and then on component will mout or component did mount you go off and fetch some Ajax request and then before that request is finished someone changes the route someone transitions away is the back button goes somewhere else you can't cancel that request so what are you going to do you have to handle that in some way because you cannot cancel it there's the autocomplete use case which I'll elaborate a little bit more in the future or you know there's simple one like the user wants you to like you're making some then you want some button to let you to let the user actually cancel the request that that is going out first I'm just going to give a little demo of the first use case just so we all are on the same page and understand it and I'm going to do it in the context of Netflix's member home page so if you how many of here are Netflix users ok nearly everyone great thank you you're the reason I have a job so let's say that you know I you know I log into Netflix I'm a big fan of daredevil so um you know I I go and I'm like oh well watch daredevil so I click on that now when I do that immediately when I start clicking that let's say I make an AJAX request to get daredevil now again all of this is not I'm not going this is I'm not explaining the architecture that Netflix uses this is just a hypothetical example so if there are any people in the room who work on this particular UI forgive me but let's say I I you know I make that request to go to go get daredevil but then I decide wow that's still going on I've got a really fast brain and I decided I don't wanna watch daredevil what's sick of here devil so what I do is I go back and I decide now the get-down no one wants to get down now notice I didn't cancel my daredevil the daredevil ajax.request is still going out so I've got code that's waiting for that request to come back when that request does come back what are you supposed to do not only is the JavaScript that presumably you're sending JSON but there'll be the the json.parse so there'll be some CPU cycles even up on that and not just that you have to handle that in some way so if that component has already been unmounted if you don't you have to have some sort of bit that you store that says like is unmounted then ignore the response so like when you come back for the from the promise you know you'll say if is unmounted or not then just don't do anything and just throw the promise away which is not economic at all and also a waste of CPU resources particularly if you're on a very low powered device right so what we want to do is usually want to come here want to select our daredevil make our request say I'm going to go back and when I do it immediately cancels that request because I no longer care about it and when you cancel it that means that your handlers no longer need to worry about state like maintain some arbitrary state of like ignore don't well pretend the the man behind the curtains not there type of thing so I can safely make that new get down request and everything will be hunky-dory and peachy that's the that's the the problem in the solution so I bring this up and stress it because canceling is very common needing to do cancellation and it's really often overlooked I don't know how many times I've looked at people's apps they've asked me hey can you can you audit my app and one of the first things I do is I look for async stuff and I look for places in which they don't handle the case of cancellation so I try I go into places that will do Ajax stuff and then I'll immediately leave and go somewhere else and didn't switch pages or click buttons and do all sorts of things and then their Apple often get in really obscure states and when they get in these really obscure states it's really hard to diagnose that right so you get a report from CS from your tier 3 or your support and they're describing the problem and you're just like how I don't understand how they got in that and you is just a normal developer who knows how things work and your app because you wrote it go back and try and recreate it good luck you know because it was like a series of steps and you had to race it that's the key it's a race so bad we need to fix that somehow so the other thing about promise is the single value and this isn't as big a deal because a lot of the stuff that a most people's apps do is single value so for example Ajax you know it's the only one of these four things that is a single value it's request/response you only have one single value and so that's not the end of the world for for promises but there are a lot of things that people want to do that are more than that particularly around user interactions I could do bouncing and stuff like that and WebSockets for me at least so what do we do what do we use to solve this problem well we used or observables and what is it observable if you and if you've never heard of them before I'm going to very quickly go through this it's a stream of zero one or more values so keep that about zero one or more zero meaning you could literally have no value and that's okay it just completes where you have one value just like you know like that like like a promise does we could have more values and those more values are over a series of time that new dimension is what's really important about observables compared to promises over time it's a stream of things happening and we get the cancellation the thing that we really were looking for in that last demo so they are streams are a set with the dimension of time one cool thing is that they're actually being standardized right now so they may actually land in the native browser without a library don't hold your breath you know who knows how that's going to go but it's very possible just like the promise promise went down the standards track observables are going down there as well now they aren't in the browser today so what do you use when you use rxjs which is a reference implementation of that of that specification and on top of that it provides some custom stuff on top of it operators and factories for creating different types of observables and I'll touch on it a little bit but my colleague Ben Lesh likes to call this low - for async mainly just because it's a utility library but we're we're low - is is just like basically you have you have it you give it an end it gets it out this is like you give it an in giving it out but it's over time so you have multiple values and you so let's dive a little bit deeper with that crash course so creating observables there's lots of ways to do it this is not a talked about like I'm going to teach you and make your own rx master so if you get completely lost it's totally ok these are creating some of the the simple primitive values like an observable of one item an observable from our array of items setting an interval or making Ajax requests or WebSockets and there's a bunch of ways you can create completely custom observables as well which is common these observables you you subscribe to and you subscribe to them not too dissimilar to how you would then a promise so you can actually provide two functions the first one being the next function is what they call it which is sort of like a promise like the the called the success callback except for because this is over time and it can be 0 or more values that next function could be called n number of times it could be called thousands of times or it could be never called depends on how many times that observable emits something and then there's the error callback and there's one more path which is complete because we have a time we sometimes it's useful not often but sometimes it's useful to know when this thing actually is done when it completely completes nothing more so that's the basic primitive of an observable what can we do with these well well they can be transformed just like I was talking about with lodash we can map filter and reduce these things but we're not just doing it on a single item we're doing it over the stream or stream processing so as something comes in we'll map that to something else and we'll do it in a very efficient way so we're not creating intermediate arrays and things like that we're doing it very very efficiently observables can be combined we can cut them and merge them and do all sorts of cool stuff with them and because they have a time dimension we can very trivially very trivially do debouncing and buffering and throttling like so trivially it's like deceptive and for good measure and a bonus observables also lazy typically which means it's very easy to retry and repeat them because they that you basically build up a definition of what the observable should do but it doesn't do anything until someone subscribes to it it's lazy and so it makes it really easy to retry on an error or on some other condition or to repeat it like say make this HX call five times so if you're not getting it yet how beautiful can basically represent just about anything when you have all of those dimensions we don't recommend you do that we actually do that you don't our X all the things so write your entire application in or X you can do it but the real problem it's not that it's bad it's that our X is so new that it's you're being kind of a sadist you're like because anyone else who comes along he's going to have to like really learn our X really really really because if you can't you can't figure out any of your app so be careful you know it's great power great responsibility and all that so we really liked our X and we really really liked Redux so we thought let's combine these two things somehow somehow let's so we experimented with some patterns and we came up with a solution that we thought was rock-solid we thought familiar through a couple iterations but we finally figure out something we thought was rock-solid and if you've ever open sourced something the most important thing this is the most important thing is you create a good logo first don't write tests don't don't even make it work don't make documentation make a good logo it's the most important thing it's how you get stars and that's what's important right how many stars you got so what we did is we looked at the rxjs logo and then we looked at the redux logo well we tried to look at the redux logo but it turns out Redux actually did not have a logo at the time so we're like huh we want to combine the rxjs logo and the Redux logo but the redux Redux doesn't have an our logo so my buddy bin he is an artist as well as an engineer he three gether a sketch of three ducks because we always thought Redux sounded like three ducks like he was kind of our internal joke and thankfully its kind of caught on and a bunch of people now believe the same thing so he created that better illustration and we pitched it to them and although they thought it was funny they did not accept it and so we said well what can we do how can we how can we really like and we like the idea of three ducks and again Redux did not have a logo so we thought it will takes we'll take it duck well so let's kind of smash it around do some crazy things right you know just kind of playing around and then a with the rxjs logo on top of it right and then we bake it in an oven and and this is what we came up with so and I love this logo I love it because it's got the color and the flavor and the feeling of rxjs but it's got the three ducks and if you'll notice it's very the symbol means something the fact that the Ducks are positioned the way they are you know because it streams rxjs this wee doctor this library's all about the input of one is an output of another so there was one thing though I was staring at this and it's like there's one thing that's missing there's one thing that's missing about this and that is that it needs to rotate just to give it that extra possess so this is redux observable is what we ended up calling it which is kind of a boring name for that logo but Redux observables the library it's it's a middleware for redux for managing your side effects including async stuff and we do it using something called an epic which is a pretty epic word right you know so it's something we coined that's similar to a saga if you're familiar with the term but it's not the same and so we wanted to differentiate ourselves and it's just a function that it takes a stream of all the actions that your application dispatches then it's expected to return a new stream of actions that then that epic wants to dispatch so new actions so you excuse me so you've got actions in and then you want to dispatch other actions out other actions and so I'll I'll kind of get your foot wet and this idea very slowly here's a very imperative example this is not an epic but it's a pseudocode to get get you the idea of the principle so we've got this function ping-pong so imagine that this middleware called this function every time you dispatched an action and it provided it with that action and so in here you could say mmm does that action type equal pang if it does I want to return a new action called pum and if so let's just say this hypothetical middleware if you return something we will go ahead and dispatch what you return glowing so pretty simple you dispatched paying it'll immediately dispatch pop in the rxjs world however it's much more declared if we use things called operators and so this is what it would typically look like now I'm not a big fan of the function with the explicit return so I'm going to quickly switch that to using error functions with the implicit so hopefully everyone is on board and understanding what this is but this is this is a ping-pong graphic the most simple epic in the world it takes a stream that's that first action gives me first argument it's a stream of actions and then you match of type pay and when that matches if that is can is that filter is true then you're going to go ahead and map that to a new action called pong and we're implicitly returning this stream so it will the middleware will subscribe to this and keep track of that so it basically this is basically just setting up the pipes so you've got the pipe of the actions observable when any new actions come in they go through here and if they match ping then they will be the new action of Paul will be dispatched okay now this is so contrived right so contrived now let's do a little bit less contrived thing called every time a new element of that stream appears a new element so this is actually a factory that that's called once so it's called during the application setup because of the way rxjs works this is called one time when your application boots which sets up the pipes the rxjs pipes and so then action then data is actually pumped through these things under the hood this code you're seeing right here actually only executes once but then internal logic charts j/s whatever this fact that creates that thing probably a function of some kind is called whenever a new element arrives on this string this action string is you have a stream of actions here right yeah I could have a fight with them yeah and they could be showing up in anywhere yeah so then if any of them show up this is then Kali does whatever he does yeah exactly so all the actions go through here all even the actions that you dispatch so my pong action will actually also almost recruit like basically recursively pipe through this as well it will go through that action stream but it doesn't match ping so it just basically just gets thrown away because you're not looking for it it will still however reach your reducers so going back to this let's let's let's make this a little bit more interesting what if I wanted to when someone dispatches paying I want to wait a second and then dispatch the pong right so ping pong ping pong right well you can do that trivially with rxjs by just adding a delay operator delay is built into rxjs and so I can just say delay thousand milliseconds done and and it will work I can throw together a really quick reducer so anytime anytime we're pinging is true and anytime we're pausing it's false and this is what it would look like in your app you you you start the ping is true and then a second later it turns back to false pretty simple right let's look at another example and debouncing debouncing an increment and decrement button so we've got that counter reducer that I showed earlier that's pretty basic what if I wanted to 2d bounce that then I would I would instead of instead of having your application emits the increment action directly they'd instead dispatch increment D bounced that way you're not only does the app know that this is about to be debounce but that way your epics can then listen for that then they can use the rxjs operator D bounce time to D bounce them one second and then then it gets the increment type action gets dispatched same thing with decrements just inverted you know decrement D bounce wait a second decrement how that would look in an app is very simple you click on the button it's waiting for a second and then it's an updating so these are these are obviously very contrived examples right because I'm going to talk and it's it's it's tough to explain now I'm going to do something a little dangerous though I'm going out on a limb I'm going to show you some very non-trivial examples chances are good you will not be able to read the code which is fun the point of these examples I'll show is that arch this arch is and redux observable shine the most in the most complicated use cases things that are simple if they're great it's a good is it is awesome if you know R X but it shines the most in complicated scenarios so let's take a look at some look at an auto complete first this is how you would do it in just plain JavaScript it's you know it's not that bad it's it's not you know amazing but it's not that bad for for your your autocomplete so that you know the main thing is we if another request comes we want to abort the previous one so we have to store that xhr the previous one and make sure to abort it if another one's coming and and doody bouncing and stuff like that so d it's not that bad but what can we do with an epoch we can do the exact same thing with an epoch with just a couple lines of code and once you're familiar with rxjs or if you're already familiar with it this is very readable for rxjs people like this is very very readable and understandable and very flexible so we're debouncing it for 500 milliseconds and then switch mapping that to our ajax request and then when the ajax request comes back we map the result of that to an action the query fulfilled action which will get sent over to our reducers when it's done pretty straightforward adding air adding air cassoulet or using air handling is just as easy as adding a catch operator that's provided by rxjs so whenever there's an error you'll take that air payload and you can transform that into an action that you can send over to your reducers query rejected with the air and it can display that all all pretty simply cancellation there's two different kinds of cancellation if you're an rxjs person already you might have caught one of them which is i'm using switch map here and in switch map it's an implicit cancellation which means if if another in this case if another action excuse we have another query is requested and it gets to that point and the ajax call inside of it has not finished it will cancel it it will cancel that ajax and then start the new one it's just it's very implicit and you can opt into that there's if you don't want that behavior you can turn that you can use a different operator but there's the impressive cancellation but whatever I want explicit what if I don't want to have what if like I'm leaving the route and I want to cancel it then so I'm not sending another query I'm just leaving the routes but I want to cancel it we can do that what take a tail which is another rxjs operator so we're saying take this results until I match a an action of canceled query so if my application anywhere dispatch is canceled query boom Ajax is canceled now it doesn't cancel all of it doesn't cancel your epic so your epic will continue to listen for queries so if you need to query later you can do so so there are probably some rxj I'm hoping there are some rxjs users out here and if you are you're probably thinking this is great but this you said non-trivial examples this isn't that non-trivial if you know rxjs right so what can I do to show you some really non-trivial examples like really push the boundaries well what about bi-directional multiplex twedge suck 'it's how about that with automatic reconnect when the internet goes out and all sorts of bells and whistles and you mean it this is not contrived this is this is something that I've had to write in so many apps in my lifetime not everyone will but particularly in larger companies this is a very common thing doing in a JavaScript good luck reading it is a lot of code and with a lot of code comes a lot of possible bugs comes more tests because you're not really because you're not using a library and and there's just a lot of code it's it's it's really hard I pity the person who has to maintain this even if it was just me I pity myself so so it's just too much code but as an epic I can actually fit all this code and you can read it you may not understand it which is fine but you can read all of this code here and it's doing the exact same thing we're doing all while multiplexing where we trying when when when goes online or offline we're waiting a second and then retrying it when it goes back online there's cancellation and you can do multiplexing and all sorts of stuff and every time the stock taker comes back we dispatched that action all of this encapsulated in a very nice declarative fashion if this is all voodoo it totally is if you don't know rxjs but the point was just to show that you can make things that are much more declarative once you learn rxjs so let's let's summarize some things real quick here so it's our redux observables going to make it much easier to compose and control complex async tasks and I want to stress complex sure learning rxjs just to make request/response Ajax calls it's probably overkill maybe not but probably so we're not necessarily advocating for that if you know rxjs or just want to learn it it is awesome for that but probably overkill and if you are in rxjs user you'll be happy to know that we made it though the framework manages your subscriptions you don't so if you're doing idiomatic rxjs you'll never use subscribe or unsubscribe in any of your epics we will manage the lifecycle for you and you can use Redux tooling so because you're dealing with actions all those actions show up in redux and you can do time travel and you have to be careful when you do that because you're dealing with side effects so you don't want to accidentally be sending Ajax calls that you don't want and stuff like that but it all keeps it theoretically possible and and a nice in practice now anytime anyone introduces anything or pitches something there's always a but it's always a but even if they don't tell you about it there's a but right and that's what I was alluding to earlier is that you should probably know redux and rxjs before you even attempt to use redux observable now if you're if they're if this is just a spare time thing and you you're you really like to challenge yourself if you're like me you likes to like get way over my head and dig your way out by all means do it I don't want to discourage you I just don't want a bunch of people feeling like I'm sitting here saying everyone in this room should use your ducts observable don't matter how small or how simple your app is but if you feel like it the problems I described today are the type of problems you solve a lot and you're willing to learn rxjs and redux it's probably a good thing to look into and on the motive of rxjs or ICS does I will admit that it has a bit of a learning curve and that's because of something called reactive programming which is the new buzzword of what rxjs is really doing it's a paradigm that's it's kind of hard to grok your brain around but you're declaring what you're going like the transformations in which you should do but nothing's happening right then and there you're there that you're epic for example is set up and you've you've basically installed all the pipes there's no water running through those pipes yet until until a future point and then when someone dispatches in action then the trickle of water goes through and then you there that that infrastructure you set up gets processed so it's a different type of programming style called reactive programming and so now a good time to to introduce he's not going to come up a hunk stay down let my co-author of this which is Ben Lesh who's incredible guy and just happens to be also the the project lead of rxjs so the latest rxjs version 5 was rewritten from the ground up here at Netflix and he led that effort and so if you have really tough questions or complaints direct them to him he's here so thank you Ben for all the work you've been done you've done and for all of this inspiration so I really appreciate it if this is interesting to you you can do a pinko take a look at redux observable on our website a little shout out here these are some of the companies that are actually that we know of at least that are using redux observable already it's brand-new and people are already adopting it really mainly being used right now for the really complicated problems slack they're using it for their electron app that they just recently are shipping to to all of their customers of Facebook's open source project nuclide which is an IDE on top of atom also heavily heavily uses Redux observable and they've been thoroughly happy with it free code can and wrangle IO and a bunch of others so and a lot of these guys have actually contributed back as well so it's been a really great experience so if anyone has any questions I don't believe we're going to open it up but I'll be here afterwards absolutely feel free to come bug me or book Ben Lesh so but that's all I have for today thank you guys very much so much say that was super awesome

Channel: Netflix Engineering

Views: 351,186

Rating: 4.9571733 out of 5

Keywords: javascript, netflix, react, rx, rxjs

Id: AslncyG8whg

Length: 37min 14sec (2234 seconds)

Published: Thu Sep 15 2016