Functional Reactive Programming with RxJava • Ben Christensen • GOTO 2013

so I plan on talking about functions and reactive programming today but never using the word monad so the intent will be to stick to the practice of functional functionally styled programming and reactive programming or asynchronous and so basically by the end of this I want you to be able to understand the approach that we've taken to compose functions in a reactive manner and what this means is that you can take a flow of data transform it from one to another as it's a synchronously flowing through a system but more interestingly that you can start to combine that together and again do that all in a reactive manner and there's a whole lot of debate as to what we should be calling this we started using functional reactive to name it before any of us really had any clue about the academia behind it and I'm fully aware of FRP and Connell Elliott and Fran and those things if anyone can give me a better name for what we're doing here today I'd love to hear it but the functional aspects of it are you're going to see a lot of lambdas closures the functions are going to be mostly pure the only part where they are not pure is typically the very end very similar to the i/o monad in Haskell and you're going to see them as composable and the reactive side what that's referring to is that it's all async that you've got values and events are being pushed through the system and you're reacting to it as they flow through when we started working on this we wanted to target a polyglot environment and we targeted the JVM as a whole not any one particular language at Netflix it's primarily still the Java language with quite a bit of groovy and a fair amount of Scala there are bits and pieces of closure showing up and we're looking forward to what Java eight will bring as well it's going to significantly change how that language is used particularly as we've adopted functional principles most of the examples I'm going to give today are shown in groovy with a few and Java eight but there are plenty of examples of these we're and the Scalla side has really in the last six weeks is has dramatically improved as the typesafe community has gotten involved in working on the Scylla adapters to make them very idiomatic for Scalla itself so what is rx Java they it is a library that we implemented as a port not a direct port but we're trying to be quite faithful to the RX net version of rx that was built at Microsoft and there's also a JavaScript implementation rxjs and so you're starting to see the RX and then a language name show up in various languages PHP is getting one there's one for cocoa they didn't stick to the convention though they culled there's reactive cocoa and what it is is it's basically the principles of an asynchronous observable with a lot of high order functions on top of it and so I'm going to start to go through that before we get there and want to give you context is to why Netflix ended up going down this path so Netflix is a TV and movie company and we are currently in 50-plus countries and territories with over 37 million customers in North America we have a pretty big impact on the last mile downstream internet consumption and over a billion hours of video are consumed every month the API growth and this is the team I work on has looked like that over the last few years and the API is you can think of it as just it like any other API it's the facade on top of the service-oriented architecture internally to Netflix which is lots of machines and that facade is basically there to provide access to all the functionality to the thousand plus different devices that can access us and Rx was discovered when we as we were going up that growth curve and as the number of devices accessing us increased we start to run into the limits of what our restful api was doing we had a typical restful api it was as typical as you can expect of a Java stack it was using Russell it and the annotations and all imperative and it was very fine-grained API calls and the devices each interacted with them but what this meant is that it was optimized for nobody it was generic and you had devices that would have to make 12 Network calls to render one user experience and the insanity of arguments that we were starting to pass into these web service calls to try and customize the data that was being delivered for different user experiences was just getting out of hand and so we wanted to be able to take this kind of experience and reduce it down to a single network call for a single user experience and push all the the the network calls from the win to the LAN inside our data center which happens to be all in Amazon AWS services and leverage the low latency of the network communication of that layer reduce all the replication of effort on every network authenticating the user fetching metadata those things and be able to leverage the far better concurrency characteristics of servers over the small devices that we support an iPhone or an iPad or something like that is an incredibly powerful device compared with the broad scope of the devices that we target $40 silicon sitting on a set-top box is a far different experience in a laptop or a modern phone and so this nested conditional logic all had to be done concurrently parallel execution otherwise we were going to take a step backwards from where we were before where at least we had the implicit concurrency that was happening through all the multiple Network calls most of the HTTP clients from devices could at least you know sustain like for networking calls in parallel and we needed at least to be that good but we wanted also to allow anybody to create these endpoints not just the API team because if it was just that one team we would be a bottleneck for creating these customized endpoints so we wanted to allow each of the different device teams to build their own web services we want to be polyglot to support the different languages that teams might want to use and we wanted to solve the concurrency problem without everyone having to read this book over and over again I love this book it's a great book I have read it multiple times but the point was that we didn't want that to be the prerequisite for people being able to successfully build highly concurrent systems at Netflix and so what this meant for us is that the owner of the API had to retain control of the concurrency behavior whoever was exposing that facade we had to be able to maintain control of was this making blocking or non blocking Network calls was it fetching something from memory and synchronously fire hosing it with a you know microsecond lookup or was this something who's going to take tens or hundreds of milliseconds so we're going to have it in a thread or a non-blocking i/o event we had to retain control of those things and what this meant for us is that our API if we had a method signature like that all of a sudden this was baking into the contract of our service layer a lot of decisions that impeded our ability to make change and also for the clients consuming the data so what if at one point that was either a very fast Network call that we pretended to trust or it was something we were fetching a memory that had to change all of a sudden we have no choice but to break the signature to make that return something asynchronously also if the client actually wants to be able to kick off two of these things concurrently because they know that they should be what are we going to make them do do we really want them making decisions like spawning threads and the answer to that was very much No and so very quickly we went down the path of taking a look at well obviously we could start to put callbacks in we could return things as futures and if we have lists of things do have a future with a list of futures and it starts to get messy but this is where we were starting to to be and luckily we have a very diverse mixture of engineers and programming languages and backgrounds and Netflix and so there was there were folks there that as I interacted with them I was able to be taught about the the concepts of rx that Eric Meyer had invented at Microsoft through an engineer the been at Microsoft and come over to Netflix and took him a while to get through my thick java skull and get me to think like this and also 15 years of imperative programming I was not yet thinking that that inversion of thinking to move into the functional world it took me a little bit of time but over the the month or two during that discovery stage the the concepts of rx start to become clearer as a very pure theory around how to do asynchronous programming and it ends up being a very simple enhancement to the Gang of Four observer pattern to move it from an imperative synchronous world into a declarative asynchronous world and so it ends up being a dual of iterable so if you ever hear Eric Meyer talk about it he'll talk about the duality and he can walk you through all the category theory and math behind this I will skip over all that because I honestly don't even understand it all yet but everywhere that you would have an iterable that is push it up where you pull data from it you can now have an observable and push data down and you can work with it in the same way the two additions to it now are so next on an interval is on next with an observable and then because it's all async you now need two new things to tell it the terminal state so the terminal state is that they're going to be all completed successfully so that's telling you don't expect to receive any new events being emitted or an on error event which is the terminal state it says you're not going to receive any new events but we ended with this exception it's actually a throwable and so if you look at an iterable with higher-order functions applied to it such as basically all of the jvm languages that have been built in the last few years have and then recently with Java eight stream interface you can build up higher-order functions on top of an iterable so that as you are a tu declaratively can tell it how to process the data well you can do the exact same thing with the observable except it's now all asynchronous and so it's all non-blocking and the only difference between them is that instead of a for each at the end which by definition is a blocking event you subscribe to it you can also do a for each unobservable if you want to break out into blocking but this duality between them's started to lead us towards the decision as to what abstraction do we want to use over our data in our service layer and so we have this quadrant of different choices of how to deal with our data so the first and simplest one that we have all been using since the first time we touched a line of code is synchronous scalar values and you fetch it and when you're doing some conditional logic on it you get that value and you perform the logic on it imperative ly and synchronously when you have multiple values you do similarly except you iterate over an iterator iterable in a loop and then you conditionally do your logic within it and I'm showing the conditional peace because it's important in the next steps we move into wanting to behave asynchronously and futures or promises they they've existed for quite a long time and in job we've got the futures and at first glance they seem like they're a pretty good solution but they they're flawed in lots of fun ways once you start trying to do nested conditional logic the first thing any Java developer does with the future is they block on it they get the future back and then as soon as they want to do something with it they they dereference the value within it and then the system blocks and this is actually hard to avoid if you're doing any sort of nested conditional logic it's okay in the same way that callbacks are quite simple if you only have one level if you don't need to dereference it until the final output but if you need to make do nested logic at some point you need to immediately block on it and to not block you end up wrapping threads around threads and it becomes really sad really fast there are better solutions though the guava library gives us futures with callback so listenable futures and so this starts to solve that problem when we don't block instead we ace we have a callback and then within that we have callback handlers where we can asynchronously do that logic on it completable future in java 8 it gives this just with new names to everything they apparently didn't want to use any of the conventions from the industry and so they have different names for everything but perform them in the exact same way and so that you can apply a callback to a function to afterwards closure and sorry not closure Scala and akka futures are very elegant the composable futures that they have actually get is very very close to where we want to be if I had not discovered rx we would probably be using these futures and this gives a very elegant future where we can then map over the response and transform it do whatever nested logic we're going to do and it gives us the exact composition and transformation capabilities that we want on scalar values however if we take that future and just change over to an observable we get all the exact same behavior but we can now work on vectors as well as scalar responses and so what we found is that the best of the implementations of the future and the observable give us similar abstractions but the observable was one step further in the abstraction and so to simplify our environment we have chosen to adopt observable across the board for dealing with asynchronous programming because it allows us to achieve these higher-order functions for transformation and composition on both vectors and scalar responses and so what this meant for us at a practical implementation level is it instead of blocking API is every other service layer we want around and change them all to observable API s and so what this means for us is that this abstraction now frees up the implementation to say perhaps I want to just synchronously I know I've been talking asynchronously all along with the same interface I could synchronously return a value but the developer who's consuming this consumes it asynchronously but maybe I've got that sitting in cache already and so I'm going to submit it immediately without the overhead of a thread or I could back it with a separate thread from a thread pool or I can have multiple threads from a thread pull each omitting different values as they're processed or I could use an actor or multiple actors to do that work or I could do an niño call with an event loop or I could be couple a floor and actor with an event loop if I didn't want my callbacks happening on the worker threads and all of these abstractions all these implementation details are abstracted away from how the data is consumed and so the client code treats all the interactions asynchronously but the owner of that that API is free to change the implementation however they wish and they're also even in control of whether it's blocking or non blocking so this the abstraction appears as non-blocking asynchronous and then the implementation detail gets to choose what resources it is using so I'm going to start getting it to code now and to walk you through code examples and some of the more common higher-order functions that apply to this and so that thing at the top we call the marble diagrams I'm going to use a lot of those in the rest of the presentation to try and give a visual representation of how the data is flowing so if you've never done functional programming or reactive programming the hope is that you can start to visualize the data flowing through and if you start to think of the data as literally like pipes of of data the flowing together and then these higher-order functions are like the connectors that you have with the pipes and that visual image can start to help you recognize basically what's going on with the data as it flows through the system and so I'm going to start with the the basics of how you create and subscribe to the observables so an observable is really just a function that can accept an observer and when it receives that observer then it will emit via on next alright once you've got that observer and it is subscribe to you can emit 0 or more events to it via the on next event and then you have a terminal state of uncompleted of success or a non error terminal state if there's a failure and so here are some examples these are purposefully verbose in showing where the concurrency comes from normally there are much better ways of introducing the concurrency but I'm doing it here to be explicit in demonstrating that it's the up to the observable itself as to where the source of concurrency or asynchronous execution comes from and so in this case the actual interesting part is just right there and what it does is with this observable is a function that once subscribed to will fetch one video rating object data object over a network call and then emit it and then complete that observable sequence and you'll notice that in this case I am explicitly doing it within a Java executor that executes sit on some background thread and typically in our production code we don't do that so again just want to be clear that that is just to demonstrate that it's fully the choice of the observable so what's going on here's a synchronous example just showing that the observable can be blocking and synchronous if it wants to be and so this one just firehoses data back out and the subscription will in fact block I don't recommend doing this typically but if you have something that's in a cache and it takes you microsecond to fetch it out and emit it generally actually better to do that then spawn out a threader or something of that nature so in this case we're actually returning multiple values and we emit them and then complete here's an asynchronous one with multiple values generally you probably aren't doing a network call in a loop like this but if you had some reason to do so you can typically you have some more efficient streaming approach where it's a single network calling or streaming the data off the line but the the whichever implementation it is it's on next thing the values they're received and then you complete and so these are very trivial examples of the source of the data the the observable self so the simplest observer now so we flip from the producer to the consumer side is the consumer has three possible events that it can receive typically you actually don't need the uncompleted at the very end it's kind of implicit typically and so generally it looks sorry I'm skipping just shortening it up to show what it looks like when you're not doing the full verbose implementation that's what it looks like when you're using lambdas typically you don't need the completed and the error handling this is error handling just at the very end I'm going to get into the asynchronous error handling a little bit more detail later that's one of the biggest challenges in async programming typically but this is the the basic observer that you would pass in and you don't always need to have the error handler if you don't want to worry about them but generally this would be where you're at but all we've done so far is just make a glorified pub/sub and nothing interesting at this point where it gets interesting is where we start to have composable functions and so some of those types of functions this is a list of some of the more common ones and I'm going to walk you through now some of the ones that are just critical to pretty well any application so I'm going to start with how you combine multiple streams of data using the merge operator so what the merge operator does is it allows you to take two observable sequences of the same type and merge them into one sequence one observable sequence that has all those all the events combined and so here we have two sequences get data a and B and they both return the same data objects some data and so these two lines across the top represent two sequences of data over time they're both emitting values at different points of time and the top one emits an error event is it's the third event there that X represents a non error call you want to take these two and pass them through a merge operator and that merge operators is this is how it looks and that merge operator out the other side what you want is a single observable sequence that combines that output and so you'll see that it combines the first four events and then the terminal state of on error and then one that terminal state occurs no further events will will arrive and then out the other side you have a single subscription that receives all those events merged together another common one is zip zip is used when you are combining multiple sequences of data multiple observables of different types so here we have two observables of two different data types some data and string so the difference here is that you cannot just combine these together and with an assumption of two their as their type and so if we have these circles and diamond or triangles upside down triangles and we want to combine them together we need to pass them through a function that then knows what to do about those and so that function here is where we pass in a and B as the sequences and then this this is groovy syntax here so the first two are the names of those parameters they come in x and y and then the other side of it all it's doing is it's returning a tuple of x and y and this is about the simplest thing you can do with a zip and so all we want to do is just combine them into a tuple and so we end up then emitting these tuples of these values combined together and then we can get out that that tuple and retrieve the two values from those pairs as we wish error handling so the simplest way is you declare your composition of all of the the sequences and at the very end when you subscribe to it you pass in your arrow handler this is equivalent to having a big try-catch around an entire block of imperative code it's almost like having the try-catch around like in your main method just around your entire application call and or like if you have a web service in your bought in the the portion we're doing the body the try caches around the entire thing now that can be very useful and what you what you want to catch anything that goes wrong and that exception it will wherever it occurs will just be passed through and then the whole sequence terminates however it's far more common that you want to be able to catch errors surgically where they're occurring in the sequence of events and so there are a variety of different operators you can apply to that and one of them is on-air resume next and it does what it says it says on error I want to resume with this other sequence of data so let's say that I had that a and B sequences and I want to add some kind of error handling to the B sequence and so whenever if B fails at any point in time instead of it causing the whole thing to fail I want to be able to handle that in some way so this would be equivalent to having like a get method in in normal imperative object-oriented if it fails they didn't return some other value and so in this case well how that B sequences is moving along it emits three values and then fails with a non error event when that error occurs it's going to trigger this on error resume next function and what that will do is it's the equivalent of in the in the catch block it would then say execute this other sequence of data I'm going to subscribe to it and then it's almost like it just switches the flows or like if you think of train tracks and had that lever that switches the tracks it moves the the subscription to another stream of data and allows you then to emit whatever the the backup or fallback data would be so in this case that backup is here and it's passed in is this other function get fallback for be for simplicity on the slide I don't have what that would be but that would be another observable function that you pass into it and so what that function does is it get subscribed to and then it emits two events successfully and then terminates now what might those be those can sometimes you could sometimes just means that you just want to just immediately terminate you could just uncomplete immediately and you just want to take let's say that you were expecting a list of 50 movies it's common thing in Netflix but you only got to like 37 of them and failed well we might just then resume next with an immediate completion and we will say we would rather the user interface gets 37 movies and misses those last 13 rather than blow up the entire user experience or you may have some tombstone object you passed down or you may have some legitimate legitimate fallback data to return on error return is a simpler version of this all it does is that when that error occurs you've passed it an actual value for it to just immediately return so this is much more common for just tombstoning or just immediately completing so I'm going to start to dig into some actual use cases and so just going over the operators so very simple use case is HDV call so I want to handle it asynchronously so in this case I want to fetch from what Wikipedia in this case and it's going to return back to me this is all just declarative at this point nothing's actually executing when this is done I create this observable out of it and I'm in the comments I'm going to just show what the static typing is I purposefully in some places and being a little bit more verbose just so you see that the types are but most of these things is including Java AAA so I'll just inferred throughout the chains this observable then that returns an observable HTTP response you can flatmap over and i'll get back a little bit more later to what that flatmap does you what this is basically saying is when I receive back this response I then want to do nested logic on it always synchronously and within that then I want to say I want to from that response I want to get the content so that response object at that point contains it's the way HTTP works is it returns bit back like the header information see the status code the headers and those things and then you can start consuming the rest of the the data and so this two-step response I'm able to validate the the status and those things and then I can get this response that contains an observable of a byte array and in this case I just want to then map over a transform it into a string if it was data you'd want to leave his byte array do what you wish with it and then it would be in this step that you could then do your JSON transformation etc when you subscribe to it is when it actually does the work and so everything in our X is lazy it does it's all just declarative until the point that a subscription occurs so it's the equivalent of you you connect all the pipes and then you turn on the water at the end and so once everything is connected then you turn on the flow and then everything happens at that point and what this would then emit is asynchronously it would just receive the response from the in this case Wikipedia now just for to allow breaking out of an async world sometimes you need to break out of it there is this ability to switch back into a blocking observable which then exposes a lot of blocking operators such as for each this is useful for if you're just doing a little demo apps or unit tests and those types of things now that one wasn't all that much more interesting than a typical async client this one's a little bit more interesting if I flip this from Wikipedia over to in this case it's a product it's another open source library we use it Netflix I for fault-tolerance but this particular thing it's a the stream of data the metrics that come out of each of our servers and so this uses the html5 server sent of that events back and when you connect to this HTTP connection it's it will stay alive as long as you hold it open and it just constantly emits events well now what this will do is because the observable support vectors of data this will now actually just start emitting every single event that comes down the wire it will emit as an observable event and so it will just keep calling on next so in this case I want to filter out the pings in the we send these pings down every like 15 seconds it's part of the network health stuff and so in this case you can apply an operator to filter those out and then let's say that I'm just out I just want to get a sampling of what's going on I could take 30 of them and then disconnect and this is an example of how the same interface now I've just used it for a scalar response where the Wikipedia page returns a single response back or I can consume from a stream of data and it will just keep emitting the the results as they come in I want to now go into a more complicated use case which is this is the use case that actually drove our adoption of functional reactive programming this is it's a simplified version of what we do we render the UI that has the grid of movies on it and that involves a lot of nested behavior and lots of different Network calls to render it and so the marble diagram is a little bit more complicated and the way it starts out is at the very top the first thing that we would do after you've gotten all your user credentials and all those things is we ask the video service for a list of movies there's actually one level above this still where you get the grid of movies first and then you start once you get back the grid then you go over each list for simplicity and just on a single list at this point and so this video service you fetch it and this is going to return an observable the next thing we do is let's say that we're on a smaller screen UI and we actually only want to take ten not the 50 or 75 or hundred that can exist on a row and so we want to take ten well what's happening when that operator triggers is that behind the scenes this is lazily emitting data and then that take operator once it hits the tenth item it's going to unsubscribe from the previous one and so the unsubscribe event will then propagate up the asynchronous sequence and tell the the parent in the chain to stop working and then this what if this is however many levels deep is this propagates up and so what this would mean is that if you were continually doing network activity behind the scenes you'd be able to stop that network activity and so it's only doing the work up to the point you actually need it lazily rather than eagerly fetching at all now that implementation may have fetched it all in batch it may have already fetched 100 of them in one Network call in that case all you're going to do is you're going to stop the in-memory iteration but the principle is that it always emits the it carries the unsubscribe all the way up to allow the resource cleanup and the the lazy execution of the work and then it's the implementations choice as to what work am i doing that actually needs to be cleaned up when I receive that data I then want to map over it to transform it so what map does is it transforms type T to type R so it lets me do whatever I want within there to manipulate my data however we're going to do something condition nested and so we're going to use flat map instead it's also known as map many or in the dotnet world select many we've adopted flat map as a naming convention on the JVM since that's more common in the Java world and actually is more descriptive of us to what it's doing so flat map is basically what it is is it's the same thing as map except that you're returning an observable of R instead of R and so you could think of it as a map returning sequences and then flat map is just merging them together so it's the equivalent of doing a map and then emerge and so you end up with if it's emitting these sequences it just merges them into one sequence otherwise you would get a sequence of sequences which sometimes you want but generally or not and so what this what we're going to do in this flat map is that we're now going to receive 10 calls of this for each of the video objects are going to be emitted we can receive 10 video objects for each of those 10 we want to then do some nested logic and so the first one we want to do is we want to go get some metadata for that video and so we we go off and fetch that and we also want to fetch bookmarks and ratings there's other ones we fetch but these are three very common ones and each of these three things are going to be done asynchronously they may or may not result in Network calls and all of these are returning observables so they're all async and then we want to on them transform them into some data that is ready to be delivered to our client and so again we map over it and in this case we're transforming it into just as very simple dictionary of key value pairs and we want to do that for each of them and so inside that map many or flatmap we will then conditional sorry we will then in a nested manner fetch the metadata and map over and do the same thing for the bookmarks and ratings well squish all this up on the screen and so we've got the the metadata bookmark and rating what we've just defined there are three observable sequences all we've done is declare them we've declared these sequences with map transforms and if we were to execute the code like this right now they would never actually get executed because they're declared but never actually subscribe to and so they because it's all lazy I would run this and I would actually omit nothing it would be like my application did nothing I would do the initial Network call to get the videos it would emit the videos into this and then it would just kind of like return empty data so I need to subscribe to them in a way that combines them together and so I want to zip these together and you you could do a there's other there's variants that you could do like a merge with a list and things like that but it works very well in this case and so I want to zip the mbnr because I want to get them together in one place and the function that I apply to it is going to take those three and sorry and it's going to combine them together into a single dictionary and so again what zip is doing is it's taking those input values and applying a function and I'm going to I could return them as a tuple but what I actually want to do here is I want to transform them into a dictionary this is groovy syntax for the first one is the map literal so I'm creating a dictionary that just contains the key ID and then the video ID and then the the three off to the side of it are just being concatenated into it so it's very very concise representation of concatenating three dictionaries into a newly created one so what I get out at the bottom of this now is I'm going to be returning an observable map or dictionary and that zip operator is the last piece of this declaration as to how I want to consume a list of videos transform it manipulate it do nested logic to actually go and fetch other do other Network calls and out the bottom I just want to emit a dictionary of data for each of my videos and what I get out of that is for each of those ten videos I now will get a dictionary that looks like that where I have key values combined from the three nested Network calls and the parent Network called the V ID obviously in the real in real life there this is a much bigger dictionary and what this what this all shows is that all the interactions with the the API they were treated as if they are async and it was all done in a lazy declarative manner and all the details as to what's happening behind the scenes are controlled by the API and so in in our production environment that whole process would have resulted in three Network calls even though it could be done in efficiently with 3131 Network calls if we were to do it in the naive way and so the metadata is always pulled from in-memory cache because that's so active we've got that all cached the bookmarks and ratings we collapse those into single batch calls behind the scenes and the video list is all pulled in in batch as well but those implementation details are completely up to us to choose how to do and we can change that without ever affecting that what the API looks like because it's all done in a declarative asynchronous manner so as we look back on our on our architecture we feel that we have successfully achieved what we aim for what what the reality of our structure of our architecture is now is that the various UI teams actually build their own endpoints their web services and deploy them on top of us in production the API team at Netflix has gotten out of business of building web services and we now just build and maintain the platform upon which all the various teams within Netflix build their own web services on top of us and they're able to deploy them dynamically in prod it it takes about 30 seconds from the time you deploy it for to propagate across our fleet of hundreds of servers and they can Rev through and and do whatever they wish with it and all of that code at that layer is all done using rx functionally styled reactive programming underneath we use hystrix for all of our back-end bulk heading of our underlying systems and we liked the reactive model so much that we have recently retrofitted hysterics to be non-blocking top to bottom and exposed observable mechanisms for interacting with it and over time we're seeing the the approach of this reactive style pushing deeper and deeper into our stack so that we're getting closer and closer to the point now that when an incoming request comes in the Declaration of all of the work actually then connects into a next level down of all observable sequences and top to bottom to the from incoming Network call to the underlying network call out the other side and then back is one single reactive declaration and so we feel pretty good about how the observable api's have worked and I think that the most interesting part about this some of the lessons learned is the the developer training was a pretty big key for us in adopting this so when I was involved in the original design and architecture of this of this new approach for our API platform and it took me a while to to grok the the ideas and it took a lot of shedding of biases and unlearning of approaches that were just the status quo and once a core group of us would figure that out and iterated enough it took us a couple months to really feel convinced that we had it the next level of it of working with developers was the next hard part of taking developers who have 10 15 20 years of entrenched just thinking on how to go about implementing a code like this and so what we found is that the first couple of developers took a month four to six to eight weeks for them to really get up to speed with really thinking like this and that was the first two month like alpha stage of it after that we found it took about two weeks for any given developer to get over that hurdle of grumbling and mumbling and complaining and really wondering why on earth they were being asked to do functional style programming - coming out the other side going I get it I do not want to go back to the other way and to facilitate that we we actually hired a technical writer to come in and write documentation for it and it was one of the best decisions we made it some of the best internal documentation I've ever seen in a company because we recognize that this shouldn't just be a few engineers just trying to like throw some stuff in a wiki but we actually brought in someone to actually help write that up and that was a big deal and then we also do just internal training for new developers coming on board and this is not just a group of like 10 or 15 people because this is all the different UI teams who want to support their devices and so this is JavaScript engineers and c-sharp engineers and Java engineers closure Scala groovy all these different people from different backgrounds who are all potentially building web services for their devices that we're onboarding onto this the next thing is that debugging and tracing of async is definitely different setting a breakpoint it's just different in async code what some of the nice things that rx does though is sit in production you can't do this is it nearly as easily but when your are in your IDE rx supports the idea of virtual schedulers or test schedulers what removes all the concurrency and you can increment time artificially and so it makes unit testing actually very effective and so that you can have deterministic approaches to thinking through the sequences of data and incrementing time as you expect and then performing the assertions there's definitely still work to go in this space and we're looking at having a console similar to what Chrome has for viewing asynchronous events over time but being able to run that against our servers and that stuff that we're exploring today the only rule that we've had to really just enforce on people it ends up self-enforcing eventually once you've stubbed your toe enough times but is because we are on the JVM and the most of the languages were using they don't enforce this themselves is to not mutate state outside of you're generally don't mutate state but if you are going to mutate it mutate it within your function and that's it if you want to do some sort of imperative thing inside we're fine but within the barriers of a function as long as people stick to that all the concurrency and parallelism just takes care of itself and this actually fits well with some of the other earlier presentations today about mutable state being a very bad thing when you start getting into these highly concurrent systems and so in a nutshell this combination of functional style and reactive asynchronous processing has proven to be in a very effective approach for us in handling large scale high-volume systems where we have a lot of nested conditional logic and it's very network driven and there's a lot more that's been written about this and the open source projects that are available we've had we've got about we've got over 30 contributors so far to our X Java rxjs I'm not directly involved in but we use that a lot at Netflix as well and love to have your involvement on the project if you are interested in this stuff or just so you can go and just learn more about the approach even if you don't use this particular library there's lots of different approaches to functional and reactive styles and if nothing else I would recommend you learning the the ideas because I think you'll find that it really it changes your coding habits for the better thank you the short answer has been that that actually has not been as big an issue as what we first thought you unit tests in the same way as any other code at the end of the day it's just code and it actually ends up being simpler for the same reasons that earlier presentations by boat'll and others where once you start removing all the mutable state and have these these sequences of transformations it actually becomes very easy to read through and understand and then it just becomes normal coding practices if you have just a big ball of code that is like 20 transformations all in one then it makes a little harder to test if you use good coding practices and break these into functions and you test each function independently and do normal good coding it it's the same concepts the only real thing is that if you are using operators that by definition needs or some sort of concurrency like windowing or buffering those start to involve time elements and then that's where the ability to pass in all of those decisions are abstracted to schedulers in our X and so you can in your unit test used test schedulers which then allow you to remove all the actual concurrency and then you can increment or move the time to where you want assert at that point and it allows you to reason about time without all the non determinism of thread schedulers a little bit more about parallelism and obsesses parallelism and observables so by by the each observable sequence the contract is that it will not enter leave any of the on next calls and so it that's the contract that it gives you but then it allows you then to if you merge for example multiple observables it will subscribe to them all in parallel there's also like parallel operators very similar to like the parallel stream in Java eight or parallel pair map and every language has some variation of it so you can tell it explicitly that it's okay to go parallel within an observable sequence and the schedulers allow you to access thread pools or ever you may choose so the Deaf by definition because you're using a functional style without mutability you're able to be very free about about executing things in parallel and using the full capacity of the number of cores in your machine by default if you just never even think about it though it will just generally do the right thing on your behalf and it will just use however many cores around the Machine correctly can we use this on Android yes it is coded to Java six standards which work on so it we're keeping at that level specifically so it works on Android obviously you don't get the niceties of lambdas but there are some pretty big names such as SoundCloud who are using this in production on their app on that on Android and they've written some really great articles about it and done some presentations on it if you wanted to go just search for search for SoundCloud rx Java and I know there's some others at some other fairly large companies I don't think they're in production yet but they're kind of following the lead of what SoundCloud is done on Android okay I don't understand this question but how would you create a bracket bird a bracket bird yep good question you got me you come talk to me after I think we'll we'll bring the questions to a close and that's been again you

Channel: GOTO Conferences

Views: 51,874

Rating: 4.9361277 out of 5

Keywords: Software Develop, Software Development (Industry), Programming, Programming Language (Literary Genre), RxJava, Web, Services, functional, reactive, programming, Conference, Presentation (Software Genre), Great, Talk, use case, Netflix, Netflix (Organization), GOTO

Id: _t06LRX0DV0

Length: 49min 27sec (2967 seconds)

Published: Tue Apr 08 2014

Reddit Comments

[u/rostislav_c]

Good old days in 2013

[u/Exallium]

"Functional Reactive Programming" is a misnomer with respect to what everyone here is actually thinking of :)

Just "Reactive Programming"

Functional Reactive Programming is a continuous time based stream programming from FP, whereas Rx is discrete.