Moving from Imperative to Reactive by Paul Harris @ Spring I/O 2019

[Music] okay as mentioned my name is Paul Harris I'm the lead developer on Cloud Foundry Java client not as was tweeted earlier the whole of Cloud Foundry that would be a bit generous so what is this talk it's an introduction to reactive programming now I want to stress that from the start this is an introduction so if you're already pretty up on reactive and you know how it works and your bat flops and mono and all the rest of it then you don't have to stay it's okay to walk we're going to have some background about where reactive comes from that's going to be pretty short and after this morning's keynote it's going to be even shorter we're going to look at a practical example coded live and I've just realized I've never coded standing up before so that'll be interesting and just to stress again this is for your benefit so if this is too simple for you please leave if it's too complicated for you I guess please please and leave and see me later so why am i presenting this I've been using project reactor the rebooted version that was mentioned earlier for about three years over three years maybe we started on the Cloud Foundry Java client at the same time pretty much as spring boot with spring sorry project reactor was being rebooted which means we were its first customer we had a huge input I think into how it worked out the various real-world decisions that were made which I think made it better and certainly made the Java client much better because I started three years ago I didn't have any background in reactive it wasn't you know that much of a thing back then so I would guess I have made all of the mistakes it is possible to make my programming reactively and I may well show a few of them to you today final thing to say is that I'm a normal developer you've heard a lot about the rockstars I'm just like you guys I picked up this thing because I needed to pick it up for work and here's some of what I've learned so to clarify I'm not a Java ninja and not a java wizard and not to drive a Jedi or a Java blue the bear for that matter I am none of these things just just one of you okay so this all started out in 2013 with the reactive manifesto which was an attempt to codify some of the thinking that was going on at the time about how modern applications should behave the manifesto came up with four main ideas four concepts applications should be responsive so that doesn't mean that they should be as quick as they possibly can be but they should always feel like they are making progress so if you have to trade off a little speed for a little bit of feedback or you know something else like that then that's okay it should be resilient so if you have an error in a particular part of your application or an application with a larger system that shouldn't take the whole thing down you should be able to cope with that as best you can it should be elastic which is I've always thought of that as being kind of scalability really but if there's more to it than that it's about making the best use of the resources you have available so the one of the keynotes this morning Ben was talking about the having multiple threads running and being able to with existing models you tend to be CPU bound because you've got all these threads all this memory or this contention going on and you're not actually using the CPU that you've got available to you so elastic is about making better use of things like that fooling various connections all this sort of stuff and finally it should be message driven rather than event-driven so manifestos are great but they don't compile I've got a message up here tell me I've got 11 minutes of speaking time left which is a little concerning but I'll try and go faster than that and so the next step in this adventure was reactive streams which defined a set of interfaces for how we might deal with reactive streaming situations they defined four main interfaces you have a publisher which is something that emits things that could normally be data so it can be strings or whatever it might be but also you can be emitting signals to say this this signal is complete or whatever it might be you have a subscriber that's the thing that listens to the signals that are coming in if you put those two together then you have a subscription so a publisher communicating with a subscriber is a subscription the key point to make there is that a publisher is setting up the potential to publish information but until you get a subscriber attached to it then it's not necessarily publishing that information well the way we generally work with imperative code is you write a program you run it and it starts doing stuff and generally speaking with reactive programs it just gets ready to do stuff and wait for this signal that it should actually consume the results and the final type that they defined was a processor which is a combination of a publisher and a subscriber that allows you to process data okay that's the the basic interface definitions but the intention was from reactive streams that that would move on to a more useful real world implementations of that basic specification and one of those is project reactor so it's turning reactive streams into a framework you can actually use and there has all sorts of convenience methods in there that will be helpful to you I took a look back the first commit was in 10th of November 2015 I mentioned that to illustrate that this is a major project that's been going on for three and a half years now for an obvious Matt so this is not a casual undertaking it's something this brings taken very seriously and I think that's reflected in the quality of the reactive frameworks that are now available to you through spring there are many elements a couple of new ones one I hadn't even heard of before we mentioned in the keynote this morning but probably the the three big ones a main interest is reactor core so that's the the basic provision reactor nettie which is the implementation of Nathu suited to project reactor that if you're coming from the web MVC world that replaces Tomcat as the server container and the final one is react test which is a bunch of really good useful methods for and testing reactive streams which is a tricky thing sometimes so before we get into the code there's a couple of key concepts to understand we I mentioned earlier about publishers which is this reactive streams idea and in project reactor they decided to have two specialized publishers so you have a sort of a general one which is a flux and we've seen those mentioned already today as well and the flux is essentially the same as a publisher with many methods added on and so forth it will emit 0 to n elements it's unbounded so it will potentially keep emitting until the heat death of the universe I guess or until it for an active data that needs to emit there's this more specialized type which is a mono which is a publisher of 0 or 1 elements so the reason this was done was mainly as a sort of a real-world decision that not everything is a flux you know sometimes that you're making a call and you're expecting to get back a single value or potentially no value at all so for code optimizations and for being able to work in the real world with this tool it can be handy to narrow things down and say here's a mono so here's are the methods that are specific to this particular thing and I found using it that it also helps me to get into the my right mindset that if I know I'm dealing with Manos then then I'm thinking in that way of ok I need to think about one value coming back and I need to handle the fact if nothing comes back and that's a separate case so it's an both room code and from a philosophical perspective I guess it's a quite a useful distinction so we're going to look at both of those in a moment so here is our simple application we're going to use for the demo try and make that much bigger so it's a spring boot application as you can see it doesn't particularly have to be it's just a convenience I'm sure I don't need to sell all you folks about the the benefits of using spring boot I have a repository of employees I'm using JPA repository I can see if I have to do this again I'm already gonna have to convert that to spring data or to DBC we have an object of employee very simple just named pension ID and role you can see I've used the app data annotation there that's a Lombok thing just to save me having to generate getters and setters and so on again not required for this so that's all the details around it and now for the actual code so we have a rest controller just like that and we have a number of functions in here now I should point out this is a demo app it's not terribly well structured I just wanted to get everything into one class so if that's making you feel sad then just think of it as legacy code and we'll fix it later we have another number of functions in here we can do a mapping of to employees where we say give me all of the employees and it just spits out a list we can search for an individual employee and give it an ID we get back an employee or a message saying that that employee couldn't be found we can search for problem records where we go through and look at all of the employees and the database and decide if there's something wrong with their day to say they've got a missing data entry or spelling mistake or something like that we can create a new employee this is interesting because we have this pension lookup that will call out to a service and give us a pension ID based on that person's name so we've got a little bit of communication there and finally we have scuse me update pension records so rather than where with the new employee we go and get a specific record for that employee we can update all of the employees pension records so we call out multiple times to this service to get the IDS that we need and then down at the bottom we've got a couple of convenience methods so we need to take this imperative code and make it something that's reactive and step one because we're at a spring conference is we open our pump and at the moment we're using web MVC to both ride all that get mapping and goodness and all the rest we're going to get rid of that and we're going to add a dependency to spring boot start to web flux so whereas before you are all used to having a web MVC we're now using web blocks which is essentially the same thing clearly there will be differences because it's dealing in a different term a different model of programming if you like but it is the the non-blocking asynchronous the reactive approach to doing whatever the web MVC does something that I've seen highlighted a couple of times is to point out that web hooks doesn't replace web MVC where BAM they see is not going anywhere this is just if you're doing a reactive style then this is the thing that you should be using so we've changed some dependencies we'll start running and if we take a look at the console we can see down here that we've started nettie on port 8081 BV ously it would have been running on Tomcat and if we go over to my little test suite I can run the various queries that we were earning that we have available to us and we can see the data comes back from from them all so I can get users I can create myself as a user and I get back my pension ID that's great I can search for problems and I'll get a list of all the users who've got issues with them like a missing role for example I can do a search for a user that I know exists and I get back that the users details I can search for user that doesn't exist and I get back a 404 because that user wasn't found and finally I can do a pension clean which connects to a slow connection I know it's slow because I put thread sleeps in there and I get back ok I get back the results showing me the updated pension information and if I take a look over here I'd got a Asus out in there that tells me what thread I was using so I was using one fret and one thread for all of the calls I did to this low service nonetheless I am now running on Nettie I've enabled web flux so I guess that means that we're now reactive and we can all finish not quite it is impressive to me though that although this is quite a simple application we didn't have to change anything all of that still runs because all of the functionality that we're already using I'll find the class though all the functionality that we're using still exists so it's not necessarily a huge explosion if you decide to move over you can start doing it a little bit it's time does depend on if you've been using Tomcat specific or particular servlet features obviously but in general it tries to make the transition as easy as possible having said that we do actually need to start using things like flux and mono to find out what we do next so we'll start off with the simplest query that we've got which is just to list all employees and the moments it's got a list of employee and we just called the repository and it just dumps everything back to us in the form of a list and then web MVC converts that into something that we can display on a web page or in my rest client so the first thing we need to do is say well we're not dealing with lists anymore we're dealing with flux and immediately that gives us an error because this return repository findall is still doing things with a list so we need to work out some way of converting that into a flux handily something that's not particularly in the reactive stream respect because it's that's a very high level document but is added in project reactor an example of one of the things that it will give you for free we have a method that we can call to do that so we start with a flux and we choose from iterable and because repository final find all returns less that's an interval nice and easy so we can put that in there get rid of that make sure we're returning it now when I restart I should explain I've been working in go for the last few months so if I miss out some semicolons don't judge me too harshly clearly there should be a semicolon in there just do it for me if I then might get recall again get caught then I get the same list that I had before so those of you who are thinking back on what I was saying earlier will have noticed that we've got a publisher but it doesn't appear based on the code that we have a subscriber in much the same way that web MVC takes this basic gap mapping and turns it into something that we can provide to a web page or to the cocoa client the web phlox is doing the same thing it is the subscriber so whenever it receives a call it initiates that subscription and says give me the data that you got for this endpoint that's fantastic because keeping control of subscribers can get a little complicated and that's one of the things you can look forward to in your future if you using reactive project react to makes it as easy as it can but there are still some tricks around knowing what thread you're on I'm making sure that you've closed down things okay so we've done that and I have notes as well let me just make sure I've not forgotten anything on there so we started that by beginning our entire flow as short as it was with the flux now we're gonna look at a similar case but we're gonna start with a mono instead so we're gonna look at the find and employee by ID in a moment you can see finds by ID throws an exception if there's a problem I had a 404 so we might want to do something similar to what we had before and we could try something like this we say mono just so I just want you to give me a mono and I want you create that mono from posit REE I'm gonna give it an ID but it's a problem with that and the problem is that find BIID refer to returns an optional because it doesn't know if it's going to find anything or not and handily we have a method that will handle that situation which is just or empty and what that says is return the mono that you get back from whatever call this is but if you don't get anything back then just return an empty so the key thing here is that we're always dealing with Manos so we need Manos to be passed around but so if we don't get something back we don't want to have an error from it particularly we just want to say here's what I got back which was nothing in a mono form and you can carry on we still also need to have this exception and we can do that in a slightly different way to the imperative style by saying okay we know we might get back a value and that's fine we can just pass on but if we don't get something back then we want to switch what we're providing by providing a new mono that we can use and we are going to do and I've done something terribly wrong so I should look at my notes to make sure I can fix it yeah when I said that I have made most of the mistakes that you can make him using reactor one of the first mistakes I think I made was to forget that we're not returning an employee anymore we're returning a mono of an employee so we now have the same functionality that we had before but instead of throwing an exception we are returning either the value that we got or we are passing on a mono of an exception this is an important thing about project reactor and reactive programming that errors are values just the same as a string or an employee or anything else as part of that resilience we mentioned earlier you don't necessarily want to panic just because you've found an error you want to pass that on to somebody perhaps several chains down the line that has the ability to deal with that error in a sensible way so for us here we're just going to say I've got this error and you should do with it what you want so if I run that and this is doing a search for a user so here's a user that exists and that's fine his user that doesn't exist so I created this mano of an error and passed it on and I've passed it on to Webb flux which is the thing that's doing the SUBSCRIBE and it's decided that when it sees errors like that it will pass them on to the users as a 404 and employee not found so just to emphasize their errors or not they are just values the same as anything else and you need to pass them on as you would a string or an employee or or whatever it might be the next one we can look at is find problem records and again we're going to make we're going to avoid the first mistake that I made earlier so we want to return a flux of employee and obviously that causes a problem because that's not what we're doing anymore so we need a flux and we already know of a method that will generate a flux for us and then we want to do something with this list that will get rid of all the employees who are fine you are good and we'll just leave the ones that have problems with them now in the imperative code that I was using earlier you can see that I'd created a couple of lists one of the result I initially got back from my repository and then a second one where I could put all the users that were problematic to pass back there are various ways of doing that the reason I've chosen this was because other some of the other ways of doing it can cause concurrency problems that if you're trying to update a list that you're also reading from then it things can start strip over so we want a way that is not going to cause that sort of problem using a reactive flow and ultimately what we want to do is to filter the results that we get from this initial flux so that we can discard anything that doesn't meet our criteria of having a problem with it and handily there is a filter so that filter takes I can't spell an employee and we'll do something with it to perform a test on it and it checks to see whether that employee has some problems important thing to note here is that once we get into the filter and we have this employee this took me a while to fully wrap my head around I've made that singular for a reason although we have a flux of all of the employees once it gets inside the filter then it's just dealing with a single employee and we'll have if you imagine as many filter instances it's not really an instance but that's the idea working on this flux each dealing with a particular employee as it needs to process all of the data and what happens is that each of those filter instances we'll drop one of the employees if it says yeah that's fine it passes my test the the or it we don't have a problem with this particular user so what's left behind after we've done that is just a flux that contains the users that do have a problem we've discarded all the others which means that this code is now complete it's actually surprised me when I first did this I thought I would have to have another line but we've got rid of everything that doesn't meet the criteria we want so what we're left with is a problem employees and that's still a flux because we've just filtered the flux we haven't changed it in some way it gets passed on to web flux and again it presents it to whoever's calling it one little thing that will probably be upsetting some of you IntelliJ PRS because I'm passing an employee to a method that only requires an employee we can do a simplification of the code like that okay so the next thing we can do is a new employee this one's a little trickier because we're making this call out the first thing we might think of doing is saying well I need a name that's fine I've got a name and I need the pension ID for that employee okay well I'll do new employee don't let me try this again I need the pension ID and I can do a pension look upon it with that name and then well I need that value so how do I get that value and this is where it kinda starts to go wrong because I'm not programming in a reactive style I've started in in an imperative style and what I actually want to do is start somewhere with either a mono or a flux to say this is the flow that we are doing so what I want is something here that will provide me with a mono I could just create one with a mono just as we've seen already but actually there's going to be some code here that will provide me with a mono if I start making some changes but the moment we're using a rest template which is perfectly fine reasonable web MVC choice but as we saw in the keynote actually it is a blocking call which means that we act on a single thread it's not really reactive so we need to change that and the way we change that is we will have the web client that was mentioned earlier and we would like wouldn't it be great if spring boot could provide us with one of those come on spell and then we want to webclient is that except coding standing up is a very weird art builder okay so we've got a basic one there's all sorts of configuration that we can do on it we want to base URL and I know that that is localhost 8080 too so that's all I need to create my web client spring boot in the background is doing all the magic for me to give it sensible options if I go down to my pension lock up clearly this is now out of date because it's using the rest template so instead I need the web client I'm going to do get the URI I'm going to call so that's an extension to the base URL is name I'm going to parameterize that then I'm going to retrieve the result and I happen to know that the result that comes back is just a string it's nothing fancy so I'm going to body tamaño and it's a strained and again we've got the error because I'm saying that this is providing a string and no longer in our proof it now provides a mono / string and again go programming put in that semicolon so now we've got the infrastructure set up we can go back to our code here and think I've just created something that will provide a mono for me so why don't I start with that pension lockup just going to return a mono I can get the name that's easy enough it's in the new employee and then I want to do something with that and the way we very often will do something with that is to use flatmap which says I'm effectively I'm going to do reactive work on this piece of data that you're passing in in mono it's a little odd because one of the implications of flatmap is I'm going to create a number of processes that will deal with one each of the items that you're passing in to me or a mono we know that's going to be a single one nonetheless that's the same pattern that we want to use across both of them so I'm going to get the pension ID from that incidentally something that took me a while to work out I would call that pension ID you can call it P if you want it's not a particular value it's just a name that you're assigning if you've been using Java rate streams for example then you should be very familiar with that idea so I want to do something with this pension ID I want to make sure I get it right as well okay I want to do two things I've got the new employee I want to set this pension ID to be that pension ID and now an important thing I want to return something that my flow can continue with and in this case that's fairly straightforward I can use the new employee that I've got and finally I need to do something with that employee and I want to save to the repository in fact I can do that more easily and again I want to return a mono F employee oh dear that's the wrong isn't it there we go yeah as I said normal developer make normal mistakes so that bit of code now should be okay we've got a flat map there which says I will take each individual item that you send me being just one in this case but it could be many and I will do whatever process you tell me to do on it and then at the end we have a map which is essentially a synchronous call for trivial behavior if you like that just says save it to the repository we're done so we have a very similar method that we want to call but instead of just doing one pension update one call to the pension service we're going to do multiple calls so we need to do a similar conversion on that and if we first change that so I get it right were you all feeling sep 10 that I put in the wrong place I could feel a certain concern so again we know how to create a flux that will have some information that's interesting to us you can start with find all and then as we've just seen we're going to do a flat map again because we want to do something with the values that come out of that that iterable that we've turned into a flux this time it's a little more interesting because we're actually going to get multiple values out of it so if we take or we're going to get an employee out of that and we're going to want to do something with that employee [Music] again I want to make sure that I get this one what right so let's do that we into your pension lockup on it and we need the name so that it can do the pension lookup and then we want to take the result of that and do a flat map this time it's a mono because we're just going to get one value back but it's buried inside this flat map we've got going on so we'll get a pension ID come back from that I'm much like the code that we just wrote we don't want to do two things for that we want to update the pension ID with that value and again we want to return an employee so that the next step has something to work on and again the final step that we want to do is to actually save it and I need another thing there somewhere sixteen sixty sixty six thank you do you have returns Ingo I can't remember okay so this is the more interesting one because I still need my know this is a more interesting one because we're using flatmap more for the for the impressive purpose that it has that we start off we've got employee and we're going to take each one of those individual employees and we're going to do the pension look up pension look up for update the record and save to the repository so if we've got a hundred employees that we need to check the details on an update then it will potentially spin up one hundred processes that we'll be able to do those in parallel or as close to parallel as it decides is appropriate a lot of the work that's gone into reactor is making sure that it has good defaults for things like the number of threads I think it was mentioned earlier that it's typically the number of physical CPUs on your machine so all of that is something that you can tweak but you should probably think about that and first get used to the programming method in general so if I try and run that now then with a little luck we did our Creator user I should probably restart as well that will help no obvious alias I must have the right number of semicolons in there so when I do my call again it works famously did before I looked up the pension ID if I do my clean so I make multiple calls out to the pension provider you may notice that that was quicker and that's because if we look over on the contour I forgot to put in the thing didn't I I'll just steal this piece of code you know stone that's again so I make that call and if I look down in my console and the size that you can see you can see without me having to think about coordinating different things that are going on reactor has taken care of the threading for me it's used for threads because this is a fork or machine or for CPU machine I guess and has made the process faster without me having to think about anything and it just happened by magic or almost by magic ok so let's have a quick summary of what we've been doing there we've had a quick look at what flux and mono are the important thing being flux is a zero too many Manos to zero to one it's a different way of thinking about it limitless than the options that you've got helps you to think about processing individual items we looked a little bit at map which is doing that that synchronous call and the flat map which is part of the real cleverness of project reactor of sharing out this workload and then bringing it all back together so that you just get a single flux at the end of it that will think and then be processed errors are just values I stress that at the time so we want to be able to be resilient and we can do that if we're passing around errors rather than just exploding as soon as we hit upon something a key thing that needs to be emphasized this you should stay reactive so we start each flow by coming up with some sort of flux or mono and then we try and stay in that pan one of the most common mistakes that people make is there is a method on a flux or a mono called dot block which lets you retrieve the value at that time and particularly few start to get into more complicated flows that what the stuff we've seen here is quite simple you get halfway through and you think I just need that employee name if I can get that then I can do the rest of this and you put in a dot block back in the day when I started with it it was called get which was far too tempting now block we're starting to realize block is a bad thing so you try and avoid that command and there will be a way to follow through the logic of stay reactive and making your entire flow reactive one thing we didn't get to too much is it doesn't replace what you know you can still use Java rate streams and if you want to call a method that does some processing as long as that method returns a mono or a flux depending on what you're doing you can have if-then-else inside there you can do the streams to manipulate strings or whatever you might be doing so this builds on what you know it can be really intimidating at first one of my colleagues described it as when you're learning Java there's quite a shallow learning curve and then to get really good at it it gets really steep as you getting into all the complex options that Java provides reactors kind of the opposite that it starts off really complicated because it's a whole new way of thinking about how to create quo but the relatively few commands available to you I think there's between flux and miles maybe 200 something like that most many of them shared so it's actually possible to get a grasp of pretty much everything that reactor offers some resources for you there's the reactor manifesto and the reactive streams or that we mentioned earlier project reactors home itself really good documentation on there not just the Javadoc and stuff like that but it really takes you through here's what the flux actually is here's what we mean when we flatmap what we're doing underneath the covers something that I particularly like inside there is the choosing an operator page where you can sit there and say okay well what I want to do is to filter the output that I'm getting here and say all right well you need to use a filter then it's really useful when you're first starting out and you just don't know what any of these things might mean reference to the spring web reactive the code for this presentation will be at that URL that's not there yet but I'll get there and the final thing I wanted to mention as I said at the start I started out with reactive on the CF Java client and because it grew up with project reactor it is an excellent resource I'm not boasting there were many more people than me involved in it excellent resource of examples of how you actually do reactive when you've got complicated code when you've got the sort of branching that I didn't even dare to get into today wander through there particularly if you have a look through the the integration tests which are pretty extensive you'll get an idea of what you can achieve in inside project reactor so the clock down here I think has been corrected and now tells me that I have 10 seconds left so all that remains is for me to thank you so very very many of you for being here for my first ever live coding session I'm going to be hanging around the back if you need me and this one two of my colleagues who will also be able to help out with any questions you have thanks very much [Applause] [Music]

Channel: Spring I/O

Views: 3,377

Rating: 4.8805971 out of 5

Keywords: springio19, reactive

Id: vSHNBgY7MGA

Length: 45min 38sec (2738 seconds)

Published: Wed Jun 19 2019