New Concurrency Models on the JVM: Fibres, Verticles, Agents and Actors. by Lutz Huehnken

hello welcome to my talk thanks everybody for for staying this long coming through this last slide of the day I think over the last before the closing keynote so I want to talk about concurrency models on the JVM this first slide shows the whole already feels like the big stretch this talk is about it's about concurrency modeled so it's kind of you know it's kind of abstract we're gonna talk about different approaches different principles may be underlying those but I have a little code example you see there is a little github repository so we're going to talk about concurrency models but we're also going to look at a lot of code because I'm a very pragmatic guy I don't like to talk about the abstract only I want to look at code and want to see how these concepts actually are reflected in the code that I have to write in the end but before we start I'm also a bit of a book nerd I like books but books on concurrency that doesn't doesn't go well together why the book we read from and know from start to end from the first page to the end so that's very sequential you'd think but actually there's a an interesting book by my favorite author Anna Schmidt if you speak German if you were familiar with a drum language you should definitely read something by this author he's absolutely great and this book is called settle Tom and it has well it's only difficult to see because it's also the book is like 800 pages thick and it's it's copies of his typewriter pages and it has three columns because he tells three stories or he's he has three threads of of action going on them in parallel so parallel I can tell you from reading this book I actually haven't read the whole book brother I can tell you from reading this book that parallel reading parallel stuff is hard and I think when we look at programs that are a parallel and we reason about that how things happen this is also hard because there's another book there's also a very funny book it's called well it's French I forget the time you got a poem so it's ten thousand now 100 thousand billion poems so you'd think that's also very thick book it's not though but all the pages it's a fourteen line sonnet and all of and it's ten of them it's ten ten fourteen lines on it and the pages are cut so you don't browser page by page you can browse every single line so you can combine these two 10 ^ 14 poems so in now if you think about how a computer would concurrently execute your stuff just say it's one single core like I'm a single reader then you have say 14 you have 10 threads with 40 next instructions how does the computer actually execute these in which order well it seems there are 10 to the power 14 possibilities for that right he can combine it in any possible manner so to know what's actually going on in a concurrent program is rather difficult to test it the the way we think about test coverage is that every line of code is covered and has been gone through in the execution of the program but really in a multi-threaded world you would have to test to have complete test coverage you really have to test all the possible combinations well maybe not it kind of works without but I just want to sort of open your mind well I think I want to make the point that this is an interesting problem concurrency is interesting it's worth thinking about luckily we don't have to do it by hand we have the computer so because it's a bit abstract and about models I want to sort of I want you to stay motivated I want to do I want to put up this claim I'm gonna I'm gonna I'm gonna I'm gonna claim that the Java world needs a new concurrency model ah that's a bold claim I said thread so robbief we wanted to have more on there's something on top of that so this is something maybe after this talk over beer you can discuss that and I'm going to give you some material for this so why do I think why do I not like threads what are the problems for threads I think there are two fundamental problems the first one it's is efficiency or rather the lack of efficiency for the threat model it's video I don't even know when it was discovered or or introduced we had multiple processes and then somebody had a great idea and said Oh processes they are just too heavyweight I kind of just always switch between processes I want to have something within the process I want something that's in the same adverse space as the process a lightweight process and hence the threats were born so but at that time it was at the time where we had single core machines or so so this is I'm trying to visualize how computers work with Lumberjacks so we have a single core machine there and the core the processor core is this chainsaw there and we have a number of threats this is the Lumberjacks and they have some work to do the logs so obviously we all know how multi-threading works it's time sharing right so one of the Lumberjacks gets the the chainsaw can work on this log and then at some point he has to pass it on to somebody else and then this person or this thread gets the CPU this lumberjack gets a trace on the thread gets the CPU it's actually not a stupid model it's it leads to what this is how we get a good utilization of the chainsaw right we don't want the chainsaw lying around we want the chainsaw to be shared it's an expensive resource so it's good if they share it of course if you pass on the chainsaw to another lumberjack there's some overhead to that right you have to maybe have some protective gear you have to wear and and there's a there's a context switch going on so nowadays also it actually looks different we have multi-core CPUs it's not just one core and oh and all that this is about having very many tasks that we want to work on it's this whole thing is especially interesting if you think about high concurrency if you think about servers that have thousands of connections for example so if you have that this becomes ineffective if you have a number of cross and you want to split them between ten thousand Lumberjacks and you can easily imagine that the overhead of the context switch becomes so relevant that it's it's not an optimal model anymore and they're all set seems weird that each of these Lumberjacks just has one log to work on right I have this one log and I have to wait for one of these chainsaws to to be assigned to me and then it starts you know I have to wear the protective gear and start working but then I have to pass the chainsaw long before I got my work finished and then when the one log is done I go home in the another lumberjack comes up and does the work so this is not what we want this is not effective we want to go what we really want to end that is something like this we want to say okay that thread should not be our unit of work that should not be the finest lever that we work on we want some abstraction below the threat level our unit of work should be the task this this lock this log of wood this is the task we work on and the thread should be like a worker a worker that works on it on the number of tasks not just one so ideally to have no context switching at all and to be have the maximum efficiency we want to work artists to have the same chainsaw all the time and so we can just sort of you can just work on one log after the other or of course I'm still talking about computers so what we want is to threaten ocala T we want ideally we want one thread to run on one core all the time that that will probably not happen most of the time but that's our ideal scenario then we can leverage modern hardware then we can leverage the fact that the cpu will load lots of stuff in its care that will try to predictively execute stuff that will beö it will do pipelining things like that all of these optimizations that modern CPUs have get lost when we overdo the context switch thing because it's like a restart of the whole scenario over and over again so enough with the Lumberjacks actually they're gonna come up again later but on this sort of more serious level we want well if that's not I think there isn't really an established term for that I use didn't I tend to say tasks do so I saw it somewhere in some you know some online course or so but there's not really an established word for it for the sub thread level concurrency I'm gonna show some terms later so what we want is we don't want just the threat level we don't just want processes where one process has a lot of threads we want processes of course processes that have many threats and but we're going to consider these feds as workers and each of these workers should have many tasks and similar or the approach is also known as the reactor pattern where tasks are or what the deferred annuity text on the workers so we want this once at threat level concurrency and another point that is very closely related to this efficiency thing is a synchronicity maybe some of you have to have seen the talk this morning about acing everything-everything async I forgot the title but how it again the heart where we use offers as a lot of power in terms of doing things outside of the CPU basically when you do some IO you don't need the CPU for that but the traditional model we use if we use the serf that API for example and do a a database access or network access synchronous file IO will block the current thread it will not block the CPU that's fine we're not we're not producing CPU load but the the thread will be there and will have a certain locked state will be waiting or actually blocked or I saw another threat well I'd say it's waiting I await whatever so in a synchronous model where we here and this is not a sequence I'm saying requests you ten requests and then our threats are workers but it's not a sequence of threats coming in over a certain amount of time this is a snapshot we're looking at the system and at this point in time it has ten concurrent threads and ten ten requests that are being worked on in parallel and each of these each of these requests has been assigned a thread that's a traditional thread per request model that we have in Java EE in the surface API for example now if I do blocking i/o what do I block I don't lock the task right I block the worker because the threads are still our worker pool we use the feds to actually do the work so if we block we we block the worker we block the lumberjack in this case that will affect one one request right because each of the requests has one thread to it if this fare this blocked and this request is not being worked on but in a in this task model or I said okay we want many tasks to be worked on by that's and that's probably really many the we if we really want so we want to keep the number of threats low let's say we have an 8 cross CPU and we just use a threatened or worker pool to work on incoming requests so we have one threat for core but we want to work on thousands of requests so it might be thousands it might be a thousand tasks that one worker is working on so if we now block the worker with blocking i/o then we've got many tasks actually not three or four like like drawn here but it could be yeah it could be a thousand or well it's it's open-ended obviously so it's I hope you can follow this train of thought that's it's a we want sublevel concurrency we want that is a finer granularity we want tasks but of course we still use threads as the as a pool of workers so we can't block we cannot we can never block we must not block our our worker pool threads to work on these tasks don't worry too much about it because it's not you don't have to implement this that's been done but you have to know about this so I said one of the main problems with threads is the efficiency and this has actually been solved the idea is I go sub thread I have suffered from currency I have task level concurrency and I use asynchronous i/o to never block my worker threads I'm going to show you four different this is really irritating I'm going to show you four different models for different concurrency models and they are not different in this way all of the models were going to look at share these characteristics they all implement sub threat level concurrency and they all encourage asynchronous i/o and also if you see somebody saying this is a reactive system on this is reactive this follows the reactive manifesto all the systems that I know that I'm reactive also follow these principles that does not mean that by doing this you're automatically reactive but to be reactive you have to follow this so problem one is solved then we're going to look at how the different model solve it the second problem is the programming model the way threats communicate with each other the way we compose our programs other threats it's basically impaired the ideas imperative programming we call methods we create a new Auto object-oriented we create an instance of a thread we call one method on it we override stuff and then something is calculated and it has to be put somewhere and it's usually in some variable that can be accessed by multiple threads what we call shared mutable state if you see here mutable state somewhere that's usually bad if you you have to protect it you don't protect your head state you will have raising conditions we saw threads can be executed and all sorts of fashions and different orders so if they just write in something you will get a big mess you will have errors and you will be it will be because they're kind of non-deterministic because it depends on the execution all of the threads it would be difficult to find out where they ever actually came from so how what was his name again mr. Lee as mr. Lee put it in his nice paper oh well this is if there are many there are many publications I think about threads and why they're not a good idea or not a good idea anymore or not a good idea for an application programmer but well for something low level and he says well the the the one beautiful characteristic of programming languages is that we let it so deterministic that we can say these are the steps that are executed and the outcome is clear and we can reason about that and the threads we give this all up because we have no idea what all that's executed and we can't so what we do is we introduce locks and synchronization so we protect our family to the state but that of course leads to contention if I have many threads I want to write to the same area and memory then they will queue up I will have contention that level will hit my performance so luckily I'm not the only one saying this or many many people are saying this but even this JVM architecture on roses saying this he said I think in February this year threads are passe but they're not going to go away but we need an abstraction on top of threads something that's easier to handle he says it's an old model fifteen years ago it seemed like a good idea but the mechanisms we have it that's not a good fit for more than half this is related to the context switching and non-uniform memory access and characteristics that just make it any different to switch between too many facts also threats are kind of heavyweight it's when I create the threat on the JVM probably a few of you know that there's the default there's a parameter the excess I think is it is the stack size you can configure that and the default stack size that's memory that is reserved for the call stack of thread the default stack size stack memory size on a 64-bit JVM is one megabyte so we're talking about systems here with one 10,000 thread so it's 10 gigabyte of memory for my thread stacks that's in the different to your heap space so it's not it's not a really lightweight model and yeah and the synchronization the the soul locking thing just doesn't work out so when we talk about this I already said ok we want to go to sub thread and one you want to use asynchronous i/o when we talk about programming models and we talk about asynchronous we also have to talk about how do we use asynchronous api's what's the standard way of using an asynchronous API it's the callback I call something and I say when you are done I'm just gonna not gonna care I'm just gonna send you something here please open this file but I'm not going to wait for you to open this file when you have opened this file then please invoke this function or this method here and then I can add the numbers I think this is JavaScript right this is a key pasted from some copied and pasted from somewhere so it opens the file and then on the file open there's a callback when the file is there then it will go over the lines and there's another callback than through I the file this is not very readable code callbacks are a bit of a pain so when we look at different programming models we're also going to look at how how do they communicate how do the entities and these models communicate with each other what is their model as opposed to callbacks and also how can I use asynchronous API is is there a way a nice way to use callback something on top of that or some alternative to that and I might also be in a position where I have to use a synchronous API I just had three slides away ago Blagh never but you know for every rule there exceptions you might be in a situation where there's just no alternative the most prominent example probably being JDBC if you want to use a JDBC driver to connect the database here you'll have a very hard time to make it unlike an asynchronous call so we are also going to look at these different toolkits and see how can I deal with the synchronous safety eyes so just to be fair Kohlberg's I don't like callbacks but of course I am aware of the fact that this could have been written in a different way different languages have different syntactic sugar for callbacks you could say oh yeah why doesn't he use Scala futures and he can have nice four expressions around them or even in JavaScript there's the promises and I can say do this and then do this and then do this and it looks like sequential code but it's actually async just like a single rate if you've used C sharp or if you've also it's also in Scala it looks like sequential code you say a saying and then await this await this and it's actually asynchronous and the code is going to be invoked when the async stuff is finished so this can all be done sure but it's syntactic sugar doesn't doesn't solve some problems of the callbacks because in this a callback might be okay if you have one function and when the events done you want to invoke the next and then the next but you often you don't have the synchrony chain of this one function after the other you might want to spread work out you might want to wait on something to happen and then you want to invoke three other functions or you want to wait for three functions to be completed or you want one to wait for the first one of three to be completed and things like that so although there's syntactic sugar for callbacks it's still callbacks and we won't want to at least want to explore if there are not best possibilities for that so now this was all the sort of the build up why why we don't want to use threads now we're actually going to look at some models and at some code - if you're scared of code it's yeah you're gonna have a hard time but what are we going to look at if we look at different concurrency model what's ours what our criteria so from what we said now from what we don't like about first you can say we want predictability so we want to look at this this programming model gives me a different approach of thinking about my program rather than having to think of something is happening at the side at the same time in some way does it have a different approach to that I want to look at composition in the sense of how do my asynchronous units of work how can I combine them and I want to look at integration in the sense of how can I use asynchronous api's how can I use synchronous API and is there now and stay of course I said about said something about here mutable state that we we don't want to have that but of course we're not always stateless right we're not everybody not every program is purely functional and has no internal state it's very likely that at some point you have some state that you have to handle in memory in your program so if we don't want help mutable state and we don't want the synchronization of Docs and stuff how do these model egg models actually handle state so the four models were going to look at ah the first one is fibers it's gonna be language called fibers and I said there's not no term has really emerged for this software level concurrency what I call the task so some people call it co-routines - expecially to express the the the notion that is actually collaborative multitasking and not scheduled by the operating system and go of course they're called go routines which is a little play of course on go on Co butenes some people say oh it's just green threads like like the first versions of the JVM or I think in Ruby also have that it's like threads but it's not operating it's not native threads we just have native we just have JVM internal threads in the and the closure world are called inversion of control threads I think or some people call them that to express the fact that they actually use this to get around callbacks some people say user mode fast the fiber guy's face mmm the guys who do the fibers actually say it's a user mode thread so if you see any of these terms there's a certain likelihood that they're talking about the same thing that what I call tasks channels of course we're gonna look at this so I'm not going to talk too much about that now but these are the different models for and look at and the first one is fibrous and this is probably I would guess not many people have used this here it's I think it's not so not so popular what is a fiber it's pretty much like a thread right I just created a new fiber and I have this one method that I have to have to override and in the spirit of duck typing we could say it looks like a thread it walks like a thread that talks like a threat it's probably your thread and in fact you use it like a thread oh yeah I said we have this little example I have some example code so the example we we have is it's a it's a lookup I want to look up a word I hear a word but I don't understand I'm not a native English speaker and people might throw things at me and say you are a I don't know a doofus and I'm gonna know what that's a mean so I look it up on victor nary on the merriam-webster dictionary and on the urban dictionary because it might be a slang term and I want to do this in parallel I want to do I take this word I want to do three HTTP requests in parallel I want to aggregate the results and then when all the three are done I want to see the results so I have the code on my machine but the whole switching thing we can do if we have some time at the end not bad is very exciting but I could show the code so you would see it's that these models actually do something um so the funny thing here is if you look at this I said it looks like a thread it does it's not it is what it does it uses bytecode instrumentation to detect these at startup time and relax me I have its own scheduler to run them to run money for many fibers on the thread what's interesting about this code is that it also looks like imperative code that we write in a thread for example this HTTP client this is the regular or the interface as a regular a pet fee or well no matters ng ng and and during a sink a TV client I think but this one are the close ability of its line interfaces even I think some standard from factory HTTP or so but the interesting but here is the fiber if you client builder so I get an HTTP client that I can use in a synchronous manner when you look here I just do a response equals client execute there's no call back there there's no async no asynchronous messaging whatever so what it does is this fiber HTTP client builder gives me a blocking API but it does not block the thread it's only fiber blocking so this is actually a smart thing they give you some additional API so they give you some utilities that give you a synchronous that give you synchronous API on top of asynchronous API so you can use you can basically program like you used to program and driver you don't have to the mental switch is very very small the some things you have to keep in mind like you can't lock with read all you shouldn't and there are some extra features but basically it's like multi-threaded programming so it looks like blocking it's and it is blocking but it's only Fiber blocking it doesn't lock the thread which is a good thing but in the end you have to aggregate the results somehow so at the end I just call a method where I put in the result and that's gonna write in some variable that it will live for the one time with the program so that's this prep mutable state thing right there's something that would have to be synchronized or I would have to use a data structure that is thread safe because my different HTTP results will come in concurrently and will all void to this this space in memory basically so when I want to aggregate the results I have this yeah I just I use a concurrent half I'm here just because it's less safe obviously and to start my parallel retrieval of my definitions I I start the fibers like I would start fats so the retrieve info was my fiber so I'll just say a new retrieve info or this is the key the key I think is the name of the website so at the end I can assign to make the connection between the website I'm looking up and the definition my search term of course and the source is just a list of the URLs for key so it looks very much like multi-threading and in the end it's about it I have the software level concurrency but I still have have mutable state it's not really a different programming model in terms of the approach so the major advantage you could say of the fibrous is it's just like threat based programming if I'm used to that I don't have to learn something new the main disadvantage though it's just like threat based programming all of the disadvantages we have in the programming model and that is not having a sane where you're thinking about the code and not having a sane way of dealing with mutable state we preserve this we take this with us this is a this is how it works with the or the idea behind the this ATP client for example what they offer and I have to give it to them they have some nice things there in the in the draw they have this the idea of this fiber async that actually does what the what we saw with the HTTP client this wraps an async all that that takes a callback right it will take something where you can actually put in a callback and have a completed method and we'll wrap this and something that looks like a synchronous call but as I said the idea or the funny thing behind it is it will block it will be a blocking call but it will not lock the thread it will only block the fiber and that's about all this to say about fibers they're like threads but just much more lightweight you can actually oh this is something you can easily try if you run the program that creates thousands of threads for example on my macbook if I run a program that attempts to run ten thousands of ten thousand threads it will just crash at about 2,000 or so well this is actually I think an operating system limit maybe it can be configured but standard wise at 2048 threats or so it will just crash the JVM will crash and will say I cannot allocate more native threads if you use fibers or any of the other things we're going to look at you can easily create a hundred thousand fibers or more probably I haven't haven't gone up higher but they're really lightweight so you can actually set the the efficiency problem is actually soft with it you can have 10,000 connections if you will and each connection can have its own fiber or 10,000 events or whatever you have and you can use this programming model it is felt like it has some nice tricks as I said it works with instrumentation so you you have to add a little jar to your to your JVM options you have to say the realm of since Adrian something-something and at this fiber jar and then it will so that's not of time it will detect the fibers and will instrument the bytecode and put in this mechanism and has something like a suspender bill annotation because they are co-routines they the operating operating system scheduler will not switch between fibers you have to let go you have to do have to make an async call or you have to stop your work for other fibers to be able to take over they have an annotation suspending where you can say on a method okay this is a suspension point you can we scheduled this fiber at this point so you give back control to the scheduler it works nicely with threads as well you can still create new threads they have a common abstraction on on threads and fibers so you can program things that take it's called a strand that work on threads or fibers you can use things like thread local some people are used to using thread locals which is of course a problem if you use sub thread concurrency because it's not really local anymore than is it if one thread works on ten thousand events then its local environment its global for these ten thousand events so with that local might not do what you wanted to do in a sub thread level concurrency world but it works with fibers fibers alone I think our a bit low level it doesn't really give you a new programming model it's almost a drop-in replacement for thread so some things you need to learn if this is what you're looking for if you say threads are great I just they're just too heavy then fibers might be good for you the English Channel the White Cliffs of Dover sound like thing is candles knots getting more interesting in terms of concepts there's actually a different way of communicating between asynchronous routines can let's go back way to 1978 and I think it was the papers this that about communicating sequential processes where there's a whole theory behind it in some a lot of mathematics and algebra about what works and how it works and why works and what the laws are of it but as I said I'm very pragmatic to me it's more interesting how it works in code but it obviously works it's very popular and go it's the standard model of concurrency and go and go you have goroutines and channels and somebody indeed the closer world with Hickey took the go basically the go functionality and said let's put this into closure and created core async a concurrency library for closure which was a lisp like scheme like language as you probably know on the JVM and he brought go routines and now they're called go blocks and closure and channels to closure and some nice syntax of operators with it so what happens here so this is we define a channel and here we define a go block so a go block is a lightweight asynchronous thing it's it's like a fiber well it's not a threat there's another another expression that starts with thread another function thread that will create a new thread here we say we want to create an asynchronous routine but not it doesn't even have its own thread and we say ok in this go block what's supposed to happen we just want to print something and the parameter you know the parameter for print alone is the string right and here but you say okay instead again I I pass a function here and the function the functions name is this you can if you have any good ideas how to pronounce this but it's a busier take out of the channel and of course a parameter is the channel so I say print the result of this operation the operation this takes something out of the channel again this is a blocking operation or it's it takes something out of the transit endless empty it's going to run rate there it's going to block but again it's not going to block the threat it's going to block the fibre and here we put something on the channel and this is we're not in a go block here that's a subtle difference so in it's actually about the threat we use the to deficit not one but that's actually of no importance to at all for us now but I think the the basic principle is probably clear we decouple the one execution from the other by putting something on a channel we say listen listen basically to this channel and then I put something in and then execute this it's not it's not the bust of a channel it's just the decoupling this channel just takes this one element and kind of synchronizes things it's not it's not an asynchronous model per se I can make it asynchronous more asynchronous by making the turning the channel into a buffer and say I take a channel with two elements then I can put in this case two things in the channel and then it will block again until I take something out but of course of course I can make the tents much bigger than two so the idea here is I say it's a buffered channel so the things can actually sort of be decoupled again from the the producing of output can be decoupled from the consuming of input I can put stuff on the 1000 this was only the one bubble there um and and this way I can already have some asynchronicity or if I combine this with the go blocks let's look at our example again so if I want to get results from websites and I want to when I aggregate them and I can use channels for this so we define a let's look at this first so we say we want to get a URL and then again this uses a callback so the way we can deal with callbacks is here that we put in a channel instead of a callback or ask the callback function we put in the put it on the channel function so we move from callbacks to channels here so this function actually expects a callback it's like if I get the I make an HTTP request and then when the result comes back I want to execute something pretty much like me on the within like an acing HTTP client in Java I would also do and put in the function here and the fact that's gonna be executed again asynchronously in the go block is put something in this channel that I have passed earlier as a parameter and what I want to put in is the body of the HTTP response so now use this to call three web sites in parallel and all of the responses are going to be put on this channel and then I have this go block that will just sort of it will go through the loop every time something is on the channel because of a block here right it will recursively call itself but at this point if nothing is on the channel it will just block but again so I like it's you know it seems a bit repetitive to me to say it over and over again but I just have to say it again when I say blocking it's not bad in this case it just blocks the go block not the thread so that's how it works in here of course this very simple code I didn't even write that thanks to Vijay this when it has three values when it has aggregated three values because we know it's three websites we are creating here every just that we turn this and then this print on would get the values and would print it out so what we did here is we define the channel again the buffer oh we can we can take a couple of responses there to have nicely a think we would aggregate the results recursively by by here some blocking Lee waiting on something coming from the channel and then just put it back in into this function so it's a nice way of kind of the state is basically the state in this application is handled in the channel like we don't have a variable where we aggregate the state we don't have shared mutable state we have a stateless function that once that loops over this channel so the channel becomes our keeper of state that's actually pretty smart so I think it has some very nice ideas it's well obviously well if it's it's very popular in the go world as I said it's a working model but also it's about the whole thing is about exploring things on the JVM wide thinking about concurrency and exploring what what models we have and what's there and what are the ideas behind that so here the idea the focus of course is on on a nice compositor model that I and say I put something on the channel somebody else takes it out of the channel so it's more I don't think of programming as function function composition or method invocation definitely not a sequential sort of imperative style um maybe but even a functional style would not not naturally be asynchronous right but this is more a flow style I put something on the channel and this might trigger something else it actually has a lot of nice options too I can listen to a couple of channels I can select from channels with the Alice statement so I can I can I could start these three and then say I just continue with the first one that's finished things like that it gives me sanity sanity in terms of predictability on when I reason about my code because a message comes in and then this is I said it's a it's a blocking operation but it kind of synchronizes on the queue on the on the channel so it takes one message out of the channel and the code will be executed and when that's done that's going to take the next message but it's not going to be it's not going to be so I'm going to take two messages in parallel and and we don't have this this weird thread parallel execution we have our our different blocks that listen to different channels and do different things of course this can all be that the requests can run in parallel and then the other things can run in parallel but we don't have this real situation where we where it's a bit unclear or what the state of something is or who's just accessing it it kind of sequence as sequences sequence Eliza's and another word for that our message processing we saw the go I don't know I think it's a Mac word we saw the go macro that will create the the subroutine the fiber likes are 14 the co-routine whatever you want to call it the go block and there's another one thread so you can do the same thing with threads so I said never block and that means that your your main worker pool must never be blocked but you can use this to put to offload work to an ace eppard thread if you want to use a synchronous API you can use a thread and say I I'm going to create in a different thread to work on that and not not contaminate your work of love you will and to a certain degree channels can be seen as the keepers of state you have many opportunities now to keep the multiple state out of your application by aggregating it in channels but let's move on event loop if anybody has worked with nodejs then that's a very familiar paradigm the ideas well I consider everything coming in as an event and I register event handlers that will react to that that we the event is put on the bus and the loop works on this bus or a couple of loops and this will trigger event handlers to be executed so it's a bit like nodejs but it has actually it uses multiple threads nodejs the main event loop and notice this uses one thread as as far as I know in the vertex is actually going to use multiple threads vertexes our example here for the event loop model what's one thing that's funny about vertex that's independent of concurrency is that it allows you to use any jvm language really what you do see deployed little units called verticals and you can write these verticals in closure and Java and Scala and JRuby I think probably drivin definitely JavaScript so it's very polyglot some people like that and the the way the communicate is usually a JSON what does a vertical look like it looks like this so I have a class that I have to inherit from and then I register myself on the event bus I get this event bus I get so this is like a global okay in the Indian epic vertical the vertex is available to me as a member and there's also there like static utility methods to access the sort of the the environment it comes with some infrastructure to write it's not just maybe it goes a bit beyond channels because it's like a a one-time if you will and then you register yourself you say ok this is the address I'm gonna be known as ping address and somebody's got to put a message on the event bus and it's gonna say this one is for ping address and then this is gonna be invoked especially this lambda here so again callback style obviously this is like a go the world example I get a ping and I'm gonna reply with a pong let's look at our example I want to have three in parallel I'm gonna have three verticals running that fetch the URLs and I want to aggregate the results so this is the our actual vertical that takes the that will do the HTTP request and again it's interesting what they offer as a utility library we saw this in the fibers with the fiber blocking HTTP request but here well again we register as a consumer so I have to put a message on the bus saying this is a method for define or task and this is the vertex HTTP client that takes a call back again and when I get the response I'm gonna take the result and gonna put this on the event bus again because this is now the way we communicate right I don't invoke there's no no callback that has been passed to me that I could invoke or there's no channel I put things on the bus I don't even know who's gonna get it right it's a of the the utility that's interesting the utility API is that you get with vertex they're very callback heavy so you can really feel that it's coming but it's kind of following the node approach you don't have to use the event bus as much you wouldn't you could actually sort of you can do a lot just with callbacks if you prefer that style but there are of course there are advantages to using the bus I think the sort of the general principle is probably rather easy again this is the creator of our tasks we we have to deploy the verticals right we have to tap vertex create these verticals that we want to use to get the HTTP responses so this is the static affinity method I talked about you get the vertex one time you get the event bus when you want to deploy verticals you can actually pass a couple of options in this case I just say okay I want to only want to deploy three instances I don't want to deploy one vertical I want to deploy three verticals because I want them to run in parallel and and then I just deploy them and then I send this or your once they're done when the vertex is done with the deploying it will again invoke a callback and I can send the message here this is usually Jason I created a synoptic here and then I sent this to define a task and then that's interesting that's kind of um now I have created three verticals that are all called definer tasks they all have the same address now I put something on the bus for definer tasks which one of my tasks is actually gonna get it where's it gonna end up I don't know and and vertex won't tell me it's by default it's going to use a round it's called a non strict round robin so it's kind of conveniently trying to round robin the messages so it's actually most likely each of my verticals are gonna get one message and it's actually gonna be done in parallel I could do it differently I could get the identities or I could configure them I could generate a name and call them task 1 task to task 3 and configure them when I deploy them to have this name to listen to that address but it's actually a very common pattern to say I want to use this I want to use this for scaling I want to use this for for providing things I just put multiple verticals to and on on the same address and I have automatic load balancing around that automatic round-robin load balancing over my verticals so that's why I chose this for this example in the end we saw the vertical sense again since the message on the bus so I need another vertical that collects the the results and here I chose to make its faithful so this is something you have to do sort of that doesn't vertex doesn't give you this automatically by the vertical just by looking at the name of the vertical you can't say oh this estate fold is a stateless so here I actually the the identity is actually important to me i encapsulate state in the vertical so if I want to send the message I want to make sure it actually goes to this specific protocol in this case this is so far just deploying at once I take the collector and it's just kind of going to be deployed once so that's not going to be a problem but so being encapsulate the state in the vertical and this is a very that's a legitimate approach that's if you want to if you want to handle mutable state and vertex either you encapsulate it in a vertical or you kind of break out of the whole vertex system altogether and just reference some some globally visible data structure somewhere which is of course in worse so well the other ones not bad so that'sthat's worse this is good you encapsulated in a vertical but then you actually need the identity of the vertical obviously another nice thing about vertex is this what they call the hybrid threat model so it actually gives you a concept of background workers I said in closure we have the threats and the go blocks so you want to offload a synchronous work or cpu-bound work to another thread pool in vertex I've actually implemented the three types you can have work our vehicles and these vehicles will be acts butter kills work of verticals and these workers will be executed on a different thread pool so you have your event loops which is already multiple threads that work on the event bus now you have a long-running or synchronous thing you can say well this is actually a work of vertical I don't want it to spoil my my event loops here so it's gonna be executed on a different thread so there are a lot of interesting concepts in this it's very loose coupling I just put messages on the bus and these messages are gonna picked up by some event handlers this in terms of reasoning about my code and what's being executed and what order or how my code is executed again it sequentialized it for me and gives me the sanity back the predictability back I get the event and then my code is executed like it's a so-called single thread illusion at this point I just have to care about this one thread that my vertical is running on there's not going to be any sharing of a vertical of multiple threads at the same time so I can use things I can encapsulate state in there things like that the hybrid threat model allows easy integration of synchronous api's if it comes to a single TV callbacks it's very cold bank heavy it's not my something like my taste but well other people might like it and we already saw there is of course syntactic sugar for that state handling and captivating state and verticals need some care you have to be aware of the identities of verticals or there's of not to be identified and you just use as a general address for load balancing and this is y and this example wasn't room for this but something you get with it is distribution because vertex this event bus you can actually run it in the network and this is actually a very powerful feature because you have a unified model now you don't have to in a so if you do that based programming you're gonna have multi-threading in your local environment and then you want to go across the network to another machine you have to use something else you have to use messaging or you have to use a rest call HTTP RPC e or what have you so usually we have different models for distributing work locally and distributing work over the network with an event bus like this it's a unified model so you can build your your application in this way and if you want to scale out you just add another machine to your event bus and deploy the verticals there and and can subscribe your system over a number of machines without having to care much about it I think that's this is a very strong feature of projects of course actually a feature it shares with actors the the last of the four approaches I want to present so actors also there's a good theoretical foundation for that even older than the communicating sequential processes it's from 1973 from Carl Hewitt and he says I have to read this for me because I don't know berhad the actor model the model that treats actors as the universal primitives of concurrent computation that's a bit like an object orientation we consider objects the universal primitives of our computation everything is an object right all the way down until it's Turtles but so I can I get a message I can make local decision I can change my state I can create new actors I can send other messages and I can change my behavior I can that's that's an interesting part but we're not going to go into that too deeply but compare it with object orientation and in the object-oriented idea a method is invoked as also often described as getting a message objects send messages to each other you receive messages you can change your internal state you can send messages to other objects you can create new objects or maybe destroy them so it's it's kind of similar but actors of course all communication with actors is asynchronous the way it works is when you create an actor you don't get a pointer to the actor you say system give me an actor and then you get a so-called actor reference an actor ref and that points to the actors mailbox like a channel every actor has its own mailbox every actor has its own mailbox ID is a bit like a channel in CSP and the the asynchronous part is the sender sends the message to the actor reference the actor F and the message is going to be put in the mailbox and this is kind of this is one thread and at this point the sending thread is done the message is in the mailbox and yeah we don't know about any any anything else and then there's a so-called dispatcher which is a thread pool and and so the manager of the mailbox if you will and this dispatcher will be notified when something's in the mailbox it will DQ in the message from the mailbox and invoke the actor so again a bit like with channels or with events this asynchronous messaging is obviously a principle that is a has some universal appeal so you put something in the mailbox you're done another thread from another thread pool or from wherever will pick up the message will actually invoke the actor and we'll work on that how does this work in practice it seems a bit it sounds interesting it's often confused people see this and think oh yeah message thing our team messaging that's like JMS messaging or wherever them cure whatever but here as we said earlier it's our principle modelers are primitive for the computation so we do everything in actors we it's very lightweight it's just putting something like in the Java basically your cue and that's it's all local possibly what does it look like so I have to create an actor this is the sort of V again remember the example I start something I want to spread it out I say I wanted a definition for this term I spread it out I do three HTTP requests and then I want to collect the results so here we have two actors we have a coordinator that will spawn this the child actors that do the HTTP request and will also itself receive the results and it will collect the results here again we encapsulate stayed in something else not on a vertical in this case but in an actor and here it gets so this is the receive block an actor gets messages we saw that and the dispatcher will call the actor and give it the message but in the actor this Scala code I hope it's just more concise than the job of Earth and so it's simpler so in the we see if we have different commands either I get the start command that gets me going the start command will will make have me create the other actors so I have a list of websites I want to query anything that's the sources I have a list of sources Easter's the sources contains a key and the URL so for each of these sources I create a new actor I initialize this actor with a key and the reference to the squadron a drunk self is myself I get this but you can't use this because you know we don't work with references here we work with mailboxes so self is my actor F self points to my mailbox well not my butt to the coordinators mailbox and it sends a command to the child actor it creates a trial actor here initializes it with the reference to itself and censor the command then just for just for fun it also schedules a message so if it takes more than 5 seconds and will actually get a message and say cut it short and if it receives a result that will probably be from one of these sub actors here from the children I will just add the results to the map here and at the end we'll send a message to itself and say ok or I had three sources I received three with odds so just send a self-admitted and then and done it's know it knows it's done so it could close system down and print out the results or whatever so that makes sense so the trial actor is also super simple again a receive block we saw this message here this message is something we got from the coordinator so the cue on it coordinator started this child actor and send this commodious and definition so again this is this topic about asynchronous API again we have an HTTP API here that this callback based or in this case we I think we get a future backer Scala future which is as I said earlier syntactic fragra for a callback based API but the actors have a nice mechanism to deal with this you can say take this and instead of putting in another function that's going to be executed as a callback I say pipe to myself so what that's gonna do is it's going to put in a function that will sort of when the the HTTP request is completed and this the uncomplete handler is executed I will take the result and wrap it in a message again or actually will not wrap it and the message will just take the real out and send it as a message again to myself in this case the message is going to be an HTTP response so I'm just in my receive dog I also say okay I can also receive HTTP responses so this is the one that's gonna come from here and in this case I'm just going to take the body of the HTTP response and send it back to the coordinator because when I was initialized the coordinator passed his reference to me at the sector if I can just use that and then okay there's some error actually there's no error handling but in theory you could build in some error handing as well so you see again this is very again this gives me some sanity back it's not weird things happening in parallel I get a message something happens some other messages send right it's asynchronous but when it comes to processing a single message it's super simple right so it's a bit of a strategy that it's of course it's still difficult to reason about a complicated system if you have a big system and a lot of messages are send and you have to follow the flow of messages then you might still have some difficulties but when it comes to the single some of the your single unit of code you don't have to worry about your code being executed in parallel by different threads and of things having to be synchronized or or locked using locks and things like that so it's again we get some sanity back we just sent this out to the aggregator and another little bonus we get here you saw if you remember that in the coordinator when it got the start command it would create the task actors as child actors and actors form a hierarchy if I create an actor I become the supervisor of the actor so the first actor we created we said a system actor of coordinator so that was that's the user actor at the top here the system Act well the guardian actor would say and that would be the supervisor but then when I create sub actors like the tasks I created I can I'll be the supervisor that means if something happens in the task I can be notified I'm even except from this throne although this exception is strong on a completely different thread and usually I would not be a concern with stacked face at all anything I can actually convert this into a cup into a handler being called on the supervisor so give me very fine-grained and very flexible error handling in this case yeah you could just retry the HTTP call or well depending maybe on what the error is so summarizing actors again like I think like all the approaches except fibers it kind of sequential eise's things its events or messages or whatever in this case it's messages it gives me this predictability back when it comes to one message its messages it's not events that was probably a bit too quick to sort of see it in detail but if you if you remember the event bus we would put something on the event bus with an address but it's on the bus the receiver is the bus and we just kind of put a label on it and we're not when you show what picks it up and it can be we don't even know who picked it up if we have multiple verticals with the same address because it's going to be rounded in actors its messages and in the sense of there's always a receiver I sent a message to somebody I don't just put it out there on the bus it's not it's not out of my control who gets the message because somebody listens to these events I control who gets the message in terms of the actor F when it comes to what is after the actor F there can still be some routing on the things like that but that's necessary of course for scaling but it's a different it's got a slightly different thinking I say this is the coordinator I want to send it back to the coordinator I need the coordinator to give me a reference first so I can send something back to the coordinator violin and eventbus we just create an event and put it on the bus it's a nice there are nice tools to integrate with asynchronous api's namely the pipe tool and if you have the other way around if you wonder if you have an act and you want a callback you can use the ask pattern um for synchronous api's we saw the dispatcher that is dispatching messages and invoking the actives you can define different ones and you have as a bonus again it's distributed it's a unified model for local and distributed and you have supervision as an extra for error handling so if we look back at the models we explored here maybe it made sense and in terms of it's a story that builds up we looked at fibers which is a very simple approach which gives us the efficiency and the subject level concurrency but basically that's all we get then we looked at the at the channels that gave us a new way of composing things and of interacting between asynchronous code so on top of the sub threat level concurrency it added whoo thumbs up it added asynchronous messaging vertex as a bonus gives us a unified model for distribution and local concurrency and akka has all this and also gives us a special way of supervising things with the supervisor hierarchy so to summarize I think concurrency is interesting I hope you feel this too and maybe got some ideas of what to look at and what to explore or what to discuss over a beer when it comes to new concurrency models for the JVM threads are old we don't want that anymore they're not going to go away but they're not going to be the user level API they're not going to be the application developers API we need an abstraction on top of that to work with we need event buses or actors or fibers or channels something that that gives us a sane way to think about our code and luckily on the JVM alternatives are available today they're all production ready and nice and good I personally like akka a lot so if you don't have the time to look at all four three or what others there are I think akka is kind of it's so mind-boggling too because the sole messaging approach is so different from the way we're used to program that I think it's a very inspiring thing to look at I think that's about it oh yeah and there's if you're like me and you like books and you want to read the printed book I highly recommend this book it's called obviously seven from currency models in seven weeks it covers all the concurrency models I talked about and three more but it's not restricted to the JVM it also looks at other one times of course and that's it if any we don't really have time but if there's any pressing crack or maybe you just come down and ask me in person if you have any questions so we can get set up for the next speaker or

Channel: Devoxx

Views: 17,073

Rating: 4.9853477 out of 5

Keywords: DevoxxBE15, Devoxx, DevoxxBE

Id: EMv_8dxSqdE

Length: 63min 2sec (3782 seconds)

Published: Fri Nov 13 2015