Featherweight Isolates in Flutter

when we build our flush wraps we want to make sure that they are available and responsive for our users this means we want to make sure our apps are able to draw their ui at 60 frames per second and respond quickly to user input but everything in dart is strictly sequential it runs in an event loop on a single thread yet we often want to need to do or want to need to do some work that may take longer than the 16 milliseconds we have to draw each frame such as fetching data from the network or reading a file stored locally for these situations we need concurrency the ability to keep rendering the ui and responding to user input events even while we wait for long-running operations to complete dart provides concurrency to us with futures and asynchronous functions which we can use to initiate some action and then allow us to continue running our sequential programs while delivering the result of that asynchronous action at some later time but asynchronous concurrency is not enough these days pretty much all of our devices are multi-core cpus and sometimes we want to take advantage of that and need to execute long-running code that is not itself asynchronous such as calling a library that does some cpu intensive operations and so will block our sequential program from continuing to run for far too long if we simply execute it sequentially this requires parallelism and for this dart provides us another means the possibility to have sequential operations executing simultaneously this mechanism in dart is called isolates dart isolates are an implementation of the actor model for concurrency which actually dates back all the way to a very famous paper by carl hewitt at all from 1973. isolates are just that completely isolated independent sequentially executing programs isolates have their own event loop and their own heap memory so importantly they share no memory with each other but they are not operating system processors though they very much resemble os processors in their complete isolation from one another being so effectively isolated from each other means that isolates need some form of communication mechanism in order to be useful to us so isolates are in fact only able to communicate with each other through asynchronous message passing the fact that the message passing is async and not blocking means that isolates are not able to interfere with each other's normal execution even via the message passing mechanism itself it turns out that unlike in other languages that use the actor model such as erlang and go isolates do not have an explicit mechanism for communicating in a bi-directional manner between themselves instead there is only the ability to pass an initial message when an isolate is spawned if return communication from the child back to the parent isolate is required by convention the spawn message contains a synport instance which provides the means to send messages to an associated receive port which is listened to by the parent isolate as a way for the child isolate to communicate back with its parent if the child isolate wishes to provide a means of receiving ongoing messages from its parent it can again as a convention send another send port as the first message through the send port that it received listening on that senport's receipt port for future messages this two-step dance provides a means to set up bi-directional communication between parent and child isolates while this sounds like a cumbersome process it's really only a very small piece of boilerplate code that's required as you can see here but it's not only the ability to do parallel processing that isolates give us because of their strictly isolated nature we can use isolates to make our apps much more reliable and robust in the face of errors if we look at the isolate api we can see that we are provided with callbacks that can inform us when an isolate encounters an error and when it exits either normally or due to an error we are also given a means to terminate an isolate via the kill method these methods give us the tools we need to make our apps be able to continue running even in the face of errors let's now look at an example app to see how this can work here we have a simple app demonstrating the use of isolates we have a number of background isolates in this case simulating a computational load by generating a random number between 0 and 99 and then sending it back to the parent isolate while and then sleeping for one second all in an infinite loop in the apps ui each square represents one of these isolates however just like in the real world our app is not perfect and we have errors in our code for us generating the lucky number 13 causes an exception to be thrown while generating the number 99 will cause the isolate to stop sending results and responding to input from the parent isolate when i run the app you see that while these errors begin to occur our app does not crash or stop being responsive even in the face of encountering these unexpected errors in fact we are able to detect both these conditions and display them as such in our ui and in fact it also allows me to restart the isolates simply by clicking on their ui representations of course in a real system we would not have the user play this kind of whack-a-mole and simply restart the isolates automatically via our code once we detect issues but what is important to note is that the code does not have any specific or general exception handling try catch code which if we had used it without isolates could potentially leave our apps state in an unknown inconsistent manner instead we know our isolates are completely independent we are able to stop and start them as we wish knowing that they are unable to corrupt the global state of our app's main isolate while we can see from the app demo just how useful it can be to transition to partition our apps processing into separate isolates to provide a much more resilient app for our users we currently cannot make full use of this kind of approach due to the memory requirements of dart isolates let's look at another app demo now to see where these limitations lie here you can see me running a very simple flutter app that i've built that allows us to create multiple isolates it simply displays the number of isolates that we create and the current memory usage of the app the isolates themselves are very straightforward and simply consist of a function that sleeps for one second fetches the current time and then continues and then continues in an infinite loop to sleep again for one second fetch the time and so forth if i add one isolate worker we can see that the memory usage jumps quickly if i then add subsequent isolates to the running application you can see that the memory steadily usage steadily climbs and is reasonably significant even once we've reached 10 concurrent isolates if we now just start adding them isolates in batches of 10 we can see the memory increase dramatically again where at 50 isolates we're already well over 360 megabytes of memory usage if we then were to attempt to have perhaps even as much as an extra 100 we see the memory usage skyrocket well towards one gig of memory as can be seen from the demo we are currently severely restricted in the number of isolates we can make use of in our apps and so we need to resort to doing things like having only a small pool of isolates among which we do need to distribute and multiplex work making our code more complex than it needs to be luckily we now have a solution to this with a new feature called isolate groups that has has very recently become available in dart and flutter this new feature greatly optimizes the memory usage of isolates let us have a look at our demo app again running with isolate groups enabled to see what a difference it makes now let's have a look at the app again this time running with the latest version of flutter which now includes the brand new isolate group functionality which allows us to use isolates in a much more effective manner by greatly reducing the amount of memory usage that creating isolates contains if we try using one two and more isolates we see that memory usage first grows to a rapid amount just as before but then stabilizes if we then add isolates in groups of ten we see that the memory usage does continue to climb but much much more slowly even when we start adding hundreds of isolates in batches we see that while the memory does climb it again does so slowly and we can get to a thousand concurrent isolates again very simple workers within just a basic while loop return and sleeping and returning the current day time but still we can have a thousand current isolates running at a mere 170 odd megabytes of memory usage with such more efficient memory usage we can even now contemplate using vastly larger numbers of isolates so if we start adding them in batches of 1000 we can see even at 5 000 concurrent isolates we're still below our previous memory usage of 50 or around the same memory usage of the same app running under the current previous version of flutter using a mere 50 isolates and we can continue adding new isolates even up to 10 000 and we're still only at approximately 350 megabytes of usage at 10 000 concurrent isolates which is probably far larger a number than you would normally have in your flutter applications as you can see the new feature makes a huge impact on the memory usage of isolates this table shows measurement of memory usage for basic app we saw running in release mode on desktop linux showing the memory usage for 1 10 100 000 and then 10 000 background isolates first with the existing isolates implementation and then with the new isolate groups functionality enabled from the table we can see just how dramatic the difference in memory requirements with and without the availability of isolate groups is having this feature opens up to us the possibility of using almost whatever number of isolates our app functionality requires without the need to worry about having to limit ourselves to some small arbitrary limit then having to work out how to make use of the small number of isolates while the new isolate groups feature is great it is still a work in progress at the moment so there are some caveats to be aware of in making use of it in your apps for now firstly it's enabled by default currently only in the flutter dev channel secondly isolate groups are only implemented at the moment for release builds and so will not work in mode which is what flutter uses for your debug builds please be aware of this as currently the memory usage in jet mode is an order of magnitude larger than with release mode this talk would not have been possible without the prior work of many people including carl hewitt who first wrote the paper on the actor model joe armstrong and his work on creating erlang and the beam vm sasi uric whose talk the soul of erlang and lixa was a direct inspiration for this talk gillard bracker and his design of the dart language and isolates martin kusterman who's worked on the dart vm isolate groups functionality and who greatly helped me by answering my questions about them and brett morgan who gave me invaluable ideas and feedback for this talk links to the source code of the apps i've demonstrated in this talk are available in this video's youtube description along with links to the relevant documentation for dart isolates thank you very much you

Channel: Flutter Melbourne

Views: 4,496

Keywords:

Id: SXT7nir1B48

Length: 15min 22sec (922 seconds)

Published: Wed Mar 03 2021