MICHAEL THOMSEN: Hi,
I'm Michael Thomsen, and I am a product manager
on the Dart and Flutter team. We're here today to
tell you about when you can benefit from
using concurrency in your Flutter apps. As a developer,
you'll want to create apps with fast and smooth user
interfaces with no UI jank. Sometimes other code can
block the UI from updating. [PHONE RINGING] Sorry. Sometimes other code
can block the UI from updating,
triggering such UI jank. Concurrency can help
you move that code so it runs in the background and
no longer causes interruptions, like when someone interrupts
you with a phone call in the middle of a video shoot. MICHAEL GODERBAUER: Hi,
I'm Michael Goderbauer, and I'm an engineer on
the Dart and Flutter team. As the other Michael
said, concurrency is a great technique for
running code in the background while keeping your
app responsive and animations running smoothly. I'll show you today
how you can offload heavy computations in your app
to a background worker using isolates in Dart. Isolates are a construct
for running multiple threads of execution concurrently. MICHAEL THOMSEN: Our
first topic is synchronous versus asynchronous
function calls and how this relates to the
event loop of a Dart-based app. MICHAEL GODERBAUER: All Dart
code runs in an isolate. Each isolate has a
chunk of private memory and a single thread
running an event loop. Lots of events happened
during the lifetime of an app. Events are, for
example, generated when the user taps the
screen, I/O finishes loading, or when Flutter needs
to repaint to update the UI shown on screen. As new events happen, they
are added to the event queue. When the isolate runs,
it grabs the events from the queue one by
one and processes them on a single thread
in its event loop. For smooth rendering,
the Flutter framework adds a repaint request to
the event queue 60 times a second, one for each frame. It's important that
these are processed by the event loop on time. Otherwise, the
app's UI will become unresponsive and
animations will stutter. MICHAEL THOMSEN:
Here's something I worked on recently, which
illustrates this issue. This is a small app for
doing image processing. You can load an image
from disk or the network, and you can apply image effects
like the sepia color filter. Let's take a look
at the build method. First, we check if
there's a current image. And if so, we render it. Then we have a
progress indicator, which is shown when the show
progress variable is true. This is used to provide feedback
for long running operations like loading and
image processing. Then we have a number of
buttons, like the Load button. Let's take a look at the event
handler for the Load button. This starts by
showing a file picker, and then it does the actual
loading of the image. As you can see
here, then it starts by setting the show
progress variable to true to enable the
progress indicator. Then it reads the
file, and then it unsets the show
progress variable and updates the current image. Here's the running app. Let's start by loading an image. We'll select photograph
and click OK. That's odd. We was supposed to get
a progress indicator. We clearly set the show progress
variable to true in the code, but it never appeared. Michael, can you explain
what's going on here? MICHAEL GODERBAUER:
Let's start by reviewing what happens behind
the scenes when you load an image in this app. Once a file has been
selected, the event loop of the main isolate is running
the code of the load image function, which
as you mentioned, sets show progress before it
reads the bytes of the file, so you might expect that the
UI would paint the progress indicator on the
next repaint event. However, the read as byte sync
call in the load image function runs synchronously and blocks
the event loop until all bytes of the image have been loaded. This can take a few frames. The event loop
doesn't get a chance to process the pending
repaint event until after show progress has been unset again
at the end of the load image method. As a result, we end up with
an unresponsive app that misses a whole bunch
of frames and we never get to see the
progress indicator. If you change the
file loading to await the result of an
asynchronous read call, the loading of the file is
done on a separate VM thread. While that thread is
busy, the main isolate can continue to run and
process repaint events to update the progress
indicator animation. When the VM thread
is done processing, it adds an event to the
queue of the main isolate to signal that the
file has been loaded. The main isolate then
resumes the execution of the load function,
which removes the progress indicator again. MICHAEL THOMSEN: Got it. Let me see if I can make
that change to my app. We will start by reading
the bytes using async. This returns a future,
so we need to await it. Oh, and then whenever
we use async, we need to add the async
keywords to our function. Let's hard reload and try
the load button again. We will take an image, click OK. Great, here's our
progress indicator. And it looks like
here's the loaded image. Thanks for helping me, Michael. A second feature I've
been working on for my app is the ability to perform
image processing, for example, to apply a sepia filter
to add a warm color tone. Here's my current code. First, I set the show
progress state true, then I do the image processing,
update the current image, and set show progress to false. But that's odd. When I click the
sepia button, it looks like the
whole UI is frozen and nothing seems
to be happening. There's certainly no progress. And wait, here's
the finished image. What's happening now, Michael? MICHAEL GODERBAUER: Your app
is doing the image processing synchronously in the
main isolate again. The apply sepia
function never yields to allow the event
loop to process other events like drawing
the progress indicator. The next time the event loop
can process a repaint event is after show progress has
been set to false again. Therefore, the
progress indicator is never actually
drawn on screen. MICHAEL THOMSEN: Based
on the previous example, maybe I should
try and use async. Let me try to wait for
a future, for example, for the duration of, say, one
second and see what happens. Oh, we need to add
async keyword again. Let's hard reload
and try that instead. We'll click the sepia button. Here's the-- wait, we got a
little bit of the progress animation, but then the
whole UI just froze again. MICHAEL GODERBAUER:
In this attempt, the asynchronous
Future.delayed API allows Dart to wait until
the specified time has passed without blocking. In that time, the event loop
can process other events, like repaints, and draw the
progress indicator on screen. However, once the time
has passed the code after the awaited future
is scheduled to run in the main isolate again. While the filter is applied
to the image in that block of code, the main isolate
is blocked for a few frames and cannot service the repaint
events on time to update the progress animation. Hence, we see a frozen
animation on the screen. Just because an
API is asynchronous doesn't mean its work
is done concurrently. It can still end up
blocking the main isolate, as we've seen here. Whenever you have work to
do that cannot be completed between two frames, it's a
good idea to offload the work to another thread to ensure that
the main isolate can produce 60 frames per second for
smooth user experience. In Dart, you can offload
work to another threat by spawning a new isolate. This worker isolate
then runs concurrently with the main isolate
without blocking it. When it has completed
its processing, it can return the result
back to the main isolate. The easiest way to
do this in Flutter is to use Flutter's built
in compute function. This function will
spawn an isolate, pass it some data to
kick off a computation, and terminate the isolate
once the computation is done. While the computation is
running in the work isolate, the main isolate is free
to process other events, like updating the
animations shown on screen or responding to user input. MICHAEL THOMSEN: Great. Let me try that. First, we'll remove
the future code, then we'll add the call
to the compute function. I believe that takes two
arguments, the function to call and then the state to pass
to that function, which would be our image. Let's go ahead and clean this
up, and we can hot reload. Oh wait, this returns a future,
so we need to await it again. Now we can hot reload
and try the sepia button. Great, here's our
progress animation, and it appears to be running
smoothly with no jank. And we get the processed image. Our image processing
app demonstrated how to use the compute method to
move a function to a background isolate. That's a great technique
for short-lived functions, like processing a single image
or parsing a large JSON blob. But sometimes, you
have processing that runs continuously
throughout the lifespan of an app. Michael, do you have
an example of that? MICHAEL GODERBAUER: Yes. I've been working on a new game
where you can play tic-tac-toe against Dash. However, whenever the advanced
Dash AI powering the game is scheming up its
next dashing move, the entire UI freezes
and Dash stops dancing, as you can see here. To fix this, I need to move the
game engine to its own isolate. That frees up the
main isolate to keep the UI responsive and
animations running smoothly. Since the game engine
needs to continuously keep track of the game state,
this is not suitable for a short-lived
worker isolate. Instead, we need to
keep the isolate running for the entire
duration of the game. MICHAEL THOMSEN: In the
image processing app, we applied a filter to the
image with the compute function. This automatically handled
creating a worker isolate, passed the input, and
then returned the result and exited the isolate. This was an example of
a short-lived isolate. For a long running isolate, we
need to do the setup manually. First, we need to
create the isolate. We can then pass messages
between the main and worker isolates using ports. And finally, we can
exit the isolate. Michael, what does that
look like in your game? MICHAEL GODERBAUER:
Here's an illustration of the architecture of my new
and improved tic-tac-toe app. We still have a main isolate
responsible for drawing the user interface. It spawns a long
running worker isolate, which will own the game engine. When we need the game engine to
compute the next move for Dash, we send a message to
the worker isolate. While it is calculating
the next move, the main isolate
can continue to run so we avoid any interruptions
or jank in the UI animations. When Dash move is
ready, a message is sent back from
the worker, which we can process in
the main isolate to show that move on screen. This continues for each
move until the game ends. Let's see what this looks
like in actual code. When the game starts, we create
a new concurrent game engine and the start method is called. This first spawns a
new isolate passing at two parameters, the entry
point for our worker isolate and its input. As input, we provide it with
the sending endpoint of our receive port so the worker
can send messages back to the main isolate. We've also wrapped the
received port in a queue, so we can simply await
the first message from the worker isolate. Typically, the first
message is a send port, which allows us to send
messages back to the worker. With the communication
channels established, we send the first message to
the isolate to start the game and await the initial
UI state as response. The other methods
of the game engine are implemented similarly. We forward the method
call by sending a message to the isolate and
await its response. The actual computation
is done on the worker. This doesn't block
the main isolate, so the UI with all animations
continues to render smoothly at 60 frames per second. Let's shift our focus to the
code for the worker isolate. Its entry point
receives the send port we provided to the
spawn method as input. This allows us to send messages
back to the main isolate. To receive messages
from the main isolate, we create a receive port
and sent its send port back to the main isolate. Next, we instantiate
the existing game engine and listen for incoming
messages on the receive port. For each message, we call
the corresponding game engine method and sent its
return value back to the main isolate
via the send port. That's it. Let's run the game with
the modified engine again. As you can see,
the animations now continue to run smoothly
while our advanced Dash AI calculates its next
dashing move concurrently on the worker isolate. MICHAEL THOMSEN: In
summary, for Flutter apps, the main thread is responsible
for rendering the user interface. To ensure that your UI
runs smoothly without jank, your code must
produce each frame within the frame budget at
least 60 times a second, or even more frequently,
on higher end phones, with faster refresh rates. Any code that takes longer to
run than the frame budget will cause jank, and should be
moved to another isolate. MICHAEL GODERBAUER:
In this talk, we've shown three ways
to run on another thread. For system APIs exposed
in the .core libraries, use asynchronous calls
awaiting the result. For short-lived
background tasks, use the compute function. This takes care of spawning
the worker isolate and handles the communication between
the main and worker isolate. Finally, for a long
running background tasks, use the isolate APIs to
spawn a worker isolate. The isolate doesn't
share any memory-- they are isolated. So to communicate between them,
you send messages via ports. MICHAEL THOMSEN: You can
find more information and both app examples using the
links shown under Resources. Thanks for watching. We hope these tips will help
you create user interfaces that run smoothly with no jank. [MUSIC PLAYING]
