[MUSIC PLAYING] BRANDON DEROSIER: Hi. I'm Brandon. I'm a software engineer
working on Flutter Engine. LEIGHA JARETT: And I'm Leigha,
a product manager for Flutter. If you tuned into
Flutter Forward or you've been reading
our release blogs, you've probably seen the
name Impeller thrown around quite a bit. But what actually is Impeller? And how does it
work under the hood? Well, today you're
going to find out. Impeller is a new renderer
within Flutter's Engine. Until now, Flutter has been
using something called Skia. The problem is that Skia
wasn't designed for Flutter. It has a ton of
rendering features built for a wide range
of devices, which means that it's not always
optimized for Flutter's needs. Enter Impeller,
Flutter's new renderer. We built Impeller to
specifically focus on the rendering needs
of Flutter applications. And our main goal
was to eliminate jank or any stuttering that's
happening inside of your app. This way it'll always look and
feel great for your end users. Now for everyone who's watching
who's not a graphics engineer, you might be wondering,
what's a renderer. Well, a renderer
is software that helps you translate your
UI code into the pixels that you actually
see on the screen. So say you have a simple
Flutter app like the one that I'm showing here. In the Flutter framework,
your tree of widgets is backed by a tree
of render objects. Render objects contain
the instructions for how to actually layout
and paint the widgets. These instructions are
given to the engine and stored in an ordered list
of simple commands called a display list. And now this is where
things get interesting. The engine is
responsible for using one of the available renderers,
either Impeller or Skia, to draw this display list
to a surface texture, or a grid of pixel values that
can be displayed to the screen. It leverages the
GPU by setting up a collection of things
called render pipelines that can be used to render
everything in the display list. So let's dig into the details. Before we can use
a render pipeline, we first need to take all the
paths drawn by the display list and tessellate them
into sets of triangles. Then each vertex or
point on the triangle is passed through something
called a vertex shader. Now a shader is just
a small piece of code that gets executed on
the graphics device. And in this case, the
shaders are internal, meaning that they're a part of
the Flutter engine's code base. But us Flutter developers
can author our own shaders too so more on that later. So the vertex shader
takes the vertices that make up the Flutter logo
and it moves them to the place on the screen where we
actually want to draw them. And now the Flutter logo
is in the right place and we begin iterating
through all the triangles. So let's zoom in a little more. For each triangle, we figure
out the specific pixels that are inside it. And this is called
rasterization. Then a few checks are run
for each of these squares to determine if we actually
need to compute a color for it. For Impeller, these
checks are really important so we'll be coming
back to this in a bit. So now all those pixels
that are in the triangle are pasSED through
a fragment shader. A fragment shader is
another snippet of code but this time it takes
the vertex shader's output and computes a color. And like I mentioned
before, Flutter developers can override this by
creating our own shaders to create really neat effects
using the fragment program API. Anyway, back to our
render pipeline. Lastly, the output colors are
blended with the color that's already been drawn. In this case, our
blending was really simple and we just replaced the color. And that's how things
get rendered on the GPU. Now over to Brandon
to talk through how these render
pipelines are actually implemented in Impeller. BRANDON DEROSIER:
Thanks, Leigha. Impeller has a
layered architecture where each layer uses the one
below it to accomplish its job. Before, Leigha showed an
example of how the Flutter logo widget gets converted
into a display list. In this example,
the display list just contains a few
operations to position it in the center of the screen
along with four draw path ops, one for each of
the quads in the logo. So let's take this display list
and draw it using Impeller. First, all of the
display list operations are dispatched to
something called Aiks. Aiks is the topmost
layer of Impeller. And it mainly consists
of a canvas drawing API. By the way, the name Aiks
is just Skia in reverse. The job of Aiks is to
take high level commands like Draw Path and Draw
Image from display list and turn them into simpler,
self-contained drawing operations called entities. And this is where the
next layer comes in, the entities framework. In our example, the four draw
path ops from the Flutter logo each end up getting an
entity of their own. Each entity contains
a bunch of properties that every single
drawing operation needs, like a transformation
matrix that encodes the position,
rotation, and scale. Every entity also
gets a content object assigned to it, which contains
the actual GPU instructions needed to draw the entity. There's a contents for
drawing solid colors, images, gradients, text
clips, everything that Flutter apps can draw. Sometimes we even have
multiple specialized contents that draw the same thing
using different algorithms with different performance
characteristics. That way Aiks can
pick and choose the most efficient
rendering algorithm depending on the situation. In this case, all four of
the Flutter logo's quads are just filled
with solid colors and so each of their entities
get a solid color contents assigned to them. Now we need to spell out
these instructions in a way that the GPU can understand. Because Flutter apps can run on
a bunch of different platforms, Impeller needs some kind
of translation layer to communicate with the GPU. This is where the layer
below the entities framework comes in. I'll refer to this as the
hardware abstraction layer. This layer is just
a thin abstraction that talks to the
graphics driver through various
standard graphics APIs like Metal on iOS
and Vulkan on Android. Now we're ready to start telling
the GPU exactly what to do. Remember those render pipelines
Leigha mentioned before? This is where those
come into play. Each contents uses the
hardware abstraction layer to draw itself by instructing
the GPU to execute the render pipeline containing the shaders
we actually want to use. And really that's it. Those commands are executed
using the graphics API and then the resulting texture
gets displayed to the screen. But there's one huge detail
I haven't covered yet. Those shaders and
the render pipeline also need to get compiled
down to instructions that the GPU can execute. And this process
is very expensive. In Skia, this
compilation process happens at runtime
right on the frame. The pipeline actually needs to
be used to render something. This usually causes the
frame to go way over budget, resulting in a
noticeable stutter. We often refer to this problem
as shader compilation jank. Impeller vastly
improves on this issue by performing the most expensive
part of this compilation ahead of time. And this is where the last
component of impeller comes in. When the Flutter
Engine is built, all of Impeller's
shaders get compiled into bundles using Impeller's
offline shader compiler called Impeller Scene. In walking through
Impeller's architecture, you've seen that Flutter really
has complete control over how it renders graphics. Flutter isn't subject to
the architectural or design limitations imposed by
platform-specific UI toolkits because Flutter
doesn't even use them. Instead Flutter's
engine talks directly to the graphics driver. This is what allows you to
build rich, unique apps that behave exactly how you
want on any platform. This lack of dependencies also
makes Flutter a great solution for embedded use cases like
automotive dashboards, IoT, and digital signage. LEIGHA JARETT: Now that
you have an idea of what's happening under the
hood, let's highlight some of the key
architecture decisions that we made that makes Impeller
a great renderer for Flutter applications. So like Brandon mentioned,
one huge benefit that Impeller offers
is eliminating jank from compiling those
programs called shaders. To do so, Impeller
doesn't generate shaders in the same way that Skia does. Instead impeller has a set of
handwritten shaders compiled in advance. But you might be worried
that initializing a bunch of precompiled shaders
could mean slower startup times or bigger Flutter app sizes. To prevent this, Impeller
uses alternative rendering techniques that leverage
a much smaller, simpler set of shaders compared to
the many specialized shaders that Skia dynamically generates. Another reason why Impeller
offers great performance is the way we implement
anti-aliasing. Anti-aliasing is the
art of smoothing out jagged edges of drawn elements
so that they look more natural. Impeller uses
something called MSAA to perform this work, which
is present across all devices supported by Flutter. It's pretty cheap to perform. And it produces
excellent quality results on modern mobile hardware. Another thing we
focused on is how Impeller implements clipping. When rendering widgets,
Flutter frequently uses clip masks
to cut out shapes so it's really important that
this process is efficient. Impeller specifically
takes an approach that leverages hardware
features to ensure that clipping is super fast. Remember earlier when I
mentioned those special checks? One is for something
called a stencil buffer. Now we won't go
into too much detail but basically, as Impeller draws
the pixels for the clip thing, it tells the GPU to
check the stencil and filter away all
the pixels that aren't part of the filled shape. This means that even if a
clip is really complicated, like the ones used to create
these kaleidoscope animations, it's still a pretty
cheap operation so we can see some really
huge performance improvements. On the left hand side, we
see Flutter pre-Impeller. And it's rendering at about
seven frames per second. And on the right
side, Impeller renders at about 60 frames per second. BRANDON DEROSIER:
So we just talked about some of the design
choices in Impeller. For most Flutter
developers, this means you'll see more
predictable frame times when you use Impeller. But for those that
are interested in more advanced rendering use
cases, Impeller also aims to offer a launchpad
that allows you to solve for other graphical needs. For example, earlier this
year at Flutter Forward, we showed off a proof of concept
3D scene graph built directly on top of Impeller's
hardware abstraction layer called Impeller Scene. Impeller Scene has its own
contents in the entity's framework and it
draws 3D objects using the same kinds
of render pipelines we talked about earlier. Now there are tons of
consequential decisions that go into building renderers. And while we can certainly build
3D renderers that are useful and make smart tradeoffs,
no one renderer can solve every
use case perfectly. And so as a next
step for this, we're actively experimenting with
ways to surface more control over rendering in
the Flutter framework so that anyone in
the Flutter community can build their own
rendering packages and create custom
render pipelines that integrates seamlessly with the
rest of the widget ecosystem. Impeller's scene is
a proof of concept and a showcase for
the kinds of developer experiences we're seeking
to enable with this effort. With that in mind, let's take
a look at the demo widgets. Basically, everything
is centered around a special widget
called scene box. Scene box takes a node
which can be created from a 3D model that's added
to your Flutter app's asset bundle. Nodes can be composed
together to form a tree just like normal Flutter widgets. Here I'm wrapping the
asset with another node that applies a rotation. You can add the same asset
to the scene as many times as you want. And Flutter will know to only
load the asset once and reuse it to inexpensively
render copies, kind of like how
image assets work. And you can even
play animations built into the asset that were
authored using 3D modeling software like Blender or Maya. And since Scene box is just
a regular Flutter widget, any of these node or
node animation properties can also be animated
in exactly the same way that you would animate anything
else in your build methods. And of course, hot
reload just works. Simply export the file from
your 3D modeling software and Flutter does the rest. With just these tools,
it's possible to construct rich interactive scenes composed
from many different assets. Again, this is the kind
of fluttery experience we're seeking to
enable as we surface more control over rendering
thanks to Impeller. LEIGHA JARETT: So we just gave
you an overview of Impeller. And we went under
the hood to show just how Flutter renders your
widgets to the device screen. But that's not all. Today we're super excited to
announce that Impeller is now the default renderer on iOS for
the stable version of Flutter. So upgrade Flutter
and try it out. And if you want to see
Impeller in action, you can install G.
Skinner's Wondrous app. Right now Wondrous
uses Impeller on iOS. But we're looking forward to
the next version of Wondrous which will include Android
support for Impeller. We're really excited about
where Impeller is headed and the new rendering
capabilities that we can start bringing to
Flutter and we hope you are, too.
