[MUSIC PLAYING] THOMAS NATTESTAD: Hey, everyone. My name is Thomas. And along with my
colleague, Tom, we're going to be covering
how WebAssembly finally enables the dream write
once, run anywhere. I'll start by covering some
of the basics and motivation around cross-platform
development as well as some of
the specific partners that have found success
with this approach. Then I'll hand it off to
Tom to show some sample code and how to get started. [MUSIC PLAYING] Most of you probably know what
cross-platform development is. But to briefly review,
its computer software designed to work in several
computing platforms. In a traditional per
platform approach, you have to build your code
distinctly for every platform you wish to target. Whereas in a
cross-platform approach, you compile to
some intermediary, for example, the
Java virtual machine, that then further targets
the other platforms. This means that the
intermediary only needs to get ported
once per platform, and then all the code that
targets that intermediary can be run there. This dream of write
once, run anywhere is something that has been
chased by many in the industry since basically the
dawn of computing. Now, you might be
asking yourself why we're talking about
cross-platform in a Chrome talk. The web is, after all, the
cross-platform solution that can run on any
operating system. On top of that, you
can make your web app even more native with
technologies like PWA and advanced capabilities,
which is a truly great solution and a viable option. However, the big drawback is
that it limits you to JS, HTML, and CSS on all your platforms. This can be a big
limiting factor if you have existing
code in another language, want a fully
natively written app for some platform like
Android, or just simply prefer writing in another language. This is where WebAssembly
unlocks your ability to use the same non-JS
code from other deployments to also power your web app. [MUSIC PLAYING] So let's briefly cover
some of the advantages of cross-platform development. One simple reason to use
cross-platform development is to reduce costs. By using a single code base
for all your platforms, you avoid having to
re-implement every feature. And you can bring
existing investments from previous
platforms to new ones. One example I love here
is the Google Ink library, used for low latency drawing
in a variety of Google applications. They had dozens of engineering
years invested into this library and told us that without the
ability to compile to Wasm, they just simply wouldn't have
a web solution for Ink, which would have left all
of these products without features enjoyed by
their native counterparts. Another way
cross-platform development can save you cost is by enabling
you to use the same code client and server side,
such as with AI. Running AI on the
server can be expensive, and many computers can handle
the operations locally. So by enabling the same
code to run on both, you can dynamically decide
where to run the code and even shift that over time. Here you can see
Photoshop Web performing AI-driven client-side
object selection, while other more intensive
operations like Generative Fill are still done on
the server side, though this could also
shift in the future as client-side
technologies develop. Another advantage
is Feature Parity. When you use a
single implementation to hit all your
platforms, you also get the same features
on every platform. Google Photos benefited from
this with their photo editor, which uses C++ for
all their deployments. This enabled them to quickly
bring all the features from their native
app to their web app and keep those features aligned
across deployments as they developed. Goodnotes similarly told us
this quote, which I really love, that illustrates
the advantage well. "Every day our iOS developers
contribute something new to our Swift codebase, and our
web app benefits from that." Lastly, aligning on a
cross-platform technology allows you to
future-proof yourself. It's tough for anyone to say
what futuristic platform might emerge. But by untethering yourself
from any specific platform, you safeguard your investments. Right now, C++ is the most
mature language for targeting Wasm. And most of the previous
examples you saw used C++. Part of what makes C++ such
a great choice is its high performance, especially when
paired with threading and SIMD. Wasm, along with SIMD,
is used in XNNPACK, which is the backing for
both TensorFlow and MediaPipe for on-device AI operations. It's responsible for enabling
blurred or altered backgrounds, as well as other
fun effects in Meet. It's also responsible
for noise cancellation. And you can really hear the
difference in this example. [AUDIO PLAYBACK] - Can you hear me clearly? - Just kidding. - It's critical that we update
the figures in section 11.7 by 9:00 AM tomorrow. [END PLAYBACK] THOMAS NATTESTAD:
Another recent feature that really impressed me was
AI-driven studio lighting allowing you to set
up artificial lighting with different
colors in your scene. All of these features are
performantly enabled on the web thanks to WebAssembly. This was just a brief overview. But we have an
entire talk dedicated to AI on the web
with my colleagues Deepti and Austin
that I encourage you to check out through
the link in the description. Moving on from C++, there are
now some additional language options that can
target WebAssembly, enabling you to use
them on the web. While there are many
languages targeting Wasm now, I want to focus on
Dart, Kotlin, and Swift. Before going into each language,
a consistent theme you'll see is whether to use cross-platform
development for shared business logic or also using it
for shared UI logic. So let me explain each. In the shared business
logic approach, you use some common piece
of code on each platform but still remake
the UI per platform. In the example of Goodnotes,
they use their shared Swift code for the central document editing
but recreated the surrounding UI in React. Photoshop, Google
Photos, and many others used a similar approach. In contrast, the
shared UI approach uses the exact same
code across platforms to generate identical UI. Sharing UI can clearly
be a huge time saver. But currently, almost
all of these approaches paint to a canvas by shipping
a rendering system along with the web app. This gives you great
consistency across platforms but also means a
larger bundle size and does impact
some functionality you get from the DOM, such
as accessibility, extensions, auto translate, and
other browser features. Coming back to each language. Flutter uses the Dart
language to deliver beautiful, high performance
experiences across mobile, desktop, and the web. Flutter on the web
uses WebGL to create user experiences that are pixel
identical to native platforms. Now that Dart targets WasmGC,
Flutter web applications have faster, more
consistent timeframes. Check out the "What's
New in Flutter" talk to learn more
about Flutter, Dart, and WebAssembly through the
link in the description. Another language
option is Kotlin Wasm, which also uses WasmGC
and is now in alpha. Kotlin Wasm also
allows you to share business logic, such
as from an Android app or reuse your UI logic
through Compose Multiplatform. Lastly is Swift and the
Swift Wasm toolchain. I covered some additional
detail in our last year I/O talk, which you can reference
for further specifics. But this is a
great choice if you have existing Swift code that
you'd like to reuse on the web. It supports both
shared business logic and has support for SwiftUI
via the Tokamak project. Tokamak lets you reuse your
existing SwiftUI logic. And unlike the other
shared UI logic solution, it renders DOM elements,
which sacrifices some pixel perfection for
the benefits the DOM offers. Now I'm going to hand it over
to Tom to show you how to get started. THOMAS STEINER: Now after seeing
all of these amazing examples, you might be wondering how to
get started with your code base. Obviously, your code
base is your code base. And I can't just look into it
and understand the constraints you may be operating under. This is why I decided
to abstract away the low-level concepts and
simplify the overall ideas based on a work example app. Meet Delayedgram. It's a fictitious iOS and
Android photography app. Don't actually look for
Delayedgram on the App Store or Google Play. But just pretend it is
downloadable from these stores. Delayedgram is all about
award-winning image filters like invert or grayscale. The user interface
of Delayedgram on iOS is implemented with SwiftUI. Apple presents SwiftUI
as a modern way to declare user interfaces
for any Apple platform. On Android, Delayedgram's UI
is built with Jetpack Compose. Jetpack Compose is Android's
recommended modern toolkit for building native UI. The award-winning filters,
invert and grayscale though, are implemented in C++ for
maximum portability and the best possible performance. Take the grayscale C++
file as an example. In its main function,
it checks if it's called with two arguments, the
name of the input file, argv1, and the name of the
output file, argv2. By convention, argv0
is just the name of the command with which
the program is invoked. It then calls the
convertToGrayscale function where the real filter
action happens. The function first
obtains input and output streams for both files. It then reads the magic
number of the ppmp3 file that the filter
expects and writes it straight into the output file. Same for the dimensions, that
is the image's width and height. The next value is the maximum
color value, which in this case will always be 255. Finally, the code hits
the actual filter logic where the secret happens. Here, using some magic
numbers, remember, it's an award-winning
filter, the code converts the red, green, and
blue color channels to grayscale and writes the values
to the output file. C++ compiles natively
per platform. You can actually take
each Delayedgram filter, compile it, and run it
locally in isolation. I've compiled both filters
on my Mac to illustrate this. You can see that I can provide
an input image file name, a desired output image
file name, run the program, and out comes an image
with the filter applied. Award-winning, right? So how can Delayedgram,
an iOS and Android app, use these C++ filters from the
native world that you've just seen me run on my Mac? On Android, it uses the Android
Native Development Kit, NDK. IOS apps can get access to C++
through a platform-specific objective C++ interface layer. And this is exactly
what Delayedgram does. Android and iOS only can get
Delayedgram that many users. But there's a ton of untapped
potential for new users by bringing
Delayedgram to the web. Nothing is more frictionless
than clicking a link. So what's needed to bring
the full Delayedgram experience to the web? For the user interface,
of course, on the web, there's HTML, CSS,
and JavaScript. The real question is, How
can the award-winning filters be used on the web? This is where
Emscripten comes in. Emscripten is a complete
compiler toolchain to WebAssembly using LLVM with
a special focus on speed, size, and the web platform. It allows the Delayedgram team
to compile the C++ filter code to Wasm so it can
run on the browser. After some experimenting,
the Delayedgram team settled on the following
compile command. emcc launches the
Emscripten compiler. -03 optimizes for production. grayscale.cpp is the C++
input file, in this case, for the grayscale filter. And -o grayscale.mjs
tells Emscripten to compile to a JavaScript
file with that name. By using the dot
MJS file extension, Emscripten automatically
creates an ES6 module. Delayedgram runs filters
exclusively in web workers, so the environment
is set to worker. sFILESYSTEM enables
file system support. The exported runtime
methods are FS for file system
access and callMain to call the main function. Finally, INVOKE_RUN=0 makes
sure that the main function isn't automatically called
but only when needed. After compiling the
filters with Emscripten, they are then ready to
be used from JavaScript. To not block the main
thread, each filter runs in a web worker. The filter gets the
image's pixel data as a string in ppmp3
format from the main thread straight from the camera. It converts it to a
file and writes it to Wasm's virtual file system. So the filter can access it from
the Wasm filter's main function just like on the
command line before. After calling the Wasm
filter's main function, the result is in a
uniquely named file in Wasm's virtual file system. So all that remains is
taking it out from there and sending it back
to the main thread so it can be drawn on a canvas. This is Delayedgram
running on mobile as an installed
progressive web app. Look at those gorgeous
award-winning filters now running in the browser. You can try this at home. Just point your browser at
goo.gle/delayedgram and play with those filters brought
from C++ straight to Wasm in your browser. Next, this is Delayedgram
running on desktop with Chrome DevTools open. Here, I'm looking
at the Network tab. And I'm inspecting the
grayscale filters Wasm file. Had I compiled with
Emscripten debug flag, I could even debug the Wasm code
map back to the original C++ code thanks to the dwarf
debugging standard. And with that, I want to
hand it back to my colleague, Thomas, for some closing
words and takeaways. [MUSIC PLAYING] THOMAS NATTESTAD: Thanks, Tom. So just to summarize
some of the takeaways, Wasm enables
cross-platform development by supporting languages
like Kotlin, Flutter's Dart, and Swift on the web. Many products are already
getting the benefits from cross-platform development
like cost reduction, feature parity, and a future-proof
platform strategy. And it's easy to get
started with some of these different toolchains. Try them out for yourself. Just search for any of them
and dig into the documentation. Thanks, everyone. [MUSIC PLAYING]
