Flutter, Dart, and WASM: Shipping a new model for Web applications by Kevin Moore @ Wasm I/O 2024

[Music] do I sound okay oo I'm really loud okay alsoo now I'm here to talk today about Dart flutter and web assembly a model for building web applications first info about me I'm a product manager at Google I just passed 10 years it's gone really fast and the whole time I've been on the dart team so it's really been cool to watch The Evolution when I started on Dart there was no we assembly we were trying to actually ship Dart as part of the Chrome browser um Native client was a thing and so it's amazing looking back the uh progress we made I'd say actually philosophically and we can talk after if you're curious um the days when we thought about adding Dart to the Chrome browser um I really feel like web assembly is kind of where we wanted to go which is how do we do better faster um in the browser and the fact that we have a standard thing now just makes me super excited so you can find me kevm on most of the things on the internet even the blue skies and the mastadons if if you're curious um and there's my weird vanity domain which is mostly just links to all that stuff plus talks I've given if you want to learn more so Dart and flutter how many people here are familiar Dart and flutter how many have actually implemented anything with dart and flutter okay good to hear that we have some fans here um I'll do a brief introduction of both because I'm guessing a lot of people here don't have some deep context and then we'll talk about web assembly so first Dart uh a programming language meant to be approachable portable and productive for apps on any platform so here's the canonical Hell Out World um pretty short and sweet last year I went through a bunch of kind of basic language features and I didn't want to repeat myself this year but I think it's good to point out some things here we're object oriented plus a bunch of really great functional features the syntax is very similar to folks that use those languages listed there we have a sound type system which we're very happy about um including no safety which was added in the last year when stable so our n safety thing is not just a semantic or syntactic sugar actually is sound null all the way down um which adds a lot of benefits happy to talk more about that um we have Futures in streams records and patterns and are very rich model for inference which is great for running code quickly I want to give a preview so this is a feature we just added in last year everyone's familiar with switch statements right you might switch over strings or you switch over integers or switch over Eno values um this is a feature where we're switching over Jason a map and so look just real careful at the code so you see the switch statement on the top line we're switching of adjacent and now the match for the switch statement is not a string literal or an integer an enum it's actually a map literal what this is saying is match any map that has the key type with a value text question and the next four things are saying you know match over a string category a map a string a string but it's not actually just matching over the values it's saying match over the type and then capture the value and then below I can instantiate an object there and so extending the notion what we can do in switch was something we added in the last year and it's been super productive so we keep evolving and pushing Dart and I think it's super useful um another fun example of where we're using new language features how many people have written code like this where you kind of have a nested if block dealing with a bunch of booleans um if you actually count all the booleans here there are five and so if you do you know the combinatorial math 2 to the 5th there's actually 32 possible cases you have to handle right for these five booleans are active and correct and you know all the other things and so this is actually tricky like have I you know in this case I've actually covered the is active case twice and it's tough for static analysis to tell that you've missed a case or you might have duplicated one or forgotten one um this is the feature in our new Switch syntax and this is actually using a feature we have called records so you see here is I'm actually instantiating a record that's just a Boolean set of values so positional five Boolean values instantiating This Record C we call this Anonymous types and then I'm switching over it and look carefully so what we're doing is switching over all the possible values that could exist in this record and since all the fields are booleans we know all the type possible values that can exist there and actually statically we can guarantee that you'll hit every case so if you're not if you don't care about the case like you only care about is active true and is pressed true you can say I ignore those other values but in this case we'll actually give you static analysis that if you forget a possible value will yell at you and say you've missed a case just like if you miss a value in an En num and if you duplicate a switch context will yell at you too so I just wanted to highlight some new features we've haded in Dart that have added a lot of good ergonomics um and super useful so we keep evolving the language quickly um if you're curious about just new things we're doing in Dart how we're pushing the language I highly recommend watching this talk and one feature that's actually brand new as of February is something called extension types and so this is the ability to define a type that's basically a zero cost wrapper so I can say I want to make an extension type on Integer in this case and maybe if I'm doing an ID I don't want people doing math on the ID I don't you adding two IDs together or subtracting them I just want to care about equality and so you have a feature that is much more ergonomic and safer than using um an integer for an ID or maybe I'm trying to think a string for URI for instance but fundamentally cheaper than doing a wrapper value and actually we use this in interop that I'll talk a little bit later so that's just some stuff about Dart um we don't just use it for flutter our common front end for all our languages written in in Dart along with our analyzer and our formatter so thousands of lines of code um both of our JavaScript compilers and a WM compiler are also written in Dart um our package site which I think at Max handles 200 QPS um I think more now is completely in Dart um and you can compile Dart code aot to a really tiny binary and have a single binary to deploy which is actually great for containers you can actually have a 10 megabyte container um and do you know millisecond um cold starts on containers and dart so this is actually tweeted yesterday I'm like I'm going to totally steal this tweet and publish it um usually when you think of Dart you think of flutter but we used all a bunch of places um and um I think there's at least three production companies now basing on offering server offerings in Dart which again is a small number of total Dart users most of our users are flutter um but it's exciting to see people using it in other places so if you want to go try out Dart dart. deev and dartpad doev if you want to hack some code together now you can do that right now um it runs online so let's jump to flutter this is by far the most popular framework for writing um or the most popular usage of Dart um a framework for building apps beautiful fast productive open on any screen so flutterdev is the URI um this is actually a really great slide for me to do because I was looking at my slides from last year and last year I said 700,000 and we've topped over a million I think we're actually closer to 1.2 million I don't have the exact number um but the number of apps we have in stores is substantial um the percentage of new apps on like the Apple I store or the apps store um I don't think I'm allowed to see the actual number it is really surprising like there a lot of new apps I'm sure most of you with a mobile device have a flutter app on your phone and you just don't know it um we're used all over and over 30 teams at Google use flutter um so not YouTube itself but there's a YouTube create app out in beta um is uh uses flutter Google Earth classroom a bunch of things um um Family Link which I use all the time with my kids pieces of the cloud offering on mobile devices all use flutter and usually you think about doing three bullets we're crazy and do five in terms of kind of the things we're happy about beautiful fast productive portable open all open source um I want to talk about two of these today which is portable and fast first portable being able to run anywhere so most people that have done flutter we started out was mobile apps Android and iOS um but you know so not just arm but actually we now have support for risk processors which are excited about obviously deploying to web with JavaScript today I'll talk about web assembly you can build desktop apps Windows Linux Mac and even embedded scenarios um uh I still don't have the tweet from last year if someone is actually written and running um flutter on a lanyard which is kind of crazy Toyota uses us for um infotainment systems and cars um we run everywhere um my favorite example and actually this is something that I knew about last year and couldn't talk about but this released in October uh Google Earth recently ported all of their common UI to flutter I'm going be very careful there so obviously Google Earth has a 3D rendering engine that was written in C++ but the problem was they had this common code that could use across all the mobile devices and the web with web assembly but all the common UI you know the widgets you see on the left and the right here they had to rewrite in three times in three different platforms and of course this is tough for a team you know have to maintain it stay consistent and they were able to substantially reduce the amount of code they had greatly incl increase their velocity because now all this common UI you see the buttons on the bottom and the side panels that's all written in flutter so this is a great example of of flutter web and actually so if you have Google Earth on your phone or you go to earth. google.com um you're running a C++ app with um Chrome written all in flutter so besides portability fast is important we want something close to Native speed and I highlighted this app last year um Wonder us we built this as an example of be able to have high fidelity apps that run on iOS frankly and Android um but this year I thought I'd actually show you so here's the wondrous application running on my desktop you know I'm doing a swipe gesture oops wrong swipe gesture two fingers there we go so I can swipe between these um famous um places in the world you know you can see the pretty animations even like the little flipping around here Integrations with things like YouTube and Google Maps so this is all written in flutter and running on desktop alth actually it is not a desktop app and I did the same same thing last year and I still think it's funny um this is actually a web app so they be to have a super high fidelity beautiful user experience um the kind of customization you would really want something that looks really good native and be able deploy the web that's what we do with flutter we're super excited about it of course now my presentation's messed up let's see if we can get it going so I invite you to go now although if you go to a mobile device on the web you might notice that the per performance is not spectacular it's actually pretty solid for a JS thing but we want to have this kind of Rich experience be really beautiful and fast and feel native even on mobile web and so how do we do that well actually before we get into that though I'm going to say if you want to try out flutter as well um darpet Dev also has flutter content and so this is an example of a 2d game engine called Flame that's available so if you want to try out if you want to try out what our copertino witchet are you everone see the date time picker in a web browser on a Linux laptop you can do it on dartpad um so if you want to play with dart and flutter dartpad is the place to go but now I want to talk about web specifically um first a slight aside about sites versus apps and this is actually really important in terms of how we think about what's important and how we target things with flutter web um I won't read through all of these but like um if you Google the term um bodal distribution um I think there actually kind of is um two Islands here that matter certainly you can switch between the two but I think there is this notion of kind of a traditional notion of what we think a site is like I think about Wikipedia um which is text based more linear more static not so much user content or user state right um maybe not so much code and then on the app side often much more than text it's interactive you write lots of code to maintain it you have user state so I think there is kind of like two clusterings of experiences on the web here and I'd say you know most content on the web is still sites but being able to deploy a rich application experience is super valuable on the web um and that's what we're really focused and so certainly when you think about sites you want to load really fast you want to be indexable um you want the broadest reach possible and on apps it's really about having flexibility the type of experience you can offer the richness of the experience and maybe portability the ability to actually deploy it natively on desktop or web or desktop or mobile and so obviously HTML and CSS JavaScript were great on the site side and what I'm excited about especially in the flutter team and we're seeing this with other Technologies as well is thinking about a new paradigm for experiences that are based on accelerated Graphics so this is using canvas webg and hopefully in the future web GPU and web assembly and flutter web specifically is for applications so when you think of sites the examples I'll bring up you know not surprising are our sites so if you go to Dart dodev or flutterdev or a package site pub. these are unapologetically HTML and CSS and JavaScript I think in some cases we actually Source some of the JavaScript in Dart so we compile the dart to JavaScript but they're Dom sites they're HTML sites and that's perfect for this scenario this is documentation this is you know a search engine but you think about rich experiences so I showed you wondrous um the developer tools that we have for flutter are all running in flutter web um we have companies that are betting on us this a super list is an amazing to-do app collaborative to-do app RVE is actually mentioned earlier does an amazing um has a whole uh 2D Vector animation experience that runs completely on the web you can install natively as well and things like Google Earth these are apps so we don't care necessarily as much about indexing these maybe your homepage w't indexable but you know I don't want my bank content indexed right necessarily and so um and maybe it's okay if you have a splash screen and a little bit of load time because you have a rich experience and you're going to spend minutes or hours in the experience and so this is kind of the place for targeting and let's see how we make that better with web assembly so today we talk about Google Earth um and some of the other apps we talked about um we're compiling to JavaScript so we do have a low-level Library called skia and this is actually we're in the process of migrating some of this right now to a new technology called impeller but at least on the web right now we're using a technology called skia This is actually the 2D rendering engine that exists in Chrome if you're curious um we compile that to web assembly and then we take all of the engine code the framework code for flutter so you think animations all of our layout all of our widgets all those things that's all sourced in Dart along with your application code or packages that you use and that now is all compiled to JavaScript and what we're excited about is moving to a world where that's web assembly so we still have the canvas kit bits in this case we call it sasm this is what happens when Engineers name things that's okay and that's linear memory WM and as we discussed last year and I think Thomas and Tom from the Chrome team will talk tomorrow the GC stuff is where we're targeting um Dart and we're excited that as of I believe it was November both Firefox and chrome have stable support for GC so why would we do web assembly implementing an entire compiler is not easy and we did all this work for JavaScript so why' we do it performance and in some cases correctness we'll get to that later but first performance so this is a actually our performance dashboard we're running the same app I think it's a material demo we had we still have an HTML renderer which actually you would like to get rid of because there's ACH reasons for that um but I'm comparing performance here so this HTML renderer we have Legacy our current GS implementation and then web assembly and as you see here we're over twice as fast for initial load the unit of time is microsc so we're talking about you know 100 versus 50 milliseconds so not only is it definitively better and this is across multiple releases of or multiple builds multiple commits effectively a flutter notice the noise as well so obviously if you scale that up all the little bumps will get bigger but the the bumps are much higher in the JS version and obviously performance benchmarks there's always a little bit of noise so not only is web assembly faster it's consistently faster it's reliably faster you don't have as much bumps in the performance and in fact you can see this I took the uh the Wess app I showed you earlier I made a custom build that actually deployed to web assembly um someone on our team made this little Benchmark button we actually ran through about 10 seconds of slipping between um flipping between screens scrolling doing all the transitions so about a th frames we captured and then we looked at the analysis so as you see here for so p50 p80 you guys know what I'm talking about when I say this you know the 50th percentile so the median frame time is about 60% faster with web assembly and what's really exciting is when you get to P90 P95 talking about 110% 120% faster so obviously if you're trying to Target 60 FPS 15 16 milliseconds is your goal right like that's how you don't Jank when you're rendering frames um obviously on my M1 Mac you know 8 milliseconds is fine you start talking about mobile devices you know the difference between 15 milliseconds and 30 is the difference between janking and not janking and so you have so much more overhead on the web assembly version and what this means is either you can do more on the more interesting animations you know more interesting effects richer content with the exact same behavior or or you can go much lower on your devices you Target and still have a great user experience um we've done some of the benchmarking with this wondrous app on a six-year-old Chrome device a Samsung device um and it was amazing the difference you can it's night and day what you can see with web assembly um so one trick we do is obviously just web assemblies faster the second thing is and this is one of the benefits of using um canvas based Graphics accelerated Graphics is actually we can do off- thread rendering which is kind of crazy to think about doing on the web so this is a zoom in it's kind of hard to parse I apologize but um you can see the main thread and then the worker thread and it's not all of the work but a huge chunk of work we're actually able to do off thread and of course anyone who's done any UI programming especially on the web a lot of things end up happening on that main thread and that means you're blocking UI you're janking and the ability to actually offload rasterization to a separate thread means you just have a much more interactive much more smooth user experience and this is the benefit of using Hardware Exel graphics on the web and web assembly the second thing we want to talk about is correctness so here's a quiz A or B which column is correct for this High bit math I'll take a shout out and you don't need your phone B is correct how what's the fast way to figure out that b is correct less zeros it turns out when you do the powers of two you should not have trailing zeros they should all end in 2486 2486 forever so why column A why are there zeros any guesses no 64-bit integers on JavaScript so JavaScript doesn't have the notion of an integer it's just JavaScript number and so it's effectively a double and the problem is after about 53 bits you start getting rounding errors it goes to double math and so obviously we're able to correct for this right so if you're doing cry you know crypto cryptography I should be very clear maybe crypto too if you're doing Bitcoin um you have to do a bunch of special tricks on the browser to make sure that you have correct numbers you can't go beyond 32 bits basically um which is really unfortunate and so we tell folks in dark that the number Precision is a factor of the runtime um but actually be able to have reliable numbers so I actually have verified that on the dart VM and on WM um the powers work all the way through and of course 2 to the 63rd goes negative for weird reasons so I cut that one off um but actually wm's correct which is great so actually we're in place now we had a number of places in our um our tooling even in the last few months that we're just doing math like comparing um 1.0 to one and if they're the same or not to figure out if we're on the web we can't do that anymore because on the web might be WM and so numbers are correct now which is super nice so how do you take as a user of flutter how do you start using web assembly it must be really difficult actually it's super easy barely an inconvenience you just had the flag so as of today or yesterday we now have beta out with flutter so flutter beta now has the ability to build with WM and it's as simple as a web as an author um if you don't do any interop so if you're using all the widgets and things they just work and compile the WM um and other than perf they should be basically identical implementation assuming you're not doing High bit math and actually what we do now by default there's actually no way currently to deploy only WM so if you pass the WM flag we will compile both so we will run both our JavaScript compiler and our web assem compiler with every build and we do the work to do runtime feature detection to switch between the two so you can do one build and deploy and on browsers that support won GC you'll get great performance um and actually even the JavaScript version generally you get pretty good performance it works well um you can turn it up to 11 and so the idea here is there's no Cliff you can support all browsers today umow that have support for GC and the features we use will be faster um and then my aspiration for this calendar year we'll see if it happens I'm a product manager not an engineer is I'd like to get to the point where by default if you build for web we will always build web assembly there's some things we're working on um and in fact if you're really curious I'll show some examples here in a bit um you can dig in and look at the MJS file which is how we bootstrap things and you actually can see a numbered list of every function that we ex we um import into the WM module and this is kind of how we Bridge the browser stuff so if you're curious how we do um talking to browser apis you actually see it in the generated MJS file so there two examples you can look at today I showed the material demo last year so it's the same URI and you could open it now in Safari or Firefox and chrome um and if you look at the dev tools you can see which version you're using keep in mind that they both load a wasm module which is this canvas kit wasm thing so you know you want to look for main. dart. JS versus main. dart. WM um but in Chrome at least you know you can look at the wasm you can expand it out it shows the the wasm text t format you can set break points if you want it's all there this vote thing is actually the first thing I ever wrote in Dart um if you're interested about election methods and ranked voting we can talk later it's an interest of mine but I was very excited to get that up and running and it's a more Simple app written um and deployed with web assembly of course there's one detail left which is how do you handle JavaScript right in this new WM World um before when you all only cared about JavaScript a Bo in Dart effectively was compiled and used as a Boolean in JavaScript exact same thing so really no conversion was necessary but now we live in a world where a dart bu compiled to web assembly is not the same thing is a JavaScript Boolean right it's actually a WM i32 we use zero and one not surprising it's a good thing to use so now we need is a mechanism to talk to browser libraries other JavaScript libraries correctly and have it work the same in JavaScript and web assembly so there were four things I wanted I wanted it to be sound meaning like from a computer science perspective you know don't mess up typing consistent you want the JavaScript version and the web assembly version to work the same you want to be fast and I would love I would have loved it to be magically trivial you just do the right thing and will convert for you it turns out that's really hard to do and do fast and do consistently so we punt it on magically trivial there's some work you have to do um um although what's interesting is we now no longer kind of gloss over the difference like JavaScript is a different Beast than Dart um and especially now that we're in web assembly we don't assume that so we actually expose an entire hierarchy of objects that represent things from JavaScript you know you know big in symbol numbers even things like promises so in Dart there's a notion of a future and JavaScript is a promise and we expose a hierarchy here and then mapping between it and so what's cool is you have to do the work to call tojs to Dart to pass things around you need to convert a dart function to a JS function you pass things and effectively in Dart tojs it's mostly a no op and some cases there are differences but for things like Boolean and string we just pass it over there's no all these operations are are no Ops then obviously in the wasm side we need to convert like we need to convert the notion of a JavaScript number to the right kind of number as well on the dark side right like do you want us to do this like an integer or a double and we you need to tell us because at runtime it's just a JS numb and that could be you know it's eff floating point so we have an explicit notion of conversions this is well documented if you're curious go take a look and today we're super excited to say that we're now in beta so last year I you know a year ago I was here talking about this as an experiment we are in the process now of flying towards um stable um stay tuned the website I have here flutter. dwm will have all the details um as of today it's beta and obviously if you're watching this recording in the future you want to look at that website for the current story is as far as browser support So chromium based browsers so obviously Chrome and Edge things are fully supported I mentioned before we have this uh cross stud rendering it turns out that Firefox has what we consider a bug which is they do extra copying when we're copying bit Maps across the um the web worker boundary and it's insanely slow and so we need to figure out are we going to basically turn off the cross thread rendering and just use a single thread rendering on Firefox or hopefully Firefox will fix their bug so we need to figure that out I'd recommend if you go to either of these bugs please don't spam with me too but you know the plus one Whatever star thing is they know these are important bugs they appreciate that same thing with Safari um and actually was great to see the cotlin talk it seems like um a web kit is much further long than I thought um they're cranking along really well um but obviously once we're stable in safari with GC we'll be excited to see um flutter on WM running in Safari as well so I hear this a lot and I see this in discussions right like people like I want a web assembly UI framework like give me a web assembly UI framework I have one for you it's high performance it's a rich UI framework that covers material design and um Cupertino which we call a ios's design language so if you want to make a app that looks like a native IOS app or native Mac app you can do that in flutter today full support for localization internationalization accessibility it's battle tested production by Fortune 100 companies in the states all over the world a bunch of Google products and as a bonus from this you know this beta framework for web assembly it's also stable in JavaScript and lets you deploy natively to iOS Android and desktop so I think it's a pretty good solution if you're interested in doing um a web assembly UI framework and we'd love to have you so flutter. dev. the updates on the where we are with WM is fl. dewm and of course you going to find any of my other random things I've done go to my personal website and I'm open for questions thank you [Applause] much thank you so might be a naive question but like the performance is all great um in terms of power management for mobile devices are the implications the same in terms of like sleep States and stuff when running web assembly applications I'm thinking things like Progressive web apps and like background threads and all the rest of it um or are they higher performance but there's still work to be done in terms of efficiency Etc I don't know whether that is hope it makes sense so it's in some ways flutter web apps are just other web apps right so if you're thinking about doing background processing obviously we have our background fed we're using for rendering here in the WM case if you're doing other background processing or whatever it's just like any other web app so if folks install a web worker or wpwa and they register for background events it's I I can't speak to the details of managing performance or expectation of that but it's just like any other web app we are no different in that respect but the the web assembly version specifically it uh versus the JavaScript version yeah for the same code I mean it's it's kind of what we talked about right so like fundamentally we're getting the same pixels on the screen it's actually kind of crazy to go through all this work implementing an entire new language Target and all the tests and then we have the same we have you know our our um our screenshot tests and they're the same pixels like our goal is pixel identical and so what you get is like half the CPU time so you're doing the same work but you know you're spending half the CPU and even doing multi- thread or multi-core um you get obviously benefits there in terms of Jen and everything else but in terms of the total amount of CPU work happening it's half and so you're either getting more frames on the screen drawing frames faster obviously or you're just using less battery which is great sweet thank you going once I'm curious if you're using webgl for rendering the UI um how do you achieve accessibility there do you have to Define specific accessibility attributes and all of the UI elements separately and does how much overhead does that bring to the final binary size as far as anant you would have to reimplement some of the things that are already available in the browser for Dom if you did not use webgl in the first place right so for accessibility we actually use the Dom and so if you there's certain signals you get from the system to realize that user wants to use accessibility and then we hydrate a hidden Dom underneath the canvas effectively that has all of the Dom elements you care about and um you know has the right ARA tags and everything else there is a runtime per hit there and probably you know a compile time hit like because you know we have a bunch of logic there to instantiate the tree the nice thing is because we're not worried about the presentation to the user but just the accessibility elements um It's relatively lightweight especially compared to doing a full UI framework in HTML um but there is obviously compile time pit to include accessibility just like on any platform and we're all happy to do that work because it's good to have an accessible application um and there is a runtime hit because you know now if you're scrolling or interacting with the app not only are you manipulating the pixels on the screen you're also manipulating this Dom this kind of virtual hidden Dom prevent accessibility to present accessibility um but generally we find it okay like we get bugs on the accessibility side for you know just correctness things that were fixing very quickly but um in terms of actual end users using accessibility of the flutter app I've actually never gotten a bug about performance so it seems to be solid thanks it is a early problem do you know doing a canvas app and having accessibility It's Tricky anybody else thank you so much

Channel: WASM I/O

Views: 15,118

Keywords:

Id: qx42r29HhcM

Length: 31min 18sec (1878 seconds)

Published: Fri Apr 12 2024