""Flutter: How we're building a UI framework for tomorrow at Google" by Eric Seidel

so my name is Eric Seidel and I'm here today to talk to you about flutter before we get into that since most of you probably don't know me I should give you a tiny bit of background about me I have been a software engineer for coming on 15 years now mostly worked on the web on browsers worked on both Safari and Chrome etc about three years ago I helped found the flutter team which is what I now manage and what we're gonna be talking about today so flutter if you haven't heard of us our goal is to make it easier and faster for you to get to Iowa's and Android in fact our team's mission is to build the best way to develop for mobile and you might have stuff why don't we have lots of ways to develop for mobile we do there are a few rough edges and in fact we spend a lot of time at the beginning of this project talking to other teams who work on mobile and understanding their concerns and we heard a lot of the same things and maybe some of these will resonate with you we heard that getting to mobile can be slow and expensive we heard that doing delightful things on mobile which their designers want to do they want to do their users want can be hard and that the platform diversity story of current mobile complicates this and finally that if you really want to reach all users at least in the US market you have to develop for multiple platforms and that can mean two or three teams so we said to ourselves in doing this there must be a better way the development should be fast I mean here it is we're in 2017 custom should be easy you should be able to trust that what you saw on your phone while developing is what your users see and finally that you should ideally be able to build things once and take that hard work to more than one place so let me show you a taste of what we've built so I'm gonna switch over I'm an Android and an iOS phone I'm gonna switch over to the iOS one here and so we can fire we have this iOS phone and I'm just gonna launch one of our little demo apps and the first thing that you'll notice you may be it went by too fast but of course it just flies open Scrolls beautifully hits every frame there's lots of pretty little demos lots of the Lanham Asians etc but I can also show you this same app on Android buy switch over here's an Android phone same story boots up beautifully scrolls beautifully you notice that this now looks more like Android different scroll behavior different title alignment different back behavior etc so these just look like iOS and Android apps and they aren't just iOS and Android apps but one thing you may not notice is that flutter is painting every single pixel you see on the screen handling every single touch ending every single gesture and so I can do crazy things like this so I'm on Android here I go into our little debug menu and I turn to iOS and so now when I scroll I get different physics I get a bounce behavior and when I go in so this is for example an iOS style control and I have a back gesture all because we control every single pixel every single gesture and we're gonna talk a lot more about how we do that but that was a taste so let's go back to our slides so again our goals here we are in 2017 we want to build something beautiful something fast let's talk about how we went about doing it so we thought about these goals and what problems we were gonna solve in the mobile space and we found some alliteration maybe a thesaurus and we came up with this list of things we wanted to solve and I want to talk about particularly because this is a tech conference the tech that we built to solve these problems so let's start with fluid why do you care about fluid we want to go the fluid developer experience we want to keep you in flow we want to get you home early this is the thing that most personally excites me about this project in part cuz I am a software engineer and I like saving time but I also like looking around my colleagues and thinking about the hours of their day that I help save that I gave back to them because I have plenty of things in my life that distract me out of flow I feel like fighting my toolchain should not be one of them as so elegantly expressed in this comic so let's talk about the tech what do we build well first big part is that at a very low level our system we built in a technology that we called hot reload you've seen other text perhaps with a similar name I believe ours is a bit different let's talk a little bit about what this does and why you care so hot reload typical dev cycles is you spend seconds or maybe minutes we talked to teams inside Google who had tens of minutes dev cycles waiting for Xcode to turn etc and then you get plopped at the first screen of your app and then you scroll around and fiddle around inside the screens and you find down in the eighteenth screen your bug and then you go and you blindly try and fix it and then you repeat this cycle ha reload is about staying at that eighteenth screen or fourth screen on the slide and just fixing it right there and so I can show you this is our actual dev cycle our actual time between code and device inside flutter and you can see in this gift I'm changing the color and hey saving and it's immediately reflecting on the device similarly I'm changing the logic of my app and again immediately reflects on the device this is very powerful saves you a lot of time uh and also I should say this is because we built it so low-level into our system and we built the rest of the system around this is really works this is the default way that you do development and we test and we record stats from the wild it's fast you know that it works we know that people use it and in case there's a lot of very interesting technical details which we could spend an hour talking about but here's a taste and there are several youtubes on this topic that you can watch and learn a ton more about how this works how we make that magic happen so another thing that I should talk about one of the lessons that I learned in doing this is how much language and toolchain matter if you're trying to accelerate developers and so we started out in one language and actually built two full copies of our system before switching this is something I've never done before here we were a year into our project we were switching languages we started out actually with JavaScript it didn't in the end work for us and so we switched and we went on an exhaustive search tried lots of languages basically anything we could bring to mobile and ended up with one that I didn't really expect but has turned out to work out very well and that language you may never have heard of because it's kind of small it's called Dart so this is a language that's actually in heavy use inside Google it's one of the accelerated growth languages right now it's used to build really big things multiple millions of lines of code that drive some core business for Google we didn't really care about that except that we knew that this meant that it was battle-hardened it could scale and then finally this team and this is a part that I did care about cared a lot about productivity and they had already seen big productivity gains in the web platform which is where they were currently targeting and we worked with this team to help bring this language to mobile so we ended up really liking this decision it took us a while to get there but there's some really neat things that this language does that I haven't necessarily always seen elsewhere like for example this one this language both has a fast development cycle which you've seen but also when it's time to ship compile straight to arm code so you just ship straight to the CPU straight to the metal to your users I also has an optional but strong typing system which again those million line apps or multiple million line apps leverage heavily and we also leverage heavily it has a tree-shaking compiler which helps you to use a large code base but then only produce a tiny binary out of it only the parts that you use all right has generational garbage collection which basically means fast we do lots of tiny tiny allocations and they're very cheap to make very cheap to let go of and finally we found it familiar easy to learn if you use Java or JavaScript this will feel totally natural to you final thing that I learned through this process was how important tools were and I'm not going to read off all this to you but we have spent tons of time building tools if you want to accelerate developers you have to build and invest in tools and we have done that so I want to show you what this workflow actually looks like so I'm gonna pop out of this into an editor so footer supports a variety of editors I'm just gonna use IntelliJ because that's what I have a nav on this machine this is a bit like a cooking show I already followed the template and did a create over this is about two minutes into the process we just didn't have to wait for Xcode to submit up for the first time so here I've run this very first template app I just have this little counter again of course it works on iOS or Android I happen to be using the iOS simulator but you could use whichever and I just want to show you again that dev cycle so if we change this to red right and I just save and boom if I go and I say want to change this text right because that button is actually a fab a floating action button there again appears right away same thing I could add some text say things like hi strange loop and there you go in case this is the dev cycle this is what you live in this is what you do keeps you in flow all day long and we'll show more of this in a bit let's go back and talk a bit more so our second goal the second thing that we wanted to solve was that we wanted to produce a system that was flexible so that developers never had to say no to their designers this is was in part due to direct requests from these teams that we worked with earlier and part to us just watching the market and seeing what was happening go on are the days of your where you just produce a cookie cutter checkboxes and buttons OEM widgets app now much more common is for folks to do very custom brand forward design and these are just live apps pulled from the iOS and Android stores and similar we also learned through this is that even teams which told inside Google but no we're just gonna do material design we're just gonna follow the spec would still do custom things perhaps without thinking about it and so there's at least three things that are violations of the material design spec in this screenshot that we're very easy for the team to create because they had tools and matched what their designer wanted so another thing that we learned or how we went about doing this how we created a flexible system was layers lots and lots and lots of layers so this again came from direct developer feedback as we were starting this project one of the developers put it that they felt like dealing with some of these systems was like they were dealing with an iceberg that they had this tiny little API surface that they were allowed to use and they knew that there were big powerful things buried under the ocean they could see him but they couldn't touch and so we wanted to build a system that in a sense flipped the iceberg and put all the good stuff up under your control in the same language that you're writing in so you can jump to definition all the way down you can edit it etc and so I'm gonna talk about some of these layers that we built this is an extremely simplified layering diagram we have lots of layers but we're gonna walk through it so starting at the bottom this is our runtime api these dart colon api's we built one called dart UI this basically gives you a canvas some accessibility hooks a way to layout text some networking api's and that's about it you can write it this layer your typing a lot of code but you certainly can if you want to on top of that we have a rendering layer which should be very familiar to someone who's worked in UI programming before this is a stateful view model very typical way the job of this layer is to do caching let you put a box on the screen while you move it around to do layout to do paintings do compositing that sort of thing one of the drawbacks of working at this layer and is typical for other view systems is that there are some complexities a common one is that this sort of stateful view model has a create update separation so you both when you create the view you have to write a second code path to up the view and those can get out of sync and that can cause all sorts of weird bugs and so folks build layers on top of this as you've seen for many years the one that we chose to build was a widgets layer and this widget layer basically handles composition for us everything that the rendering layer tends to have a widget as well but widgets are how you compose those how you assemble those into new widgets that you can reuse it cetera our widgets layer layer is very react inspired these are immutable widgets short-lived ephemeral objects and they help solve some of these limitations of working at the view layer it's much shorter to compose at this layer it also has a very simple data flow you just always create new widgets you don't have to worry about updating them and then the final layer I should talk about is the sort of layers of opinion and we have several of these we for example have some material design which you saw already we have a set of iOS widgets which your so saw a little bit of called Cupertino and typically when you use this framework you sort of build your own opinionated layer so let's go back down and talk about some of the technical changes technical innovations we made at these layers to do this kind of flexibility to do the kind of speed that we wanted so one is rendering to understand rendering you have to know a little bit about our pipeline so flutter has a very strict pipeline of how data flows to the system starting to the vsync or user input and ending out when we actually push pixels to the to the GPU the part that I want to talk about starts in this rendering layer so rendering handles things like layout again painting and compositing and one of the things that we learn and working in this is that simple can be fast if you start with simple well understood algorithms or algorithms with well understood properties you can build simple things and take advantage of those properties to go fast so let's talk about for example how we do layout in painting it's typical in other systems to some times join layout and painting into a single paint phase or it's typical in layout to have a multipass layout where a single widget or a single view will walk all of its children to gather some information and then we'll walk them all again to lay them out this isn't how things are done in flutter or we have intentionally separated layout and paint and do a single pass to reach we walk all the way down through all nodes and all the way back up and this allows us to scale to much deeper trees and you might be used to in other widget a systems another way in which we found simple is fast is that we found that with very simple constraints you can generate expressive layouts so basically everything you've seen is done through a very very simple constraint model I've just basically min max width and height this is in contrast to say how you like it functions UIKit uses a much more sophisticated or much more complicated linear constraints model and that has a general purpose constraint solver to solve that and we found that we can do something simple and very fast instead another way that we did things differently here is that typically how you do repainting is that you collect the dirty wrecks and then you repaint based on based on that and we found that in the modern mobile era it was simpler and faster to invalidate sub trees because modern GPUs are very good at compositing and we could take advantage of this to go fast this was actually a big speed win for us I wanna talk a little bit more about more innovations this time at the widgets layer so one of the actual things that we did differently here I don't know of any other react style system where both the reactive layer and the view layer are written by the same team and we got to take advantage of this in building the system but before we talk more about widgets you should know again a little bit of history so I said I worked on the web for a better part of a decade when we started flutter I was still sitting with the chrome team at Google I was still steeped in the web world and one of the things that was going on in that time was the extensible web manifesto and if you've worked in the web you might have run across this this was an effort at the time and there were a bunch of related documents to convince standards organizations to focus on explaining the web platform rather than bolting on more features and this is something that we really took to heart in building our system and maybe to the extreme and strongly favored composition over inheritance let me talk about what that means and what that changed so a case study would be padding so it is very typical in these systems to have these sort of container objects that have lots of little properties that you set when you want to build up so like you know dibs or this way when when you want to build up a complicated UI in our system we are pass re art our container class if you look at it it's this cascade of if statements or if you've set one of these properties we instead just wrap the subtree in another little simple widget and I perhaps crazy example of that I don't know of any other system that has one of these is we have this very very simple padding widget and thus none of the rest of the system except maybe container have a padding property because they don't need to you don't need these cross-cutting concerns when you have this composition sure you can build your own widget and rap and padding but you build other widgets commonly out of lots of these little widgets and this composition is all over the system so here again in our animation system we have the animation system is built of lots and lots of little pieces that again you can replace and you can add - there's no locked box here there's no like fancy animation system that would be so amazing if you could just specify your own curve of course you can specify your own curves here so in our an efficient ism you generally start with an animation controller which knows how to produce a stream of doubles you then might connect it to a curve or a series of curves that consume doubles and produce another stream of doubles and then again you can chain those two tweens which know how to consume a stream of doubles and produce a stream of whatever Rex or colors etc a pattern that we find strong enough so flexibility let's talk flexibility I want to just show you a live example here very briefly if I switch back to my iPhone I can just show you an in production app that takes advantage of this so this for example is an app built with flutter shipped and doesn't really quite look like iOS doesn't really quite look like Android but rather is their own different look and feel and this is them taking advantage of this flexibility to build what they wanted to build so let's go back and show you some of this composition so if we continue down our little tiny example that we started with hi strange Lupin again we never lost state we still have our number seven in the in the corner we can build more complicated things using this composition so say for example I sent this to my designer I'm ready to ship my counter and my designer was like oh I want a different floating action button that is more styled in the way that we do things I want an explosion or something like that we can build of course our own floating action button there's no secret so of course we can just jump to definition all the way down and read the code but you'll find it's very simple it's not very mean you can just keep going jump jump jump jump all the way cuz again all the code is written up at your layer of the system but we can also just make our own so we can just make this my fab right don't get a tooltip but we'll just implement this I'm gonna use my fancy editor to splat out a widget I don't talk about exactly these classes but there there are more talks on YouTube that can explain all this to you if you if you so desire we're gonna type one out my fab actually I wanted a state less widget my fab save hot reload oops boom we crashed a little app it's telling us that we haven't actually implemented a constructor for this we're constructing am i famine we haven't handles and we could have learned this before we even how it reloaded because we could have looked at our live analysis that happens right as we type because again I talked about the focus on tooling before but we can just fix that it's very easy we just do a my fab this unpressed this child we should I guess actually to find those so we'll have a void callback for on pressed and we'll do a final widget for child right now we send the sucker oops we have another bug what's it say aha it's because we made them required arguments when it expects us to have named arguments so we'll fix that won't send it again and there we go so I'm back I never lost state I went all the way through that error I never stepped out of flow but we're not drawing anything and that's because if we looked at our at our build method we're just returning an empty container so we could take this container at our child that we're passed in and now again milliseconds later we're drawing something on screen we have our little icon so now we could say wrap this again I'm using my fancy editor in a new widget because when we have we think of this budding we want it to float right so we're gonna have to the way you float things in the material side of the world is you make a material and we'll tell this thing to reformat if I can remember the keystrokes whoops okay we'll we're gonna go ahead and add an elevation to this elevation will make it I don't know 12 sounds like a good number and we'll give it a color cuz our designer told us that they want fab to be green so we'll do colors green and we'll send the device and now we have our little fab with the shadow and it's green but we can keep going I just reformatted so this is all easier to read for one this fab doesn't yet handle presses so the way that we do that in the material world is we add an Inc well which is something that knows how to here I'll use my fancy editor again to wrap in a new widget Inc well we have a child and the inkwell also knows how to handle our on press so we'll do our on tap the passer on pressed I know this feels a little magical because I have to do this so quickly to fit into our time but now we have you can see we've done very very little typing we're up to all of ten lines and we now a functioning button and we can complete this by giving our container the desired width height and I think you know we can send that again immediately changes and we go back to our material and we're gonna make this a circle because I think that our our designer didn't want a square button right that's it we're done in all of whatever that was two minutes and ten lines of code we have built we've rebuilt some core cost core control that we can now customize to our hearts content this is just how the system works this is how composition works in our system so let's go back to the slides I hope you guys are as excited as I am about working in systems like this I like when the Machine does what I tell it to so the final thing that I want to talk about the final part that we wanted to build was something that was faithful something that you could trust would deliver the designs that you worked hard on implementing on your phone to all the phones of your users and so I want to talk about well I guess we should start about why why you care about this well again this came from those early discussions we were talking with all these teams and they talked about things like that their apps would break do the carrier-based theming that they would have to fight with platform and hardware differences that would cause their apps to look weird sometimes that they also had frustration with waiting for their users to upgrade they wanted to use some fancy new design or something like that but they couldn't push that until enough of their users is upgraded so for example I actually have kind of a counter example of on this slide where you see this is a phone from 2012 two years before material design ever existed that is running material design via flutter and you could imagine doing your designs and having that level of control and pushing to all your users now and so the insight the tech as to how we got there was to go to the metal this is something we realized relatively early on that we needed to build our own runtime but we didn't want to invent reinvent the wheel and so we actually spend a lot of time on this searching far and wide for systems that had already been battle-hardened so we took some graphics a part of the graphic stack out of chrome we extracted a bunch of text logic out of Android and of course as we talked about before we took a language that had been a successful on the web and we brought it to mobile and I should talk very briefly about some of the changes that we made to these systems because we didn't just take them as they were but for example for skia which is our 2d graphics library we only use the GPU parts because again we're working with 2017 right all these mobile phones have pretty good GPUs in them we also worked with this team to add color correction so for example even when you're shipping to really old phones you can trust the ear and just look right and similarly we worked on adding a text library on top and OneNote sort of a thing that fell out of having had built our own runtime was that then we were faced with all this portability and it was kind of neat thankfully again so much of the heavy lifting is done by these component parts that we salvaged from other products for example dart is our cpu abstraction and has a lot of different technology in order to behave nicely on all these different configurations and similarly skia which implements a bunch of different backends to do the same thing for our GPUs but then our tests worked everywhere and our product ended up working everywhere now we're focused on iOS and Android today we don't really talk about this but flutter really goes all sorts of places I've seen it run on watches I've seen it run on set-top boxes I'm just kind of waiting for somebody to send me a YouTube of them running it on their refrigerator because I'm sure it can be made to work which is interesting and maybe we'll do something with that in the future but a final insight I talked about how it was important to get your designs intact to your users but it's also important to match the expectations of your users be faithful to their expectations on the platforms that they're running on and we spent a ton of time on this we started by making our pixels match and it was important that our our buttons look all right and we actually found that users didn't really care about that when we studied that we cared and it's stuff we still to it but users what they cared about was that the feel they cared about that when they tapped the the targets were big enough they cared about when they moved and then when they scrolled that it felt the same way that the rest of their apps felt that the gestures felt the same and so we spent a ton of time on this about all sorts of interesting tools to make sure that we understood that say like this one is for scrolling that we could tell you exactly how many pixels our scroll velocity differs from the OEM controls for example okay so that's basically what I wanted to tell you I hope that you enjoyed learning about the different problems that we're trying to solve the tech that we use to go about solving them I should talk very briefly before I go I know this is a Tech Talk but we could talk a little bit about product which is that this is a very new product we just released or announced to the public our alpha in May of this year at Google i/o and since then we've seen an explosion of usage we're already up to over a hundred apps you know with this brand new thing push to play and we've seen an ecosystem develop over 80 packages in the dart ecosystem that are very specific to flutter and again as you saw we've seen some big brands already adopt us one was launched last month this was Hamilton featured on both stores over a million installs and very well reviewed finally I guess what I'm really here for since this is again a tech conference is that this is all open source it's been open source since day one I like working in that environment but open source is all built on community and we do have a strong and growing community and I'm here to invite you if this sort of stuff gets you out of bed in the morning like it does me you please should come check us out again we have over a hundred and twenty-five contributors who are main and posit Ori and probably a lot more than that if I counted up all the little ones over a thousand people in our dinner this is a very active project and we love to see you there so thank you very much for your time

Channel: Strange Loop Conference

Views: 56,694

Keywords:

Id: VUiVkDpikDI

Length: 31min 40sec (1900 seconds)

Published: Sat Sep 30 2017

Reddit Comments

[u/harald_haraldson]

Ah yes, Googles next abandonware project

[u/matchymtrader]

i dont understand why this library is more futuristic (i.e. for tomorrow) more than any other.

[u/flekkzo]

I've seen cross platform ideas come and go since Tcl/Tk, and that's just my personal reference point. They all try to be better. They all want to negate any advantages of the OS they are running on. They also have deep flaws the devs never wants to talk about.

I'm jaded about seeing these things with their new flavor of party tricks.

To me it would make more sense for Google to spend their money on a complete window server replacement for Android.

[u/c-smile]

Amalgamation:

1. Skia Graphics with OpenGL/Vulcan backend - so it does not use native platform's GFX, widgets and behaviors. Skia is not that small.

2. Java/JavaScript alike language, VM and runtime - interpreted but has JIT.

3. There is a layer that emulates look-n-feel and basic set of widgets for target style systems: Material and iOS (Cupertino?). ( because of #1 decision)

4. Styling is embedded into UI initialization code. (That's highly non desired, IMO).

De-facto each Flutter application is a browser/player with its own dart/flutter runtime. Good for particular app, bad in Electron-ish sense.

Actually having each application to contain its own GFX layer sounds terrible for Android ecosystem in general.

There is simply no standard native GFX 2D api there - so each more or less performant app contains its own set of graphics wheels. Android UI - its own, Chrome - its own, and now each flutter app.

Wearing my UI architect hat: Good UI system must be build from reusable blocks.

If to wear Google/Android shoes I would start from designing native 2D layer with stable API. That can be shared/reused by many applications. And build the rest on top of it. Then set of lightweight native styleable widgets that can be used in React-way on top of that. And then to add Dart runtime on top of that.

So if any of these will go away the rest can be reused: like to have Java or Kotlin instead of Dart.

Such "Lego-bricks" foundation makes feasible to migrate from Java based UI widgets to native UI with Java thunks. So Java with its GC pauses will not affect basic UI operations like rendering and transitions .

[u/kilroy123]

Slightly off-topic, but I have to say this guy is a pretty good public speaker and his presentation was pretty good.

[u/stesch]

Note: Flutter is an alpha, open-source project.

https://flutter.io/

[u/Purple_Pizzazz]

r/FlutterDev is a subreddit for flutter in case anyone's interested.

[u/wavy_lines]

I'm actually quite impressed, and I tend to be pretty skeptical about new technologies.

I'm not sure that having tons of layers is a good idea, but I like that you can customize every part of the system or completely change it.

I hope they can find a way to implement this on the web as well (hijacking a canvas and painting on it, for example).

[u/[deleted]]

That doesn’t look like anything to me.