Painting with Rive and Flutter - Umberto Sonnino | Flutter Europe

[Music] hello and welcome umberto from rival a software engineer and today we'll be looking at how to draw widgets and build custom behavior in flutter and also a drive so what is rive you might know rivals flare before we were called flare but now we've rebranded everything as rive rive has announced cede state a seed funding stage round from a 16 z last month and for those of you who weren't familiar with flare at all flare is a real-time interactive design tool that allows you to build custom animations we have a web editor that you can use to export animations and run it through our open source runtimes we have a number of runtimes one in JavaScript with a with a react counterpart there's a swift runtime that is running on Mac OS and iOS and there's also of course a flutter runtime which is our most popular runtime it's gotten more than a thousand seven hundred stars on github and all of these runtimes are open source so you're gonna be if you want to take a peek and help us out and figure out problems that you might encounter please do our journey has arrived or before that as two dimensions with flutter began a couple of years back we started working with flutter for IO 18 we built the flutter hot reload game at i/o 18 and for those of you who were there you might have seen us screaming next to the flutter tent trying to gun garner your attention then we built later that same year the history of everything up for flutter outlaw flutter live and that was downloaded more than a thousand hundred times on the Play Store and it's also available on the App Store as well of course last year at Google i/o 19 we built a developer quest which was used in its example for a number of flutter stalks and last month at flutter interact Tim Sneed presented on stage the - demo that was built by our team so what about today today we're gonna take a rendering overview an overview of what flutter does for rendering so we're gonna be looking at custom paint the flutter trees how flutter trees are used to be efficient render objects and the render box so later I want to take you through a number of things that we use in Drive for painting our run time and especially flare render box what our animations and controllers so how does flutter render it happens in four stages there's a layout phase in coolant in which the primitives are gotten and widgets are placed on the screen and their sizes are computed then the framework will pass a canvas object through those widgets and we'll tell the widgets to paint themselves on it later the flutter tree structure will try to build a number of layers so that the widgets will know which layer to be on and then those layers are going to be composited and the pixels sent to the GPU for rasterization so for example if we are in the layout phase constraints are going to be passed down the tree and there's a tree structure for all the tree you you might have built widget tree structures in your app of course and constraints are going to be passed down and once the tree is going to be explored down the widgets are going to be passing up their sizes and after that we paint so let's take a look at custom paint because some paint is a widget and the idea as we saw in the painting phase is to access the canvas if we can get access to the canvas then we can perform drawing operations and how does custom paint do that well it's a widget so you can put it in your tree but then as a parameter you can give it a custom painter custom painter is an interface that you can implement with a class and you can override the paint method to be able to access the canvas and perform your drawing operations flutter as you might know uses ski eyes it's rendering engine so all the primitives that you see for accessing the canvas and all the methods that are on the canvas are obtained through skia and this is there a flutters rendering engine it's a C++ open source engine owned by Google and it's a pretty impressive engine so if you want to take a look at that too I encourage you to do so custom paint is very plug-and-play it's very easy to use and I want to show you a little example that I built let's see if we can get it on screen so wait we gotta exit this I can show you there you go so this isn't easy sorry guys so we got a main this is the simple app that you'd get from built using flutter create and as you can see there's a custom paint widget in the tree so this is part of the of the tree structure the center we're gonna see a circle painter and that's a that's an implementation of the custom painter and this is just gonna display a circle on screen so if we go to the right folder and we slaughter run this you can see the widget on screen let me get there okay so what is circle Feiner as as you can see that's a wonderful circle in the middle of the screen but a circle painter is an extension of custom painter and what I want to show you is that if you override the paint method you get access to the canvas object and a size because the widget that's got to have its size and what's relevant here is the draw circle call this is what draws the circle on the screen you've got to give it a position so we calculate the offset from the center to get to the center of the switch it and then we got to give it a fill so this is the way that the circle is going to be painted on the screen so this is fairly simple and as you can see you can do all kinds of crazy things because canvas gives you access to a lot of primitives a lot of methods so let's get back to the presentation and I can go through what flutter does with the with its multiple trees so there are three tree structures that I use by flutter the first one is the widget tree structure which you might have seen and you might have built using your apps creating your apps and this is the the widgets are responsible for creating and updating elements and render objects I think Robert or Norbert might have explained this yesterday morning if you were at that to aqus elements on the other hand are efficient at comparing objects so they're meant to be something that sits in between what widgets do and what render objects do and what do render objects do they are responsible for laying out elements for painting and for hit testing so let's say that you have a simple text widget on screen and you want to to replace the text that sits in that text widget with a new text instead of rebuilding every time all the various trees flutter tries to be as efficient as possible so the widgets can be being lightweight and immutable can be replaced really quickly so the widget is going to be rebuilt from scratch but the render objects which are responsible for rendering rendering which are heavier and mutable we want to keep them there so the text in the text widget that you have in your widget tree is going to be replaced the which is going to be recreated but the render objects that that's underlying that structure is just going to be updated with a new set of parameters so we've seen what render objects are and render objects are responsible for laying out again painting etc and they're a pretty abstract concept so flutter provides an another implementation of render objects which uses a render box and it's a render object in a 2d Cartesian coordinate system so the origin of a render box sits at the top-left corner of the widget or if the render object as you may and it's given a width and a height if you want to create a new semantic model for laying out for painting for hit testing you can't subclass it and define your own things and it's used by our runtime too so I wanted to show you I prepared also an example for how you can create a render box and extend it to implement it in your own app so we get started from the idea that you first need to have a widget in your tree right so in order to be able to do that you need to get to create something like a circle render object widget we're going to be creating exactly the same app as before with a circle rendering in the middle of the screen but again we need to place a widget in the tree so first of all you you gotta extend a leaf render object widget that as norbert mentioned yesterday is a widget that doesn't have any children and this as i mentioned earlier is responsible for creating the render object so you override the create render object method and you make it return the render object or the render box that's gonna be responsible for rendering the circle on screen so over here we got the circle render box that a set extends a render box and as you can see you we override a bunch of things well we gotta be telling it that its size by its parent so that the that it knows its size the hit set self is gonna return true because we want it to be responding to hit testaments and what's relevant here is the paint call the pink call this time around doesn't have a canvas object as its parameter but it's got a context but we can solve this free easily because the context has got a canvas field and we can extract that then from the size cuz size is a field in the render box it knows its size we can get its width and its height and at the bottom of the call we of the method we can see that there's exactly the same call as before so we got a draw the circle we give an offset but this time around instead of the offset within the widget we gotta give it an offset globally from the screen and we we compute that through the offset parameter that's given through the paint call in the render box method and then we give it a radius a size basically and a fill paint which is exactly the same one as before so if we go inside the folder and we'll you run it hopefully we'll see the same object as before and again as you see this is fairly simple you create a leaf render object widget and then that render object widget gives you access to the possibility of creating a render object so as you saw hopefully that was correct you saw the the orange circle on the screen and everything came together so for those of you again not familiar with rive rive is an editor it allows you to build animations and i want to go through a couple of concepts here basic concepts you might see the animation on-screen and you can switch in the editor between a design mode and an animate mode in design mode you can build your hierarchy and that hierarchy is gonna be your objects and you can add objects such as circles rectangles vector shapes but also raster images you can perform vector deform a vertex deformations on those shapes and in order to animate those objects you switch to animate mode you're gonna see a timeline pop up and and you can set keys in your animations to let's say move around on objects change its scale change its vertex deform and so on so those objects those concepts the art board of I missed the artboards the artboards are the over-the-top containers for your hierarchy and the the artboards and the animations are gonna have counterparts in within our flair flutter package which is our runtime and those are going to be actor artboard and actor animation of course so we saw with the render object that if we want to play something in our widget tree we need to have a widget and this is what flare flutter provides we got a flare actor which is a widget that loads a flare file a flare file is a file that you export from the writer and once you get that in place you can place it in your widget tree like this you give it a path that you got to import your assets put those in your pub spec yeah Mille of course and you can play your animation simply by giving it the name of the animation the name of the animation gotta it's gotta be compliant with what you created within the writer but you can also control your animation but using a custom controller and we're gonna see how that works later in this in this talk so just to give you an overview of the of the object structure we've seen the render object and we've seen that the render objects get gets extended by the render box which is less abstracted and we're gonna use an extension of render box exactly as in the example before which is the flare render box in our runtimes you few peek through the code you're gonna see that the flare render box is an abstract class that's responsible for displaying and displaying the the flare file and also advancing the animation so this is it's what scheduling a transients frame callback which is a callback tactic it's called every frame and it gets the animation running but being a an abstract concept it provides the painting painting call so you're gonna see the painting implemented in there but and it advances frames but the concrete implementation the final can creat implementation of everything is in flare actor render object which is an extension of layer render box and it fulfills the advanced call so again you get all leaf ran object widget and in this case our flare actor is an extension of that and that creates the render object in our case the flare actor will create the flare actor render object and this is what it renders everything on-screen the animations that you see in the editor so lastly let's go through the controllers the controllers are an interface that the flare flare flare flutter package provides and you can extend it or mix it in to implement custom operations how do you do that well there's an API first of all you want to tell the framework if the controller is active if it is active then the framework will continue advancing the controller and it will continue providing the operations that you've implemented then there are three methods first initialize which lets you fetch information about the current state of the hierarchy so your flare file will have its hierarchy inside and you can fetch references to nodes and stuff like that or references to animations actor animations for example then there's a set view transform call and for those of you in the crowd who are a bit familiar with with graphical engines and this in this call which gets called every time that the controller actually is advancing so when the controller is active we're gonna relay the view transform so that we can move from from one transform space to another and then there's an advanced call you got a return true in the advance call if you want the controller to still be and understood as being active and in here you're gonna perform the advancing functions of your animation so let's see a bit about how this can work so we've seen the render object we've seen the various parts of this and then let's get to the controller sorry there's a bit of delay with the mouse so it's not super easy alright so again we start with a or create project it's fairly simple there's a really simple build call and as you can see here there's a flare actor widget with a with the flare file and we're giving it an animation so this is gonna play the walk animation so let's go to controller and flutter on this and we're gonna use flutter hot reload to see how we can improve on its behaviors so this is gonna this is one of our examples if you go to the flare flutter github page you can see in the examples folder and away we've implemented a controller to make this walk around so this is fairly simple this we're just playing back an animation and you see a penguin walking but what if you want to actually do something stranger with the penguin the penguin is a fun is a fun element so we want it to be doing strange movements so we load and you can see that now the walking phase of this penguin is sinusoidal I've implemented a simple algorithm in here in which we use the sine of of the time basically to make it walk fast and really fast and really slow so you can see that the animation is controllable at run time so let's go through an example that's a bit more fun we have wii u bolts in philip the google dev wrote from the google dev rel team and we got a wonderful philip animation here and it's been labeled as our mascot for arrived and you can see philip dancing around when throwing his phone up in the air and catching it again but what if this is too simple again and we want to use a custom controller well I've decided that it won't Philip to do this on an infinite loop but back and forth so instead of giving it the animation name we're going to play and verts the animation so it's gonna play once forward is throwing its phone in the air and then it's gonna play backwards so we're gonna see it do some crazy magic here and so and this is an example of a controller it's gonna play obviously infinitely and but this is not just it I wanted to show you one last example which might make you understand even more the idea of what I mean by modifying the hierarchy in Rive so this is the rival earth at I've shown you before in one of the slides and this is on the left on the left yeah on your left you're gonna see the hierarchy and take a look at the name of the nodes the node that we're gonna be controlling is this look node which is a node that allows you to make this guy look around so we got a lost space man who doesn't know where to look and what we can do is move to the last demo and if you're a bit familiar with linear algebra this is gonna be easier to understand and all of this code is gonna be available on github so again flair flutter prod flutter create projects really simple we got a gesture detector and we include the space flare file and we pass in of space controller so let's go to the space controller demo and flutter run this what how this shows the power of Drive is in the fact that you can actually manipulate that node at runtime so anything you see in the flare hierarchy can be manipulated and this is awesome wait I've got the example on my screen so you can't see it and you can see that the animation is playing is there's an idle animation in the background you can see as arms moving but what's more interesting is that you can look around in real time and this is not this is not a video playing this is not like a video playing you can move stuff around take a look around there's your dad companion over here sadly he didn't make it and and this is all moving in real time right so you can't control by just controlling that single node in the hierarchy you're achieving a complete manipulation of the visually visualization on screen and this is amazing because your designers can have all sorts of ideas and this is exactly our goal to simplify the designer developer interaction so developers will have all sorts of crazy ideas and they'll be responsible for building all the sorts of crazy math in there but your designers can have the possibility of creating amazing animations and those two those two elements in your software house are going to be able to work together seamlessly instead of having to recreate everything from scratch from one platform to the other and as you also as you can see the the representation of the the flare file is exactly the same as the one as you saw in the editor and so this is it this is ARF walk through the flare and the rive collaboration so if you want to reach out and you want to you have any questions feel free to ask them now or we have twitter we have email talk to us and let us know what your thoughts are [Applause] hello I am thank you for your talk I am going to say I am a newbie on flutter I am appalled him for a year now and I was also interested in life and I was looking for tutorials on the internet and I found one tutorial for mobile and I was asking him perhaps I missed it is it somewhere else but are there more tutorials to learn wife sorry I didn't catch the question could you repeat it I got I'd have more tutorials on the internet too long guys yeah we're working on that with our with our design team we know that we're lacking on that regard and we're trying to be to do the best effort we can to fulfill that that gap and do you need tutorials for creation for design or do you need tutorials for using the API both yeah we have we have a number of medium articles about using the flare flutter API and and again we know that we got to do better we absolutely know of this and we're trying to put the best effort we can but those will be coming more tutorials well I've heard you at the booth saying that you need zero to hero kind of tutorials for I've we we want to get them out there at some point in your future yeah and start with simple simple short short tutorials because the beers is a very good one but it takes about two hours or something I don't know exactly but I stopped stopped it and I was looking for simple right yeah and one thing that I can't suggest I've seen there's this guy called Mark Thompson is an active flutter a member of the community and he's been doing a hundred days of Rive and he started from scratch his first image is just a radial gradient a circle with a radial gradient and he's slowly well quickly becoming really good at building simple interactions in drive so if you want to go and take a look at his profile on Twitter you might be able to find some interesting stuff in there oh thank you lutein oh you're welcome yeah but but I know that we need to do better with the tutorials we will we will for sure thank you you're welcome oh yeah the to your right oh yeah yeah oh yeah thank you thanks for your talk very impressive animations and possibilities and obviously when you are implementing like animations or fancy stuff in your app as soon as it gets very intricate and complicated you think like okay the performance must go down somewhere so what the limit of what is house right doing performance wise on photo reps is there something like okay if the number of elements and the fly right editor is going to high you know slow phones can't deal with it like what experience you have there well so we've seen that the performance is generally really good mostly because skia does a great job at rendering so we try to be as efficient as possible just as flutter does so we render at vsync so we try to render every frame as fast as possible then there are some concerns if you use huge images the fill rate up to GPU is obviously gonna have problems if you have a lot of really tiny small elements in the backgrounds if you create a sky filled with stars obviously the rendering engine will have to to render I don't know thousands of small dots that's gonna take a hit on performance but in general we try to be as efficient as possible you you'd need to talk with your artists generally and try to be smart in what you do if you have a background with the sky maybe try to use an image instead and try to be trivy into the smart about which resolutions you're going to be using instead of using a 4k image you might use a full HD image and that's going be enough maybe on on all screens so be mindful of this but in general your your rive hierarchies should be rendering at full speed hopefully okay thank you welcome okay another question thank you for the presentation it's great and question here you generate flare file what sorry you generate flare file with the editor yeah of yours but it is possible to import from other like after effect for example or something like this so the designers use the tools they prefer and just import that is it possible well right now we have a workaround obviously if you have a after if you're building a new animation and you have Drive and after-effects we hope that you're gonna start building your animation in Drive instead but you if you have a lot of after effects animation you might have heard of Vlade which is a tool used by your Creator I think by Airbnb and we have worked with Hernan Torrisi which is the creator of body move in the plugin that allows you to export to to Lodi and there's a lot e to flare converter right now in place it's an open source package on github and what you can do is use body move in create a lot e file and then drag and drop that file into the flare editor and it should unless there is some miss compatibility with some features because After Effects it's got a million of features and we're missing some of them it should be fairly fairly compatible so you can just try that but this a lot of it sounds perfect so thanks Alice okay hello thank you for your talk was amazing and in the space controller demo there was a small element and not which was just rotating in 360 degrees oh yeah I'm just wondering what sort of black magic is that consist it's obviously 2d what sort of what black magic well that that's all on our artists the JC tune you might have seen this work if you followed us a little bit he's done the the robot that you saw in the first slide he's done a number of other artists work and also by the way our website rived app has got all of this stuff open to the public so most of JC Tunes files are for Keable you can go on the website fork them to your own files if you create a profile on rival app you can import them into your own into your own files and if you need to learn how for example JC tune achieved that simple rotation you can take a look at that directly yourself and learn from that so we know the tutorials are still missing but there's there's an open design initiative in there there's an open design mentality where you can learn directly from the artists so I don't know if I answered your question yes yes thank you you're welcome so thank you for your talk it was great so I have a question about do you think it would be ever possible to have we to have widgets insert inserted into the rife animations for instance when you showed us the philip animation jumping with the phone I was wondering if it would be ever possible to insert some flat row widget into the into this animation I well I know that we've been we've been used our flair actor has been used with a number of other widgets with opaque as a hero widget and so on but inserting a widget within a flare hierarchy I don't think that's currently possible you'd need to you can do some hierarchy manipulation with the controller's so if you want to create nodes or swap in for example there's a thing called the solo node which is the node that allows you to decide which child is going to be visible in the next frame so for example if you have an eye and you want it to be open or closed one frame to the next you can switch between its solo children but we hope that you use drive to build those things if you have some strange thing that you want to pop in and out I we hope that you use right for that so I guess not at the moment I'm sorry hi great talk it's not a father question but I'm interested in how the rifle app app actually works is that with assembly or do you mean the the editor that you saw yeah so yeah it's well it's not a web assembly it is using canvas kit which is a web assembled version of skia so it's skia the engine that runs flutter again it's an open source C++ engine and on their website they provide some ideas on how to build it so yeah it's a you use some script and you build in you and you can run the wasum version of the engine so it's still using a GL context underneath because it's it's attaching to that to perform rendering but it's using skia primitives and this is why actually our our faithfulness is so good between the editor and slaughter because the engine underneath is exactly the same so the so if I import animations in flutter does it have other extension points and not controllers but maybe to insert your own window boxes or your own canvas paint algorithms sorry I didn't understand the question can you say again so which other extension points are there for painting like for example when the box items which you insert from the flood shape yourself like not from the wife editor but from the application logic alright we can talk about that later maybe I'm sorry I didn't I think's well I haven't used write any yet yeah but maybe sometime question but I'm interested when I download some a file yeah right file can I get the list of animations maybe from it or I have to read the documentation for this file to get you know which kind of animations can be played or do you mean encode with the API yeah I just found a samurai file yeah without any information about this yeah and how I can I know the kind of animation it can play right yeah well there's a way yeah you can access the actor animation so the art board is the container of the hierarchy and this is an actor art board in the flare flutter package once you get access to that and you should because you're accessing the file itself you can grab access to the you can list those animations and print their names if you don't know what their names are so you should be able to get there yeah absolutely otherwise you can try to import if you've got a there's two modes of exporting in Drive there's an exporting a flare file and then there's you can download a copy of your own file so as a backup get a backup copy if you download that then you can import that in the editor and see what the what the animations names are but otherwise yeah through the API you can definitely get the names of the animations and by the way this is absolutely not a dumb question there are no dumb questions thank you thanks and so my question is how does the flare file looks like that is it some binary format or XML or something alright well there are actually two formats I can show you if you click here you go export you can choose between downloading a binary file which is our most supported and compressed version of the flare file so obviously being that binary it's super efficient and there's a JSON format so JSON format is going to be human readable you can peek through how we build the hierarchy and in the file and that is exactly what you see imported in in flare flutter and if you want to make small tweaks Jason is perfect for that so you can you can you just use JSON to read what you're actually building and use binary in production maybe if you want to be more efficient it's gonna be faster at loading of course so and I can manipulate the JSON by like manually and adjust some content of this fat yeah you can because what's being read in the file is what's been reproduced in the in the runtime we don't encourage you to change up the whole hierarchy structure in the JSON file I think that that's gonna lead to some pretty bad results do it in the editor but if you want to see if a color is better the druther than the one that you already created instead of having to go to the editor and change it up again and export it again you can simply do those kinds of things you want to change the scale maybe you want to see if an object scale the simple things yeah you can definitely do it you can also do hierarchy manipulation but then if something breaks it's up to you keep a backup copy or export again Thanks you're welcome hi how does arrive arrive animation handle its scale how do if I let's say I wanted to build an asset like a switch or anything it's fixed scale does it scale to the dimensions of the container or its surrounding it or does it have a absolute scale and I'm aiming at different resolutions well it's using in in flutter at least what it's using is the size of the widget right so being a render box it's got its own size and then it being in its own transform space that the scale is going to be relative to that transform space so let's say what can be an example well I there's everything is contained within a certain AABB which is an axis aligned bounding box it's basically a box that surrounds all the objects and currently in in in view and and that's basically where the scale is computed right so the scale is computed in flair space that's why also we pass through the controller the view transform matrix because that way you can't get from one transform space to the other in the space controller demo for example moving the object in screen space that needed to be computed from the flare space so you invert matrix and then you get from one space to the other but the scale yeah it's in flair space so if you if you place a widget on screen and you've got a certain size of that widget the scale is going to be relative to that size okay thank you hi my name is Reba I'm very new to arrive app I just looked it up once but I got a question to the Xbox looking like controller it it was fully a 3d object and I just wanted to know in ryf which components do I have to use Oh which tool do I have to use to create an 3d model object well everything we do is in 2d but there are some things that you can do to make things look 3d and the controller in particular was built by one of our animators so I can get in touch with him maybe and let you know but you can since you can't the form vertices you can obtain some pretty believable 3d animations also our my first slide which was the robot this guy this has been built by JC tune instead and this also gives that fall through 3d look it's still all 2d but it kind of looks 3d right because it's moving a bit forward a bit backwards so I you might want to talk to our designers if you want to know better how to achieve those looks but there are some good examples on the rive website thanks welcome I have a question so I'm here and at the back in the back oh yeah sorry you currently rebrand it from flair to arrive so can we also expect the class names change absolutely yeah yeah we are aware about that we didn't want to break everything we want didn't want to go from flair to drive in all our documentation right away because we feared that a lot of people a lot of people came to the booth today saying what I strive is it similar to Flair and it's exactly Flair it's just now called Drive and so we we still want to go through a smooth transition it's gonna happen at some point yeah all it's gonna be arrived in the know that means when updating the package this might come one day yeah well we're thinking about smart strategies to do that so that it doesn't kill everybody that using the package right now thanks ok thanks Umberto I'm used rive quite a long I have experienced it that iOS performs quite better than Android so does does do sun emissions on iOS performs better because of Swift runtime he used in that or no well in depends did you use the the Flair flutter runtime or also on iOS or we're using our Swift runtime it's all made in slaughter so that's supposedly we should see the exact example in this case I think it's either the device or maybe the emulator it depends on what you're running it it on - generally apples GPUs are really really good so if it's a physical device we know that iPhones have amazing GPUs and they can perform better so it's entirely dependent on that if the if the runtime is the same if it's still Flair flutter then it I don't think that there's anything in the code that's making a difference it's more a device related thing so under the hood Flattr doesn't profits and this Swift runtime you have what sorry so when I use a flatter app it doesn't use this Swift runtime you have no no no it's all it's all flutter it's still using flutter so the the Calder draw calls are in flutter but we do have a swift runtime which is native to two Swift and and which runs on iOS and Mac OS that that's still using skia so under the hood it's still the same engine but that's a whole different piece of software and it's still open source so if you want to take a look at that to your absolutely we're thrilled that you're gonna do that if you're gonna do that so I think we're finished with the time thanks a lot for your questions if you got more please come here [Music]

Channel: Flutter Europe

Views: 7,978

Keywords: flutter

Id: FY9f34a0tV8

Length: 45min 9sec (2709 seconds)

Published: Thu Apr 02 2020