Flutter INSIDE React? react-native-skia is wild

it's no secret that I open videos with it's no secret too much I've made it a point to not do that anymore I've also made it a point to not talk about flutter as much but uh I guess I have to kind of this is not your usual flutter video this is a react native video why am I talking about react native when the video titles about flutter it's because react native is becoming a lot more like flutter with this recent project that's just about to ship 1.0 what the hell am I talking about react native skia what the hell is skia what am I talking about here what is all of this well don't worry we'll be diving in before this is going to make much sense we need to start with skia if you're not already familiar with skia it's the core that powered flutter for a long time that somewhat recently changed and we'll talk about that in a minute but skia is the rendering engine that flutter uses so when you're rendering some text that's not using the native text renderer on iOS or Android it's rendering it in skia if you're rendering a picture it's not using the native picture whatever the hell your platform has it's rendering an image within skia this engine was built to have a standard platform where things would always look the same regardless of where you're using it think of it kind of like a game engine like Unity or Unreal Engine where if I make something look a certain way in unity and then I play it on my Xbox I would expect it to look exactly the same as when I play it on my PlayStation and those types of things where you're trying to make sure all of these platforms behave identically and you're trying to do things different from what the native platform allows skia is a really good solution for that and I've seen it used for some really cool stuff one of the coolest use cases I've seen for skia is RVE R is a competitor with things like after effects specifically when you're using it in web apps with lah I'm not a big lah fan it's a mess it's an absolute mess and RVE was built to be a nice alternative using these fancy new tools for rendering it can be used for everything from games to fancy animations which is the main thing it's used for if you want really elaborate animations like what dual lingo has Rive is one of the best ways to do it performant and it really proves out the power of using the ski rendering engine and it's also my sighted go-to of the only good flutter app there are some there's now there's exceptions but this is the only app I've seene where flutter actually makes a ton of sense for what they're building and how they're building it because they're building an animation studio the way that text renders is no longer as concerning to them it's everything else and I'm actually really pumped with what these guys have built so huge shout out to arrive they've really pushed the limits of what you can do with the stuff and their whole studio is built in flutter for web although I've heard they're rewriting it with some crazy news stuff I haven't looked into that too much yet but I trust these guys and if they're saying this is the way to do it is one of the better ways to do it for sure Taylor from chat just linked a really useful article here that will help break this down a little bit further not going to read the whole thing but it will be linked in the description if you're curious It's a breakdown of how skia and flutter relate when you're building the core points here are that flutter is an open source UI toolkit developed by Google for building natively compiled apps and skia is a powerful open source 2D Graphics Library by Google which has various apis that work across various hardware and software platforms for creating high quality Graphics in their applications flutter is almost a template for building on skia with a bunch of to build tools with a bunch of UI layers with their material UI kit that I know a lot of flutter devs don't necessarily enjoy flutters all of these pieces on top it's kind of the glue between UI components Dart and the skia rendering engine where things get even more fun is that flutter isn't always using skia anymore they've had enough issues using it on iOS that they've kind of given up and they built a new engine from scratch to try and solve all of these issues specifically the early onset Jank which is when you first open a flutter app things are rough for a while and that's been with this new rendering engine which is very different and it also means that the very least right now on iOS you're not using skia for new flutter apps using the best practices skia is still used on Android as far as I know uh I was asked to not quote Luke on a thing so I will not do that but uh yeah skia seemed to have a lot of issues on iOS so they're building this new model on top of impeller instead and I will say from the apps I've played with that use this on my iPhone it feels comically better than the flutter experiences i' had in the past we're not here to talk about impeller we're not here to talk about Rive we're not here to talk about really flutter because the thing that's interesting today is react native skia because a lot of developers in the react native World realized skia could actually be quite valuable to them why would we want this when we have native bindings in case you're somehow not familiar with the core differences between react native and flutter an important detail that is most of the core difference is that react native despite being in JavaScript is actually quite a bit more native than flutter because react native doesn't have its own rendering engine or way of doing UI it's it's just a JavaScript layer that tells the native platform what to put where so if I'm building an app in react native and I put it on iOS and I put it on Android and I'm using like the platform native tab bar bar or button those apps are going to look different on those different platforms because those different platforms have different core Primitives so react native doesn't guarantee a perfectly consistent standard experience across different devices and platforms it's not a web view like so many people like to think it is you're not actually running a browser in your phone when you're using react native on your phone you're still running a native app in fact flutter is a good bit closer to what people seem to think react native is where it's not using any of the Native Primitives or any of the Native UI that your phone has almost everything you see in flutter is either a recreation or of native UI or just straight up a screenshot of it with text overlaid on top which is why if you use a flutter app you'll actually notice lots of quirks in the UI that if you're used to the Polish iOS experience feel wrong this is one of my favorite examples of the curse that is flutter after 7 years flutter now renders text on iOS almost correctly where before when you use large font sizes they just spaced the font entirely Incorrect and these types of things add up in the experiences it feels Jank it really does and it was basically impossible to make an app that feels native with flutter because underneath it wasn't native where with react native it is native underneath which means when you render text you're using native text rendering tools when you're rendering a picture using native picture rendering tools when you render a scroll view you're rendering a native scroll view which by the way if you're using a flutter app really good way to test that as flutter scroll with two fingers and it Scrolls twice as fast I'm not joking so why would we ever want to adopt this stuff in react native land well not everything has a good native solution like animations if I want a really fancy animation that might not be a thing the native iOS platform supports and it's certainly not going to be a thing the native Android platform supports but I have a rendering engine or platform like skia I no longer have to pick and choose where with flutter text is worse but animations are better react native text native parts are better but animation's way worse react native Ski's goal is to solve that problem if you haven't heard Shopify is all in on react native like all in all in there's even been like funny tweets from Toby the CEO about how betting on react native has been so beneficial to their business and other companies that bailed like Airbnb missed out on striking gold shifi really does feel like they struck gold with react native when compared to the performance of their native apps it's often even better because of how much simpler the code and architecture ends up being and how well optimized a lot of those data bindings are especially after the architect extra overhaul which I have a video that's hopefully already out by the time this one's here it'll be linked in the description if it is react native skia was built by Shopify because there were certain things that react native didn't do great certainly not for multiplatform mostly animation and they wanted really nice animations for a lot of their apps things like the shop app which is one of the most beautiful apps I've ever used has a lot of fancy little animations and getting those details just right across platforms yeah good luck doing that in react native or even just native if you can render effectively what's a mini game engine inside of your app to do those types of animations that's a way better experience let's see what they have to say about this here before we dive into the new announcements with the latest advancements in the react native architecture allowing direct communication between JavaScript and the native sides we saw an opportunity to provide an integration for skia arguably the most versatile 2D Graphics engine we wondered how should these two pieces of Technology play together last December we published The First Alpha release of react native skia and 89 releases later over the past 12 months we went from offering a model that decently fit react native and ski together to a fully tailored declarative architecture that's highly performant one render three platforms and Counting interesting obviously react native runs on IOS and Android and not as obviously it runs on web it's cool seeing them targeting all three because again react doesn't necessarily care where you render it I'd almost compare this something like react 3 fiber which if you're not already familiar somehow uh you should watch more of my videos because I talk about react 3 fiber a lot it's a 3js based canvas renderer for react so you write react code like a box component you render a canvas and you pass it components the same way you would in traditional react but these render in a 3D space they've even made crazy stuff like 3D Flex boxes and uis and this is possible because react doesn't care where its virtual Dom gets thrown to it can go to the real Dom it can go to Native it can go to Canvas and now it can go to skia which is actually really cool especially if you have a little bit of your app that's in skia and then the rest is traditional react and whatever Dom you want to render it to this is dope and here we see really consistent animations across all platforms the synex is actually really nice this looks quite a bit like react 3 fiber which is cool to see but here they're drawing a blend Color Group and you can just render this and it works this is so cool and they check to make sure the image matches their saved screenshot because the render is not coupled with the Dom or native apis we can actually use it for testing in node that's really cool that is actually nuts on react native the skia host API available via the JS interface the JSI which again check out that react native architecture video if you haven't already we go deep on the JSI there exposing the C++ skia API to js on the web the skia API is available via canvas kit a web assembly build of skia we like the canvas kit API from the get-go the skia team did a great job of conciseness and completeness with this API it's also based on flutter kind of which showed how great the relevance was for their use cases we immediately decided to make our host API fully compatible with it an interesting side effect of this compatibility is that we can use our renderer on the web immediately in fact the graphics motions we built for the original project announcement were written using react native skia itself via remotion a tool used to make videos and react I have been wanting to make a long video about remotion for a while you can actually create an edit video with react code which is an insane concept by itself but the thought of building 2D animations with skia also with react in order to generate a proper Fancy video is so cool and it's really dope seeing this stuff people are doing with these tools and Technologies and the way that they're composing together again this is this is kind of the magic of react that no other ecosystem has where theoretic Ally could you have attached flutter to something like this possibly but what are you going to attach it to you have to invent all the other parts we're even seeing this with RVE which I mentioned before where they have to invent so much in order to make their Animation Studio whereas with this once you get skia working with react all of a sudden you can add it to all these other react things and you end up with this crazy composability that just doesn't exist in other places it's so cool and this is honestly the magic of react that just isn't matched anywhere else in software this level of composition is almost Unix like and how small and capable these parts are and I'm so excited for skia to no longer be a thing I have to adopt flutter for and instead it's a thing that I can use to make my react and react native experiences even better than they were before so cool one more important detail here is that react native skia works out of the box with the Expo Dev client and the Expo go client so if you just install the Expo app on your phone spin up a project scan the QR code and start using react native skia it works that's so cool I'll dig into all this in a minute if it makes sense to after the announcement video but I want to watch this it's a bit long but I'm actually really excited so let's do it this is the react native skia 1.0 announcement hello react native developer that cut wasn't me holy hello re oh oh oh I'm oh don't do the pan shot if you don't finish the pan shot I I'm known for aggressive Cuts but I sit still to the the best of my ability don't don't do the the Drone shot in if you're not going to hold it h developers I hope you are well William here recording from beautiful zo Switzerland thank you lending for trusting Rea native skia while working on this project we have seen incredible examples of Indie developers flourishing startups and large tech companies using it to power incredible experiences in react native Shopify of course uses it to power Rich analytics to millions of entrepreneurs void pet which won the Google Play Store app award last year uses re oh void pet let's see in chat who knows void pet and specifically who made void pet I want to see YouTube and twitch chat let's see the name there we go there we go our boy Ben getting featured totally deserves it I have to check in on him it's been a bit but yeah Ben aad the original hot react YouTuber I know that I'm building on top of the legacy of very talented individuals but it's without question I would not be doing this if it wasn't for Ben so uh cool seeing him featuring inside of this type of stuff but yes that is Ben awad's work and I also interviewed Ben over a year ago it's probably close to two now yeah year and a half ago where I interviewed Ben aad to figure out what the hell he's been up to lately and I'm surprised this video didn't perform better it's actually really interesting just chatting with Ben all about the craziness of what he was building and ended up hanging out with him after helping get some Investments and cool stuff really cool to see him featured in this back to this void pet went all in on react native which honestly I was unsure of initially because it's a game I would have guessed that it would use more game friendly Tech you know like Unity but since Ben was already so familiar with react and specifically wanted a lot of like the hot reloading and like remote overthe a type stuff that you get from going all in on react ended up being a really good choice but also animations in react are no nowhere near as complex and capable as they are in something like skio with flutter so having access to this is a huge win skia to bring us these rich and delightful creatures tlink also uses reative skia to power analytics and other delightful effects ton brings sport fans around the world the best of the six Nation okay that was a smooth animation it's silly to to overhype that but I don't care that was really cool watch how the board shifts on its like Z access ton brings sport fans around the world the best of the Six Nations Orca the leader in C navigation uses re native skia to bring Rich navigation data to its users stocked the leading app for indoor climbing uses Kia to guide climbers with their problems and the Zenit Academy Plus app uses Rea native skia to provide incredibly engaging studing experience and flip cards the photo matching game is just incredibly playful we are absolutely Blown Away to see everything you have built using reag Kia so this morning we would like to take the next step in our relationship with the industry and to get things started here are the people who are championing Rea native skia from Shopify Coen gray principal developer of mobile and Mustafa Ali director of engineering good morning and welcome to this special announcement if yall don't know Colin he's actual wizard he gave my favorite talk at infinite Reds conference last year chain react this video technically isn't sponsored by them but I do have a code that gets you money off it's one of my favorite conferences I've ever been to if you're into this mobile posting stuff it fundamentally changed my understanding of how mobile devs think and approach problems and Colin's talk was incredible we chatted a bunch after it's the conference for people who nerd out about the stuff I'll have a link in the description with my discount code in it the code is just Theo so chain Rex in July it's in Portland Oregon it was an awesome event and you can use my code for Discount 50 to 100 bucks don't remember exactly highly recommend it check these guys out anyways we are celebrating a significant milestone in the world of react native development the release of react native skia 1.0 2 years ago shopi saw the huge potential of a small open source project from Christian felge and William Candon and we've been thrilled to see how it has grown to become a main stay in the react native ecosystem within Shopify we' watched it evolve from an interesting proof of concept to powering our most visually stunning views there's simply no better way to render custom graphics and interactive animations as we'll see from today's demos we have the privilege of having five members of the react native ski Community walk us through five features that are part of this release thank you to all the people who have contributed bug fixes and reported issues you have helped make this a community-driven project and we couldn't have gotten to 1.0 without your support watch this whole video if you want to hear about all of these inner workings in details the stuff is really cool I want to see if there's any like really good Demos in here that are worth showcasing but react native developers wait a second what unites us is our love for reacts declarative model there is something so incredibly joyful about the GSX syntax and reacts composition model and we benefit from a strong type system a type system which by the way has become lips and bonds her head of programming languages which were typed from the ground up but I digress so when we are looking at these examples what we really want to be looking at are this beautifully deeply nested it's really hard for me to not go on a type safety and jsx tangent right here because everything he said there was based in true and it will get me in trouble if I go any further okay this one is funny because I know this type of thing seems relatively trivial to like drag and drop stickers onto a picture but I know from experience this is not the case specifically I know that it took twitch a lot longer than you would imagine it should have in order for them to add the stories feature on IOS and Android because they had to have a way to add emojis and stickers and things on top of an image that was platform consistent across IOS and Android and that ended up being a massive challenge for them because getting that type of layer to work on both platforms was nearly impossible whereas here you do it once with react and react native skia and it just works it just works that is magical there's whole categories of industries that have been basically impossible to build outside of just doing the IOS app that are now not only possible but like economically viable and worthwhile this is possibly the biggest release in terms of letting Android be there day one of almost anything in the modern application development world it's it's silly like this one example just made a lot more of it click for me but I know how painful it is to make something like this and I know many developers who have chosen to not support Android at first or even at all because of how hard it is to do things like this by the way that literally includes Google Google has had many an instance where they shipped an app that was really interactive on iOS first and then delayed releases for Android and then gave up cuz maintenance was too expensive like the inbox app which was their alternative interface for Gmail that was actually quite good but maintaining it and all the animations and gestures and stuff was enough of a burden that they said it and act the project those types of things become so much more viable with a platform like this and I'm way more hyped now that I am seeing practical use cases that aren't just animations but are actual new ways to render and create things as a developer and even as a user it's also particularly ironic that the people who tend to hate on react native the most are like the anti- web technology people that are huge Linux and Android neck beards so uh eat your platform is only viable because of electron and now because of this forance we use Lear on the skia canva now let's follow the thread of integrating with reanimated and let's have a look at three kinds of animations interactive animations like diving into analytics using gestures for instance large animations where we have a simple animation driver hero gesture and we use it to interpolate the state of thousands of elements and fully scripted animations there are no surprises there the whole world is known in advance the art team at du lingo is so cracked did you guys see that animation what the hell well hereo gesture and we use it to interpolate the state of thousands of elements and fully scripted animations there are no surprises there the whole world is known in advance that transition what the how did they do that so seamlessly there are no surprises there the one where it turns into blocks I'm going to like frame by frame so you can see here oh it's just the next frame does that but their eye levels are the same which allows it to feel smoother world is non in but that was so good though and using the color background changes to justify like bigger shifts that's such a tasteful subtle animation the amount of things that happen in the like 4 seconds there is just insane and also limited by the 30 FPS on YouTube cuz I know that's smooth as hell on mobile one more big announcement that is part of the react native skia 1.0 is that canvas kit JS is now also released and in technical preview very exciting stuff today we are making canva kgs open source still a very experimental project but we think it has strong legs and the way it is built is very interesting there are two layers the first one we call it canva 2D next we asked the question how would the canva 2D API would look like if it could power reative SK experiences out of the box it would contain support for image filters and layers as well as providing support for basic path operations and this is effectively what we have built with this layer once we have our canva 2D on steroid up and running all we have to do is provide a compatibility layer if we do our job well a skia client should almost be unable to distinguish between canvak kit wasm and canvak kit GS we wanted to put this ID to the test and run canvak gs on the flutter web app which has absolutely no prior knowledge of can pretty hilarious that they're testing their developments to make react skia stuff better by using their new engine to power a flutter demo because skia and flutter are so tied if you want to know your skia polyils are working don't test it on react test it on flutter it's actually really cool that the flutter ecosystem is going to get much better because now there's a bunch of web experts diving in fixing all of the problems with flutter and his web compat shout out to Samir he always finds the craziest stuff in these tools the issue with flutter web not using web standards and using its own crazy rendering engine is that all the things you expect the web to do for you it just doesn't like selecting text in order for flutter web to work with text selection like when you drag and drop to select some text to copy it since flutter web's using canvas not native text rendering they actually will put a Dom element of text in front of whatever you're doing here overlaid to select exactly what you had selected in the text and it misses like 30% of it like this is what it looks like to select some text in a flutter for web app because the flutter team know to them is just not that interested in building things in the web flutter is trying to build a new rendering engine and a new way of thinking of applications they're not trying to follow web standards react developers are generally speaking a little more interested in web standards I know they get a lot of for it but they are they are trying to use the web in the browser for what it's good for and as such when you render some text you can render some text the reason I'm bring this up here is it seems like a lot of the things that were kind of missed by the flutter team because they're not focused on web are going to be improved meaningfully because a bunch of react native devs and react as a whole devs are going to start using the engine that powers a lot of the flutter stuff it's just cool to see the web being embraced by flutters engine even if the flutter team aren't the ones doing it once again it feels like react just kind of always wins not because it's by default the best solution but because its composability and its gigantic Community result in these cool innovations that happen elsewhere being brought in and it is really cool to see the strengths of what flutter could do through the skia engine being brought all the way over to react native and also react web so once again huge shout out to everyone at Shopify and all of the other contributors who made this possible shout out to everybody who built flutter and skia to the point where these things are even usable and everyone in chat help me find all the sources throughout this if you want to learn more I highly recommend watching that whole video by William CU it was really really good until next time peace NS

Channel: Theo - t3․gg

Views: 62,134

Keywords: web development, full stack, typescript, javascript, react, programming, programmer, theo, t3 stack, t3, t3.gg, t3dotgg

Id: 2l9Wm-8TxlA

Length: 24min 17sec (1457 seconds)

Published: Thu Mar 28 2024