Bringing webgl to react - Paul Henschel aka @0xca0a at @ReactEurope 2020

my name is Paul Henson I'm a German open source developer I am employed in a small Swiss company called awv and today I'm going to talk to you about a react library that I made contract three fiber basically started off here because at work most of what we do is related to CA d 3d industrial design and at some point I decided to push my problems more into the open to develop as to develop in the open source scene and this is how we act the fiber first started and I give you small impression one of the first streets are made it treats web by CD system we develop instrument written in reactor and freeze taking shape still baby steps there was a very early beginning of what we were doing on the client but you can see the makings are out there the front end was mating react and at this point the scene related stuff was made in three.js plain three.js eventually the problems became grave enough so that we have to scrap it because complexity grew in scale it was very hard to work with nobody understood how to maintain this anymore and the project that we are working on now Scott Billy be you er Li you can check it out blue dot IO and this is completely based on react V fiber on that client and this is where really it comes from the sistine region and you can see here some 3d stuff floating around a perspective shift but what it really does at work what really does is it's like a small CD system for the web something like if you have ever heard of it blender like a modeling software you can make complex constructions models you can cut down stuff even I don't know I can play together your own CD system if you want to so this is the history of we had three fiber this is how it all started you can check the project out here it's on github it's completely open source it's on the react spring organizations organization there's lots of fancy demos on here lots of informations I don't think I will go into the basics and fundamentals because you can check out these links and start learning yourself I want more focus in this talk on how react helps us to bring some sanity into our work and in order to explain what reactive fiber is basically in technical terms it's a react reconciler for three days I know that cause that sounds complex but bear with me for a moment so three days for those of you that don't know what this is it's in itself an abstraction for WebGL so pushing graphics onto the GPU I've heard this often that some people say 3d on the weapons is like gimmicky but I don't think it's true 3d doesn't have to mean I don't know flying dragons and fireballs it can be subtle you can make stuff into your HTML you can basically do a lawsuit if you can say one thing about WebGL it's very very fast it's super performance FPGAs is an awesome project they abstract it to go to pitch because what jail is very prototyping very low-level web tier doesn't know what a camera is there's no scene and stuff like this so 3GS basically abstracts it in a very nice way gets class-based and each of these constructs is completely self-contained and it's great that way so when I started to look into this I came across this project it's called react 3 renderer by toxic fork and this is I guess what most people will call a binding what is a binding it is when you've wrap each and every constructor objects into its own component the problem is at 3GS hundreds and hundreds of components so this is what they are doing here all of the shapes of the geometries or the lights material objects it's a it's one hell of a task to maintain us and I guess the problem is also 3GS because they introduced breaking changes practically every month so you figure out that something is wrong when when it's crashing that's how it works they don't have proper versioning so to figure out what changed you need to go to change sets and stuff like this so I guess I've must have been the reason why this project eventually left off at night I think three or four years ago so the problem with finding sisters as a user you hand over the success of your application into the hands of someone that may or may not maintain this forever and if they stopped maintaining it it's it's it's basically over so as an example every geometry that was merged previous over the last two years is meant here so this one is tied to a very specific 3GS version so we knew that this couldn't be the solution to our problem so I looked on and I found this let's call it react reconcile and it's an awesome project one of my favorites actually I was listening to some podcast of the original react again so to say and desert react is something very special they saw the potential that it has because it's not bounced to a singular platform that can render everything and I'm so glad they follow through with with this vision and this project is what allows you or anyone to go through with it because you basically create a smaller config and as config teachers react how to render on a foreign platform react Dom for instance is a reconciler that you record services I have spans and divs and h1 I have certain colors in a teacher's react and says please this is the data and now expressed this in HTML and as many more like react native expresses views in iOS and Android and so on so react free fibre is basically this it's a reconciler then I guess the most beneficial is that it's not reliant on a version it's basically future proof I showed you the first-ever per commit that I made I copy pasted us in a code sandbox so this was I think 50 months ago it's how much 169 small s and in these 160 lines I could do everything that reactor here and read it and more and the way it works is as follows as you can see here we are now able to declare something that used to be imperative previously you would write new 3 what's that group and in reactor j6 you can just place an element in in here it's a group or mesh or camera material and so on and the way it works is as follows this is the conflict this is the conflicts that are that you need to to feel it's this punch and I guess the most important function is this one create instance that you get past a type this is the type that you get here in dj6 so if I do anything it doesn't make sense no because treat analogous doesn't have this but then the records are that node crashes because it cannot find anything in 3GS so we moved us very quickly my job in here is to create a object a real 3js prototype for instance group i do this by just passing in the type in here and I had a back to react react doesn't know what is this it does know how to deal with it it just maintains it for me dorsen and this there are a couple of other interesting fortunes like at the end it says what happens if someone offends something and they hand me the instance this will be the group or the measure of the camera and then it's my job to upend the child and the same happens for removals inserts prop updates and so on and with very little code you have something that's future proof because it's it works forever actually it's pretty cool because this red cube it's using the newest three.js version in this state was like I said 50 months old so it could still handle everything that we just did without any change whatsoever so that's why I went for the record server and obviously it grew from there but this is how it started the original problem that we faced at work I had to do with user interface making separation of Concerned neutral attractors made to small demo to showcase it so what we have here is a cube that's the first property you can easily recognize and it's yellow orange so that means it has a material it's also spinning in three directions or I think it's 2 X & Y you can hover it if you have a little gets red and if you click it it gets enlarged it's bigger in scale so these properties are very easy to see they're obvious to describe but to express them in three.js is not so easy I needed 70 lines to do this and obviously most of it is typical 3GS boiler court 3 Jessa's very boiler plating so you can forget that you can just sit down if you have an awesome idea and prototype it we need to way through through through more and stuff like this it sucks a little bit but really fiber abstracted away completely and here's the same version in future every fiber I night a small comparison side-by-side and you can already see that the difference is pretty big really fibers is not even half of it and if you have ever used 3GS you will have noticed that your application is split into a couple sections I have calibrated them so we have staged your variables live there this setup when you create other primitives the scene cameras and so on then you have the view this would be are you filling the scene in this case our scene just consists of just a two cube right here and then we have a section for responsiveness responsive means that 3js is not responsive by default so if you have a canvas that needs to adapt to size changes you need to code it there's a section for events so you can hover it you can click it free Jess does not have events they only after a casting so basically you also have to do it manually and then next the render loop around 60 times per second and this is where you commit to the e GPU and the problem is right here we know what you wanted to do we had a cube Orange spinning hover State active state and I'm searching for the cube now you know what you can see it everything is lighting up the cube is involved in every piece and corner of the application it's basically involved in everything obviously it's involved in the creation because we need to create it it's involved in the responsive section and it mutates it because the cube grows and shrinks if we change the viewport its involved in the event section for Mouse morph and mouse click because we can hover it and click it and it's in the render loop because it gets to to spend a little and if you look on this site though on the right side you can see that we don't have this problem the entire complexity has been encapsulated unpacked arranged into this one component and that's completely reusable at this point this component is managed by react it's bear sits on State it's responsive it has access to the Aranda loop it has some events these events are the same that you know from the Dom they are pointer events which is also super beneficial because you get to use any library that can use events you can use it now with meshes and materials and so on and it's reactive because you can react a free response to properties so we have the same thing over here running in in react and because it is react we can now reuse it now I have two cubes on the scene and so that's push this one elsewhere and this they're completely self-contained they have their own events to have to run back to state they could share sate I could use Redux basically everything and doesn't say huge you chop set for for us and I guess for everyone there's a there's another perspective I wanted to talk about and this goes a little bit into performance and mixing 3d with HTML someone linked dispatch to me and I thought it's crazy because I've never seen a experience like this and I just knew that none of the tools that I have had my disposal could do this and at this time I was already involved in react spring so I knew a thing or two about performance and animations but this was something completely out of the ordinary I couldn't do it and I started to investigate and ask around I have couple friends working in agencies and they basically explained to me that people that do this aren't in called front end developers they are three 3d talents demonic okay go on under and I guess the splits in coming in the communities is because fronting since Nadia seems so different there's a huge gap in between the two and I try to to soft us a little bit with really fiber make it a little bit easier for people like me for front-end developers to tap into this this huge potential and starting off with performance there's this article from written by Paul Lewis in 2012 and it's teaching how to make a parallax looking at this and I guess my short attention span wasn't as short as as it is today so I'm reading this really line by line and I have a code editor and I'm trying to realize the examples that yes and he says of course you have a different you have a sphere and some text and just use the most obvious property which is top so you place these elements on the screen using top property I'm doing this and it's basically lagging like hell and he says of course that's your fault because you didn't use transforms sunshine transform still lacks he says yeah because you need 3d transforms trying this it's still lagging and he says you haven't still you for a fault because he didn't use the will change property I'm trying will change and it's still lagging like hell and back and forth in the end he says oh why don't you just use canvas and WebGL and I felt a little bit betrayed or pay status because I thought why the hell did I just learn all this if the dorm basically just chokes trying to shift ten elements on the screen and I think it's even worse than us I think we have forgotten how fast computers really are I found this example it's not stand by me but it's pretty awesome what we see here is just a cube running up and down and then how we can scale it so we have two by two by two cubes and now we have five by five ten by ten twenty by twenty this is already an amount I don't think the dome code hello if you have twenty by twenty diffs and you're shifting them around all at the same time it will just choke but now I can go higher I mean this already looks crazy to me 500 by 500 and now we have 1 million objects if I go out appears like a flat surface what if it's the wind you can see that it's actually still nice all these cubes are moving at the same time 60 frames per second so computers are very fast and the GPU is very fast and I don't my guess because of years of backward comparator I don't know even what is the reason the Dom was very slow so I started to investigate how to make it easier to incorporate 3d or it doesn't even have to be 3d can be 2d but on the shader on the GPU how to incorporate this into webpages and the first thing I want to show you is a relatively new library average code dry it's a German form of three and it's a collection for utilities helpers and so on at one of these th Gmail component this one and the way it works is like this we are back to our cube example and now let's make it not spin how about making it go up and down so I have my mesh I gave it a he reference it's over here and with this reference I can access the actual object mutated this is pretty special because 3GS has a render loop and it's totally fine to mutate you view inside the the render loop it's safe inside use frame and if this component unmount everything is cleaned up all the side effects go away so basically that's accessors mesh currents we know we want to move it up and down so I'm accessing the wipe a property or in the position I take a sine function very easy check my clock that I gets get elapsed time and now it's traveling up and down and let's put some HTML right into the canvas I'm using the HTML element from this library called I place it in here close it again let's say h1 panel and and there you go it's very easy to mix these two so this sediment is right like smack in the middle of the 3GS scene but it gets transforms transformed automatically now so that makes it very very convenient and comfortable to to mix these two two words and create awesome experiences I could also I don't know let's try orbit controls place them into the scene and so now I can turn it around zoom it panners so that's how you can that's how you can mix so going back to this one because it adheres to all the principles I've been talking about it's mixing and matching Dom content which is fully responsive accessible with 3d content as you can see here in the perspective shift and this is a very interesting topic to me to use the Dom for typography for funds for language responsiveness and the GPU for everything else and this can create some pretty smooth and exciting speed experiences if this interests you I've made a full tutorial about this on KO drops called score refraction and shading effects in 3GS and react and that's teaching you the basics how to bind objects to the scroll bar how to make some match Dom content or 3d how to put some shader effect so much and in the end it looks like this is pretty far out I guess it doesn't have to be yes as drastic as this one but I just wanted to show that it's possible I've made another one this is a site scored on the harder you yank on it the more pronounces the effect and you can also hover over the mini-map and drag it does my dad by the way you say hunter this this one I wanted to show you because it's pretty special to me this is the first game I made and I thought that against will be completely different because I come from front end but it wasn't like this it yet everything is a state machine and you can see here how clean and tidy it is because it's react everything is packed away and encapsulated and now we can also watch reload stuff let's take out the stars the track planner roots and rocks and you see not and so everything keeps going because the state hasn't been affected let's mount and bacon and maybe remove effects because makes the most difference this is how games look without effects put them back in so basically to the only thing that's left us I'm going to show you a couple of flags because they're spend the ecosystem around every fiber has been pretty active and these are the first libraries that sprang up this one is caught use cabinets for physics using the famous lip what's the court cannon Jas and it allows you to put physics into your own applications very easy to use its hooks based and it's not as complex as you would think it's pretty cool and this is react spring a react animation library and it's also completely compatible to react three fiber for instance in this example on the backgrounds the eight that you see there in green this is HTML and this ping football and this the strap this is really 500 or 3GS and select spring us animating both so this is made possible by attic in verse 9 you can share Springs and animate everything and last but not least there are a couple of others react post-processing this is for the post-processing library on NPM which is for effects will make it very easy to use effects in a decorative way then there strike or die I showed you earlier this one could be exciting reactive SAR it's for virtual reality it will basically even deliver utilities for this to make it easier use gesture like to the name implies for gestures moving things around and so on so I am pretty much too sorry for sticking with me and my micros voice I'm completely messed up of pyro so I hope you enjoyed this talk I hope it interests you if you want to check out these libraries and let me hear on Twitter or on github if you have problems if you like it if you have suggestions we are always open to these things overcome so thanks a lot and again thanks for having me and have a good one bye so it seems like it's pretty easy to get started with this rather than three Jas and I noticed it seems that you can go to the three Jas documentation and and still be able to use that along with your library pretty easily is that correct yeah that's socially correct there's no there's no real splits like some people they immediately assume that your reactive fiber introduces new rules or something like that it doesn't you you need to know where three.js is you need to know the basics to even be able to use it and you can basically also use it in the same way you don't have to make it stick a declarative but in the actual figuring out the stats you should because makes many things easy as few as you have seen yeah and if anyone's looking to get started with this library very easily there is a talk that he does with Jason lang store up on YouTube which is like an hour or more long which I thought was really great to kind of get you introduced and we had a question actually one from the chats which was are you gonna be sharing any of those code sandboxes and is that something maybe you could make available on Twitter with some I don't know maybe you could tweet some of those code sandboxes out yeah sure that's basically my whole game on Twitter I do nothing else I don't have a personal stuff on it it's just like the libraries are working on I'm sharing code sandbox liked it so much that they gave me a free pro access because I'm sharing I'm like spreading the word yes but the problem is I I don't know if it's access something like a gallery where I can collect all the stuff that I have done there's just like hundreds that I Freight and they all floating around on Twitter on github it's kind of a problem I would like to collect them all at some point great and I noticed integration with things like 3d modeling programs like blender yeah it's not that you integrated with blender it's the stuff that we are doing at work is almost like a blender for the web we will publish this very soon then people can try it out if you've seen blended and you open it up and say software and then you start to construct your model sense you will be basically be able to do this in JavaScript this is the the work this is where reactive fiber actually comes from how about you Sean you got any questions well I I you know one thing I think that people will be interested in is using this together with react native or expo the actual version of react native I haven't I don't think I've seen ports it probably is possible but I just figured I'd ask you uh yeah it's totally possible and with export actually runs native so it's not you know in a webview it runs in OpenGL yes this was done by a company called Wayne oh and the guy that made it was Kirk here but his last name I wouldn't be able to pronounce it but it works I I'm not so familiar with it but I've seen other people do it yeah because I think I think it's very like because you know on a react native app it doesn't really matter if you're choosing Dom or canvas and if you can do all these slick animations and it's so smooth you might as well messages do it on canvas yeah I mean maybe even for the web I think can really heat write for a long time to find some like connecting ground it's an interesting topic for sure yes so I noticed that obviously it's going to work great with a react spring you're also contributed author of that library we had a question from discord what resources would you recommend to someone wanting to learn react spring animations and 3d modeling I think it fits in here I think the best advice that I can give us start simple go to the repositories check out what's written on them react be fiber gives you some some starting points like a couple of tutorials that you can check out the really easy stuff I myself I I'm not even too familiar with three.js I couldn't make some I don't know I I'm self-taught I know very little and I have no math background whatsoever I know this scares people so it's not so hard as it seems and the same goes for react string if you have a problem like animation wise I would start there I would start to solve this and then maybe go a venture further all the informations are on Nikita proposed and I think the philosophy of the react string organization is that most of these languages they should be simple as simple as possible but still explicit like no magic and I wanted at some point to make it possible that each repository lists the full API on the front-end on the github project for yet to bring we haven't made it the up he still got a little too big because it can do lots but start simple don't go overboard let's see it Viking and I think that that's good to hear and it makes it more approachable by all of us I also just wanted to thank you for the talk because we have a lot of people in the chats just saying you know jaws dropped a very great talk and so again thank you for that and and before we enjoy and what happened I think yeah paranoid android was a really popular androids firmware or wrong I started this I made most of these features for this really crazy stuff if you have never seen it go check it out it was an awesome time but I got bored because I hate and my computer was slow I burned through several laptops of mine because it takes a long time to compile the farmer takes the whole night and a cup of them burned through so I got bored and didn't want to waste my time but what's a good time was my first open source experience yes yeah well thanks so much for and we're going to get started with the next time it's all ready for the exons okay thank you why can't it sound great thank you right I am

Channel: ReactEurope

Views: 13,086

Rating: 4.9865322 out of 5

Keywords:

Id: YyqBdN71nFs

Length: 32min 25sec (1945 seconds)

Published: Thu May 14 2020