WebAssembly: Real World Applications

hi everyone I'm here to talk to you about webassembly like as Americo kindly said I did race a car across the desert I also flew on the Concord on frequent flyer points so I had this obsession with speeding which is why I love webassembly it's fantastic and what you need to understand is what it actually is right it's a new capability for the web it's a way that you can build really high performance code and have it run inside your web application it's also not a replacement for JavaScript it works alongside JavaScript so what it enables you to do is to take things like C++ and rust and compile it into what we call a web assembly module or weather module and you can load that into your web application and call it from JavaScript now when we were developing this one of the things was speed right it runs extremely fast it runs at a small percentage slower than true native code but more importantly it's very secure so what weather means it's kind of a binary format that was specifically designed to be safer than JavaScript so the instructions are very restricted it's it's proven like there's a mathematical proof in a published paper about it so it actually adds there's a whole security layer by design and on top of that it runs inside chrome sandbox so there are a couple of layers of protection there for you when you use it but probably a bigger goal was portability so we wanted to enable the compile once run everywhere model and so it actually does run across a whole lot of browsers so you might be wondering what can you actually do with it well say you already have a whole lot of native code you might have a native app that runs on desktop or even in an app store and it could be written in C++ it could contain business logic or you could have a game engine that's also like many many main years of invested time so you could have a multi million line codebase sitting out there in your native application now you might think about bringing that over into the web now during the keynote yesterday there was this whole talk of the new web capabilities if you have a native app you can now pull in the web browser without the URL bar at the top which was making the native app kind of or the web more native like well web assembly is the opposite end it's it's making your web app more native like it actually pull in these native things so picture this picture a new video format so we all use various video formats as WebM as h.264 there's a whole literature of video formats but if you were say a developer that decided to come up with a whole new codec or an eighth still image codec you might want to deploy that across the web now you could do two things you could wait for the browser implementers to actually implement it you could log a bug against all the browsers saying implement my new cool image format or my new video format or you could just compile your C++ code into a web assembly module load it in your web app and use that video format everywhere without any changes hi I'm VP I'm a software engineer and the web assembly team so you've heard Alex talk about what web assembly is why should you care why is this relevant to you today so more and more applications are moving to the web you have audio decoders video decoders game engines music applications and a whole lot of other computation intensive applications and we want these to run at neo native speeds and not need to be recompiled for each platform and that's exactly what wasum gives you so and that's exactly what was it gives you and we have Chrome Firefox Safari edge or support the first version of web assembly and it in fact just shipped shipped last week and we're very excited about that so what are some of the big companies that are using web assembly right now this is a Jif of the Unity game tanks we have a screenshot from the epic Zen garden from the Unreal Engine and so we want to take a little deeper into why all of these companies use plasm when you have a wiser module all of the module information is encoded in a dense binary format so what this means is you have smaller but smaller binary sizes you have faster loading faster decoding it's a smaller memory footprint and it's just fewer bits over the wire so everything is faster so one of the things we want to do with this presentation is to showcase a different all of the variety of applications and use cases that wasum works really well for so website is demo it's a real-time face detection demo of is a real-time performance demo of face detection algorithm that uses OpenCV for those of you that are not familiar with open CV open CV is the open version is the open source computer vision library and it's extensively used so let's switch to the demo please so as you can see here what we have is you have the webassembly version which is open CV that's compiled awasum running on the left side and on the right side you have handwritten JavaScript the performance is measured by the amount of time it takes to detect Alex's face in the video stream so as you can see the wizened version is currently at about 16 to 18 frames per second and the JavaScript version is at about 2 3 frames per second well we want to show you here is that the Weizen version is so much faster than the JavaScript version can we switch to the sides of these so like I said this is an example of a large existing C C++ code base that was compiled hawazin and ported to run in the web and you can see that these are significant performance gains but the thing to bear in mind here is that the handwritten JavaScript might not actually be the fastest JavaScript there is so the performance here might be slightly inflated but you can still see that the performance difference is actually quite large if you want to check out what is actually available that's website s IG HT so make sure you look for the right thing okay so when we were getting this talk ready I was trying to think of what we could kind of show you to give you an idea of what you could use web assembly for so it's like you know we all build websites right and so these days your every web site is probably a megabyte or so and 80% of that would likely be images so I don't know about you but my workflow is typically I'll take photos with my camera or whatever and I'll pull it into Photoshop and I'll crop and tweak and then I'll adjust the quality to try and get that image small because you know we want to save bytes on the wire we want to make our pages smaller and part of that is optimizing your images I don't know other people use command-line tools you lose your favorite things I thought what if we could actually do that inside the browser so what I did was I took the independent JPEG group codec so this is code that is in a lot of production applications it's probably 20 years old by now so it's had many many years of debugging its rock-solid it ships in production in fact a derivative of it is inside blink to decode JPEG images but we don't have a JPEG encoder in blink right so what I thought is why don't I just take the ijg code compile it into like a module that I can then load that module into my page and I can load images and start playing with the quality in size to see what happens so step one which I encourage you all to do is to download the Emscripten SDK and install it on your workstation so once you've installed it you will have a bunch of command line tools that help you work with the web assembly so this is what I do to compile my little JPEG transcoder it's nice and simple so AMC sees the inscription C compiler now this flag - s was an equals one tells the compiler to generate a web assembly module for me the next line which says - s exported functions is JPEG transcode exposes the C function jpg underscore trans code to JavaScript courtesy of web assembly so you can actually put a list of functions in here and export all of them the next line the output is jpg squash J s so what that is is I'm script and generated glue codes so that is JavaScript code that manages the web assembly module for you so it manages the interface it manages the loading at man effect men like manages all the calls in and out etc and then of course is the last thing is my source files jpg Transco dot c so once you've done that on your command line it will build you a javascript thing called jpg squash j s and a web assembly module and this is how you use it from your HTML so the first thing you do is put in this script tag which pulls in jpeg squash j s now what this does is pulls in the web assembly module hooks up all the glue logic and creates a global object called module and module is how you interface to your web assembly so the code that I actually write myself for you right as a developer is something like this the first thing I do is set a boolean saying the web assembly is not loaded now the reason I do this is because web assembly modules are loaded as synchronously so what happens is that when the first script line is executed it kicks off the load and then it kicks off the actual compile so in the background the browser is compiling the web assembly module to machine code while the rest of your page is continuing to load so they're running in parallel now in a future version of Chrome we're going to change is from we're actually going to have streaming compiled so as the weather module is coming across the wire it will be compiling at the same time which will give you faster startup so I set this property on the module object which is on runtime initialized I set that to a function and that function will be loaded when my web assembly is ready and so it just sets the boolean to say I can use this stuff and so this is the simple function I wrote to bash JPEGs so it takes two arguments the source is a typed array containing my original compressed JPEG image the second argument is the quality I'd like to compress that down to so if the web assembly is not loaded I just do nothing if it's loaded I call this function JPEG trance code which if you recall on the previous slide I had in my exports so this passes in the source typed array the number of bytes that it is and the quality I want to set it to and under the covers that goes off to the C code that's been compiled it decompresses the JPEG image it we compressors is that the new quality and returns it inside that source typed array and the return value from that function is the number of bytes of the newly compressed data now of course I can't put a typed array on my page very easily so what I need to do is convert it to a blob URL so we pass in that type array into this function make blob URL which creates a blob URL tags it with the mime type image JPEG and then I can attach it to any image node or even put it as a CSS background image so why don't we just go to the demo and I'll show you how this works okay so here's the my beautiful UI the application that I built and of course it's the webassembly logo there so I could pick a file so say I've taken a photo on my camera I can just click the choose file dialog which of course is being nice and slow on me there we go so I'll just pick this photo here's a photo of my daughter and here i'm recompressing it at quality 5 so you can see the original image on the left you can see if you compress it to q5 on the right now if i just grab the slider and bump it up to full quality I'll kind of explain why I'm doing this so this is 100% quality the image is absolutely identical but it's 935 kilobytes remember when I mentioned a megabyte for the typical web page I don't want to deliver a megabyte so I could like play binary chop or something well let's go somewhere around the middle so at 45 it's gone from 900 K to 100 K and still this image is still not too bad like you can you almost can't really tell in fact I can't really see a lot of difference here I don't know about you so let's go smaller again so quality of 23 now it's probably not so clear on the projector you can actually see if there's there are a few bands in the sky there but overall the faces is still kind of recognizable the parrots are happy you know I might just ship this because the subject is clear the skies a bit bad but I've only got 68 kilobytes now the reason I'm doing this in the browser myself because I want to actually hand up the using my eyes to decide what it's going to do because if you use an automatic JPEG compression tool it will use things like signal-to-noise ratio and decide to deliver you an image that is imperceptible from the original and that's not what I want I want to deliver less bytes over the wire so I'll just show you one more image quickly so this is an image that I took on Sunday with my pixel two so this is a butterfly at the California Academy of Sciences and again I very compressed this at 23 quality which is what I was using for the previous photo it's only 54 K but you'll notice in this particular image only the Center matters like there's this butterfly it's in focus and over the side here it's all blurry because there's a lot of depth of field happening so in fact I could even shrink this even more so we have 50 K now and I've experimented with this before I went on stage so if you bump it down to like say 10 that's still relatively clear in the middle and you'll notice the background here like you can see the blur on the left and the blockiness on the right well I don't care about that blockiness you know so I can use my better judgement to say well that's blurred anyway I'll let it be blocky and in fact this kind of gives it this artistic look and then I only ship 37 kilobytes to my users okay so we flip back to the slides please by the way if you want me to open-source that just ping me on Twitter and I'll happily publish it okay so that's what one person would do is a bit of fun right yeah and something that I would do in my day-to-day web workflow but now I wanted to show you something that's a bit more serious this is like a serious engineering company building a large product so what I'm gonna talk about is this thing called construct 3 now what this is it's an in browser game editor so it's like a full-blown IDE but running inside the browser courtesy of web web assembly now what they've done here is they've wrapped a whole lot of public open source libraries into an application that combines the best of native and the best of web and this is what I was talking about earlier like the keynote may improve native apps with web we're actually improving web apps with No if so what they bring into their application they bring in a bunch of open source libraries like they use ffmpeg for video encoding and decoding they use box 2d as a physics engine they use Z Lib for compression they use PNG crush to squash down their PNG images but they're not just using native code they're also using a lot of browser capabilities so they're using HTTP - they're using serviceworker they're using WebGL - and they fall back to WebGL one if it's not available they use Web Audio for all their sound they even use WebRTC and so this is a full-featured rich app and one thing that's really really interesting about their web experience is that the size of the thing they load is smaller than their native app now you think about that for a minute because when they ship their native app they have to compile all those native libraries such as ffmpeg etc etc but then they need to use the functionality that's already built in the browser so things like WebRTC like compiling that alone is like two-and-a-half Meg so that has to be put into the native app but when it's on the web they can actually load a smaller payload and it actually loads a lot quicker than their native app so can we switch back to the demo please I'll just show you what this thing does it's okay so here is their IDE running inside the browser like if there wasn't a URL bar there you'd probably think this is like you know Xcode or Visual Studio or something like that it's a full-featured web based game editor so it has things like you know there's the start page we have tree controls here and you know proper tree controls that work we have property pages where this property page here somewhere so like property pages we even have like a preview of the game that you're working on like they're in the window you have inspectors on the Left so at the moment see it says press - to begin I could just go over here to the tree control and click on the text that's displaying there and if I scroll down in this inspector I've clicked multiple things though they don't begin text then we can actually see here's the text for the press to begin I can just change that to whole oh yeah chain in my preview window or I can even go into the editor like in my kind of WYSIWYG mode and pull up a dialog box and change it to something Oh can't even spell and then we actually change this live in our preview so this is actually doing a lot of work in webassembly and taking advantage of the power of the browser so the people that built construct 3 also have all these great things like you can actually start playing your game so you can mouse you go hit play prepares preparing the images packing the game and it pulls up a new window running this thing so I can actually start playing this game so I can you know fly around and you know start shooting things and as you can see it's nice and performant because a lot of the logic is actually in native code running through web assembly module which is very very nice now one of the things that they've built with this IDE is their they've got all the native code that does all the magic and the editing and the tree controls etc but they've also used some of the best parts of the web now what they've built is they built this thing called remote preview and what remote preview does is it creates this QR code like I'm just gonna pull out my phone and scan this QR code quickly which gives me a URL which I can click on if I click on that URL you'll see it's connected to this client chrome 62 on Android right and what it's actually doing is it creates yeah excellent I was actually going to ask you all to do that get your phones out scan this QR code I'll leave that there for a while I get everyone many of you would want to do this as you can't see ring crash the server I think if we try hard enough we can keep going coming so what this does is this actually takes advantage of WebRTC really keep scrolling it so where RTC has this thing called a data Channel and that lets you send arbitrary binary data around and so in fact I just lose it know how many we got 33 do you want to keep scrolling and they slowed down my load I'm only a ninety seven point nine percent here people come on I'm unloading the images anyway it allows you to send binary data around the place so they're in their IDE they've actually built where bytes it's the data channel to talk to your mobile device and then send the web assembly modules across and all the images and the entire game and then you can start playing the game in your phone straight from your IDE and try it out so it's a very powerful facility and hopefully some of you will get it loaded in the minute and can play can we go back to the slides please so we've heard about projects that are successfully using plasm right now if we've heard about a whole bunch of these demos and I want to talk about some really exciting things that we're working on for the future so first up we have threads so the thread proposal provides low-level building blocks for pthread style shared memory between threads or web workers in this case and concurrency using Atomics and few taxes so a lot of the existing native libraries that are available extensively use threads so adding threads to Adam so simplifies porting multi-threaded applications to the web and as a developer this is functionality that you can use in your applications you can build multi-threaded applications and you know have them work faster and what's really exciting about this is that this is coming soon we're actively working on it and we should have something for you to try by the end of the end of the year and launch in the first half of next year so because of the nature of compilation targets debugging in Bassem is not always easy we have support for basic debugging but we're actively working and adding source Maps memory inspection and all of that good stuff so it will make debugging your application simpler and help with figuring out exactly what's going on with your application what are some of the other features that were working on we're working on adding Sindhi support what does this mean the means that we're adding support for Wecht are instructions that I used in in it video processing cryptography etc we are also working on zero cast exception handling so when you have C C++ applications that extensively use exceptions what happens is that right now we're emulating these using JavaScript but that's not really fast so we want to be able to provide a way for these to work in a way that there's no overhead so a lot of this presentation we've been talking about C C++ libraries but what about support for other languages to do that we need garbage collection we need to be able to efficiently allocate and manipulate managed objects directly from wasone court and there's a proposal in progress for adding GC support tourism so so far you've seen a variety of applications you've seen game engines you've seen image detection compression modular libraries but how big can this really get how about the size of the earth let's switch to the demo please okay so you've seen a snippet of this in you've seen a snippet of this in the keynote yesterday but we want to show you what a complex application like Google Earth looks like in practice when it's using wasum let's go to the Empire State Building you can see that this is a whole immersive application you've got some awesome 3d content you've got knowledge cards that tell you what other interesting sites are around so let's try why Joe Weider is this really interesting interesting application that lets you explore different destinations from around the world and I love traveling so let's see what interesting things we have here so unusual city streets how about that so these are curated map stories that are available so let's go so let's see if this actually brings up Street View and what I want to point out here is that this is using shaders WebGL API is you know and seamlessly interfacing in and out of the wasum runtime to give you this incredible immersive experience one of the criticisms about Google Earth when it first came out was that it was chrome only now let's pull this up in Firefox and see what happens we'll give it a second to load but what we expect to see here is that this works just as well in Firefox as well as a dozen Chrome so to do an apples-to-apples comparison let's look for the Empire State Building again so you've got like a very similar you know the same 3d content everything that's available in Chrome that's also available in Firefox so you can see that this works in a truly portable way switch back to size please I also want to give a shout out to our colleagues on the earth team that have worked on putting this to Azzam and made this such a seamless experience they're here Jordan and his team are here in person so if you have any questions please feel free to find them after the presentation cool so what we kind of wanted to get across here today especially for all the web developers out there is web assembly is ready now for primetime production applications we're super excited to see four major browser engines already shipped this edge shipped last week so we have edge Safari Firefox and of course chrome shipping this now but also something to keep in mind is that it's not just desktop browsers it's a mobile thing too so Safari on iOS 11 as well as chrome on Android both support web assembly so you can build highly performant applications using native code for mobile devices which of course means faster execution time which means less battery usage so this is why it's a really exciting technology not just that but not only the browser vendors but there are other large companies working with webassembly now so we've seen that unity and epic are both using it so it's a true game changer for the web we also have autodesk releasing autocad using web assembly so these large players are using it because they realize this is kind of a next-generation high performance web application environment and you should be taking advantage of it too because using web assembly you'll build better applications that will delight your users thank you [Music]

Channel: Coding Tech

Views: 91,754

Rating: 4.9107141 out of 5

Keywords: webassembly, c++, rust, javascript

Id: ysFJHpS-O08

Length: 27min 46sec (1666 seconds)

Published: Wed Dec 27 2017