WASM + React... Easily build video editing software with JS & FFmpeg

webassembly opens the door to building high performance applications on the web in languages other than javascript in today's video we'll build a simple video editing tool that can take a video file and convert it to an animated gif the cool thing about this demo is that it's made possible by a library called ffmpeg which is a utility written in the c programming language but thanks to webassembly we can run this tool directly in the browser and that means we can perform cpu intensive video editing jobs without the need for a backend server for example here's how a traditional web application might work we collect a video file from the end user then upload it to the cloud we use our cloud server to run ffmpeg to encode the file and send it back down to the client that works fine but it's not very efficient and it also has no way to work offline if you're building a progressive web app what we'll do in this video is download the webassembly binary for ffmpeg directly into a react app allowing us to combine the c programming language with javascript and offload the video editing work to the end user's hardware if you're new here like and subscribe and follow along with the full write up and source code on fireship io and if you have no idea what webassembly is check out my 100 second breakdown on that topic this tutorial is very beginner friendly and doesn't require that you know anything about the c programming language that's because the ffmpeg.wasm package provides an api in javascript to interact with this low-level utility now before we jump into the code let's take a closer look at the demo we have a file input where we can select a video file the video can be previewed in a video element then when you click the convert button it runs ffmpeg to load the video file into memory and run the conversion very simple stuff but my main goal here is to give you a starting point for building more complex features through the magic of webassembly to get things started i'm generating a new react app using snowpack and keep in mind here the code is very simple so feel free to follow along with any other framework then from inside your project you'll want to install the following packages from the ffmpeg namespace and lastly you can run npm start which should give you the default react app running in the browser from there we'll go into the source directory and open the app.js file i'm going to go ahead and delete all the boilerplate code leaving us with a basic starting point from there i'm going to import two functions from the ffmpeg namespace the first function create ffmpeg we can call right away which gives us a starting point for working with this library we'll set the log option to true so we can see everything that it does directly in the console now it's important to understand at this point that the actual webassembly binary has not been bundled in our application it's a pretty large file so we don't want it to block our web application right away instead we load it asynchronously over a cdn to keep track of the loading state i'll go ahead and add a stateful property here called ready with the use state hook it has a default value of false and then we'll create an async function here called load that can flip it to true after the binary has been loaded the function awaits the ffmpeg load promise and once loaded we can flip the ready state to true and now we just need a place to call this function in react we can do that with the use effect hook to run this function when the component is first initialized use effect takes a function as its argument then notice how i'm adding an empty array as the second argument this will ensure that the function is only called once when the component is first initialized or mounted then inside the effect we can simply call our load function if you go ahead and save the file and open up the browser you should get a blank page but in the console you'll notice that ffmpeg core is being loaded in the background in fact if we go over to the sources tab you can inspect the actual webassembly code there you don't need to understand what any of this code means but it is useful to know that this originally started as the c programming language and then was compiled over to webassembly what we're looking at here is the webassembly text which is an intermediate format before it becomes binary or ones and zeros now back in the source code we'll go into our jsx and then we'll use the ternary operator with our ready state if the app's ready we'll show the main ui otherwise we'll just show the text of loading now our next challenge is loading a video file into the application and also showing it in a video player in the ui the first step is to add additional state to the component in this case a video file which will initially start undefined in the html we can add an input element with a type of file when this input changes we'll run an event handler that will set the video state and it gets the actual file from the event the event emits a file list and then we use item 0 to select the first file from that list now that we have access to a video file we want to display it in an html video element we'll use a logical and operator to only show the video when the video file is defined the element has controls a width of 250 but a tricky thing here is that we can't just pass the raw file to the source attribute we need to convert it to a url that the browser can fetch we can handle that by using the url interface and then calling its create object url method which takes the file object as its argument if we go ahead and open up the demo we should now be able to choose a file on our system and then have that file displayed in the video player and now we're ready for the fun part using ffmpeg and webassembly to modify this video file or in our case convert the file to an animated gif the first thing we'll do here is add one more piece of state to the component called gif and it will be used to hold the end result which is a url of an image file from there we'll create a function called convert to gif which the user can run when they're ready to make the conversion now one thing to understand here is that webassembly is managing its own in-memory file system and in order to run ffmpeg on that file we need to make it known to that file system we can do that by calling fs or file system along with the write file method along with the name of the file and then we fetch the actual video file that we've collected from the end user what we've done is taken the video file and saved it to a place in memory as test.mp4 it can be now accessed by webassembly while it's in memory and it only stays in memory until the browser is refreshed from there we can run an actual ffmpeg command if you're not familiar with this utility it's a command line tool that can perform all kinds of video editing jobs it's been around for a very long time and you can check out the full documentation to get an idea of what it can do we can run the command by simply passing arguments that we would normally pass to the command line the i flag is for the input file the t flag is for the time that we want the length of the video to be we can offset it with starting seconds then we use the flag to encode it as a gif file when that's complete it will write the result to memory and then we can access it by again using the file system command this time to read a file called out.gif we then need to convert the file data into a url that can be used in the browser once again we'll use url create object url and as its argument will create a new blob blob is kind of a scary word but it just means a raw file which in this case is the binary that we can access from the data buffer it's also a good idea to pass a second argument with the mime type of the object which in this case is a gif image and lastly we take that url and update the state of the component from there we just have a couple of simple updates to make to the ui we'll go down here and add a button that when clicked we'll run the convert to gif function and then we'll add an image tag that's only displayed when we have the gif url you should now be able to open up your demo select a video file then when you click the convert button it uses webassembly to convert that file to an animated gif and it should log the entire process here in the console you could then take this file and upload it to giphy or use it on twitter or whatever you want and that's how easy it is to start incorporating webassembly into your web applications i'm going to go ahead and wrap things up there but in the near future i plan on making a video about how to take your own custom rust code and convert it to webassembly to be used in the browser and if you have any ideas please let me know in the comments if you want to learn more and support my work consider becoming a pro member at fireship io thanks for watching and i will see you in the next one you

Channel: Fireship

Views: 89,882

Keywords: webdev, app development, lesson, tutorial, js, wasm, web assembly, react, reactjs, react wasm, learn wasm, video to gif, ffmpeg

Id: -OTc0Ki7Sv0

Length: 8min 0sec (480 seconds)

Published: Tue Nov 17 2020