Building a Pixel Editor in Rust & WebAssembly (and Javascript)

hey folks in this video we're going to be looking at building a basic in-browser pixel editor with undo and redo we're going to be using rust compiling to webassembly and a bit of javascript this is based on a side project that we're going on at the moment which has been my first opportunity to play around with rust and webassembly i'm really excited about webassembly for a number of reasons firstly the performance aspects it opens up the possibilities of what we can do in the browser secondly that it doesn't even have to be in the browser it'd be very interesting to see what people do with webassembly runtime in other situations and thirdly that it seems like a quite a natural compile target for a number of languages that we don't normally get to use in the browser so in this situation i'm going to use rust which is a fantastic language with a great type system and i'm getting to build a web app with it or parts of a web app with it and that's really exciting i have to say though that i'm not a rust expert so if there's anything i'm doing wrong in this or anything that's not ideal please comment below to help me and the other people watching this video let's take a look okay i'm in an empty folder here and we're going to use the npm init functionality to set up scaffolding for our project so i wasn't aware of this functionality before but it turns out if you run npm init and then some kind of package name afterwards it will go and find the package name corresponding to what you've written with create dash appended to the front of it you can see down here as well so they'll do create dash and then whatever package name you've given it and then it runs the command the main command from that package which is designed to do the kind of setup for you so in our case we're going to run npm init rust webpack and that's going to go and run this create rust webpack package and that uses kind of going to set up a template from this rust webpack template github repository so ultimately what we get is everything that's inside here kind of processed through whatever sort of templating language they they have and that's going to set us set up our project for us so if we go back to the shell and run npm init rust webpack and then editor this is going to set up a directory called editor using that rust webpack template that we saw before okay so this has set up everything we want inside an editor directory if i now run npm start this is going to get the whole web server going and it's going to take us across to the browser initially but we're actually just going to come back and have a look at everything it's building so at the moment we can see that it's compiling with uh cargo and the rust compiler a bunch of the rust dependencies that have been pulled into this project if we jump back across to the browser we can see that well it's hard to tell but we can see that eti has loaded we can see a title in the tab the browser tab there and but there's actually very little page content to work with so we shouldn't be too surprised that we're not seeing anything yet if we come back and have a look at this project in a new tab we can see that we've got a js folder and a source folder and this static folder these are the main things we're going to be interacting with so we've got our initial html setup we've got our lib.rs which is going to house all of our ros code for this exercise and then we've got our index.js which is where we can kind of access the rust classes and things that we've been implementing and use them from the javascript side in our little web app if we clear this and then have a look at our index js file we can see that it's importing this package index.js file so the package index.js file is a product of the build system we don't need to create it manually the rust code is going to be compiled into a set of files including webassembly files which are ultimately accessed through this package index.js and this import that we're doing here is a dynamic import so it's not a kind of full top level es6 or typescript module import it is a dynamic import that can happen at runtime and to the best of my knowledge you can only import wasm files dynamically so you need to do one of these dynamic imports to access the compiled result of our rust code so the first thing i'm going to do is to wrap this in an async function just to make it a bit easier to access the uh the result of this import so if you do async function i'm going to call it main because i've spent too much time writing c plus and then we can just say our lib is equal to the oops a weight awaiting the result of this import now we don't really have anything in our rust code yet we can open it here it comes with you know this is the result of the scaffolding and that what this is set up we actually don't want a main function in the rust for the project we're going to be doing so we can delete this i'm going to leave these here for the moment although i don't need the console one i'm not entirely sure what this does so i'm just going to leave it to be safe and now what we'd really like to do is write some code in here that we can then start to access from the javascript side so we're going to set up a image struct something that's going to contain all of our image data for this project and we want that to be accessible from the outside so we're going to make a public struct called image and we're ultimately going to have an implementation for it as well and the implementation for the moment is going to include a new function that is going to return an image and it's just going to be an empty image we now this is obviously very basic it doesn't do much because there's no content in this class or actions we can do on it we can just make new ones um but in order to expose this to the uh javascript side we need to add an additional uh i'm not sure what they're called perhaps directive something a bit like this up here and we're going to add wasm bind gen here and here in order to get that exposing exposed up to the javascript side so if we try and compile this i'm just running kind of cargo check in my editor making sure this all goes through smoothly um it doesn't use the same build cache as the other one so this is going to take a moment it seems more or less happy there's a small error we can correct but it seems okay if i check the the webpack dev server that also looks okay so this is quite promising if we want to access this we can now say an image is equal to lib dot image dot new this is how we can call into that new method we've just defined in rust again this isn't really going to do much it's going to give us this image class and not much else we can sort of log this to see how it might be and if we run our main function down here make sure we're actually running our code and pop across to our browser tab we can see hopefully that it's reloading and yes it's outputting this object pointer which is how rust objects appear on the javascript side um we're just sort of keeping a pointer to the memory like i know the whole webassembly setup is managing that and that's how it likes to to represent things now one minor thing we can tweak here is that this is not the normal way of making classes in javascript we or making instances of classes in javascript we'd normally expect to do this instead and the rust bind gen setup has a way of doing that so we can take this directive and add constructor here and this does the necessary tweaking to make sure that it's exposed in the correct way and this should still be printing our little pointer as expected so that's nice of course in order to draw this image we want something to draw it onto so we're going to set up a canvas we're going to open up our index html and add a canvas element in here i'm going to make it 500 pixels wide and 500 pixels or units i'm not sure how it works exactly hi we need to close the tag and give it some kind of id so that we can access it easily with that in place and lined up uh we want to like have a draw function essentially it's going to draw information to this canvas so we're going to draw it we're going to draw you know maybe with the image initially um although we don't have much image data to actually draw yet so we're going to come back around to that so we're going to pass through the image for the moment and then we're going to say that our canvas is equal to document get element by id and our my canvas id that we added so we have our canvas from that we need a context to do anything in with a canvas on a graphics element and we say get canvas.getcom and ask for the 2d context and from here we can start drawing things with this so a very basic idea might be and you know we're going to remove this very quickly but just to we can check that it's working we can draw a rectangle from 0 0 to 50 by 50 and it's going to be red due to that fill style so if everything's working we get a red rectangle or square in this case um but that seems hopeful so we know we've got a canvas here we know that we can draw onto it and now we can look at improving this so as we're implementing a pixel editor it might be quite nice if we could see those pixels so we want to draw a kind of grid onto our canvas so that we know what we're interacting with and what squares we kind of we want to edit as we go forward so we're going to comment this out for a second and then set our stroke style to just be black for the moment context dot line width just one for the moment and then we need some concept at the moment we're going to change this soon but the height and width of our grid so we're going to start with a 10 by 10 pixel grid and each of our cells is going to be a 50 by 50 block essentially so we can now draw like have a loop to draw all of our lines so we've got x uh less than width for the moment and x plus plus and in each of these loops we're going to want to draw another line so we can say we want to begin a path and we want to move to the start of the line so the start of the line is going to be x because as we go across the width we're drawing the full the lines which are the full height so it's going to be at x times our cell size and zero and we're going to draw a line to x times cell size all the way down to the full height which is going to be the height times the cell size and then we do context dot stroke in order to actually produce the line and then we can repeat all of this for the other line so we're going to go y y height and y and then we want to move initially to zero the start of the line and at the correct height which is going to be y times cell size and we're going to finish at width time cell size which is the end of the line and the correct height which is also y type cell size so if we save this and have a look at our browser we can see that we have a grid now which is nice it's missing the ends because somehow the way that um canvases work you you need to kind of shift the pixels around a little bit in order to get it working perfectly we're actually going to move these all by plus 0.5 because that gets us sharper lines that look a little bit better i really no idea why the canvas api is designed in this way but these do look sharper as a result and in order to get those final lines at either end at the the right and the bottom we're going to actually we need to fix this we need to go up to the less than or equals instead of just less than and we also want to bump this up by one pixel just so that we're not going over the edge and that should give us our nice grid for drawing our kind of little pixel art on we now want to draw some pixel data on here we don't have any pixel data yet so let's go about setting that up one of the first things we can do is instead of kind of hard coding this here we want to make the width and the height kind of properties of the actual image so be much happier if we were sending them in here and then extracting them here that would be a better kind of storage for our data so how we might we do that if we go and have a look at our lib.rs we want to introduce width and height values in here now in javascript you only have a number type basically it's all all integers and floats represented as a single type that you might refer to as a number um in rust it's got a much wider range of integers and indeed uh floats to work with here we want to use an integer referred to as a u size that's the best thing that suits our future usage of this so we're going to have two fields on this image struct called you size we're then going to accept two arguments to this if i could type um both of you size and we're going to put these in here okay so that's going to allow us and the compiler the the whole set up here with rust and wasm is going to automate the process of making these two values passed in here become the argument here so that's all taken care of for us which is great we can then extend this saying we want to have a way of returning the width here so this is going to return you size and it's just going to be self.width which means we need to give it a self to access and we can do the same for the height so these two then become these two accessors that we're seeing up here so this should all work smoothly this continues to uh yes display the grid so everything seems to be okay for the moment with that width and height information we can now start to populate an array of cells within the image data so we might say that these cells are a vector of rgb values and rgb will have to define ourselves i'm sure there are plenty of ways of going about this but this is our current preference so we're going to do it very simply with kind of 0 to 255 rgb colors which are well represented as u8 and this means that our image is going to store a single dimensional vector of all the rgb values so we are going to compress this kind of two by two grid into a single vector which means we're gonna have to do some kind of lookups to get from our width and height and sort of x and y coordinates into the cells but that shouldn't be too hard what we can do here now is say that we want our cells to be and we're actually gonna uh we create a new vector and then we're gonna resize it because the vec api as far as i can tell in russ doesn't provide you with a way of creating a new vector of a particular size in order to resize it has to be mutable but we can now say we're going to resize this to something which is the width times the height and the value we want in every single one of these slots is going to be a mid-tone blue that's what i'm going to choose so if we say 200 200 and 255 this is going to give us something that's kind of light blue and we can add cells in here and hopefully that is going to compile fine which it isn't so it is expecting a semicolon at the end there and also it expects to be able to kind of clone this into the array in some way i'm not 100 up to date on exactly all the ins and outs of how russ manages the data but i do know that if we add a clone here then that should work okay so it's warning us that we're not actually accessing the cells yet but it is otherwise happy with this setup so we do want it to give a way of um getting to these cells and to passing them back to javascript so if we add a method like this again it's going to require this self and it's going to return a vec of these u8 values i'm going to jump in here to say that i'm about to very confidently tell you that you can't move anything except for numbers and arrays of numbers in between javascript and rust this is just wrong not in the least because as you can tell we've already moved an image pointer back and forth but there are pages of docs about how to move values of different kinds in between these two levels and it's you know it's possible to tackle a number of different ways what i can tell from looking at it though is that moving a single rgb value is sort of very simple to do moving a vector of rgb values might be a bit more tricky and might involve implementing some traits which is beyond my current kind of experience level with rust but all i can say is there are different approaches to doing this some might be easier some might be more complicated for the moment we're going to kind of convert it all into a single u8 array and you'll see how that that goes but you've been warned some of the statements i'm about to make aren't quite correct uh i need to do a bit more research maybe in the future i will get a better hang of it thank you so we're limited in the ways in which we can pass data from ros to javascript and back again basically it all has to be numbers um or arrays of numbers if we pass this vector of u8 values out to javascript it can happily become a u8 a u int 8 array objects like javascript javascript these days has these numerical array objects which are i guess more tightly packed numerical data than if you use a kind of a standard array and just throw number values in there so we can deal with kind of non-nested arrays being translated quite easily along with plain values being translated quite in easily between rust and javascript but we couldn't do something like send a vec of rgb values out because it wouldn't really know how to do that and i don't believe it can anyway that's the principle i've been working with we also couldn't do something like a double nested um array of sort of array of arrays of u8 where maybe each of these inner arrays are only three elements long we can't do that either unfortunately so we're going to have to flatten it all and do some indexing on the javascript side to get it correct so we want to get from this array of cells into some kind of single array of decay so we want to maybe flatten these rgb values into arrays of their own and then concatenate them all together and we can do this by iterating over all of these rgb values we can pull out each one and create a new vector of these so rgb dot rgb dot g and rgb dot b i'm not an expert in the kind of memory usage here maybe there are smoother ways of doing this but this is how i've come across at the moment we're then going to collect up this iterator of values into nested vect8 now vec u8 and i think we need a operator here and then we can concatenate all of these things together oh and return it so we don't want that final semicolon so we start with our initial array of rgb values we iterate over it and for each one we're going to replace that rgb value with a little vector of uh u8 so we now at this point have a well an iterator of vec u8 if we call this collect we are collecting the all the values from this iterator together into a kind of nested array uh where there's evacuate inside another vector and then when we call concatenate it's going to concatenate all of these nested values into one great big long array so we now should have a vec u8 array where there are three times as many elements in it as there are cells and this is because each cell here has become a rgb element in the final array we're just going to see if this compiles um okay so it wants us to add the copy trait here as well again i'm no expert i hope this is reasonable without being egregious i'm sure there are some sort of tweaks that could be applied to make this a bit more efficient but essentially we should now have this cells method which uh is going to provide all of the values back up to our javascript layer for us to interpret and draw so if we go back to javascript we want to draw them all before we hit this set of lines so we draw the the lines over the squares to make sure the lines are nice we can say that our cells are the results calling image.cells and then we're gonna do i guess a double nested loop so we're gonna go from x equals zero to x less than width incrementing again and the same thing for y and we're going to be able to say that our index is going to be y times the width plus x that's going to take us to uh well that would index into a an array that was the same length as the number of cells but we're going to have to take all of this and multiply it by three in order to get to that first r of the rgb of each cell so if that's our index we know that our color is uh going to be set by that so we can make a construct our own kind of color doing this again maybe there's a more efficient way but um we can use string interpolation and we can say that our cells indexed here plus zero which is redundant but makes the kind of whole logic flow a bit more clearly so we can now index into the zeroth first and second um colors there in order to build up this rgb string value we can now say that our context.fill style copying what we've got in that comment up the top uh is now the color and our fill rectangle is going to be dependent on where our x and y are so we're going to start at x times cell size we're going to oh and y times cell size and we're going to draw something which is cell size if i could type by cell size and if we save all of that and we come back and have a look we can actually see that this has drawn out a nice light blue grid and appears to be working well excellent right now it's not much of a pixel editor if we can't edit any pixels so we want to have a look at what it's going to mean to set up some kind of a brush tool here so you can change the color of each pixel so how might that look we would want to set up some uh if we get rid of that for a moment set up the canvas to have some uh uh event listeners on it so it's aware of our mouse interactions so we've got our setup canvas oh we're taking an image we're going to fetch the canvas the same way we did up here and we're now going to say that canvas we can add an event listener again if i could type i'm going to look out for the clicks and on each event we're going to process that in some way now i'm going to paste in some code i got from stack overflow um we have the canvas already here actually so we don't need this line but we can say this is essentially getting the total size of the um the canvas on the screen and then it's comparing the so client x is the mouse position where we've clicked and it's comparing the mouse position where we've clicked with the top left coordinates of the canvas and it's allowing us to derive an x and y position within the canvas itself which is going to correspond to kind of what pixel we're clicking on so this is going to give us a sort of screen pixel value when we divide by the cell size and floor it it's going to give us a coordinate in our own pixel space in this kind of 0 to 10 x and y pixel space of course the cell size is only going to work if we for the moment to move this up to the top level there are other ways of managing that but it's the easiest thing to do for the moment and so we now have a x and y here and we want to do something like image dot brush with x and y and some value color value at the moment i'm just going to hard code this to a green so similarly kind of light shade of green and then because we've made a change the image data we're going to redraw it so that it's uh fairly visible so this should work but of course we need to implement this image dot brush function so if we come down here we can say let's provide if brush we're going to edit it so we need a mutable reference to self and we're going to say we've got x which is a you size y which is a new size and then a color which is a vec u8 and this isn't actually under return i think it's just going to edit the internal data structure and we're going to say that our index as we've seen before is going to be y times cell times the width plus x and this is going to allow us to figure out which cell we want to like uh change in our cells array and we're going to change it to be a new rgb value where r is color zero g is color one and b is color two and if we check that that compiles it seems to we can come back we can assume this is working at the moment and if we come back to test it every time i click the pixels turn green so that's nice we can't change the back we can't undo redo we can't choose a different color but that's a good step forward to take this a little further and to allow us to choose the color we're going to introduce a bit more of a kind of state wrapper within the javascript code so instead of just having an image here that we're going to pass around we're going to want to set up some kind of state um maybe there'd be a better name for this but it's okay for the moment and this state is going to include a current color that we're painting with so i'm going to set it to that green that we've used above and i'm going to pass this state around instead so everywhere that we have been passing in our image we want our state so we can say const image is going to be equal to state.image and then we can brush on that still we've got the state passing in here state passing in here and up here const image equals state dot image this means we've got a nice place to hang our current color and to update our current color if we want to if we just go and check that this is still working it is which is nice and we can load up the html and think about adding some buttons that allow us to change the color of our uh of well paintbrush essentially so we've got some button with an id and we're going to make three of these to give us a very simple palette to work with and when we come back we can add more things in here so if we do document.getelementbyid and we're going to look for the red one first and then we're going to add a event listener uh for the click event and i think i mean it's worth saying that you know if this was a big serious project i wouldn't necessarily be you know doing handwritten html and sort of javascript you know always good to reach for react or some other toolkit i personally really enjoy elm but you know i'm trying to keep this as relatively streamlined as possible in certain ways so if we clicked on red and we've got access to state in here we can say that now our current color if i could type is now going to be a delightful shade of red instead and we can just paste this out a couple more times saying if it's green then we can change it to our green and if it's blue we can change it to a comparable blue so if we come back here we've got our red green and blue down here if i click red that is not working which is fair enough and that is because up here we have still got our hard-coded color which i should have noticed before we're going to use our current color instead so this is going to reload initially it's set to green but if i click red we get red instead if i screen we come back to green and if we go to blue then we sort of looks like we're raising it but we're painting over with blue so that's working we can change our very simple color palette here there's another step it's a little bit more satisfying if you can drag so at the moment if i click it doesn't really do anything i guess yeah there you go it appears at the end of the drag but that's about it it's nicer if you can paint in these tools by pressing down and dragging around so we can look at what it might take to add that we're going to want some kind of drag state so initially we're not dragging anything but then we can say back on this canvas we can say on a mouse down we want our state.dragging to be true and on a mouse up we want it to be false so this means at the beginning end of each drag we should hopefully have the correct state and then we can look at another function very similar to this where we can say on a mouse move we now we don't want to do anything if we are not dragging so if we're not dragging kind of in drag mode at the moment we're just moving our mouse around on top of the canvas without having clicked then we just want to exit and do nothing but otherwise we're quite happy to continue with the same logic and i guess you know i should probably extract this and share it with that function above if i wanted to be a bit cleaner but for the moment the copy copy and paste should be fine if we click over here now if i click and drag we can see that that's working reasonably well i imagine there's lots of little things we can fix about it but it's a good start okay so the final chunk we're going to add here as a kind of undo and redo um so this is a a nice thing in any graphical editor or indeed any digital tool at all you always want to be able to undo mistakes and it's interesting to look at um implementing undo in different ways quite often i think the two i've come across are use some kind of command pattern where you encapsulate every change as a command object and that object is able to kind of do and undo the change it is responsible for itself you store these objects in a kind of some array and you can walk up and down them asking them to undo and redo as requested the other option is that you can reach for kind of well i'm going to say reach for immutable data structures which helps to improve your kind of memory footprint but otherwise even without those you can just remember like the entire state before and after every interaction and then in order to undo and redo you just walk up and down this kind of array of states we um uh just you know reusing whatever state is at that point now we're going to look at doing this with immutable data structures because there's a good mutable data structure library for rust and because that means you're going to reduce your memory footprint because a fair amount of the state is going to be reused in between different undo and redo slots [Music] but you know you can approach it in different ways so this is going to be quite a big change we're going to start by adding the immutable library to cargo uh which is uh the rust package manager so this is the file cargo.tomall is the file that's specifying all of our dependencies we're gonna come down to here and we're just gonna say we're gonna use the immutable library which is just called am and we're going to use version 15 of it fortunately the server we're using goes and fetches and builds this for us automatically so that's quite handy and the first step we're going to take is to use this immutable library in place of the current vectors we are doing so there's going to be quite a few steps in this undo redo process but the first one is to look at our rust code and just use this im library instead so if we do use im vector then we get this data type called vector which is an immutable version of the vec which is built into rust that comes as very much standard in rust so if we use a vector instead we then need to change our approach because this resize method doesn't uh exist in the same way instead we're going to say we want a vector from an iterator and that iterator is going to be um us counting from zero up to the cell size and for each cell we're going to ignore the actual value and use this rgb value instead so if we get rid of that uh get rid of that rust is going to require something's not working here we need to give it a moment to build on the checking side as well um essentially rust format isn't kicking in so i must be missing some subtlety here oh yes it's going to be the end of this from ita operation there okay that looks a bit happier that's still going to be unhappy because it we don't have this in scope so the from ita behavior determines on is dependent on making this trait available in the scope so we can import this at the top in order to make that work and when we compile that that goes through fine so the nice thing here the fact that it is compiling uh indicates that this interface is still allowed so we can still iterate over all the cells and also we can still call this when we have a mutable version of the um of the data so we can do mutable updates of this mutable immutable structure if you want but it also provides efficient immutable updates as well so if we jump back over we can still see that this is working so that's great we don't have to worry too much about everything breaking and doing a lot of work to get it back into one place but we can now look at how to use this mutable data um to make a more efficient setup and a more uh friendly undo queue so one of the things with uh immutable data structures is that when you do a update of an immutable data structure then you and you do an immutable update or an update of an immutable data structure not a mutable update like we're doing here when you do a normal updated remote a data structure you get a new copy of that data structure back and that means that we would want to return a new copy of our entire image to make this work smoothly or the way i'm planning to do it anyway so we would update cells we'd get a new cell's value back and then we would return a new image from this brush and then we would manage this kind of collection of images as our undo stages so there are sort of multiple steps in going about doing this but the first step is going to say our brushwork is going to our brush function is going to return an actual image and in order to do that we can say our new cell's value is going to be self.cells and then we're going to call the update function so this is the immutable way of updating this data structure it returns a whole new cells vector with this one change in it and then we're going to return from this a whole new image object so the width is going to be oh wow i cannot type um the width is there the height is there and then cells is going to be this new cells data and we are going to compile that that seems to work fine now we could possibly work with this in the javascript code so every time we call brush you would get a new image back and we could manage the undo queue in javascript but i'm going to choose to do it in rust instead and so in order to do that we're going to want to instead instead of working with images in a javascript code directly we're going to want to work with some kind of uh other form of state that has an internal undo queue to manage all of this complexity so i'm going to make a new struct called state in this case or perhaps internal internal state just to make it different to that state object i made in the javascript world it's going to have some kind of undo queue and the undo queue is something we're going to write that takes an image and when we implement our internal state we're going to want to make new ones of these so a bit like we have before we're going to want to take our the width and the height to start this whole thing off and we're going to return an internal state and this internal state sorry about that uh it's going to have an undo queue which is a new undo queue taking a new image so we're going to call image here pass in width and height so we haven't written all of this yet so it's not going to actually work but that's more or less the structure when we start this new eternal state we're going to start it with an undo queue that begins with our initial version of the image so what does our undo queue look like um we're going to make it parametrize because technically it could work with anything and it is going to work with a cue that we're just going to base on being a vector of this entry type and if we want to implement an implementation for this we have to account for t in each case and we're going to say we're going to expect an entry of type t this is going to return an undo queue of type t and the way it's going to do this is by making an undo queue where the queue is just a vector around this entry okay quite a lot of fuss but that is going to compile it's going to complain about various things never being used but we're okay with that for the moment so we still have quite far to go with this one thing we might look at is how to return the current oh actually yes this is going to need a concept of an index as well so for an undo queue you can store multiple states and then you want to be able to walk up and down those states without necessarily removing the ones you don't use yet um and so we can do this just by adjusting an index rather than by kind of pushing and popping actual states off the queue and the fact is like when you undo a certain state you want to be able to redo and return to that state so you want to leave that state in the queue but our index is obviously going to start at zero and if we want we can add a function here to uh return the current entry so this is going to need self it's going to return a t and we're just going to return our q and it's going to be indexed and we're going to have to clone this um again my complete understanding of rust and the memory management isn't there entirely yet but uh this does seem to be necessary and i'm just vaguely understandable given everything that rust requires um so that's how we make that happy so the idea here is that um we're happy to return a clone of the image because the image is relatively cheap to clone when we clone an image we are cloning these two size just these two numbers and then we're cloning an immutable data structure which is basically designed to be cloned so it can the mutual data should actually be concluding incredibly quickly if we were cloning the original vec value then it would be copying all of the memory and would be quite uh slow for large images and wasteful of memory but with the mutable data structures whenever you clone it it can just share all the memory with its previous instance because it's the nature of the data structure itself so that's pleasantly why that works and we can say here if you want to get access to the actual image from our internal state then we can excuse me return the undo q dot current and hopefully that should all compile uh yes we have to run self.index at this point index itself doesn't exist and it wants the undo queue to be cloned all right so the image itself despite my little speech before we haven't implemented clone for it yet so i'm going to say that it is cloneable it's clonable quite cheaply uh it's happier with it being that way around um oh and not on the implementation i am embarrassed by this please forgive me okay so this is a lot of kind of warnings about stuff never being used which we're okay with for the moment so ah part of the reason it knows that it's never being used is because it's not accessible from javascript yet because we haven't uh pasted this in everywhere and i'm going to be honest like i'm not sure exactly all the places we need this and i'm not sure actually how often i need to write pub in front of these functions so i um apologize if any of you know better out there and are wincing every time i write it um but i'm now just kind of doing it out of habit and we'll come back to this later to do better so the other thing is is we're going to be making this from javascript now we're going to add that constructor line so that behaves better and okay this is actually telling me that i've got a bit too casual with this so it doesn't want to we don't need to expose those and that's not going gonna uh and it's gonna be happier if we don't so again a bunch of warnings but we can see how this is gonna work when we return to javascript so we're gonna sort of piece this together slowly to begin with we want some kind of internal states created here instead so this is going to use our internal state constructor we're going to store that here instead and then up here we're happy in order to get the state we can say internal dot and just get the image from the internal but when we get to here we don't want to get the image and then cool brush because that's going to just generate a new image by itself and instead we want to manage that image generation within the rust code so we're going to change this to state.internal dot brush and we're going to do the same down here as well and then we're going to come and implement that so down here we're going to need a function called brush which is going to look very similar to the the one above it's going to need a mutable reference we're going to call with an x and a y and our color which is going to be this vec u8 again it's not going to return anything because it's going to manage this extra image that's created internally and what we're going to say is our image is going to be equal to self undo q.current and then we're going to create a new image by calling brush on this image so our image.brush function now returns a new image so we want to capture that and push it onto the undo key so we can pass this through hopefully that should be fine and now we want to do undo cue.push with this new image now we haven't implemented push yet so we can have a look at doing that it is going to be a mutable change to the undo queue and it's going to have a new entry it's not going to return anything and we're going to say that self.q now the complexity here is that we want to actually truncate the queue and we want to truncate it to index plus one so we want to in the in the event where someone's done a lot of actions we've built up a lot of data on this undo queue and then someone started undoing those items then they're kind of walking back through the queue and if you want to push a new if you then sort of make a change they want when you want to push a new state onto this queue we want to truncate the cube back to where they've been undoing to and then start adding new states on there so we can do self.q push this entry and we want to uh set our index to uh just plus equals one for the moment so we're incrementing our index so we're going to reference this new entry that we've pushed okay so we've still got a hangover from here so this is still saying that brush requires a mutable self which it doesn't anymore because it's returning a completely new image and this seems very uncomfortable with the amount of there's a lot of things that it's complaining uh are not used but hopefully we're going to see this work all the same so what we have is that we are we've got this brush function now which is getting the latest image editing it to get a new image pushing that onto the undo queue the undo queue is incrementing the index so it references this new entry and when we ask the undo queue for the current entry we're going to get that newly sort of added entry there and that should be what we get back from image so when we return here we should see that we are kind of doing a completely internal brush update here an internal brush update here and then when we're drawing it we're fetching this state and drawing everything through so if we come and check it out this does seem to be allowing us to draw stuff which is great it's hard to tell if this is working perfectly until we started adding undo and redo but let's give that a go now so if we come back we can have a look at our index.html we can add some more oops some more buttons it's going to include an undo a redo can delete that we can come back to our setup function here and we can say that in the case that the undo button is clicked what we want to do is call undo on the internal state and if recalling redo we're going to call redo on the internal state and now we have to go and implement those obviously so if we come back to our rust code this function now needs a i'm going to add them here uh undo code so it's going to be mutable in this case it's not going to return anything and it's actually just going to pass it through to the undo queue underneath so we're going to call undo on that and similarly we can call redo in this situation we can now take these and call undo again it's going to mutate itself but it's not going to return anything and in the case of undoing we are just going to um minus one from the index okay now we want to be careful with limits here really we don't want um we don't want this to go to less than zero so we could say something like if self.index is greater than or equal to one then undoing should take us back to zero but no further and if we're redoing we're moving our index forward through the stack again so we can say if our index is less than uh the length of the q i guess less than the q minus one then we can afford to add an index to it so our under and redo a very cheap here because we're just changing our reference to this point in the queue and then this current is going to change its behavior accordingly to return that particular entry so here we've got under and redo so that's linked up this is hopefully compiling we're getting a lot of warnings so i assume that's okay and if we come back here we can make some lines and if we click undo nothing is happening of course that's because i'm not asking it to redraw after doing the code so here we now want to draw the state in each case so if you've undone it the state's going to change we want to redraw it in order to be visible so we can make some nice things here and then we're going to click undo okay it's inefficient and we can talk about why in a second um but it is essentially working back too obviously i'm having to click loads and this is because um actually when i click and drag i'm painting several times into each square so i have to undo and redo each of those several times if i'm a little more careful with how i do things then undoing and redoing becomes more obviously correct we can have a look at um fixing up that drag behavior if we want but for the moment we can see that this undo redo appears to be working quite well the simplest way of getting this um drag behavior to work better with the undo redo is to say that if the cell is already the color we're trying to paint it then like don't paint it that color like don't perform an operation don't put another state in the undo queue so we don't have to work back through it all the time there are other issues with dragging as well which we might get into in a second but that's a good place to start and interestingly that allows us to look at the rust option type so we can say that this brush operation isn't always going to return a new image sometimes it's going to return nothing at all to indicate that no work was actually done so we can say that our colors equal like the color that we're hoping to paint this is equal to this rgb just extracting that and we can say if our like accessing the cells directly to get the color if that's equal to our color uh then actually we're gonna return none so if it's already the right color we're gonna return none otherwise we're gonna do this work that we've got down here and we have to wrap our image in a sum in order to make that work so there might be some issues in here which it clearly thinks there are so we don't we can't do partial equalities on rgb values but i'm happy to add that because the rgb is quite a a simple type so i don't mind it deriving that um obviously it doesn't like the fact that we haven't added color to this update function and now we're seeing the fallout down here so we're calling new image and we're getting back an optional image or call sorry calling brush and we're getting back an optional image instead so our new image we can match on this in the case that our new image is actually none we can just do nothing but in the case that there's some image there uh okay we'll we'll improve the wording here but in case there's some image we're going to push that onto the undo so this is kind of an optional image oh we can only add that semicolon if it's within a block uh and we want to do that in order to make sure both of these sides both both of these options are returning the same kind of unit type um okay so in this case whenever we brush sometimes we are going to add a new entry to the undo queue and sometimes we're not depending on whether it matches so if we now have a look at how this plays out if i choose red and drag that around a bit if i hit undo you can see it's undoing each cell much more appropriately now of course you might expect it to undo the whole drag at the same time and we can look into doing that too that's a additional bit of complication but um at least this is behaving a little bit more reasonably so as a final step if we want to improve this drag behavior so that we undo the whole drag uh at once we need to think about how we're actually managing our undo queue at the moment every single time we change the color of one of our pixels and our canvas we add a new state to the undo queue but for the drag obviously we need something more complicated where we start some kind of larger undo block that can be undone and redone all in one go and then we can and we kind of start that at the start of the drag and then we close it at the end of the drag and we you know rely on that becoming one unit so one way to approach this is to add some kind of extra metadata to our undo queue so it understands kind of what state it's in for navigating this stuff and we'll look at doing that now so in the rust code we can add a mode concept to the undo queue there might be other ways of doing this but this is how i've come across doing in the past we can look at several different states so we can have a normal mode we can have just about to start a block mode and we can have a in a block mode so we're either kind of treating every single update as just a fresh update we are either going to or we're going to start a block with the next update that comes along or we're in a block and we need to treat each update kind of slightly differently as it comes through so when we are starting off we're gonna be in normal mode which i think is pretty reasonable um and we're gonna say what what happens when we push so when we push we're gonna inspect what mode we're in and we're gonna say if we're in this normal mode then we're happy to do this kind of standard update but if we are in if we're about to start a block instead then we want to do the normal update but also set our mode to be in the block oh we got to do mode here and if we're in a block then instead of doing this whole thing we're just going to replace the current entry so just going to use the um the new entry that's coming in to overwrite whatever we're currently looking at so i don't know if this will make sense but basically normal mode operated the way we were before if we're about to start dragging and this is the problem like we click our mouse and we're kind of about to start dragging we want to move ourselves into like we want to start a new block so that the first change that happens is still going to make a change but we also want to say the next change just overwrite our current state this means at the end of the day if you kind of you start a block with one state you then have 15 in-block updates all 15 of those are just going to overwrite the current state and then you're going to return back to normal mode at some point we're going to have to add a method for doing that and and that means this kind of 15 updates becomes a single entry in the undo queue and you uh can move past it kind of one in one easy step so our whole drag can become one step in our undo queue which is ultimately what we're looking to do so in order to enable this so we're going to have a look oh it doesn't like the fact that i've used normal somewhere up here by itself so in order to enable this we need to think about how we're going to use it from the javascript side and we'll come back here to do some further updates so the main thing is when we're looking at dragging we want to start when our mouse down at the start of the drag we want to start this block so we're going to call start undo block on our internal state and when the mouse is up we're going to call close undo block which means we need to implement these methods down here so start undo block here it's going to be immutable and again we're just going to call through to here so do something similar and we're going to have see similar functions up here but instead of doing an else it's just going to set the mode so our mode in this case is going to become start block and our mode in this case is going to go back to normal so again we start the drag before we've done any kind of painting at all we've moved our undo queue into this start block mode that means when it receives the first brush instruction the first update instruction because the undo queue doesn't actually know about brushing as such it's kind of more independent than that when it receives the first update instruction it's going to treat it normally but then believe it's moving into an in block so it's just from that point onwards it's going to be overwriting that state with this entry every entry that comes in when we uh finish our drag we're going to do the mouse up which is going to call close undo block which is going to return us to normal and so every new change after that is going to come in as a new entry in the queue if we compile that we can see that's still working and we can come back here click and drag and if we hit undo it's uh managing them all in one block which is nice so undo undo undo we do redo we do and that's how we get a nice well-behaved undo queue working with immutable data in a pixel editor written in rust on a canvas for a little online app i hope you've enjoyed it thank you thank you for watching i hope it served as a good introduction to combining javascript and rust via webassembly and to the joys of immutable data structures and undo cues if you want to see more don't forget to like and subscribe and if you have any questions or if there are any topics you'd like me to tackle in future videos and please leave a comment below

Channel: Michael Jones

Views: 22,071

Keywords: rust, webassembly, programming

Id: rHRBJKWbbw0

Length: 65min 9sec (3909 seconds)

Published: Wed Aug 05 2020