Rust + Yew + WASM + Canvas

hello there are we live hello there are we live okay we seem to be live uh yeah so long time no stream been busy for a couple of months actually but yeah i decided to well not decided i want to get back into the that streaming stuff but i decided to do a little warm-up session and i hope to to actually do something quickly today [Music] because i found this uh nice little effect that you can do with pixels and stuff um using canvas and i wanted to see how i can do this in in wasm using rust essentially so i highly recommend this channel guy has a you know some nice tutorial for javascript um so if anybody's interested have a look but i'm not really interested in the actual effect and the javascript part of it just in the uh the wasn't stuff and how we can actually start playing with canvas just using rust so i basically took my stupid web page and i ported it to you uh i both reported it to uh rust using you you is this web framework that's you know it's very similar like the um the react or the vue.js frameworks from the javascript world um and i really really enjoy using it to be honest and the reason why i ported it originally it was just a static site generated using zola which is a rust based static site generator but i have i plan to do some of some more um advanced stuff with this website and stuff that i wouldn't be able to do using just a static site generation so i decided to rewrite it so that i have full control over what's going on on the page and i did it using you so i should have this stuff actually running locally yeah there it is um so this is the the web page that's this stuff so i'm not sure how how interesting would it be to go over the stuff but basically um you you allow you to use oh jesus that's going to be massive the u framework allows you uh to basically create a bunch of components that you can use to assemble the web page so if you're familiar with react or vue.js and i don't know i'm completely unfamiliar with angular so i don't know if there's any anything in common with this um but yeah if you know react or view js you should really be comfortable using uh u because the principle is exactly the same let's do it like that okay so um i haven't looked at this in a while so let me just uh [Music] get up to speed with this myself oh yeah so basically yeah so we have the the the router functionality uh from here uh we implement couple of uh component rates and and that's it basically the u does the rest for us which is basically just generating the entire webassembly stuff i think we go through this while i start implementing this effect to be honest so let's maybe not waste any time the only gotcha here is that i'm going with the master branch of the you for reasons one of the reason is that the eu has this concept of where is it oh yeah they don't have them here we need to go towards the latest documentation they have those function components which are like pure components so i just wanted to try it out so that's why i went with the master version uh so all of this is going to be released in the [Music] uh in the latest version but basically what's currently in the in the repo is just a version 18 and one of the other changes is in the in the upcoming release is that they remove all of the services so basically they had a bunch of services that well you came with a bunch of services like the console functionality and the fetch functionality in the upcoming release all of this is going to be released so basically if you want to make a fetch request you need to go and use the website directly or any other or any other crate that you might want to use for this but basically website allows you to create the requests [Music] um for for for fetching data and that's what i'm actually using right now for fetching the posts uh so basically i'm composing the request using the website stuff so yeah so basically that's what i want to start doing now is just go and have a look if i can if i can replicate this this effect using just waxes and the reason why i want to stream this is because there's going to be probably some gotchas and issues that i might encounter because while the website is good um that the the whole what is it this whole project is is actually supported by the rust core team as far as i understand so it has a full rust backing but you know that there are some issues here and there with the full replication of the of the javascript ecosystem in in rust but yeah as far as i uh i'm following it a little bit so it seems that it's they're doing a good job in in managing to port the javascript functionality into rust okay so i just want to think a little bit uh however to do this and maybe just hold on i just want to catch up where am i currently with the stuff where's my webpage here i actually did a a little blog post ages ago about you but it it changed quite a bit since then so that's probably need to be updated as well um yeah so let me just have a little thing how i want to do this effects so the view revolves around the components essentially and as you can see in here so you just create a struct and you need to implement the new traits on it with the bunch of metals [Music] and by implementing those methods you basically give you pointers how page should be rendered at any given time um so um so i think we're just going to start from creating the struct we call it particle effect maybe for starters um so that's going to be our component where we're definitely going to need a particle so now we need to use the new stuff [Music] okay so now here we should have the uh of the completion going on uh some errors yeah that doesn't matter it later okay so the particle [Music] so we will need to the particle we need to be able to create it we'll think about that later we will need to be able to update it because that's essentially we're going to get into the realm of uh you know like uh graphics graphics rendering essentially so we're going to be able to update the particle and and if we're going to be updating it it's going to be movable itself that we will need to pass into it and we're going to need to have a way of rendering and probably well if we're just rendering let's see if we can do it like that okay and then okay and right now then let's focus on uh see how we can actually create a component which will give us this effect so for that we will need to implement the component trait that comes from you and for our particle effect so now we just need to implement this bunch of methods that comes with this trait so first is the create you prelude the rust analyzer didn't see the stuff so actually we need the type so the type it's the associated types with the the component traits so one of them is message so i think we don't have any message for now and type the properties um [Music] like we probably will need to have some properties and what we will need we'll probably need like width height number of particles stuff like that okay we'll think about this later and we need to create a function so this is one of the main differences in that master branch uh version from the current one here they basically uh lump it all together the the parameters that they were passing in previously so like properties link which allows you to go which allows components to communicate with one another right now all of this is lumped into context uh struct that they're passing into this so i don't know if i can show it here getting started examples okay so all of the examples there actually looks like they're up to date per day and project setup yeah we will just pull it up yeah so that's the old way of doing this like in the version 18 that's currently in the in the crate repository in the crates but the master branch and the future release is going to be this so since we like to live dangerously we're just going with the master branch so yeah so create um so right now we're just returning the vertical effect or yeah article effect we need the update function so right now it's going to choose false i just want to build the skeleton uh i don't think there is uh there was an update so then there is a view method with again just the context and that's supposed to return the html okay so are we compiling should be good to go yes uh okey-dokey so we just need to be able to return our own canvas in here and but normally uh we would um we would make some modification uh you know as a result of a you know in an event handler or something like that so as a result of a click we would normally just react to that click by some sort of message which we would specify in in here in depth update method we would make a match on that message and depending what the message is we would for example made some modifications to the to the dom and return true so then if the update methods return true the u would basically re-render the entire page but in this case we want to animate stuff so we not going to be doing this using the update method we will going to be redrawing stuff using the canvas itself this in theory so in order order to do this we need to be able to to modify the the canvas outside of the normal component life cycle so in this case we can use something that's that's called the reference to the um to the note to the html node um yeah so so in order to do this we need we need to be able be able to keep that reference to that node within the within our particle uh struct um so for that we need to be so for that our struct our component structures to be able to keep the reference to the node so i'm just gonna call this canvas and that's kind of us it's going to be the node so what else we might want to store in there um so it's definitely going to be a load ref and probably the particles that we're going to generate uh particles yeah so that's going to be a vector of particles okay and in here in that case we need to get the reference to that canvas and canvas requires to close the html macro it actually enforces heavily the the soundness of the html syntax um so basically any um any issues with the uh with you know missing html tags and stuff like that will result in the compilation error okay so what's going on here so because we added this stuff here we need to fix um oops to be more rusty and we want the canvas and our particles okay are we good for the time being so what else do we want should we focus on the particles yeah i guess so so we need to be able to create a bunch of them we need to be able to create a bunch of them okay so yeah let's have a look at the particles in that case so what uh what particle might need so it will need a position like that and what it will need probably size because that's going to be used for drawing um yeah so then that means that we will need to create a bunch of random values so the particles are going to be flowing from top to the bottom just like the example in here um so so we will need okay so here we probably gonna need some uh random generation stuff um so i think that's the problem because i don't know if there's any if the random generators work in in the wasm context because that's it's basically wasm is like is like embedded so there's no std uh libraries in there so let's have a look so we need this and there's going to be probably some feature that needs to be enabled what features std wasn't support it wasn't target blah blah blah it's not automatically supported or other dependency get random with the js feature and that's unfortunately what we're using she can run on webassembly support so you just need to enable yeah with the gs okay but what does it mean like add it directly with this so just either use a different target such as or other diet or other okay i think we need to add this in that case so i think it was yeah so what does it mean so x so we want to we want the particle to be generated somewhere on the so horizontally in in random places but started at the top of the canvas so okay let me call it i'm not sure if that's unwrapable or not so rng range so we went from zero to uh yeah we want it to be as wide as the canvas so let's do it like actually no that's going to be stupid okay let's do it correctly so with so canvas width it's going to be i think we want it always to start at the top and then that size again random [Music] what's the proper size maybe from one two two because and generate a range so for velocity zero [Music] two one oh so in that case velocity we're talking about a float so more yeah that okay so x y and size size and velocity i don't know what's right down here that's not here there we're building so that that means we need to be able to pass the canvas [Music] okay [Music] we [Music] on every call of the update method in here we will just take the self and y equals we will just increment this for the time being later we will change that depending on the speed and the movement and stuff like that uh so for the time being we just just want to make sure that it's fine um and then we want to be able to restart it back from the bottom so height and yes okay so that means it's going to be so we just start the height [Music] as well let's do something in the chat okay so canvas width comes tight okay so we need to take not sure if that's the best way to do this but we'll see for the time being [Music] okay so that's going to be so if it's larger and we're just gonna restart it back at the point zero and one zero there okay now we're going to be rendering it so to render it we need the canvas okay so we need to have us look at what is available in here in terms of the canvas api so canvas okay so this and this is what we're going to be needing so then basically we need to grab the context of the canvas html element which is the raw element that we that we have a reference to in our component so when we get the context we will get the object which we will need to cast into this into this yes okay so this is basically straight away you can see how rust is a little bit more verbose as opposed to what you could do in java so how can we do this so in here okay so i see that right away that's going to be needed so [Music] we will need the properties anyway so struct code and we are going we will need which is u32 [Music] which is this and which comes from the new framework and it's probably gonna have to be clone as well and since these are our properties we need to we need to tell you that this is what we're using for our properties so basically we will be able to access it from our context and now what's going on here oh yes it's the requires also the party here so now if we want to uh access the the properties we can do it so let's say our height for example yes so maybe let's mount our component so should i create a new web okay i'm gonna go lazy and i'm not gonna create uh a new page for it but i'm just gonna stick it in my little about page so i'm just gonna stick it somewhere here so basically we just need to import our new particle component so what was this um props and we will need the particle effect needs to be is this okay yeah okay so we should at least be able to see it in here somewhere i'm being silly because we need to okay let's add it just here like animals and because we specified those particle props and they're not optional i think that should give us a compilation error that we are missing our properties and missing required prop height so our height is going to be let's say 500 and the weight is going to be 500 so i think now at least we should be able to see it rendered on the page do this so now if we reload this there it is that's our canvas okay progress so that's essentially how you create a component um okay so now uh we are actually we're not putting in our dimensions in here so the width and let's just that's because oh yes because we passed it in as we passed it as an integers so we can either just simply do it like this or we can run a tostring on our canvas element oh yes no actually i was wrong well i was kind of right but that's because we specified this as the u32s are across the string okay this so with our height okay so now we need to be able to render our particle effect so for that we're just going to simply import our particle effect and we will specify a method called render and in here so in here what we need to do is um we won't be returning anything because we were simply going to be modifying the um our canvas in here so we need to get we need to basically this is just a raw reference to our html element so what we need to be able to do is just to grab it as the actual canvas element um so let's cast it and into the canvas element and that's unwrapped and so we can either specify the type in here or we could just simply specify the type in here and that's going to be html and now that's going to become and from there we need to be able to grab the context so canvas and dot that was just get context was it okay we have too much stuff here so we can just so get context and the context id as well which i think for this is just 2d so canvas that give us result and option okay so um and then we just need to cast it because this is one of the kind of weird things well not weird like once you know it you you know it you get you basically get like a rust representation of the javascript object so once you have this you actually need to uh i think it's dim cast now it's into [Music] that returns the actual uh object casted into what you want exactly which is this object called canvas rendering constant context 2d something like that and that's a mouthful cannot find canvas element oh yeah yes basically almost any almost everyone every html tag available alright almost every html object in websis is hidden behind the feature so you have to list every feature that you use every type that you want to use you have to list it in your cargo tunnel so we are using the hdm html canvas element and we are using the yes canvas rendering context canvas yeah so we need to we need to list both okay so this is basically what i was saying uh with using the websites in rust it's it's not as you know easy-going as javascript is because this strict types enforcement and yeah so essentially here now that we have the context we can start drawing with in the canvas itself so like if you're familiar with the canvas api you should be able to just now start start drawing stuff um so basically let's just start it easy and let's see if we can draw those particle particles like you know raining in the canvas um so let's uh fill the uh we need to fill the element that we're going to be drawing right now so that's going to be the those particles and so we need to set up a style of dose like i'm not the greatest in the canvas api but i did couple of examples on the using just a javascript so and in here like normally in javascript here you just basically specify the color you know like white rgb value but with rust because of the translation that it has to do between the rust type and the javascript and in here we need to specify the js value so let's just i'm not sure if black is like that and then we need to be able to just fill the correct field rectangle and let's have a look at the parameters so x y width and height here so we just want to fill the entire uh entire canvas and oh yeah so in here we need to be able to pass in the to pass in the conversation well not exactly uh we have the canvas right so actually canvas itself uh yeah it gives us it gives us the height and the width okay [Music] okay um let's see how we're getting known and not found this canvas rendering context html element this way oh um okay so here we are getting into the stuff i expected jason yeah okay so what's going on here so that needs to be expected from sos 464. what was the problem with this or something here as well yes okay like that's the last thing we need to be able to in order to use the specific trait implemented on the object you need to have that trade in scope no what was that it's not it's coming from the i wasn't being again and just cast here okay check okay so just a bunch of floorings so let's see what we have here okay so definitely there's something here because it moved the stuff down okay not invisible yet oh is it yeah because you just can't see it what was it two point two i'm not sure if that's allowed like this um let's see if we can display let's let's see if we can display the particles actually yeah okay let's create a bunch of particles or particles even and we'll see how we're getting on okeydokey so so yeah we need to generate a couple of particles and we need to render those okay so actually so this is one of the other things so we need to be able to display them so we will be keeping the the rendering context in our component so we will have access to this so we can just pass it into our rendering method so [Music] so let's just the canvas context and that's just going to be the reference rendering context and in here so how do we what was the steps that was something like yeah this begin part so context and we fill the style style so we have a black we should have a black background and again to pass in the javascript object so the js value from string and and then we can just draw the arc which will just create the our particle which is essentially just a circle and here what do we have arc there's a bunch of parameters for this oh and our result so we need to remember to unwrap it so x y radius angle angle so x and seven size and we need the start angle which is this and we need the end angle and that's all in radians so we need some pi and [Applause] we need to uh whatever four i don't remember uh yeah so we need the pi times two blah blah blah wrap and [Music] oh yes because yeah this is where i've done all this messing with types so so those are f64s all those so i think let's just keep it simple and we're going to turn those all into f64 because otherwise we will be just like messing around with this and so that means all of our random generated stuff as well ah chase it's getting messy so count this with 64. so that as well okay so this so that's our columnist height as well yes okay we could good so respect yes because the my obs just crushed so not sure where we're at where i ended okay we're we're back in business okey dokey um i kinda lost my train of progress but yeah let's see where we are we should be okay now actually yeah so we are just yeah so we blah oh yeah so we are drawing the ark and blah blah blah we unwrapping it and we need to actually now we need to uh what does it feel and the lack of our competition is a little bit annoying yeah okay so this is how basically our rendering is going to look like uh it's just boils down essentially to just drawing the the particle on the screen so here let's just generate a bunch of those and i'm not sure what's the reasonable what's the reasonable amount is but basically what we need to do is just create a bunch of particles keep them in here and then just render them in in our methods in here so particles okay so particles particles and so what do we want let's just generate a punch so i don't know 4 000 let's say so particularly and we need a canvas width and canvas so collect is canvas height the closure takes argument there yeah and that expects okay okay so we're looking good if we just there should be no i'm just reloading it to see if there's any uh crashes or anything like that because sometimes that happens as well okay so what we need to do in our render method now is that we just need to iterate and because we are going to be updating those particles we have to either remove over it and [Music] for each particle update so that will update the deposition values and then it will just update but render render but that requires our context the rendering context okay what's our problem here gonna follow us mutable okay because we need the ethernet okay okay that's good and now this is the trickiest part because in order to animate this we need to request the animation frame so requests animation frame there it is so that's a function on the windows object window object in websis so this and that requires our callback function okay so that's going to be a little bit tricky so that means that we need to request our window window so like we have access to it from the website and also you re-exports that method now just i think we're just gonna take it from the website i don't know if that matters to be honest that's request animation frame and that expects a callback that we will need to provide and that's going to be another tricky part so like you don't want to go crazy with the recursion so normally you would simply here just call the the render method back on it but it's not that easy to do in the wasm so okay let's let's think about this so that takes the callback and that's a function js [Music] this is so that's object so we need to specify the callback in here so normally the callbacks in you is what you would simply use as a link to communicate with your other components so in here we just need to specify the the closure so we're just going to we're just going to wrap our own closure that we can pass into the and the request animation frame so it's not going to take any arguments and it's simply going to be the the callback that gives that the that allows us to communicate with the u components so we will need the so we will need the uh yeah that's a problem because we don't have we don't have access to the you context okay so that's getting messy so you can text and that's the context itself and actually not even that we're only going to need the use scope yeah this which is essentially the link so if we look at the you documentation i know just want to see the api docs here scope has a bunch of methods like for example send message or send callback that allows us to send the the messages within the component and then react to those events so the send message is basically a synchronous message that you can send to the component and the send callback is something that you would call that you would use within the event handler or something like that so we need to scope this and pass it to the callback okay um message and that's essentially just going to be the message that we need to create let's just call it render or re-render possibly and actually we can do this well we can we need to box it there [Music] uh be a box of type and function mute like this and then this we can pass into our callback okay and then and that means we need to specify our messages so not here in here so jesus it's a message and it's going to be our render and so that means we need to listen for that message in our update method so we're just going to match on message message and when it's message rerender and we are going to call the render message on self particle effect render render and we need to pass the uh [Music] context to it uh that should have been a reference city x so this has to be i'm not sure if that's the best way to do this but i guess i just want to get stuff working to be honest okay so what's in here and yes because that's expelled expects the rule to be returned you can just do this expect it oh yeah because we didn't let you uh know that this is what we're using ok and wanted more error which is here one more error message who is this rock expected reference gs function there jesus oh yes there was something there function yeah it was this unchecked ref where gs cast performance zero cost unchecked cast into reference yes i think that's what this is a mouthful as well explicit lifetime required in the type oh that's because we're moving this into the closure so let's uh link equals good and one more error i fortunately cut this quite quickly nothing major was lost okay so yeah so that's and i think we should start seeing stuff but we do not okay i think that that's our problem [Music] well done because so request animation frame that gives us a result i'm not sure if this isn't some sort of handle so that's for starters and then another thing is the closure wrap looks new so that should put it into the heap okay so i think that when this function dies let's get dropped [Music] so we need to keep a handle for this so we need to keep it inside our struct [Music] like that and this is so that's going to be our cpx and that's and that's essentially our closure we just need to keep the the reference to it and then that means in here we don't need this anymore okay oh oh and the other stuff is oh yeah i'm stupid as well so probably the other thing was correct because that's on the heap with box new okay we'll see but the other thing is that we never trigger the update so we need cpx initial message to perform the render so then as soon as as soon as the the yeah when you when the create method is called on the component we send this message to trigger the rendering and that actually compiles boom and we have rendering okay nothing is happening right now because there isn't much but we see that the particles are updated and they all travel travel at the same speed so if we take this and like that it should create those all over the place all over the canvas come on there it is okay so let's try to display an image now so that's png here what's the size of it 500 by 677. okay so we'll try to display the image and rds trunk truck truck truck did they pass to avoid myself some typing ok so so what i want to do is i want to be able to fetch that image from my server and display it in here so what we'll do is in here initially when we do create we will send the request to fetch that image and i have a fetch function here that i use for fetching the post and so i should be able to just modify this and be able to fetch an image image part blah blah blah and it should be all the same but here that's not going to be a text we need a like bytes or a body or something like that and what is this response to webster's response um so we need the response um and for our post i was using the text so which just gives me basically the promise with the with the text content okay so we will need this or this dlp requires the following create features we already have that so i think it's plot and that gives us promise okay so here yeah that's blocked which gives us promise so that gives us image bytes and so but okay we're getting ahead of ourselves we want to be able to write the image request animation frame image with bitmap or image element so we need to be able to just grab the the image bitmap i guess draw image there's plenty of those is there something for blob gloves create image create image hmm create image bitmap with blob that's something on the window type create image okay that was what i was afraid of there it is like something like that we just get a blob from our request here we get image bytes comes as a moment block and window this so that means we need to so that's our so jay's future for um so here we actually going to actually get the blob and so that's our block and that's js value so that means we need to uh so there we can use this create image bitmap with block that takes a blob and gives us another promise so let's so what do we get here create image bitmap with block so and that's oh here i was using the window from uh view from you okay let's use it i need to be more consistent i guess what am i using and where so [Music] create match bitmap with block create image and that takes a lot as a reference and creating a blob and since this is a promise then our image here we need to await well we need to um we need to fulfill fulfill the promise so js future um and that's our image bitmap promise and we need to await this and wait and unwrap it up so in here we should already have the image which we can return here oh yeah and from websites and this as well yes and no image bitmap in the root okay okay um image put my progress will work but found because we need to uh then into it as well and then unwrap as well we'll get to the error handling someday probably okay we're looking good blah blah blah there yeah okay so where are we so that should compile yeah and now we should be able to fetch the image that we want to display okay so if we have the method to fetch the image that means that we should be able to run it in here upon creation of our component and how do we run this yeah so that's what we use so you gives us uh [Music] you gives us a sense future method that allows us to run a synchronous uh code from within our component and so that's what we're going to do as well here so ctx send future and within the the the future argument is basically the our async code that we want to run and that's going to be a fetch it's bgpng and the method can fail so we need to uh and that's going to be our image and then something or an error we're going to do something else so that's something and something else is going to be basically a message that we are going to create so here so uh it's going to keep our container image bitmap and png fetch error and some error it's going to be the string in the error and so that means that in here so our message okay and that's going to be our image data and our message error that's going to be our error to string like that oh yes yeah so once we fetch the image we're going to update our component what's going on here mismatch types that's because we need to await this because that's our future or asynchronous code so we need to add those to our update handler so if we have some error message we are going to just display it and like i mentioned before you removed the um all the services that allowed you know printing to console and stuff like that so right now you need to use glue for this which gives you the console functionality and so like this and just err message because that should be string and but when our image fetch is okay we should have the image bytes here and so what we're going to do then so if we have the image bytes and hold on what's going on here yeah so if we have our image bytes we should be able to take the canvas for example and just and just display it and that's what we're going to test it now so we will need to grab the canvas and because we have the the canvas reference in our struct so we will grab the canvas html element canvas because uh you remember we have the note ref in here so that's kind of like a generic reference to the html element and if you want to extract the actual element we have to cast this and that can fail so that we need to unwrap it and once we have the canvas element we need the canvas context that we can use for drawing and this is the canvas rendering context 2d and again we need to uh the graph the con context from the canvas object oh jesus tiring even to articulate get context 2d and then there's a bunch of unwrapped unwrapping that we have to do and that has to be again can fail but after all of that we should have our context that should allow to draw image from bitmap i think it was draw image with image bitmap so image bitmap and the location so cctx [Music] uh bitmap okay so that will take our [Music] image bytes and the starting location which is just the beginning of the um the compass yeah for some reason this macro is causing this loss analyzer error but it's not in fact an error and if we're lucky was there any much yeah there was an image but we just redraw it straight away we straight away or redraw it when we run the the render method so actually if we just comment this out the image should stay on the screen there it is okay so now in order to copy this this effect what we need to do is just grab the [Music] the pixel information about about the image so so image fetch okay so in here so we can do this multiple ways actually but the problem is that if we were to grab those image bytes straight away we need to be aware of uh you know of the pixel layout depending on the image file format and stuff like that um and we would have like parsers for you know different image types and whatnot but what we can do is we can just draw the image on the canvas and then read those that image data right away read it back from the from the canvas and then we have a a very uh you know very nice layout of the of the pixel data because the the format the pixel format from the canvas is like rgb and alpha so that's what we can do actually so you know instead of messing around and you know parsing those image bytes manually we will just reuse the canvas for that so so image pixels we can just read it from the from the canvas so we should be able to just uh what was it uh mesh data from the canvas from the well the rendering context here to get image data and that gives us a result with the image data okay so good we want to grab the image from the entire canvas and then again and and the height is this so canvas with canvas height okay and then once we have so image fetch okay once we have that data we can extract the the pixel data which is the basically the brightness of each of the pixels and depending on that pixels depending on the brightness of the pixel of the value of the brightness we can control the speed of those particles that we want you know uh floating floating around or you know descending to the canvas um okay so that's going to be a little bit messy should we maybe no doesn't matter uh brightness map is what we're looking for and that's going to be a vector right brightness so that's that okay and now what we need to do is just iterate over all of the pixels so first first we iterate over the each of the row of the pixels and okay we need to make a little break are we back online again okay we are yeah i had to uh pause the stream for a little bit got to i had to pick up the the child from the preschool came up unexpectedly i didn't realize that you have to pick them up okay anyway where are we um oh yeah so what are we doing we successfully are picking uh or not picking fetching the image from the server we agreed on that the picture is going to be one of guts and oh yes so we're displaying it in the canvas and then we grabbing it again just to save ourselves the hustle of uh actually messing around with different image formats and stuff like that and raw parsing the uh the image for pixels so essentially we have the so what we want to do now is just get the the brightness map of the pixels so yeah so we're going to iterate over the over the rows so that's going to be from 0 to the whatever the canvas height and then we're going to create another look for x in 0 to canvas width will be going to iterate over each uh cell or each column uh so we're iterating over the the row and the column of the image and so that's this is going to be the uh the whole map of the brightness values of every pixel and in here we will need to have to store the intermediate row and like that so basically once we are in the inner group loop inner loop and we are going to grab each of the values for the red green and blue so red is going to be the so we're fetching it out of the image bytes so we should be able to we should be able to index it i think directly into the image bytes because what we get here is the image pixels let's have a look get image data and we get image data and that should have some sort of iterator on it no oh but we just can simply get the data which is clamped vector vector of u8 okay so so that's not actually the image pixels that's just the image data data that's our buffer so this case is and we should be able to just index into it now as we uh as we need it so we're going to access it by the row times four because we have the four bytes per we have four bytes of information per pixel and the width and that to index into the cell itself x times red um so the fourth value is the alpha um so we could probably just you know alpha we we don't actually need this yeah so actually i'm not gonna bother um okey-dokey and then so here we're going to have to uh just calculate is um so we will need to implement this calculated brightness and that's actually i'm just going to steal from the internet there's a there's a whole bunch of uh you can actually steal it even from the video from frank laboratory it's somewhere in the code uh actually this is just a way to extract the brightness from the from the raw rgb data okay and once we have this we can stick it into the row the row the row vector like this and once we have the row and once we do this for each of the rows we will add this to the brightness map itself so brightness row and then once we have this we will put this into the small self where do we have this okay so we need to stick it into our particle effect so brightness map and that's going to be a vector a reflector of whatever the brightness is going to be which is probably like f64 and the reason for that is is that we're going to have to calculate this using the square root function function but we will going to have to fetch the um we will going to have to fetch this from the jss library not from the website from the jss because that's the essentially the the wrapper around the mat library in javascript so brightness map like that so that means that in here once we have the brightness map uh we going to have to actually sd no we will need the either swap swap the value to the location without the initializing the other one if you want to swap with the default now we want to replace the place move source into the yeah there it is and we're going to replace the the brightness map which right now is just a default empty background with the brightness map that we have here locally uh and now because so far we were triggering the update method here so actually we won't do this in here anymore what's going to happen now is that this future is going to get executed in the create method and that will either get us into the fetch error or image fetch okay so once the image is fetched successfully we're going to populate that brightness map and we're going to trigger the update rendering method from here like that okay and now we need to do this calculate brightness stuff calculate brightness so actually going to be before as well okay yeah so here this is you should have a map in here [Music] so it's here so so js and this is so what we want is essentially we want the red yeah times zero points what was the function uh yeah and then plus and you know those functions they're out there if you google it it's there it's something like relative bro brightness or if you want more details going to that youtube video in here we're just focusing on the canvas the website stuff and the rust and and that divided by this so that should give us this so that means that this is going to be okay so get image update method not found oh image data are we missing it that's what it is okay and yes yes for indexing we need view sizes so this and the same those oops are we winning so in here you can either do this like this or just do exclusive cast like that and i think this is more readable at least to me so i do it as that and here [Music] what's your boss [Music] let's uh yeah okay so where are we and this get image data what's with this oh yeah that's expect the f64 as well okay and okay what's going on here is that's building okie dokie okay so i think we can just uh okay what's going on with the image here let's check okay so we're generating this but now we want to make uh the speed of those particles dependent on the uh the the brightness of particular pixel uh of the image so what is this enter okay that's one of the other rust analyzer problems i'm not sure why is this happening okay this doesn't matter it's not actually a problem uh men replace that must be used value okay okay so now we're going back here we need to update the particle itself um okay so we want to update this stuff in here actually it's not in our render and what that essentially means is that we just want to change the movement depending on the brightness okay so in that case because in here we change this to float so in order to index each of the pixels uh so position x is going to be [Music] x as new size because the usage is only used for indexing and we need to do the same for y like that and that means that now we can get our [Music] speed equals okay so in here we will need to pass in the brightness map so that's our brightness map and that's going to be a slice of vector of e8 okay so now that means that where is this coming from that's an error because this should be the brightness which we're going to index it with the position of first the row and then position of the column and then you will calculate movement so let's see two minus the speed what we just got and then plus variable velocity and then the self y is going to be the previous self because we want to advance it and plus this movement value that we've just okay oh yeah because that's a float so that should be it then oh okay so actually we just need to pass the self dot brightness map because we required it to to get the speed what's going on for ourselves so here we borrow us mutable okay because we have the because we only get with rust we can only have one exclusive borrow if that world is immutable so that means yeah we just need to map because we wanted to do this okay let's have a look it's building that's looking okay okey-dokey it's almost something is almost happening so i think what we just need to do is just we just want to change the what's the opacity here so from something between the update and the render for the each pixel to get the speed of the of the particle oh so that means we need to also store the speed so f64 like that so that means in this case to keep the speed [Music] uh like [Music] okay i don't know what the story is but under sway anytime i switch to the obs to check everything is okay it crashes and it happens something like three or four times already on the stream so i'm not sure what this story is okay that's annoying but that hasn't happened before okay anyway we're almost uh we're almost close to the uh to the conclusion and i'm to be honest i'm just uh trying to get this working as quickly as possible because i need to do the baby stuff in a second [Music] okay i'm just gonna try to clean up some of the warnings okay so just to recap what we have so we have the particle create particle with some random stuff oh yeah and we don't actually want this to be random uh we just want to we want the particles to fall from the very top or from the bottom it doesn't matter then we just need to here we just need to increase the here we just need to then increase the or sorry subtract the the y value yeah it doesn't matter uh so this this this okay so we created we updated depending on the brightness of the um of the pixel and then we render it oh sorry we rendered the particle uh but here this is just uh it has nothing to do with the with the image it's just how we're rendering the white stuff okay it's building building okay so it's just stopping on the outline of the image so with that because here we're going to make this dependent on the ore they're dependent on the speed so that's why we put the speed into the particle structure yeah here we have the speed there sorry it's not so speed the speed belongs to the particle there it's party [Music] okay so we have our velocity this render method that's all fine so the issue could be somewhere here [Music] when grabbing the the brightness data [Music] [Music] oh that's what it is i didn't spot it because this stupid highlight and i just ignored this like that we divide by 100 the result of the square root perfect there it is okay so that looks a little bit better and then again this effect can be you know improved a little bit that's not the point of it it's just messing around with canvas and and the you and the website stuff and whatnot um yeah so that's it probably we can velocity it's might be very fast so but anyway all of those values can be adjusted and that's how you do stuff in you and and websites and wasn't uh in rust it was a little bit painful i guess to watch uh especially with those stupid crashes it actually crashed again what the hell and i'm just fresh after updating the arch just this morning so there's probably some of that causing it um yeah so here we can do also ccts like you can clear the image here image now let's where's the canvas rendering context or clear clear rectangle there this so we can the same arcs and what else can we do here yeah i guess this is it there's not much stuff like anyone can play with those values as needs make this a little bit smaller as well here we can specify the particles as a property particles that's going to be coming from the props as well [Applause] [Music] i think that's going to have to be yeah but that means then we need to specify it in here as well [Applause] and let's try it 8 000. so that means we can control it from through our component parameters or arguments there yeah so there it is more particles so this is like kind of this you know a sonar effect or whatever almost yeah so that's about it to be honest um the rest of the stuff seems to be working as well and closure for the course of the year this trade already that's because of the page reload and yeah so basically what we've gone through is just you know operating on the html elements through websis crate uh we've we've done some fetching from the server uh we manipulated some data uh you know from blob into image data you know into image bitmap we play with canvas a little bit uh we saw how to create an uh component uh we saw how to add it you know into a web page and how to call it um we saw how to handle handle different uh messages within the the u component handling we saw how to execute a future within the component what else yeah we saw how to convert the different js objects you know what's the words transform them from one into another so yeah i think uh i i hope that it was useful to someone to be honest um like this is so all over the place so i'm probably just going to try to assemble it into a coherent one video and just stick it on youtube or something and so yeah and definitely check out this dude's channel because it's excellent if if anybody is looking to play with some graphics on the web it's frank's laboratory yeah so take care hopefully see you soon at some other stream i'll probably will be doing some more embedded stuff soon because i have a little side project that i want to look after okay take care bye

Channel: Vers Binarii

Views: 1,928

Keywords: rust, wasm, yew, canvas

Id: 7Smco8araSo

Length: 172min 46sec (10366 seconds)

Published: Wed Sep 29 2021