Serverless & Databaseless Event Sourcing with Cloudflare Workers & Durrable Objects

hello friends i'm mike kan and today i'm going to take you through a project that i've been working on lately called clouds and edges um it's a project that's been built on cloudflare um cloudflare workers and cloudflare durable objects so if you're not familiar with cloudflare workers they are kind of similar to aws lambda or google functions except that rather than running in a single data center they are spread out all over the world at the edge so that they run very close to where your users are going to be calling them from which makes them really quick so and durable objects i think are really really cool technology they're brand new from cloudflare and they're kind of um they're in beta but i i've been thinking that's the whole reason for this project is so i can tinker with them and they basically provide the stateful part of serverless that's the way i like to think of them which means that they allow you to have an object which persists in the cloud in the void that you can call from a worker and get the state back that you previously stored in there they kind of remind me a bit of the actor model if you're familiar with that and they're a little bit like that but i think they're going to be really really transformative in the architecture of multiplayer games specifically because it allows the game to remain serverless which is great for scale but it allows to have a certain persistence but before going too deep into that um let's keep going so um one one example that they used to build an example they built to demonstrate durable objects is they built doom upon cloudflare workers which is pretty impressive they do multiplayer so you've got the full classic multiplayer experience of doom running in the cloud in a serverless manner which is very impressive to say the least so i'll provide a link down in the comment in the description so you can check that out anyway back to the project okay so i'm going to start off by just taking you through um the functionality of the app and then i'm going to give show you like a a brief high level view of how it works and then if we've got time i'll take you through a simple example an end-to-end example so this is clouds and edges this is what you're presented with when you first arrive here and you haven't used it before so you just chuck in a name and this would be like your normal sign up with like email password or whatever but just for this example for this project it's just very very simple it hasn't really got any authentication you just check your name in it so let's put mike in here and click sign up and here we are we're taking straight to the user profile section so we've got user profile here which is your name and an id uh sign up button and a sidebar which contains this section which is your profile the matches and admin so we'll get to those in a second you can change your name so my key c and then yeah it's saved in the cloud so if i was to refresh then you get that so um pretty simple but it kind of allows me to just kick the ties and tinker and test with durable objects so the next section is the matches so this is a game that's based upon a classic well pen and paper game called dots and boxes if you're not familiar with it it's a game for two players and more but basically you just draw like a grid of dots on a piece of paper and then you take in turns to draw a line each so in different colors and if you if the last person to draw a line to complete a box gets that box so you get a point so it sounds pretty simple and it's pretty simple but you it then it lends itself to some pretty interesting strategies where you know you build up chains of these things and you want to prevent somebody from [Music] you know finishing off the chain and then you kind of like yeah you do bits and pieces and it gets interesting towards the end of the game but i thought it would be good to do rather than like a simple notes and crosses i wanted to do something a little bit different but um still kind of interesting so um yeah so these are the matches so you've got two different kinds of uh my matches which lists all the matches that you have active obviously i've got none and open matches so these because it's a multiplayer game it's the last or lists all the matches that are available so we can click new match and you can pick the size what kind of size grid we want three by three five by five seven by seven like for example five or four five five create that and then it creates um your match here and then we can uh we can all we can do currently is uh just cancel it so it's cancelled yeah pretty simple really um but to obviously show demonstrate this we need uh somebody else so let's create let's stick this on the side here and let's create a new person chuck them on the side here come on windows thank you let's open up this url get rid of that bit check this in here right now when you assign it here let's call this person righty righty tighty okay right so um it's automatically assigned me a different um emoji currently can't change that but i'm just using that in place of what would be i suppose your uh your profile picture your avatar from google or wherever um but it's just a random emoji that i i select when you create the user so um there's no matches running so if we go to matches there's no open matches but if we create one here so we create it here we go back to here when we click on it it refreshes and pulls down the new match so then we can join it with this person so now it's created here we go back to here we'll see it's open and it's their turn shorter and their turn so we can open up the match and you can see uh the game board so let's open up on this side as well so it is there so it's your turn because you joined the match so now i can pick a line to go so i could say like i want to go here and then i go over here and i can see that they've gone now okay i'll go here okay i want to get here and i'll go here and go here and i can you wouldn't normally finish it off like this but yeah you can finish off and then you get your point so i've now got one point and obviously the aim of the game is to um you know when all the lines all possible spaces are filled uh have them more score than your opponent so i'm going to um just go through this now and just finish the game just so you can see what it looks like at the end of the game okay so we're back and you can see that um almost all the boxes are filled in it's just the last one now so when we click it the game finishes and says you won go back in there you lost so that's what you expect um so then if we go back to the matches um page you can see that the battle's finished you can still open it to have a look but yeah that's the end of it so that's basically it that's the basic functionality of the game it's pretty simple but you can see that it works it's multiplayer so let's uh minimize this guy and then go back to full screen and let's start diving a little bit deeper to underneath the covers of how this actually all works so let's let's go down to the third and final section which is the admin section so this section admin page wouldn't be available obviously in a production app and i'm only including it here just really kind of just for demonstration purposes and just showing what's going on behind the scenes a little bit more so so what do we have here well we've got a few different sections we've got events aggregates processes and projections um if you're not familiar with event sourcing then i'll put some links down in the description but i will try and give you a brief explanation as we go hopefully it makes sense but um there'll be better explanations in the comments uh in the description so the first section is the events and this is a big giant list of every single thing that happened in the in the game so you can see the very very first event zero zero user created and it's it was the name first when i was testing it and then they created a match and match was created is joined matches cancelled so what this is basically a huge immutable list of everything that happened in the application and mutable means it can't it's never changed so much like blockchain technology if you can have a huge immutable list of everything then you can replay all those events and get the current state of the world so in blockchain it'll be like you can get somebody's bank balance but in this you can get the states of anything you can get the state of the user you can say to the match for example it's very useful because it means that you your back end is flexible so for whatever purposes that that changes the business requirements change so now you know the rules for the game change but the you can just process those events in a slightly different way and producing a slightly different output but the events stay the same it's very useful you never lose track of a change for example so um what we can do with this admin interface is we can reverse it so we can see like the latest event was uh was a match finished which is what happened we were playing before and we can see some various things about it so aggregate is a match which we'll get to what that means the id of it and who the winner was and then we can see there's a whole bunch of turns taken as we're playing the game um so other things we can do in here we can query the uh query using uh durable objects queries so we can just select just one for example um or we can just limit it to just a number of just a couple of uh entries oops um or we can offset the query from a given key but yeah makes sense um so once you have this immutable record of all events that ever happened in the world you can then as i mentioned before you can then collapse all those events down to a given state so this is where projections come in this is what a projection is it's a projection of your immutable list of events so for example there's only two projections in this particular application but in a normal application you could have a projection that represents a very specific part of your app and is obviously um it's designed for reading just purely for reading so it's obviously read optimized so it looks it structures it exactly how it needs to be rendered in the client which means you don't have any experience or queries or anything that's why i really like it so in but in this example we have just two we have users and matches so for example uh in the users there's only there's three users there's first which is the debugging queues i created and then there's me mikey c and then there's writing i noticed that it didn't doesn't have the name mike because we then changed the name later to be mikey c we can do the same thing as before we can query we can limit we can start from query it can reverse um and then we've got matches as well so the same sort of thing where this is a column a condensed version of all the events and it just represents the state of a given match so this one must be the match that we were on which because it's huge it's got all the the lines that were added um which represents the state of the world so um those are projections so the other part is processes processes um are what happens if you want something to react to an event so when you cause an action in the client that will create a command which will execute an event but then say you want something to happen say we want to email a user well we don't actually do that immediately instead first we make an event put that into the store into the event store and then we have a process pick up that event and then it then goes to a series of things that happen so one of them might be we email a user to say match has been created here you go come and join it or you've forgotten your password or you know any any sort of like um side effect that we that that would happen in the application in this case we've got two which is match creation match joining they're not strictly necessary but i wanted to include them just to kick against tires and event sourcing on durable objects but basically um in this case they just it limits the number of active matches you've got so it won't allow you to it recreate another match if you've got more than a certain number already active and match joining um is also similar to that so the final one is aggregates i haven't actually implemented these yet because there's some nuances around what i actually want to show there but aggregates work like i mentioned before where when the user performs an action clicks a button it creates a command which then gets turned into an event to illustrate that further i created a mirror a whiteboard of the the system of a high level system so if we dive in a little bit here we can see that the user sends api requests which are structured as rpc rather than rest to a cloudflare worker and this is your entry point for the whole system and it provides the api and then it dispatches commands well it it passes it in turn calls rpc methods on these aggregates and these aggregates are durable objects so think of them as a class an instance of a class that just exists in the cloud and you just ask for that instance back and then you can call functions on it and then these durable objects these aggregates once you issue a command say set name we'll get to an example set name for example it will then create an event name set push that into the event store then the event store then lets listeners um which in these case are projections and processes as listeners know that an event has been added and you need to handle it you need to update your internal state and then finally the user wants to be able to get that updated state so then they can just send another request give me that updated state and then it goes to a projection and projection comes back with the object so it's it's pretty simple but it's really powerful and it allows you to have a lot more flexibility as your app develops to change the business requirements so what happens if the business requirements change and our projection we want to update our projection well we can simply add a new handler for an event um for if if it's a new type of event but one's if we want to process old events in a new way and produce a new kind of state well we can do that by rebuilding or rehydrating our projection and if we flick back to the application and we go to projections we see this action up here called rebuild and what that's going to do is it's going to wipe the state out of this durable object this uses durable object it's then going to ask the event store for in batches give me batteries of events and then it's just going to rebuild its state back up so if we click it don't actually see anything happen but what it has done there is it's rebuilt the whole state of the projection um going through one by one one by one and reconstructed our state so that's about it for now um if you'd like to check out this project um a little bit more in a little bit more detail then it's all going to be up on github and by the time i publish this video um so you can go through and i've read uh i've written a bit more about sort of the caveats and the things to do and the edge cases and other issues with this but um and you can also obviously check out the code and you can also uh check out the example that we've been going through here um but yeah so it'll it'll all be there um on the github and i'll leave a link to this down also in the description of the video okay well thanks for watching and i'll see you next time

Channel: Michael Cann

Views: 568

Keywords:

Id: W7PmZmVmsDo

Length: 17min 32sec (1052 seconds)

Published: Fri Oct 15 2021