Exploring Event Sourcing (part 1)

hello everybody and welcome to the stream we are going to talk for an hour about event sourcing this is a subject that's uh my colleague brent and i uh have been yeah researching and uh we have been creating uh a course and um yeah the the meaning or the the purpose of this stream is to give you like a very short introduction to event sourcing and yeah to answer any questions uh that you might have first let me bring on my colleague uh brent hey brent hey hi everyone hey so yeah brett this this stream was basically yeah your idea i think uh to do uh i think it's very great to have like an informal uh context where we can yeah talk about these things where people can can ask questions yeah yeah i was actually asked some questions on twitter and i was like that's not the place to answer uh those questions in depth so let's do it this way yeah yeah so if anybody in the in the chat has any question uh let's let us know now to get the ball rolling i'm going to take like five or ten minutes to give like a very quick introduction to to event sourcing and yeah prince you can help out as well if i say anything where you want to comment on uh just do so and yeah we also have another demo um that we might do if the questions lead us there we'll see how it goes um let's do a little bit of window management here so i can share my screen share my screen the screen and that is my screen i don't need where's my pointer i don't need to have this i just want to have safari open let me drag it on and there we have it um if there is any problem with audio video let us know i'm going to assume that everything is in order unless sold otherwise so first let me show you the thing that yeah mostly brent has been working on with feedback from our team so we're creating a course on event sourcing in larval it is a it consists of an a book of a fully functional application in which we have modeled a shopping cart that's like an example that everybody can use i think we're also making a new version of our popular larval event sourcing package we are going to make it php8 only we are going to embrace all the features that it has and we're also going to include two hours of uh of videos so it's also a premium video course that builds upon what is explained in the book and what is the um yeah the target audience of this book this book is written for people that already know laravel or at least php and want to know how to event source a larger application in there so i think that even if you do not have like an application where you think you'll use event sourcing this might um still be a good resource to have to have a little bit of a perspective i'm pretty sure that every developer will come across a problem where yeah ideas of event sourcing can be used now um let's see where are we going to start with this um let's hit this link first so this is the documentation of our laravel event sourcing package this package makes it very easy to start with event sourcing in the larval application it's actually quite sizable package it can do a lot for you and in the documentation we have like a mini course on event sourcing as well so if you're not familiar with defense sourcing reads read this i'm going to do a very quick overview of two important concepts of event sourcing namely projectors and aggregates and yeah we'll use this package in in the background so let me go over to the projectors uh thing first uh i already did a couple of conference talks using this application but i thought it would be good to get everybody in the chat a little bit on speed about what event sourcing is so event sourcing in short in a traditional application you probably are going to store all your results in the database and if you want to store a new result you're going to overwrite the the previous results and the previous result isn't accessible by your application anymore you can't make decisions based on the previous result whereas with an event sourced application you're not going to only store the results but you're going to store all the events that lead up to the result and this way your application still has access to all of the history and you can make decisions based on the history that's right right brent yeah my mic was muted yes uh absolutely right yeah all right good so let's see a little bit of an example of this and i'm going to log in my lara bank application and this is basically a small larval application in which i have already installed the uh event sourcing packaging so in this bank you can have like accounts oh and this one asks for autocompletion that isn't needed my private bank account we're going to add an account here and i'm going to add some money so yeah i'm getting very rich but then yeah i saw this uh apple monitor and i want to uh yeah spend it all on a monitor plus uh plus a hanging mechanism so i'm going to withdraw money then i work a little bit again i deposit some money and then i'm back here now let's take a look at how this looks like in the database so we're going to go to valle we're going to the projector laura bank database and here you can see that we have a table called accounts and here you can see that we have a private bank account and we have a balance here and this table is projected out of other events so let me show you the events table and here you can see all the things that were that have led up to the balance of that account so first we created the account i added some money and i removed some money again so in an even source application this is your source of truth and you'll use a projector class to go over all these uh events and to create a projection in this case it's called accounts let's maybe go to php storm and show you a little bit around so this is lara bank projectors i think yeah phpstorm let's open this i should have probably opened it before the stream because php storm always takes a little while to load even on a fast computer but yeah there we are so i'm not going to show you all the details of this but i'm going to show you the projector itself this is what the um yeah what the package this is the power that the package can can give you so in your application codes instead of creating account yourself you are going to raise an event and this is a projector that listens for those events on account created on money added on money subtracted you can see that it listens to those events and it will create an account and it will just adjust the balance of that account now you can also um replay all your projectors so your application really what it needs what is the base of all of all truth in your knowledge are these events and you can basically throw this away if you want to so let me just throw this away and if you want to build up the state again then you can replay all the events to create the state of your projectors again so let's let's try that so we are going to replay all events yes are you sure yes i've replayed five events and if i refresh now then you can see that i've rebuilt my state so the interesting thing that that this unlocks is that you can after all events have been recorded and you want to have like a different view on your table on your events or you want to have like a different report you can create another projector and replay all past events to it imagine that i want to have like a screen with not only the balance of the account but how many transactions uh were there made for this account and i think in this case it are three transactions or four transactions and yeah management says we want to have that report so what do you do in that case well you create another projector and i've already created here i'm going to uncomment this that just uses the same events to build up another state and this is uh here's a model where we just store all the counts of a certain of a certain account and i can build up that state again by replaying all events so here is that transaction counts it's empty now and if i rebuild this i'm going to rebuild all projectors it's easy and you can see that i now have some state here and i've um yeah got the count here stored and i can use this state to create a view i think i also have it i think transaction it is to just display the count here and a projection like this um is uh is very handy because right now um reading the status is very easy i think in a transitional you know in a traditional application where imagine that you have hundreds of accounts and thousands of transactions maybe you can write an sql query that gives you uh all all of the right state but your database needs to do a lot of work and if you have like millions of accounts and millions of transactions that won't work anymore but if you have like a projector where you build up the state that a view needs then yeah creating a view like this becomes yeah very easy and the view is very lightweight because this table is just built for this view so yeah reading the data is is really really cheap in this case so that's in a nutshell i think what projectors are um do you have some thoughts on this uh print did i miss or do you want to see a certain example or well i think the most powerful aspects of projectors is that you can have several imp um interpretations of the same source of truth the stream of events and that's like you say uh what's making them uh what makes them so powerful also the fact that you can replay them uh to correct mistakes or to build new features two years from now uh all based on on that same data set it's yeah it's a very powerful technique yeah i think so too i think that um even if you don't use the other concepts of event sourcing that projectors already bring a lot of power to the to the table do we actually have some um some questions on projectors itself because i can go on and explain aggregates as well but i've already been talking so much so maybe it can be yeah that's actually interesting because the very first question we got on twitter was was sebastian asking when does it make sense to use event sourcing and when is it just just too a too complicated solution for the problem and it's true that there is some overhead when you use event sourcing um but like like you've shown right now with just events and projectors we can already do quite a lot of interesting things and i think one of the the common misconceptions about event sourcing is that it always should be used in very large applications and it's true that the event sourcing is often used in large applications because it's a very good fit for that kind of applications and distributed systems but also in in very small projects where there's the aspect of time that you're dealing with a linear flow of things happening in your application and you want to generate reports based on them or you know that your your read models might change over time um event sourcing can also make a lot of sense to use over there and on top of that uh the the mindset of event sourcing is actually a very natural way of of thinking about a problem when it comes to time if we if we have normal crud applications we're often dealing with adding columns like created ads and updated and visited ad and stuff like that we're always dealing with with dates while events themselves encapsulate the the date and and a thing happening at a given point in time and that's that's a very powerful concept so the the mindset of event sourcing shouldn't be discarded to be just used in in very large applications you can do pretty cool stuff uh with storing events and replaying them already in small applications with little to no overhead actually yeah indeed it is always ideas and the mindset are that are mostly more important than like the actual yeah technique i think um you also mentioned that in the book right isn't that like a quote by martin fowler i think that yeah that said that yeah that the mindset is pretty much the most important thing with events or thinking that you really think in events yeah so in essence event sourcing isn't more than than storing events and using those events as a way to to build your application stay with and it's actually very simple principle but you can build on top of that and and create very complex and large systems and add patterns but it's not actually needed to if we're only speaking about event sourcing itself yeah now i want to ask a question to our audience and i'll keep an eye on the chat is there anybody in the audience that is already using event sourcing and what kind of problems are you solving uh with event sourcing so if you're using that letters uh let us know yeah and while we're waiting uh frick i maybe i want to to give another example of of uh how you can use event sourcing in in real life applications uh yeah like for us we're we're building a rather large client project um but it was very clear to us that while event sourcing would solve some of our problems uh it would be overkill to to event source the whole application and frequent i we did quite a lot of research on the topic yeah and it so happens that josh i think asked on twitter a similar question about how we can add event sourcing in existing applications i think here i'm going to stop you a little bit then chris s is also um asking a similar similar question here what do you think about the hybrid style so yeah yeah how great i can see it on the screen now yeah it's actually it's actually very important it's it's one of the major pitfalls that uh greg young who is the inventor of event sourcing says uh if you're if you're going to start with event sourcing and build your whole application on it you're doing it the wrong way event sourcing is a technique to solve specific problems and uh you really shouldn't think of your whole project as an event source projects because it's going to get you in trouble because there is some overhead associated with it and there there are cases where the overhead is is uh actually adds value i mean event sourcing as a whole adds value but in other cases it's just it costs way too much and so what we did is we we were a little bit inspired by the the microservices community where there are strict boundaries between an event source part of your system and the non-event source part of your system because imagine uh let's go back to the do the laravel uh the lower bank example imagine you created an account and we to account uh there there's an a user associated with it right a user id it's stored in the event and let's say that the user management part of our application isn't even sourced now what if the user is removed for example maybe possible gdpr related stuff now imagine we throw away all of our uh accounts account projections like you've shown and we want to replay our events now we're in trouble because the user doesn't exist but there is no event uh showing telling us that the user actually was deleted right so so there's a kind of mis-synchronization between the event source system and the non-event source system so what you've got to do there's there's no solution to that besides creating strict boundaries and whenever you're in your event sourced application part of your application you've got to keep to the rule that events are the only source of truth so if you want data from the outside other parts of your system you will have to copy the data over in events yeah indeed and we explained that further in the book right um yeah definitely there's a whole chapter on the topic yeah because it took us quite a while to to get right i think yeah we did like uh yeah four or five uh sessions yeah um yeah code pairing sessions where after which we got like a good way of of dealing with combining a non-sourced part non-event source part of the application with the event source part of the application and and the reason for that is because most of the talks and and documentation about event sourcing assume that you're working in distributed systems because that's where where event sourcing really shines where you get the most value out of it but that doesn't mean that there's nothing to gain from event sourcing in an event driven mindset in smaller applications but there was some some thinking needed in order to get it right in order to not fall in in the trap of event sourcing the whole application can you imagine uh if we would it's probably i would say 25 of the application is event source maybe 30 percent and you wouldn't want the other 70 to be event sourced as well yeah that would be too much overhead yeah this makes me think of like the the quote by frank the young aware he says like uh event source event sourcing makes easy things hard and hard things easy so if in a part of your application that that is easy where you're just mainly doing kerdi things don't use event sourcing because yeah it will be a lot more work but if you have like a part of your application where you need to make decisions based on the past uh yeah that that are the things where event sourcing can shine now this is a good segue in another uh question that i that i see here uh by uh pushback are there what are like the good use cases for event sourcing that's that's an interesting question i think so uh there's of course the the aspect of time and when you start thinking about it there there's actually quite a lot a lot of business processes that are time time um sensitive right take the example of creating an invoice or a shopping cart or a bank but we're not building a real it's it's a it's a traditional example that most of us don't build banks no yeah but but if you think about maybe maybe blogs stuff like that i mean there is always the aspect of timing in in the sense that you create a draft and and only a few hours or days later you publish the post but the the data related to time progressing probably isn't all that interesting in case of of a normal blog but when you start thinking about it there there are quite a lot of processes uh where where time is really at at the essence of the question where time is uh of of great value to the business and i would say those are the cases where event sourcing can probably help you out quite a lot yeah imagine for instance like that you have like an onboarding process for your sas application and in your onboarding process you have like multiple steps let's say five steps or something like that and people have to create to fill in like five forms to progress each step now i think in a traditional application yeah you just want to move forward like you want to get people through the end and that's what you store you store what they submitted and if it's all correct you store it but with event sourcing what are you going to do you are going to store like the events like hey um people got past step one i'm now into step two but you're also going to record hey they validate something and it wasn't valid then you can start event like form wasn't valid you can even store events like user got got went to step back and then you can do like interesting things like you want to have like a report how many percent percentage of user users did a step back how many users did two steps back did it hap when did that happen are those users from like a specific country if you fill in the country field like that are they are those the ones stepping back and you can you can analyze those those events after the fact you don't need to think of like which report or which view do i need when you're building like the uh the the little onboarding wizard you you just store the events and you can think afterwards like what uh what knowledge do i need from this and this is already like a small example where event sourcing can shine because you can just event source your little onboarding wizard and not even source anything else about uh about your app yeah it's actually yeah go ahead go ahead good uh it's actually interesting i was thinking about a project i i started working on maybe three years ago and it's a project that handles hotel bookings basically and it's it it isn't event source because we didn't really see a need for it at the time and the project has been in production for um a year and a half maybe two years now and it's very interesting because almost every week the client comes to me comes to me and says hey i want this kind of report and i want to know uh this guy this kind of of time related data like uh when people uh how how was the the um checking behavior of uh habit yeah clients in their hotel um stuff like that and and i'm almost almost every week i think oh event sourcing would have made this so much easier to solve and we don't have it in that project but but i mean we can still solve all most of the questions of the client but it takes it takes much more yeah adding strange columns to keep track of time and um having situations where uh where i have to say yeah okay we can we can build this feature but you will only be able to use it as soon as it deployed and and we won't be able to go back in time and generate reports based on what happened in the past because we didn't keep track of uh that happened yeah so that are very interesting use cases by the way frank did you know that you give the example of a form um i'm not really a front-end developer but um i did have heard the talk about redux a long time ago probably react redux and i believe uh if i if i understood it right those are actually event sourcing principles it's a container that where you apply events to and it's it's uh definitely an uh a whole other setup than what you might think of from a backend developer perspective but event sourcing in itself as you can see can also happen in the front and in on on little forms yeah it's it's actually the the very same principles being used there uh i think if you use uh a few eggs uh it's called a mutation you know you mutate your state and basically what you do is you say hey if this event comes in then i want to mutate my state like like that and your state is basically or yeah if you jump to to react uh your reducer that's basically uh yeah your projector that creates a state out of events that uh that come in and yeah because you have everything like in inside of events you can go back in time as well that's how like the stepping back in time works it's it's possible because you have those events now for um examples where yeah event sourcing can shine i think a shopping cart is also a very good uh good example um you can for instance just track like how much time is there between one item uh being added in the next one being added uh how much time is there between like in um shopping carts getting getting abandoned um yeah there's a lot of things you can track in like a cart and yeah but maybe this is like a good moment to demo the cart if you if you uh yes well can we can we answer two more questions first because i'm yeah sure really excited about this those last two one by by robert but like can you show them on the screen yeah about uh yeah the gdpr stuff and that's very interesting because if you're uh if we say that events are the source of truth and we're storing sensitive information in those events how are we going to redact that sensitive information when a gdpr request comes in right and there are of course several ways to handling that the most obvious one would be to rewrite your events and that's a little tricky uh there are there are documented ways of doing so greg young again he has written a whole book on the topic of migrating events event versioning i think it's called and but there are also alternatives for example you could choose to um to not store sensitive information in the events themselves but store them in a file or a non-event source table and have a reference within your events to that table and whenever a httpr request comes in you can simply remove data in in those external sources right and your events can still exist and maybe you'll use the null object pattern or or some kind of defaults to to re have rejected uh information if that's needed for your website to work um so event sourcing really isn't isn't a blocker in with regards to gdpr but you have to think about it but you could make the same argument for for credit applications non-event source applications gdpr is it will probably always be a little bit of a hassle if you you really have to start removing stuff in bulk um whether you use event sourcing or not yeah i think you could use like the same solution for if you want to track like large uh large files or something like that you want to don't want to have like 10 megabytes files in your events what you do then is you just have a separate table with all the media uh and you store like the reference to a media inside of uh inside of your events and you basically do the same for like any uh sensitive information i guess yeah and so the next question was about performance and that's that's a very valid question if event sourcing won't be slow if you have millions of events and in production to the end user event sourcing will actually be much faster um if you're using projections then you're never aggregating data uh at runtime so to say well if you're you're building uh very specific projections freak showed the example of of uh mysql tables but you could also have rendered html views as projections so from the point of view of the user event sourcing really won't slow you down but if you're speaking about uh projection replays um and uh well aggregates maybe we'll come back to that later yes it can take quite a while to rebuild millions of events and there are several strategies in dealing with that actually i i was writing uh the chapter of of the book that deals with these kinds of questions today and let me actually grab it here so that i i'm sure i don't forget anything um yeah so if you're if you're if you want to uh rebuild existing projections real time and your application needs to be uh tp running at all times you will have to either spin up a copy of your application and send real-time events to both of them and and wait until all the rebuilding is done you could you could opt for temporary creating a new kind of projection version two of the same projection and rebuild those from scratch and while it's going your application still works just as intended and maybe a few hours or or days later you can make the switch once everything is converted that's a very very short answer uh like i said i've i'm writing a chapter on the topic right now and if you if you dive deep into the microservices world and distributed systems there's a lot of of great information about horizontal scalability out there of course as well i want to show a little demo to drive our points a little bit again in the event projectors let's do a little bit of screen management here and show my screen here so again on the topic won't event sourcing be slow i want to state that event sourcing writing something can be expensive but reading something is really cheap yeah you don't have to imagine you can see it here if i deposit something here then yeah some some operations are going to take place to uh yeah create uh this uh this sum so let's deposit something and yeah i also have like like those uh transactions here where we have a uh where we have account so and there might be yeah some some logic being performed while you're writing these things but once they're written yeah reading is is insanely fast because it's just stored as a value in your projection so your views will be much much faster than if there would be uh complex queries here um so yeah i think that's something that you should keep in mind that writing something can be expensive but reading something is really cheap um yeah we haven't dived into uh aggregates yet um when we get to that then it unlocks another dimension uh i guess but there the the the rules is again that yeah reading something is insanely insanely cheap and by the way uh i don't think i have the example set up but right now my projectors are just executed in the same in the same requests as i'm yeah doing the deposit here but you can also cue your projectors so if your projectors are doing like expensive work some somewhere then you can just yeah queue it up and yeah our package makes that uh very easy it's just i think implements should queue i think on the projector there and then eventually your projectors will become consistent maybe that transaction counts it's a very simple example but imagine that this would be like a very difficult formula to calculate maybe this would update then after half a minute or something but in most cases that would be would be okay and yeah that's something that yeah it's called eventually consistency what is basically yeah an entire world of its own but uh you shouldn't worry about that that's like the tl dr of of that well maybe it's good to to discuss aggregates for a minute because i i i can see there are some questions related to it uh so so what freak has been doing with the lara bank example is is just sending out events and they all are uh they all come together together in the large table um but actually some events are related to the same concept for example an account and imagine that frick has a user in the larabank application i and i do as well and we both have several accounts um and each account has some kind of id associated with it and so there's only a very uh small set of events in the millions of of events in the data set there are only so much that are related to my account to frick's account and we're only interested in those very specific events and so what happens if we use aggregates is that we're going to to create groups of code concepts that belong together and we're going to assign ids to them and whenever an event is is triggered we know from what kind of context that events was triggered and where it belongs to so we assign an id to those events and that makes because there are ids we can index those columns very well that makes that uh performance really is an ish isn't an issue even maybe uh freak has like a few thousands of of transactions you know maybe hundreds of thousands maybe he's very rich you know that that's that's still within within uh there's still a reasonable amount of of data to process at runtime in php yeah indeed it's just a subset of the data and databases are insanely fast when you want to query on like an id like if even in a table with millions of records give me the tables with this id if that is indexed it'll be insanely fast you shouldn't worry about that at all i think maybe i should um yeah give the demo on what an aggregate is because we've already mentioned it a few times yeah i got a simple aggregate example here again from the the lara bank yeah it's a little bit of a silly example but i think it's because it's easy to understand that i'm always reaching back to this one so this is basically the same application um but built using aggregates and what is an aggregate i always find it very difficult to have like an exact definition of an aggregate for me it's a it's a it's a class where you can use the the past to model business rules uh that's like my yeah fake kind of naive implement uh definition but how would you define it uh uh brilliant aggregate what's like a formal definition there yeah well aggregate is a collection of of concepts that belong together like if you have a cart and a cart item uh those two together can form an aggregate and from the outside you're only interested in interacting with the cart as a whole you're never interested in in using a card item on its own because that doesn't make sense card item is part of of card and that's where why every aggregate every collection of domain logic gets an aggregate route and that's the class where you're talking about we can build those aggregate routes we can build their internal state based on the events that were recorded and it makes it very easy to to make decisions within that aggregate route um based on what happened in the past and i i think that's what you can show with your demo application as well yeah so in this larabank application we're still going to create accounts but we've also implemented one business rule and the business rule is that if you go below the limit of your account and in this case the limit is minus five thousand if you go try to go below that limit three times in a row we are going to send you a mail to propose a loan so after if you have tried three times in a row then a male should be sent now in a traditional application try try to think how you should implement this then you probably have like some kind of counter in your accounts table where if you hit the limit and go below then you increment it and if it's good again then you just reset that and if your little incremental number goes above three then you'll you'll send the mail and reset the counter that's that's how you would do it but with with an aggregate this becomes yeah much a much more natural thing to do now let me show you that this is actually work so let's add some money here so um oh i've named my account thousands but i really want to put a thousand on there so now we have a thousandth of a thousand accounts uh let's go below zero so i'm going to redraw four thousand and like i've said the limit is uh minus five thousand so we didn't hit the limit yet so let's hit it for the first time so i'm going to draw another five or let's let's do another three thousand so i would end up minus six thousand which is below the limit of minus five times five times so let's retry it nope can't do it because you can't go below five thousand so that's the first time where we have hit the limit now let me open up the ray application this is just a little debugging tool and um let's where can i drag it like this and in this window uh each mail that the application sends will be displayed so we hit the limit the first time no mail has been sent um let's actually keep this on top let's try it a second time i'm going to redraw even more money that doesn't work this is a an exception that is handled that is written to the log and let's now hit it the third time in a row and then a mail should be sent for the loan let's withdraw it and here you can see that the application actually sends the mail okay let's see how this is implemented and i think aggregates i don't know how it was for you print but for me it was like one of the most difficult kind of concepts to understand uh before uh when i when i researched uh event sourcing because the name i don't know it it sounds a little bit hostile to me even like aggregate this is like a big difficult thing and in my head it was like whoa what is this but once you get your head around it's actually quite uh quite simple so let's [Music] right he once uh said aggregate roots uh within event source contexts they are state machines that are driven by events and i found that a a rather interesting definition it made it clear for me in any case yeah that's i think that's a correct uh correct statement there are kind of state machines so let's go to the aggregate route that where we implemented this business rule so an aggregate route is a class that where you call methazone and it actually guards it should i say this the aggregate route it will fire off events if all business rules are are being followed or it can it can transmit events when yes certain certain business business rules um yeah should should be applied i guess regards invariants free and cards in variance yeah that's like the the formal way of saying yeah that's that's the way uh we're we're getting scared of it but yeah so let's first go to the controller so here what uh what i do here is in the controller when i update something i'm actually going to retrieve the account aggregate route i'm going to if we're adding something so if i my amount is above zero i'm going to add money otherwise i'm going to subtract money and after that i'm going to persist the aggregate route and persisting that means all events that should occur should be fired now so let's let's see what happens when we when we add money here so whenever we add money we are going to record a new event money added and this event isn't being fired just yet this event will get fired when we persist this and with ad money it's really yeah very easy it it always will pass there are no kind of kind of checks here now you can see here that we also have a balance property on here and this property is used internally in this class to always re recalculate the balance and we need that balance to make decisions on zone [Music] and it's important to understand that this balance isn't from the projector it is internal state of this aggregate root now how does it calculate the balance well whenever we retrieve the aggregate route all the events that we that have occurred will be replayed onto this class now that can be confusing so i'm going to show it to you so you feel it so let's go to the other database so lower bank aggregates here you can see that we have our stored events here so here you can see that yeah i try i created the accounts and i try to subtract some money and the first one occurred because it has been recorded but here you can see that i recorded that the limit was hit and here i even have like an extra event more money needed i'm going to show you where these events came from in a bit but let's first discuss about that replaying or that reconstituting an aggregate route from events so inside of the aggregate route you see that there are two kinds of methods they're like methods that you might expect there create account at money and then you also have these apply methods and you can see that for each kind of event we have also an apply and then the name of the event here so whenever an aggregate route is reconstituted all events will be fit to those apply methods let me let me let you feel that so i'm going to send that event to ray so event amounts and here i'm going to say that this is money added so you see that money added and let's take a look new screen here let's add something here so i'm going to add one two three deposit and you can see here that we got money added from a thousand and then we got the money added that i that i just added because yeah this was the event that we just stored and that's being applied to as well let's add something else four five six close it and you can see that the previous events they are being replayed as well on the on the aggregate so what's being done behind the screens is that the package will select all the events coming for this id because every for every account we basically have like a separate aggregate uid so if i have another account it will be another uid but we get all those events and we feed them to the apply mids of the aggregate and you can see that in the apply method what do we do there we are keeping track of the balance so for apply money added so we are going to um increase the balance for apply subtracted we are going to decrease the balance now let's see how we implemented that business rule that after three times that the account limit was hit that we should propose a loan so we in the controller if if the request is about not about adding but subtracting money we are going to call subtract methods on the aggregate route and if there are not enough in if there are not sufficient funds to subtract the amount then we are going to do this branch if there are sufficient amounts then we are going to record that the money is subtracted and then it succeeds but let's take a look at how this is implemented has sufficient funds to subtract amount you can see that this implementation is really easy we have recalculated the balance so we know how much money is on the account and we basically are going to say hey if the balance minus the amount that you want to subtract is lower than the account or is is higher than the account limits then we have sufficient funds if it is lower then we have insufficient funds so because we have recalculated that balance it becomes really easy to do this so let's see what happens when we don't have insufficient funds so if we have if we don't have sufficient funds then we're not going to record that there was money subtracted but we are going to record another event account limit hit and that is what you saw in the database here account limited now it needs more money let's see what happens happens there and i can already say that that more money needed that is what will trigger that mail that we've sent so we are going to record more account limited needs more money needs more money is implemented like this account limit hit in a row is greater or equal than than 3. now how do we calculate that account limit hit in a row so i've told you that when the account limit is hit we have recorded that event account limited and we also have an apply account limited so here you can see that in the internal state we are going to summarize how many times the account limit was hit in a row you can also see that if there's more money added that we are going to just reset the account with limit in a row because yeah the account limit wasn't hit and basically yeah if we hit it three times in a row then more money is needed and then we are going to fire off that mail and so everything about deciding what should be done based on what has happened in the past is being done in this aggregate class because this aggregate class is being rebuilt by history it can basically say hey we need to track this based on on the history so you can build up any state that you want using all past events and by using that state that we've built up you can make decisions and that is i think in a nutshell what an aggregate is i think it can be confusing at first but if you yeah really yeah try to yeah wrap your head around this a few times and toy with logic a little bit uh this uh resource uh this um project is is open source i'll share the link later then i hope it'll it'll click for you is there something print that maybe you can do to make this more clear or well i think part of the confusion comes from from the definition of aggregates within the ddd community and the fact that you don't need event sourcing to be able to talk about aggregates and we're only using events to rebuild that internal state of the aggregate route to be able to make decisions based on that state that was that's what uh marco meant with uh saying it's a state machine um using events as an implementation detail uh yeah i don't know that that's what made it click for me because um yeah it's it's it can be confusing too yeah i think uh frank is here too and he said um he talks about that aggregates provide boundaries uh that allow you to maintain consistency across various business rules yeah boundaries that's that's probably a good keyword um yeah now frank is here i should also mention that uh in case you don't know frank made an excellent uh library for event sourcing in php as well called event source um it's basically an alternative to our larval event sourcing package or i should say that ours is an alternative to event source because frank was uh was first on the yeah an event outside of laravel so yeah indeed it's a framework agnostic one i think that is like the the biggest difference between between the two um i think that yeah we said we only wanted to keep this going for for an hour yeah uh we're almost hitting the the hour and we still have like a lot of other questions that we noted in our in our docs document right uh yeah yeah okay we can go a little bit uh over an hour don't worry are there any questions that you want to tackle now otherwise we can do like a next next session next week or something um let me quickly scroll through uh the one here um maybe an interesting one yeah someone uh yeah so so um evan asked about how we can query the current state if it's only kept as a sequence of events and uh yeah that's a good question because you've shown aggregates aggregate routes that can that keep their internal state but it's important that that aggregate roots themselves don't expose that state to the outside world because that state is really only meant to determine the process flow uh that goes through the aggregate route but there are there are other ways to uh expose the state that's the result of a bunch of events that happens over time and of course we already covered projections and you can really think of projections as more than tables in a database projections like i said can be uh can be rendered html views in a way that it's just a cached version it's it's a an optimal way of looking at the event stream for a specific to solve a specific problem projections can also be for example an index in algolia or elasticsearch so so those are other ways to to interpret the stream of events and normally that stream would be it would be difficult to aggregate on it on the fly so so that's why projections can offer a real value sorry there's there's one more solution and there are event queries and actually that's one of the features we're going to add in the new version of the package that comes with the course and event queries are you can think of them as in-memory projections they only work if you have a filtered set of events so if it's a relatively small set of events and what event queries are going to do is exactly the same principles as aggregate roots and their internal state but since event queries are only used to read data we can expose that state because we know we know for sure that nothing else will be done internally with that state um and so event queries they they just they are simple classes where you can sorry where you can query a bunch of events you you filter them down maybe on um aggregate id maybe based on time you only want events that happened within a specific time frame maybe uh based on their type stuff like that and you apply them one by one on the event query class and in the end the event query class produces a result and and you can use that result in however whatever ways you want so so that's the third approach so projections aggregate roots for internal state and event queries and i find that those three together are a really great tool set to to use event sourcing in a very performant way yeah i think we can in the next stream maybe we can show a few examples of that because there are a couple of concepts that we didn't explain at all um yeah like yeah process uh yeah different kinds of process managers um yeah the thing that we call reactors um a thing that we call uh aggregate partials i think that uh that we that we landed on there's maybe one thing more that i want to want to show in this stream and that is something related to like having like the event source mindset without going full blown event sourcing and that is uh yeah the package that uh our colleague alex has created i have i've created a little demo application where we use that so i'm going to do a little bit of window management here and i don't know if you've already seen the package print uh i haven't alex showed it internally but i had to take care of my baby son at the moment so i i miss internal presentation so no it's new for me i'll uh i'll show you i'll show you um so i've already created a little demo application here and it's it's it's a laughably easy um uh package i think it's it's it's very small and it is event sourced uh but like in a very lightweight way so i'm going to go to the um repository and get them first uh larvae stats it's called i've just tagged zero uh one one and i say i tagged it but that's uh mostly mostly coincidence uh i should stress that i think uh eighty percent of the work of this package has been done by my colleague alex and so what you basically can do in this package is you can track changes in your database over time and the example that is being used here is if you want to for instance want to know how much more subscriptions that you have in your application and how much subscriptions did you lose over time and this is something that the application makes very very easy so with this package installed you can basically create a stats a stats class where you can yeah call the increase decrease and also set method on and then you can basically just yeah create a query where you get a result like this now i want to let you feel it a little bit so i've created a very small demo so this is basically a larval vanilla larval application where i've installed the packaging and i'm just using a database heater to just see some time so what are we going to do we are going to go back two months in time and we for the next um yeah for the for the next month we are going to see some data for each hour some random number and also some we're also going to record some some decreases and in the loop we are just going to always add an hour and do that step again so let's migrate and see the database and it's actually very fast so let's go to the other database that's starts up i think it is and you can see here that all statistics are in a stats event table and yeah this is basically lightweight event sourcing so we don't store the results but yeah we store the changes here so we can yeah make a summary of a change uh for a specific time period so using the package you can do query like this where you say hey i want to have a result starting from now um for all data for new and two months ago um and we are going to group all the data for a week so let me execute this i'm going to make it one line otherwise i'm going to get into troubles i think when i copy paste this so let's get this and hopefully this works uh in a tinker session let's do it yeah and here you can see let me scroll up that between so it is summarized by uh by week you can see that in this week with this start and this end we had uh so much uh extra subscriptions uh these ones unsubscribed and yeah this is yeah our starting value so it calculated everything before that date as well and this is the difference between that um and this is basically all the package does and to me it's an example of like having the um yeah event source mindset where you just want to use store events and use those events to build up a certain a certain result so you don't have to go all in on this we are going to use this package on our own sas on flair which is an exception manager exception tracking service for larval applications to track not only subscriptions but other kind of metrics as well so we build it for ourselves but i hope it will be useful for for others too um now that my screen is shared um i'm also one time i'm going to finish off with just showing um yeah the promotional side of our event sourcing page again for those that missed it in the beginning so brent and i and mostly brent i should say uh we are creating a course consisting of a book demo application package and a video course all on yeah how you should uh start with event sourcing in a larval application and i think yeah i'm i'm probably biased but this is probably the the police sorry the the book that's that i wanted when i want to yeah learn about event sourcing um i i learned different sourcing in a very fun way as well uh by the way uh i think um yeah my buddy buddy dries vince he actually i think he mentioned event sourcing to me first we went to um a workshop by frank maybe still in the in the audience here yeah no he's not but he's gone okay then maybe he'll he'll he'll hear it later so in frank's workshop yeah i first learned about uh about about aggregates okay i think that were done for the stream unless that you have something more to share brent i just want to say that now that i saw the package that alex built it's it's such a great example of how how simple event sourcing in itself can be there's there isn't even a serialization layer to store events in some kind of of generic table it's it's so great because uh essentially what he built was the event query implementation i was talking about earlier and and used it to solve a very specific problem a problem that's that's very time sensitive because you want insights over time and yeah just it i'm really excited about it like i said i hadn't seen it before and it just makes me very happy to to know that a complex a complex pattern and everything that belongs to it as event sourcing can also be used in in such small and and very contained examples and still be very useful yeah yeah yeah i'm really excited about it and that's that's by the way um how we start the book as well uh we really build from the ground up we start with the event-driven mindset and we work slowly towards um the very complex patterns but it's it's so important to realize that it all starts with the very basics and the basics aren't all that difficult i'm very excited about it yeah me too i it makes me happy that like uh in our company like that spirit of event sourcing it really lives like we have like a big client project alex making this um yeah you're creating uh the course or doing a stream like this and there's also like an unannounced secret thing that we can't talk about yet but uh also harnesses the the power of event sourcing yeah maybe if the secret thing has uh evolved a little bit maybe you can share it in a in next stream or something but yeah we'll talk about it uh uh off screen but i'll i'd definitely be up for like a a next stream because i feel that there's still a lot to talk about here we've got a lot of questions still to cover and uh i'm looking forward to it yeah me too okay i think that uh we're done here yet thanks all uh for watching i hope you enjoyed um i hope this yeah uncovered a little bit at yeah the magic behind event sourcing and it's it's it's not magic it's basically very simple um i do get that some some things uh might still be confusing um but be sure to yeah watch our next stream and subscribe to the mailing list to get notified when our course is uh ready that's all we have for you uh thanks all uh for uh watching and i'll see you in the next stream bye-bye bye this thing wait and broadcast bye bye everyone again

Channel: Freek Van der Herten

Views: 11,045

Keywords:

Id: vLd8BGzFeK8

Length: 71min 30sec (4290 seconds)

Published: Wed Apr 14 2021