LARACON EU 2024 // MATEUS GUIMARÃES :: UNVEILING THE MODULAR MONOLITH

so a big big big Applause for M gim so um today we're going to talk a little bit about modular monoliths um modular monoliths have have recently come to rise again and people are talking about them quite frequently and they are super super interesting um before that a little bit about me um so I'm a software developer I'm also an educator I do powerlifting I do photog graphy content creation spend time with my wife um this is my YouTube channel you might find some lcons there on lar of journals software architecture that kind of stuff I also do powerlifting it's going sad so there are some videos on myself as well this is me and my wife at Disney my wife is here with my mom actually I don't know where they are exactly but they're at the audience so that's pretty cool um thank you but uh we're not here to talk about me we're we here talk about software architecture and modular monoliths so before we talk about modular monoliths we got to talk about what a monolith is and a monolith is probably what most of us use at work it's our regular applications our desktop applications it's one big application one artifact everyone knows then we use regular method calls to communicate between components in the application we have database transactions and we're going to talk about them real really soon and why they're so important and we don't value them and usually we only have one artifact so they're easy to deploy they're easy to scale as a single unit but they also have their problems the biggest problem is probably tight coupling between components um you have no well-defined boundaries you don't know where component starts and where it adds um if you have a big project a medium to large project it gets really hard to understand you join a new project and you have this huge repo with 700 files there's a lot of cognitive load um this is what a monolith usually looks like you can see that we have different components talking to one another and they're gr by color just so you can identify whether they belong to the same module so here's the thing about a monolith even if you have all of these components talking to each other stumbl uh stumbling on each other's toes you can still say okay this is part of the product module this part of the shipping module they're all spread throughout the application but you still can Define them you can say Okay um this relates to shipping this relates to the warehouse system and you can tag them if you will now this is what you usually end up with in medium to large applications we call this a big ball of mode it is a complicated monolithic application you have tight coupling between components it is hard to understand you touch feature a you break feature B on the other hand a another common architectural pattern is microservices and microservices have some benefits they're easier to scale individually they have well defined boundaries and they're resilient so if microservices a breaks M all the other ones are still going to work right they do have lots of trade-offs so you have multiple repos you have a much more complex infrastructure what was previously method calls turns into Network calls and you guys know that you cannot trust the network it is more complex by default you have to deal with disty so we have database transactions in mic applications you don't have them in microservices you need to do compensatory events you need to do sagas for example and they're much harder to debug just because you have multiple applications so the infrastructure itself is more complex um this is what a microservices driven application kind of looks like so you have those boundaries you have multiple databases usually um and you communicate through the network now the interesting thing is you can see that with microservices you are obligated to think about boundaries because it cannot work otherwise um the shipping service does not know of the data on the product service or the warehouse service you you're you're physically separated from those services so you're obligated to think about those boundaries and how your services communicate to one another a m ground between those is the modular mon so although on in microservices we we do have the separation clear separation between boundaries it is not exclusive of a microservices driven approach it is required yes you don't have to do this in a monolithic application but it's not exclusive and you can apply that whoops how do I go back sorry um I thought I was going back you don't have to apply that um to a monolithic application with a modular monolith you kind of have a middle ground so you still have one artifact you still have one repo you still have one deployment unit um you have a infrastructure you can have multiple databases but you can also have one single database you get database transactions back so you no longer need compensatory events well you can if you want to if you have event driven communication we're going to talk about that but you can have everything in a database transaction and that's amazing for consistency um and instead of having those physical boundaries you have what we call logical boundaries within a mon uh a modular monolith we want to have independent modules we want to allow teams to work independently in different modules without affecting one another and we want to have low coupling between these components so this is what a mod monolith looks like you you can have multiple databases but in this example we have one um we separated those boundaries between method calls you can also do events and we're going to talk about that but this is what we call a logical boundary you're not physically separated between Services they can talk to each other if you want to you can do a method call to another service it is going to work it is an option so talking about coupling who here has worked on feature a you thought that was good and you broke feature B raise your hands okay yeah me too so what is the reason for that does anyone know coupling okay cool coupling yeah so when you have tight coupling between components it means that when you change one component and you have an object graph you can affect all your components in that object graph so you touch feature a you deploy and this happens right and you have to roll back and it's a mass we want to tackle that with modularization it's one of the things we want to tackle um so in a modular application we definitely want the modules to be somewhat independent and the thing about coupling is it's not zero or one it's not your either titly coupled or your losely coupled it's a spectrum you can have some pieces of the application that are more coupled to some other pieces and you can have certain pieces that are pretty isolated so let's say that you have a system that deals with payments you probably want the payment piece to be somewhat isolated maybe that's your core business you want to give this a little bit more attention you want to give it um a little bit more interaction right uh we want to empower teams to work in parallel between different features without affecting one another just like you have in microservices that's a huge benefit from microservices that we can have in modular applications and with modularity you decrease the surface area that a component reaches uh it decrease the blast radius of changes so they they're concentrated in a single place of your application so um this is an image from a talk on YouTube there's a source there it's a great talk you should watch it the big difference between monoliths and microservices really is the number of deployment units it's not the databases it is the communication to some extent you can see the network calls on monolithic applications but the big difference is deployment units and with a module monolith we still have one deployment unit but you can think of as a big application with smaller applications inside communicating to one another versus with microservices multiple applications orchestrated to communicate with one another so the first step to modularization is code organization and I want to add a few remarks here this is not a domain driven design talk I don't want the Dutch to go after me they can go after Daniel um this is not a talk about coupling from the framework you can be super coupled to the framework it doesn't have anything to do about that it is about coupling between your components inside your application and this is not a recipe I don't want you to watch the stock and think this is a recipe you can just apply to your project I want to give architectural ideas that you can use when you see a problem that fits those needs so code organization um we have two types of mostly organization you have what Bobby spoke about yesterday grouping by colar or by type where you have your modeles together your mables together your controllers together Etc and you have grouping by context so you group everything in a single context let's say that you have a shipment service everything is inside shipment now this is not not following the framewor conventions you can still follow the framewor conventions inside your modules and you can treat it as a Min of application or you can choose not to and we're going to talk about that as well um so you have grouping by type like I said and you have grouping by context and you can think of the context as some people are going to call it mod modules some people are going to call it domains features context um you can have context within context those are those are subc contexts and um let's talk about grouping by type real quick so imagine that you have a few models right you have payment support tiet site template uh and shment why are they together what do they have in common they're models right and then you go and you have some actions or jobs or commands whatever you want to call them why are they together because they're actions this is just a provocation does it make sense for them to be together I don't know maybe another thing you can do is to group them by context so you have an action with a model and obviously in a real application you'd have many more components and they're grouped by their contact so you have a payment with a refund payment action a site template with a refresh site template action that kind of thing uh this is an example of what a module might look like this doesn't look like like a regular application this is what we call vertical slices so you slice a feature and you get everything together in a single folder another option that you can do is the regular lar structure you can have that inside your modules just like your regular lar applications you don't have to change anything um this is symphony you can see that it looks modularize right you have several components they talk to one another maybe uh sometimes through an event bus this is LL it also looks pretty modularize to me you have different components they talk to one another they deal with coupling they have that in mind how do I do this in a lar application well the first step is you can do this in your app namespace if you want to I prefer to add a separate namespace you can go into your composer that Chason file and add a new namespace it can be modules it can be domains whatever you want the folder can be modules Source that's up to you and you can start from there um the most interesting thing to me isn't the code organization because I think that's pretty subjective if you want to have a regular lay of application grip by type that's completely fine you can still have those things in mind the most important thing to me are boundaries so how do your components communicate to one another where does a component start and where does a component end let's talk about stripe whenever you call an API like stripe you have a class and a response right when we think about contracts we usually think about interfaces right but that's not the only thing a contract is if you think about the stripe endpoint it is a contract you have an input input is expectation and you have an output expectation and stripe offers both so the provider offers you a contract it tells you okay we expect to receive this and we're going to give you this that is a contract and you accept that contract you have to fulfill Stripes requirements otherwise the API is not going to work let's look at a very simple example let's say they you have a user class really simple you have a token user class that does the magic and then issues a token you also have a contract so you expect an input you expect an output and the consumer knows about that so the provider is giving you that contract and then consumer is fulfilling it so the provider offers a contract the consumer has to respect that contract and you can see that throughout the laral framework as well you have components offering contracts and you have other components consuming those contracts so we go to something called cross-boundary communication and I know this sounds really fancy it it isn't it's just communicating between boundaries so how do you communicate between component a to component B let's look at a few examples so if we imagine each component as a square the way I like to think about that is you have a square you have a box you define whether that is a black box or a white box whether the internal implementation is going to be known by other boxes or not the way I like to think about that is you have the center of the box which is your internal API and at the border you have your public API which is just like stripe stripe offers a public API right you don't know what's going on ins strive you make a call to their endpoint you don't know which models they have which jobs they executing you just know that you pass some input and you get some output back and that's how I like to think about modules you have implementations internally at the module and you have those contracts or translators or whatever you want to call them at the border and they're responsible for representing the module's public API again couping the Spectrum so you don't have to follow this to the rule you can you can change things you can see what fits your project but it's good to to have this in mind um and then uh you have a consumer a producer and then you have the consumer consuming something the producer offers so you have your public API at the border and you have your internal hidden API at the core of the module some useful guidelines with modules we wanted to realas some organization and we cover that with code organization that's the simplest part but we also want modules to have some Independence if we do all of that and we don't think about independence we're just giving up a lot of the benefits we can have with modularization as R of th theoretically modules should not know about other modules internals because you're violating the contract um you're violating encapsulation sometimes you don't have that big of problem and it's just easier to do that and you can go with that it's fine it happens um and you want incom messages to a modu to be transformed into something that's meaningful to The module's Domain so you want the producer to also own the objects that are going to go inside it usually or you can have specific objects that are meant to be passed around so we're going to talk about data transfer objects um sounds fancy it isn't it's just a bag of data it's an object that is meant to be passed around that's why it's called a data transfer object and we can use those to pass information between modules the thing with dto is they represent a contract so you might think okay why can I use an array if I just have a simple Contra simple object like this because you don't have types and you have no guarantees that the array is going to match what you expect right and you can fail early unless you do lots of checks and it gets really complicated so this is an object that's meant to be passed around this is an example of an object um I have a pening payment which is going to be passed to a payment service which is going to then process the payment and the payment service knows what to expect that's the beauty of it the consumer knows what they have to send and the producer knows what they're receiving so let's take a look at some code I don't know if you can look that way all the way to the back but you have a relatively simple controller here and I know it's m we're going to refactor it so you have an order you're creating an order you're adding some uh items to to the order lines you're decreasing the stock of the product you're saving the order you're processing the payment etc etc etc see how many different modules we're touching in a single controller so we're touching something that relates to cart items probably product we're touching orders we're touching stock management we are touching payments we are touching I mean we're touching payment providers and payments we touching shipment so we have a lot of CR communication here even if your application isn't modularize you still have that they still exist you can still determine them so we can ref facted this a little bit and we can maybe create an action called you know fetch card an item by product IDs and purchase items and you kind of encapsulated that into another class and it's it's more readable right but if you go into that class you still have the same problem you still have a lot of boundaries being touched so you're still decrementing the stock you're still making a payment creating a payment creating a shipment and as you can see there's a lot of data modeling going here there's not a lot of intent you're saying create a shipment you're not saying start the shipment for this order you're creating a record in the database um a way we can make this a little bit more digestible for our different modules is and this is what we have on the purchase items class by the way so you still have you have the class you abstract that into a class but it still holds all of the data um we can do something like that so we can encapsulate that into some Services owned by the different modules and you might look at me and say okay you're just stupid cuz you you added those services but they just do the same thing and that's the thing as an application grows complexity also grows and our models have to change so where us we were just creating a shipment back then in the future you might have you probably have much more complicated shipment logic you might be shipping to different stores you might be using different providers so it's very likely that it's not just one shipment model and that's it you have more complicated logic and what we're doing here is we are modeling Behavior so we're no longer creating a shipment record on the shipment table we're saying start a shipment for this order uh pay this order and you have well established contracts um one of the problems and whenever I say problem I don't mean literally problem but something that could affect you here is you still have a lot of coupling between components because these components know about each other so if you were to break something in the payment service what would happen to this code it would crash if it were to break something on the shipment service it would crash um and this is what we kind of have so we have those Services being owned by the different modules but purchase items still knows about all of the modules so you're still you're still somewhat coupable to them and if something there it's going to break your code what can we do to remedy this well we can do event driven communication so instead of talking to these modules directly we can talk to a bus so instead of me going to James and saying hey James take this message I can go to kako and say hey kako can you deliver this message to James to Tim to Joe and kako is going to deliver it to all of them and I don't know about it I mean I just give it to kako kako decides who he wants to give it to so it would look something like this instead of talking directly to these modules we could dispatch a message an event and this message says okay an order was placed and then everyone who's interested in doing something once an order is placed is going to listen to that event and now the consumer doesn't exist you only have a producer you have someone dispatching an event and then you have a bunch of other modules listening to that event and doing something with that and you no longer have direct coupling between these components if the shipment service for some reason isn't working that's fine the order is still going to be placed if the payment service is broken sure the order won't change State into PID but it's still going to be placed and you can fix it and apply it to all of the pending orders let's say that your boss says okay we want to give a reward to everyone that purchases over $500 instead of you adding some code to a class synchronously you can just add a new listener the events already there you don't have to do a lot of things you just add a new listener and you can add functionality at HW so you also have the benefit now if you look at this you can have different teams or different people working in different modules and since they know about their API they know about the events that are being placed they can work independently they can run their own tasks they don't have to touch the purchase items class if they they want to for example task that an order is going to be paid or shipped or that a rewards going to be given because they have the event and the event is just a dto it's just a bag of data um let's look at an example so what if we have a purchase items class and all it does is add some adline add some lines to the to the order and then dispatch an event saying an order was place that's all it doesn't doal with doesn't deal with payments it doesn't deal with shipment um Amazon does this so if you purchase something at Amazon you might notice that your charge your card isn't charged right away your order goes through and then at some point your card is processed in the background and anybody fail and you got an email saying hey your payment fail um so you dispatch an event and then you can uh in the pay orderer for example you can listen to that event so you you listen to order place event and then you can do something with it okay so and order was placed I probably want to try and charge the user's card and if it succeeds you can dispatch a payment succeeded event if it fails you can dispatch a payment failed event so with event driven communication you kind of go back to what we said about microservices you need compensatory events because it's no longer synchronous you have eventual consistency um and then you can have a fulfill order class that listens to the payment succeeded event to fulfill the order you might have a start treatment for order class that listens again to the payment succeeded event to start shment and then you might have let's say a decreased product stock the listens so payment exceeded to decrease the stock of a product in real life you probably want to decrease the stock as soon as they make the purchase um so that you don't have multiple people buying something out of stock what are the trade-offs well you're Crea another layer of interaction so you can no longer look at a block of code and know everything it does because it's not there it's somewhere else um you have eventual consistency if you have here listeners which I would recommend um that is at the point an action happens the database might not be the changes might not have been applied yet the the listeners might not have been executed yet so you have to take that into account so for example if you have if you're in e-commerce you have to expect that by the time the user completes their order they might not have paid for it yet they have not necessarily paid for it um you need compensatory events so if the payment fails you have to dispatch a payment fail event if it's suce you have to dispatch a payment succeed at event and again because of the layer of interaction it's harder to reason about everything in a block of code do um we're running out of time but real quick one important piece of modularization is data ownership um let's look at example let's say that you have your application and you have one separate module this learn module and you have a course and lessons and um Can a user access a given course that's a simple question right you want to make sure they can access a course and this is how it is often modeled you have does the user have an order that has an order item that is a product that the chord is attached to so you're touching a lot of boundaries to determine something and you're licking your modules concerns so it often looks like this the user has an order the order has an order item that has a product that has a course right um what would be a simpler way to do this well you could publish a product purchase event and then you can have the learning module listen to that and say okay someone purchase a product I'm going to unlock the courses related to the product and then you can have a relationship between courses and products probably belongs to menu so you still have relationships between modules right but you still have Independence if the app doesn't exist nothing changes in the learning module and vice versa um you can you can go a little bit further so you still have a user to class relationship right so if the user model is gone your learning module breaks your user module model is probably not going to be gone but sometimes you have to this in other modules um so what you can do is you can introduce context specific entities or models so what if you had a student model inside the learning module that has courses instead of a user having courses um that gives you a much smaller API who who here has ended up with a god user object that user object that has 30,000 relationships 30,000 methods you know 30,000 accessories and this is not just for users I'm just using users as an example you can get a much cleaner API a much more concise API and a much smaller surface area now it does come with trade-offs now you have to listen to an event you have to have a student record for example uh to unlock the course for that student but you no longer depend on data from a different module all of the modules data is contained inside it so it can work in isolation imagine that you're creating a package inside your application you plan to open source it later what happens if you create a package the package in a separate namespace but you still have calls through L of application direct calls you won't be able to open source it because it won't work in isolation um so can a student access this course simply becomes does a student have the course unlocked and that's it um this is an example of a real life app application uh that has no sub contacts and the trick with with this is you're kind of obligated to keep your contact small um and to keep your API very concise so you end up organically creating those new models and new sub contacts and whatnot um modularization encourages you to actively think about your systems and their relationship so that's um an unrelated thing that is really good with modularization that is you have to take a step back and think about your components and boundaries and that gives you a little bit more time to think about how your system is going to interact um how your systems are going to interact with one another sometimes when you're dealing with with monoliths um we just keep creating models and controllers and listeners you know real it's super quick it's super easy so sometimes we just kind of do it without thinking too much and obviously this is a side effect because it's lower um and because you have to think about those things but it gives you those benefits all right so let's recap the first thing we do is to split our application in different modules even if it do not exist in your directories you can think about them right you can still think about them the second thing we do is to enable a public API for each one of them then we start to communicate between the internals of a module to the public API of one another so imp uh implementation to contract implementation to contract we never want to touch we never want internal of a module to talk to the internals of a different module um and this is pretty much what it looks like another option is to instead of having that communication we can dispatch an event right we dispatch an event and then the other modules can listen to it the other components of your application can listen to it and there you have um much less coupling because you don't know about the other components you know about the event bus you know about the event itself your dto but shouldn't know about anything else um so some suggestions coupling inside the module is much less concerning than cou between modules so if you have your module and things are tightly coupled that's probably fine it's still within the module you're still going to Deo that you know together frequently um if you have couping between modules then it becomes a little bit more problematic because you can you can run into those situations we talked about earlier um within your modules always try to expose a very small as small as possible of a public API and modules allow you to to use different solutions different problem so you might have some components that are more complex intrinsically so if you're in Ecommerce payment is defly complex shipment must be complex and you can use more complex solutions for those problems and keep other modules you know uh uh simpler to apply modules to larvo the first thing we want to do is to have probably have a modules namespace or whatever you want to call it to group them um we want to leverage service providers to load routes configuration files and you know that kind of goes against what Taylor said yesterday you know we don't want to use service providers or we don't have to but uh if you're doing modules you probably want to have everything related to the module you know in its box you don't want it licking outside the box um we should think about whether module should hold all of its data sometimes it doesn't have to sometimes it have simple modules that's fine and we usually oh okay T all right thank you all right cool uh is it working okay cool and sometimes we also went to Leverage The Event bu to pass messages to different modules um some packages real quick we have a l ofal modules which is the most popular one David is actually at the audience right now I think uh we have internachi modular uh which I think Daniel worked on with Chris morale um and one of the trade-offs that going with modules is you cannot you can no longer useing uh use scaffolding commands by default or autodiscovery and this packages will give you that back so you can use autodiscovery within modules using those packages and you can also use da track and pass architectural tests to Define your boundaries and have for example your test fail if you're crossing a boundary you shouldn't uh so I suggest you look at those some good articles and I'm going to have the the slides published later but Shopify has a really good article about modularizing their monolith um there's a good book called strategic monoliths and microservices there's a Ros com talk really good as well and a little bit more hardcore than mine um that you check out that you should check out and also my lacast course modular laral um which is available on L casts um so you can also check that out to have a little bit of a deeper Vision on this and also I did a podcast with Chris moral who was mentioned yesterday by Daniel as well um about modularization and we have very different opinions so you can also check that out and that's pretty much all so thank [Applause] you I should have invited you to hold the mic for Taylor oh yeah for sure you got a bigger arm than me you can know it better so I got a couple of questions okay so first Jose Manuel Cardona okay we got here at least a Hispanic person how do you scale the monolith um you can scale it as a regular monolith just you know either vertical or horizontal scaling um you can also apply reverse proxies to direct traffic to specific modules based on the endpoints which is a little bit more complex it's definitely not as easy to scare individually as a microservice architecture um you can still do it but you're still loading the entire application in memory uh regardless of whether direct traffic or not so um in general just like a regular application thank you Martin is asking one thing I'm going to answer okay myself he's asking can mat deadlift can Echo I'm saying no okay that will settle okay and the anonymous user is asking did your wife enjoy your talk uh I sure hope so I mean I don't know if she understood much but I hope she enjoyed it do you enjoy it see she's shy she's shy a big Applause for M than

Channel: Laracon EU

Views: 7,587

Keywords: laracon, laravel, php, laraconeu, monolith

Id: WqL6wgH1fKs

Length: 34min 25sec (2065 seconds)

Published: Fri Feb 23 2024