Database Replication Explained | System Design Interview Basics

relational databases have been around for more than 30 years now during this time the requirements for large-scale systems have changed multiple times nevertheless in most modern large-scale systems you still find relational databases and they play a central role one of the reasons why they have withstood the test of time so well are effective replication patterns and today we're going to look at the iconic main replica pattern and ask how does it work and even more interesting how does it scale let's start from the very beginning to better understand the underlying problem imagine you just have set up a database instance on your local machine it already meets all functional requirements you would also need in a large scale system for example you can create the tables you need and you can already read and write from and to them really the full story however we really didn't waste any time on anticipating any kind of failure but you need to know failure is inevitably when you run a large deployment it's not a question if servers fail but when and how often they will fail as a system designer it's your task to anticipate failures and prevent them from taking the whole system down just imagine it's two weeks before christmas you work for a large e-commerce company and someone pushes a buggy software update to all your databases due to your naive design the whole system crashes you can imagine the revenue loss created by the hours it takes you to get the system back online is well a lot more than five times your annual salary it's really like designing a car without crumble zones these areas are engineered to protect passengers by failing first with replication you can create similar safe failure modes that contain the damage and protect the rest of the system put simply replication is nothing else but creating copies of your data and distributing those copies over multiple physical servers sounds easy in theory but in reality as always it's much more complicated and with this additional complexity a lot of problems appear you need really good answers to for example you need to think about in which way you synchronize the different copies this question is closely coupled with another question how users are supposed to interact with a different database servers now which ones are used to read and which ones are used to write and well how do we make sure our users always read the latest data and not read from an outdated copy and another one is what happens when there is a peak in requests let's say it's black friday or well just before christmas don't worry you don't have to come up with answers to those questions all yourself engineers have been thinking about them since the very beginning of relational databases and the best of the answers are distilled into design patterns we still know today the main replica pattern is one of them so let's discuss it and see what kind of answers it gives us historically the main replica pattern was the first widely adopted approach to leverage replication to improve the availability and fault tolerance of relational databases for a long time the term master slave pattern was common in recent years this terminology has been phased out for good in fact there are two types of database notes one is called main the other one is called replica sometimes you also hear the terms main and secondary the main accepts read requests but it's also the only node in the cluster that accepts write requests the other replica nodes only serve reads and replicate the latest changes written to the main typically a load balancer is added to the system to route the read requests to the different replica nodes just like any distributed system databases cannot escape the problem of consistency it's a whole subject of research by itself to keep data across different nodes synchronized in the main replica pattern replication of newly written data happens asynchronously this means that the data on the main and the replicas may be not consistent for a while because replicas haven't caught up with the latest changes yet if you need a fancy term to describe this phenomena in your system design interview you should call it replication lag the consequence of those lags is that your relational database which applies the main replica pattern has the property of eventual consistency it's not strongly consistent and this is a big difference and depending on the use case this can be critical however there is an intriguing advantage of using asynchronous replication it's much faster and scales better than synchronous replication there's also a middle ground solution called semi-synchronous replication if you want to learn more about the different replication methods i have a dedicated video on this topic and you'll find a link to it in the top right corner or in the video description below in a typical setup you would distribute the notes over different physical servers in close geographical proximity most likely within the same data center that way you reduce the impact of hardware failure but still you can keep the latency relatively low otherwise your design might come with a replication lag and grain to it even though there isn't even load on the system yet now after we work through the basics i want to show you a couple of failure scenarios so you better understand how this replication pattern works in critical situations let's start with the case that a replica node crashes what happens then i give you a moment to think about it well not a lot in fact the traffic that would have been served by the node will be rerouted to other replicas that increases the load of them the maximum read capacity gets reduced but as long as the system isn't under full load already the user isn't affected by this incident at all the straightforward response is to simply spin up a new replica note to replace the faulty one i say oh simply spin up a new note but you would actually need a backup a fairly recent one even to do that because otherwise your new note doesn't know anything about the latest state of the database and would need to replay a lot of events to catch up with the main that means it would come in with a big replication lag it would take ages till it actually could replace the faulty node which just went down if you don't know a lot about backups don't worry it's on my backlog and i'm gonna do a video on that topic as well very soon now we get to the most critical situation that can happen your one and only main note goes down due to a hardware failure what can you do about it again i give you a moment to think thanks to replication the system remains able to serve read requests which is great as most modern applications are very read intensive they have a high proportion of reads compared to writes but the system still lost all abilities to serve write requests and that's really really bad because it affects the user experience so the clock is ticking and the system really urgently needs to get back its capabilities through serve write requests if you lose your main node in the main replica pattern you can promote one of your replicas to act as the new main and start to accept write requests the big question is how to decide which replica should be the new main due to the asynchronous nature of replication in the main replica pattern all replica nodes are in a different stage replaying the latest changes once the main note is down all those replays stop so the most reasonable choice for promotion is the replica that is most up to date once promoted all other replicas need to start replaying the changes this newly promoted main is ahead of them i said the most up-to-date replica is promoted as in most cases no replica is actually identical with the main and that means all data which isn't replicated in the moment the main goes down is lost last but not least to also get back its full read capacity the system needs to spin up a new replica that replaces the just promoted one that's how a replicated database reacts to failure scenarios it's very straightforward to see how it significantly increases the system's availability but we are not done yet with exploring the main replica pattern availability is just one of the desired properties a system needs to serve large scale applications another crucial one is scalability let's investigate which methods you can leverage to tweak your main replica architecture in order to keep up with an ever growing user base to give you an idea netflix just grew their user base to 200 million users this year and this is still a small user base compared to other large scale systems like youtube what happens if you don't scale your system alongside a growing user base is very simple to explain first your replicas get overwhelmed with more and more read requests the one point write requests also increase and that means that replicas need to replay more and more rights and they also need to serve more and more read requests that leads to an increasing replication lag because they simply can't catch up a massive replication lag again leads to poor user experience as users are presented with outdated data and it takes increasingly long to actually show the latest changes eventually nodes run out of memory and crash which increases the pressure on the remaining nodes and it speeds up the collapse of your whole system to prevent a scenario the system needs to be tuned to be always the head of the increasing read and write requests and you still need a good amount of backup resources to handle peak loads how do you scale read traffic i'll give you a hint this one is the easy one and it's also the reason why the main replica pattern and relational databases have withstand the test of time so well you simply add more replica notes each replica node runs on its own physical server the fancy term to describe this approach is horizontal scaling it's great that we can achieve this because it's a cheap way to extend the system's read capabilities of course there's also constraints to this approach as these new replicas need to be synchronized the network traffic would increase significantly but that's a story for another video for now i want to focus on the second scalability challenge here making the system handle more write requests and here comes the tricky part we only have a single main node and while we theoretically can increase the amount of replicas to infinity the main node remains to be just one so over time the main node really becomes the bottleneck of the system what can be done is to increase the hardware capabilities of the physical server the main node runs on again here's a fancy term it's called vertical scaling however this approach quickly gets to its economical limits because adding more and more memory and cpu power to a single machine eventually becomes very expensive if you want to learn more about the different approaches to scaling you'll find a link in the top right corner to another video of mine and again in the video description below so there must be another way to scale write requests but it's impossible in the main replica architecture to add a second independent main node how would you maintain consistency over the system if you would have two nodes that accept rights but what if we would create multiple fully independent data sets each of them with their own main and replicas with this architecture it's possible to scale write requests all such requests would need to be directed to the partition that is responsible for the unique subset of data that is intended to be updated that way all right traffic can be split up directed to different independent main nodes our original bottleneck is gone let's look at an example imagine you have a single large database handling all data of a social media platform like instagram you have many write requests as people keep commenting and liking posts but also post themselves are write requests by dedicating one database to only store comments and likes while the other one is handling the content of posts you can reduce the load on each of the main notes to describe this approach people use all kinds of fancy terms the most common is sharding but there's also other names like splintering or horizontal partitioning each partition is also referred to as a chart for today there's really nothing else to do but wrapping up all the new concepts and the new fancy terms you need to speak the same language as your interviewer in your system design interview first database replication is a technique to increase the availability of a database by replicating and distributing the data over multiple nodes that way the risk of data loss and downtime due to hardware failure or human error gets reduced a common replication pattern is the main replica approach it creates two types of nodes one single main node that handles write requests and replica nodes that only handle read requests the replicas receive the latest data changes made to the main in an asynchronous fashion and then they try to replay all changes as fast as they can but there will always be an individual replication lag for each replica that's why the system only can provide eventual consistency replication lags become worse with increasing workload and can affect the user experience at one point in case individual nodes fail there is really two options if a replica fails simply replace it with a new instance if the main node fails the most up-to-date replica will be promoted to be the new main due to asynchronous replication there's always the risk of data loss replication is a great way to horizontally scale the read capabilities of relational databases by adding more and more replica nodes it comes at the cost of increased network traffic though scaling write capabilities is possible to a certain extent by increasing the memory and cpu power of the physical server hosting the main node that approach is called vertical scaling once the economic limit is hit database sharding is the next best option database sharding follows the idea of partitioning data into multiple independent databases called shards thereby the load on each individual main node is reduced i hope you enjoyed this video as much as i enjoyed creating it if you want really deep dive into the preparation for system design interviews i might can help with that on this channel i fully focus on teaching you all the theory you need to nail the system design interview at any big tech company you can support the channel by simply hitting the subscribe button but be aware there is a maybe unexpected side effect that you won't miss any of my upcoming videos anymore most recently i also launched a system design interview preparation course i walk you in the course through all the large-scale systems like netflix youtube amazon dropbox instagram facebook twitter whatsapp and many more i want you to pass the interview with confidence and ease probably also important made the course fun to watch because learning something new shouldn't be feeling like a grind not at all so if you're interested just check out the link in the top right corner and again in the video description below thank you so much for watching this video and see you next time [Music] you

Channel: Big Tech Coach

Views: 8,425

Keywords: system design, interview question, amazon interview question, google interview question, system design interview, system design tutorial, amazon, google, coding interview, distributed systems, microsoft interview questions, what is scalability, scalability explained, system design interview questions, scalability basics, learn system design, engineering blog, replication lag, replication vs backup, cross cluster replication, database replication, database, relational database

Id: WG6k74VSOOU

Length: 17min 52sec (1072 seconds)

Published: Wed Apr 20 2022