CAP Theorem Simplified 2023 | System Design Fundamentals | Distributed Systems | Scaler

hi my name is anchorman and i'm one  of the co-founders of scalar academy   if you're attacking chances are you  would have heard about cap theory   today we would talk about what c  a and p and cap theorem stand for   why it works the way it does and what does  cap theorem mean in the modern ecosystem i   was fortunate to see tech at scale at facebook and  was able to design products that served millions   of queries per second and billions of users i'm  here to share a few of those learnings with you   if you like the content please like the video and  subscribe to the channel all right so let's first   look at why do we even need to learn cap theorem  capturum is one of the most basic principles of   distributed storage so if if at all you think  about building or working on distributed systems   in the future cap theorem is a must-have must  know so that's why you need to learn cap theorem   now before i get into explaining cap theorem and i  would try to do that with using as simple example   as possible i want to get give credit to kaushik  whose blog i have borrowed a few ideas from uh   so thank you kojic for coming up with a very very  innovative example so let's get to it right so to   elaborate uh cap theorem i'll take example of of  a real world example of let's say you imagine i am   an entrepreneur i'm looking for ideas and i feel  you know what like people need reminders let's   build a service that just stores reminders right  which is i get this very premium number imagine   i named my company as abcd and i get this very  premium number which is one two three four one two   three four people could just call me on one two  three four one two three four and they can tell me   what they want to remember and then when they call  me back i tell them what they wanted to remember   and then things work it seems to me that this is  a great idea so what i do is i set up this line i   i sit in the office imagine this is me forgive  my drawing imagine i sit in the office every time   somebody tells me let's say i got a call from  x and x says hey please remember that i have a   flight tomorrow at 9 00 pm i say great next time  they call and they ask me tell me what i wanted   you to remember and then i tell them you know what  do you have a flight tomorrow at 9pm and then this   seems great i charge people very normal amount of  money to store this information and service takes   off everybody needs this there's a big market  everybody starts using it and then i start to   hit the problem and you'll notice most systems  they start to hit the problem when they hit scale   then i realize that i'm getting a lot of calls  and i'm not enough to address all of those calls   myself so then i start to think and then i think  sure let me let me get my wife also involved   so now this number actually has me and my wife  both of us both of us sit with our diaries where   the call gets routed to either one of us  whoever is free and when i get a call i just   very quickly if the person is asking me to take  a reminder i just take down that reminder right   and when the person calls me back i respond  back to the reminder great i have suddenly   now doubled my capacity if i was able to take  1000 requests in a day before now i can take   2000 requests because i have my wife to help me  as well very soon i got my first unhappy customer   mr x calls me one day and says hey can you  please remind me when is my flight and i   look through my diary and i realize i don't have  an entry corresponding to mr x and i tell mr x   hey look i don't have an entry responding to your  corresponding to you mr x seems very disappointed   very angry and then disconnects the call now i  start to think what might have happened either   mr x was delusional or maybe there was a fault at  my end and then i see that there might have been   a problem it is possible when mr x wanted to store  their reminder they might have called my wife and   my wife would have stored this entry that hey x  wants a reminder for a flight at 7 00 pm tomorrow   however that entry is not with me because that  call never came to me however when x wanted to   understand when their flight was they ended up  calling me instead of my wife because they don't   know they only know this number one two three four  one two three four and i don't have that entry   and this is why x is very unhappy this is what  is called as consistency problem data consistency   problem which is an end consumer might have  stored some data with me which and when they   ask me back there is no guarantee that they will  get the data back because it depends on where the   request comes to if the request comes to my wife  they'll get their data back if the request comes   to me they'll not get the data back if my wife's  diary has that entry this is a big problem if not   solved my company will die so i start to think  i start to think and i come up with a solution   i say hey look i mean here is what we  could do when x calls me to say that please   note down that i have a flight tomorrow  at imagine 6 pm i'll note it down in my   diary i'll also tell my wife to note down this  entry which is also that tomorrow 6 pm flight   and only when both of us have written the entry  in my diary in their own diaries then i return   success then i tell mr x i have noted down  your entry that way there is guarantee that   both of our diabetes my diary and my wise  diary they'll have the exact same entries   consistency of data they'll have the exact same  entries and great my consistency problem is solved   now when x calls any one of us we can respond back  with the right data points things were going great   one day i again start to face a problem one day  my wife calls in sick she's not keeping well so   she's not in the office what that means is that my  wife for the time being is gone for a day is gone   and at that very moment this another gentleman  why calls me and tells me to note down an entry   i say okay sure what's your entry and they tell  me you know what i have a flight tomorrow at 3 p.m   i said great let me note it down let me by the way  also go and tell my wife to note it down oh by the   way my wife is not here she can't note it down and  therefore i'll have to tell why that hey i can't   note down your request your request actually has  failed which means for the entire day i can't take   in any new request that is a availability problem  which is my system is not available to take   all kind of write requests in the case of  software systems for example like facebook   you could think of facebook saying that for an  entire day or for an hour or for a few minutes   you can't make any new posts or you can't  post any new photos on on facebook that is an   availability problem so again i'm very stressed  what do i do i don't want my business to fail   so i think and think and think and i come up with  another solution i say you know what when my wife   is not here so i have my diary and imagine my wife  is not here so i have written a bunch of entries   this new day comes whatever entries i get  mr x says to note on something i'll note it   down here mister y says to put down something  i'll note it down and when my wife returns back   because i want both of our diaries to be identical  i tell her look you'll not take up any phone calls   till you have noted down all of the additional  entries that i made in the previous day   or if she was gone for last two days than in the  previous two days so you first note down all of   the additional entries and then you start taking  calls and then my problem is solved now because if   if x calls me and says that please note down that  i have a flight tomorrow at 8 pm even if my wife   is not in the office i can still note down the  entry and therefore i am still available since my   wife will come back and she'll she will actually  catch up with whatever new entries are there in   my diaries before answering any phone call so  we're also consistent which is great things are   going super good and then comes another problem  one day me and my wife we both have fight and we   stopped talking to each other so imagine i get a  phone call x says hey look please no down i have a   flight tomorrow at 9 00 pm i can't note it down  because i'm not able to talk to my wife and if   if i noted down then and say tell x that hey look  i've noted down your entry then my system becomes   inconsistent right because my diary has now  different entries my wife has different entries   and she's still taking calls so when this network  partition happens when this network partition   happens where i'm not able to talk to my wife  or other systems then i have to make a choice   do i want to stay consistent or do i want to say  available if i want to stay consistent then if mr   x calls me and says please note down the entry i  will have to say sorry i can't my system is down   it's not accepting any new write requests if i  want to stay available then i can take down the   entry but then i become inconsistent so i if  i choose consistency then i become unavailable   if i choose availability then i become  inconsistent and that is exactly the   cap's theorem caps theorem says that if you  have consistency if you have availability   and if you have partition tolerance then at a  time you can only guarantee two out of these   you can't promise all three at the same time  which means when this network partition happens   i would have to choose between consistency and  availability i can't have both that's the gap   theorem now i have two bonuses to add here most  systems that are available and partition tolerance   which is called ap a for availability p  for partitioning c for consistency right   most systems that are ap can still make sure that  they become eventually consistent what does that   mean that means eventually my and my wife's diary  they have same entries how do i make sure of that   whenever we start talking again we can exchange  notes think i mean or maybe just imagine that   there is a clerk here that i've hired who just  takes up all of the new entries in the last one r   tells me what those entries were so i take those  down also by the way asked me what were the   entries that i made in the last one are goes back  to my wife and tells here are the new entries that   were made in the last one are so there might  be periods in between that i was inconsistent   but eventually we become consistent so that is  eventually consistency there is one more extension   to cap theorem that was done very recently  actually not very recently about 10 years back   which is called as p a c e l c theorem and what  this theorem states is that when network partition   happens when my systems can't talk to each other  then sure you have to choose between availability   or consistency you can't have both however  there are going to be cases when my systems   can talk to each other which means there is  there is no network partition which is else   so else i have to choose between  latency or consistency i can't have both   if you notice when i made this decision that to  stay consistent i will make sure that on every   single call i will make sure my wife also notes  down the entry and only then i return success   then i'm actually waiting for my wife to note down  that entry my wife could be doing something else   at that time my wife would be answering some other  call and in that case it the entire request might   end up taking a lot of time so the latency could  become high because i want consistency however if   i was willing to let go of consistency which is i  wanted to say that hey look i'll just note it down   and maybe my my wife would probably catch up later  on then latency could stay really low so then you   would actually have a balance between latency and  consistency then there is no network partition so   that's what this lc theorem is which is just  an extension to the cap theorem because cap   theorem does not talk about latency i hope that  made sense to you and i hope that was helpful   this cat theorem is one of the most basic and one  of the most useful theorems you'll come across   when designing distributed systems i'll keep  doing more videos like this if you like this   video please like the current video and subscribe  to the channel thank you thank you so much   if you like the video please like this video  and subscribe to the channel leave a comment   if you want me to do a video on on a topic  that you find interesting thank you so much

Channel: SCALER

Views: 212,179

Keywords: system design interview questions, system design, system design interview, cap theorem, what is cap theorem, introduction to cap theorem, cap theorem for dummies, cap theorem explained, how to beat the cap theorem, distributed system, software engineer, tech lead, software architect, software skills, software developer, coding, system design fundamentals, system design basics, system design essentials, distributed systems, cap theorem system design, scaler academy

Id: 8UryASGBiR4

Length: 12min 47sec (767 seconds)

Published: Wed Apr 21 2021