What is Distributed Caching? Explained with Redis!

hi everyone this video will be talking about what caching is and how we can use it to design systems so at the end of this video you should be able to tell what a cache is and the situations where a cache can be used and of course how you'll be using it so what is a cache imagine you have this server and you have a database from which you pull out results so whenever there's a user request to your server you actually pull out results from the database there two scenarios where you might want to use a cache the first use cases when you're querying for some commonly used data for example you have your profile and that is stored in this database the same user is asking for the profile multiple times and how do you stable it in this box then you have saved a lot of computation so within this box maybe you can keep a cache and in-memory cache which is in your memory it stores the profile or user so you have a key and a value and so that is the first reason why you want a cash to save Network calls the second use cases when you want to be avoiding computations for example you want to find the average of all users the average age of all users that you have in your system in this case one of the ways that you can do this is every time a person asks for you for the average age you go to the database get all the users find the average of the age and then return that now this is very expensive of cost so instead what you can do is you can do this once find the average age and then store it in the cache so that's a key and a value so average and then 26 28 whatever be the thing it's a key value store it in cash the final scenario where you want to be using a cache is when you want to avoid load on the database so if you have a lot of servers and they are all hitting the database to get information it's going to be putting a lot of load instead you can get some caches maybe two or three distributed system you know after all and then you can keep that information in the cache you can hit the caches avoid hitting the database but the first two points are the key ideas of a cache which is either avoiding a network call or avoiding computations and both of them are designed to help you speed up responses to your clients so client makes a call and you immediately give a response if you have it in cash instead of making a query to the database so now that you know the benefits of a cache or first instinct might be to just store everything in the cache like cram it up with all the information because the response times are much faster when you're pulling data from the cache you can't do that for multiple reasons firstly the hardware on which a cache runs is usually much more expensive than that of a normal database the reason being that this cache is you runs on SSDs which are expensive compared to comparative hardware that a database runs on the second thing is that if you store a ton of data in the cache then the search times will increase and as the search times keep on increasing it makes less and less sense to use the cache why don't you just go and hit the database yeah so you'll be saving on a network call yes but eternity down the cache is not just expensive but also counterproductive so a pond becomes storing information in cash such that the database has almost infinite information while the cache needs to have the most relevant information according to the requests which are going to come in future so we need to predict something the predict things we need to says just two important questions when do I make an entry in the cache which is when do I load data into the cache and when do I evict data out of the cache the way in which you decide for loading or evicting data is called a policy and because this video is on caching this is called a cache policy so a cache performance almost entirely depends on your cache policy there's multiple policies you can have LRU is probably the most popular least recently used what this says is that whenever you're pulling in entries to the cache keep it on the top and whenever you need to kick out people take out the bottom most entries which means the least recently used entries in the cache so the top entries are going to be maybe seconds ago where you seconds ago and then you keep going down the list minutes ago hours ago years ago and then you kick out the last entries when you do need to ok for example if there's a celebrity who's made a post made a comment and everyone wants to pull that comment so you keep that on top of the cache and while it is hot while everyone wants to read that comment it's on top of the cache it's performing really well and when it gets cooler when people stop asking for that comment it keeps getting pushed to the end of the cache and then it's kicked out all right so that's the LRU policy the lfyou policy least frequently used is not frequently used in the real world so you can read on it of course it's a description below and some policies have started to perform really good I mean even better than LRU so they are sliding window based policies one of the Google developers who made caffeine has implemented this in their caches now imagine you have a really poor eviction policy which means that hitting the cache is of no use I mean you ask the cache for some data it always responds with I don't have it and then you have to go to the database again the first problem over here which is with a poor revocation policy is that it's actually harmful to have a cache because you are making that call that extra call which is completely useless if you have a poor eviction policy so the second problem can occur if you have a very small cash so users X&Y need their profiles and this is a profile cache so X asks for the profile you get it from the database you put X in this cache and let's say it in just store one entry why ask for their profile you have a cache miss you go and hit the database get that entry populated in the cache and now X makes another call which means they need X so there's a problem here so this concept is called thrashing where you're constantly inputting and outputting into the cache without ever using the results so it's again hurting you more than helping you the final and most important problem I'm sure you guys have already realized is that what about consistency so if a different server makes a call to the database for an update and says update profile X yeah so there's an update oh sure but the cache doesn't have that entry updated so server one pulls from the cache gets the user profile for X outdated profile and then serves it back so maybe the user has changed the password and you have the old entry the old password is still working maybe there's a hacker who's actually hacked the old password this becomes an issue of data consistency now let's try to figure out where to place the cache the cache can be placed close to the database or it can be placed close to the servers there's benefits for both and drawbacks for both if you place the cache close to servers how close can you place it you can place it in memory itself yeah so you can put the cash in memory in the server if you do this what's going to happen is that the amount of memory in your server is going to be used up by your cache but if if the number of results that you are working with is really small and you need to same on these network calls then you can just keep it in memory a few other things over here are that what if this server fails if it fails then this in-memory cache also fails and we need to take care of that the second thing is what if this data over here on s1 and this data on s2 are not consistent so they are not in sync in that case it depends on your application if it is a user profile it's really not that big a deal but if it's passwords if it's financial information if it's something critical critical data then you can't keep this so benefits are actually having closer to the server is that it's faster here and also the other simpler to implement but the benefits of having it separate are going to be seen now you can have something like a global cache and since we always work with distributed systems it's going to be a distributed globally but I'm just drawing one oh here the benefit of this is that also as our hitting this global cache it's limited in size so if there's a miss it's going to be querying the database otherwise it's going to be returning what's the possible benefit well your wedding queries on read database and increasing queries on the cache but more than anything actually this is a faster disk read so it's going to be something like Redis I think a lot of people are actually asked me what this is what is Redis and this is one of the common use case of Redis it's like a assistant storage which is like a cache yeah so this is a cache basically you can keep a few boxes which are just going to be storing memory and whenever person asks for a key you return a value in this case what happens if s2 crashes nothing really because everyone is going to be quitting this global cache anyway and because it is distributed and it's going to be maintaining that data consistency we can offload all that information all that headache to this guy over here although it is slightly slower it's more accurate in this choice range of placing the cache I would rather place it somewhere over here as a global cache the main reason for this is there's higher accuracy in the sense that if a server crashes it doesn't take its data to its grave yeah it's it's more resilient in that way the second reason is that you can scale this independently yeah you can scale the Redis caches independently while keeping the services as containers running on these boxes so there are certain boxes which are just running services which don't have their data being eaten up by a cache so my preference would be a global cache however for simplicity and for extremely fast response times you can keep it in the local memory we are now on to the final important characteristic of a cache which decides how consistent the data in this is and that is whether it's a right through or a write-back cache a write-through cache means that when you need to make an update to the data that you have let's say you are making a update to the profile profile X you hit the cache you see that there is profile X in this and so there is going to be some data inconsistency if you make an update directly to the database remember that person X has changed their password so I'll make an update to the entry for user X make that update and then push it to the database so that's a write through cache I wrote on the cache before going through it and hitting the database the second option of course is to hit the database directly and once you have hit the database make sure to make an entry in the cache so either database can tell the cache that this entry is no longer valid or you hit the cache you see that there's an entry for X just kick it out evicted delete that entry once it's deleted update the database when there will be a query on the cache it's going to come here it's going to see that X does not exist and so it's going to pull from the database and then send it back to you okay so this is the right through mechanism I mean the original one where you hit the cache make the update and then send it to the database and the right back is where you hit the database and then make sure that the cache is having a null entry what is the possible problem in the write through cache well you do the right over here you go and write it on a database but what if there are other servers having this cache in memory yeah the profile entry for X is on s1 and the profile entry for X is also on s2 so this is an in-memory cache and you are doing a right through so you actually update this entry but what about these guys so in that that case basically you're right through is a little silly so you kind of implement this algorithm you have to go for right back however there's one issue in the right back and that is performance when you update the store the real store the one source of truth that you have and you keep updating the cache based on that what's going to happen is this is going to be lots of entries in the cache which are fine I mean your consistency is not so important so invalidating them is going to be expensive because whenever there's a request you have to send a response from the database and then update so this is like the thrashing thing that we have again so right back is expensive how can we do better as usual just use a hybrid sort of mechanism right through and right back a booth giving you advantages and disadvantages what you can do is whenever there's an entry that has to be updated on the database don't write back immediately right onto the cache if it's not a critical entry if it's something like the person has their comment changed edit it just right under the cache keep serving requests through this box and when I mean there might be data inconsistency but do you really care ten seconds have passed twenty seconds are passed and when you have actually seen that there is some scope for you to persist entries make entries in bulk and persist it on to the database so that's one network call and you are saving on hitting the database consistently so what we can do is instead of taking one core policy we take a hybrid and then we take the best of both worlds depending on your scenario if it this is financial data if this is passwords you can't you can't have right through cash you need a write back cache but if it's not critical data then you can go for saving Network calls this way so caching is extensively used in system design one very good online course for system design is grokking the system design interview I have a link in the description below and of course I will explain to you what it is this is a part of the set of courses that educate of dot IO offers grokking the system design interviews a very famous course a lot of people before their system design interviews they go through this course in detail the best thing about this is that it has a lot of topics when it comes to load balancing or caching so on and so forth it's got a lot of those simple topics and it's got a lot of systems that it has actually designed so with this you get hands-on experience on designing systems so that's it for what caching is and what else use cases you can use it in system design extensively it's actually used in real world systems so if you have any doubts or suggestions on this of course you can leave them in the comments below and if you liked the video then make sure to hit the like button and if you want notifications for further videos like this hit the subscribe button I'll see you next time

Channel: Gaurav Sen

Views: 583,296

Rating: 4.8827519 out of 5

Keywords: system design, interview preparation, interviews, software interview, design interview, gaurav sen, distributed system, distributed systems, caching, cache, cache management, cache eviction, cache types, types of cache, caching in os, caching in database, distributed cache, redis cache, scalability, consistency, cache consistency, cache invalidation

Id: U3RkDLtS7uY

Length: 14min 4sec (844 seconds)

Published: Sun Mar 31 2019