AWS re:Invent 2018: ElastiCache Deep Dive: Design Patterns for In-Memory Data Stores (DAT302-R1)

hello everyone and welcome to reinvent I hope you guys are having a great time I name is Michael Labib a principal solutions architect here at AWS and I'm super super excited to be talking to you guys about Amazon ElastiCache so today what we're gonna do is we're gonna dive into the service we're also going to look at various usage patterns that you can use with kind of injecting in-memory data stores with your architectures so for our agenda today we're going to start with what's new then we'll dive into the service itself we'll review some of those common architecture patterns will dive into different caching strategies a lot of people ask how do you cache data using rightists or memcache T and then we'll conclude with best practices so gone are the days where you have one monolith database serving all your data needs right and for good reason with the diversity of usage patterns and data access patterns essentially you really need a database that's that's optimized and purpose-built for a particular use case and in the case of ElastiCache sort of fits in the in-memory space partly because the data is in memory and because it's a memory it's incredibly fast orders of magnitude faster than retrieving data from disk and in addition to the speed you you have specialized data structures that can really augment your applications and architectures and really speed up things with respect to those whose cases speaking of speed you know when you're building an application typically you know speed is one of the things that should be when you're your top design principles right so you'd be hard-pressed to think of a use case that you want to build a slow system and you know the opposite is true and and that's partly why we see such a demand for a memory data source and it's not really a sort of a binary you know do I choose in-memory or do I choose a different type of database it's really to augment your architectures and we're gonna really dive into that when we look at the architecture patterns so what's new a lot is new with the reticent M cache D engines Redis 5.0 was out huge huge announcement so run as streams really being the highlight of Redis five rhetta's five is supported with ElastiCache the other thing that's new with Redis is sort of Seth sort of gotten got married with lists so if you're familiar with both of those data structures so what it says can now do pop and block capabilities which is pretty awesome memcache be one point five ten is out also supported with elastication they're really between both of these engines a lot of optimizations enhancements algorithms things of that nature we're not going to have time to dive into each one of these but if you have any questions afterward please you know let me know so since streams are such a big big topic I want to spend a few minutes talking about the value of streams before we can get into that it's important to level set on how time series data was sort of built and use with Redis so there was generally a few different approaches that you would take with with Redis with respect to time series the first one is you'd leverage a sorted set this is probably the most common approach and so the way you would do it is essentially a sorted set is you have a key you have a collection of members and each member is a has a value and a score and the score essentially allows you to rank the data and the score for time series would be say a timestamp and that would give you the capability of you know sort of sorting the data either by sending a descending order the problem with using a sorted set is that you know number one the values have to be unique right in the time series use case you know you might have duplicate values right and so having a constraint on the uniqueness is it could be problematic the second one is the scores themselves can be modified right so they're mutable and that also can be a problem so it's not truly a time series data structure another approach is using a list so a list is a obviously key that's mapped to a collection of members and those members allow you to push in pop values in the the order of insertion now the issue with using a list is essentially you you typically have one can and that the consumer could be a blocking consumer that's just popping elements off the list so there's no fan-out capabilities with a list and there's also no way to recover from you know say you popped a value and say that the message and get processed properly doesn't really have that true capability in terms of recovering from a message another data structure that people would use is pops up and so pops up is not really a data structure but it's a capability it's embedded in Redis and the thing with pub/sub is it does support fan-out right so you have a channel and then you're you define a channel then you have a you publish messages to the channel then you could have subscribers subscribing to that channel and as our subscribing you know basically you're fanning out those messages the problem is is the data is not being persisted in a structure in a data structure so if a client is not listening to the channel it's gonna miss the message right and so it's not really ideal for you know critical data or truly you know message driven data or time series data so the lack of you know a true time series a data structure is really what birth the need of a reddish stream so I'm gonna kind of dive into the anatomy quite a bit and we'll kind of break it down to some degree um so you have three concepts right do you have a producer you have the stream itself which is the data structure and then you have a consumer so from a producer standpoint you can add a value to the stream using the X add command basically syntactically would look like this right you define the name of the stream in this case is called my stream the star is basically saying you know Redis hey generate an ID for me a unique ID in this case the idea would always be a time with the sequence so it's guaranteed to be unique and then the message is sort of a pair with the message and the value so as soon as I persist that into the stream Redis returns back that unique ID that ID would be used for me to know what was the last message that I persisted so I can use that to iterate over the mess is in the stream now the stream itself is append-only right so the data is immutable right and so it's ideal for you know capturing the true sequence of events and in addition to that you know there's a variety of different ways that you can consume that message and this in this example we're just using an X range I'm saying give me all the messages with the min Max and so in this case I got my message back but there's a variety of other ways and even more interestingly as you can define something called a consumer group and so a consumer is consumer group is really a collection of consumers that are all participating in retrieving data from the stream itself right so incredibly powerful in each one of these consumers is really just getting a unique value out of the stream so in addition to the Redis updates a lot of ElastiCache updates since last year I'll kind of touch on some of the bigger ones so in place version upgrades say you're on Redis 3.2 or Redis for and you're using cluster mode enabled or or not you can upgrade the version right and so you can do this without any manual steps or application changes and basically what we'll do is we'll upgrade your replicas and it will just fail over to those replicas so we'll essentially we'll take care of that operation for you the other one is if you remember last year we talked a lot about the our fours well now m5s and our 5s around much greater performance from the new version that the new version over the previous version in terms of performance CPU and on top of that what we've done and that's really the gray bar is we added additional optimizations on those instance types so just another you know I guess case for using ElastiCache over running Redis on you know in the vanilla ec2 instance but generally what you can see out of this graph if you compare like the blue which is the previous gen our fours the gray would be the optimized on for ElastiCache m5r 5s you really see a hundred and forty-four throughput additional throughput that you can achieve which is massive another big announcement is you know previously we still you know the standard configuration is 15 shards and a shard again is made up of a primary and up to 5 replicas which make up a max of 90 nodes we're changing that right if so if you really need have the need to have much more maybe larger data or a much larger cluster use case the 250 nodes support is available today if you want to white get white listed and so if you can see do the math depending on how many primaries shards that you have or how many replicas associated to that shard you can have upwards to 170 terabytes of data in your cluster that's massive right so you know it's that might sound crazy but maybe not right because people aren't just caching database data anymore right it's not just database it's not just session data it's a lot of other types of data we'll talk through that when we get into the application architectures and in addition to what I just mentioned a lot expect a lot more optimizations that are going to occur on the actual the instances that we we we host for you and we're also gonna support the rename command a lot of you have shared that requests that you want to be able to hide a specific commands maybe like the keys command that prevents your developers from you know bringing down the the cluster and then self servicing self-service patching so we're gonna give you the capability of really selecting you know within a certain window when you want those patches to take place all right so let's kind of level set we'll dive into the service and kind of just break it down make sure that we're all on the same page so what is Redis Redis is the most popular in memory key value store in the market and you can check that you know just check maybe DB engines or other sites and you'll see the rankings and what people are really saying about Redis and they say that and it's so popular for a lot of reasons and really you know in addition to the speed and you know you can see here sub-millisecond performance on average we see somewhere in a neighborhood of 400 to 500 microseconds in terms of speed but you have additional things would say like memcache T doesn't have which is really the H a right so you have H a you also can do backup and restore you also have atomic operations and in where it gets really interesting is the additional data structures that you have so if you're a developer and as you're building your applications and you're thinking about the different objects and you know collection classes that you eat that you use a lot of these are familiar to you right so whether you're using a list or a hash map a set a sorted set a lot of these are things that you're currently working with so you no longer have to think about what type of how do i serialize the data structure that I'm using in my current application you can actually just persist that structure into Redis and operate on that structure right so it actually eliminates a lot of lines of code with respect to you know what's supported and then the cool thing about having backup and restore is you can essentially up to 20 times per day as a soft limit you can take snapshots right create an RPO of the data that's in memory as well and from a syntactical standpoint incredibly powerful the api's are very rich with respect to sorting ranking and a lot of different ways that you can manipulate data and retrieve data in Redis so additional background here so like if you were to stand up say open source Redis on your own maybe in a dev a POC environment you might find it to be you know relatively straightforward but if you want to productionize that there's a lot that goes into production izing Rutter's number one is how do you make sure that this thing is truly AJ right especially if the data is say critical data or really you know it's enterprise level application it's also difficult to scale right because you know if you want to change the sharding whether you want to add shards and remove shards there's some limitations with open-source writers I'm gonna talk a little bit about that in a slide and then from a from an investment standpoint you know how much resources you want to invest in putting people behind managing Redis and when you do the TCO on that with comparing that with a managed service doesn't might not make sense right I'll give you one reason is if you are hosting it on your own you're paying for say data out between the ACS so if they add the data out charge plus the ec2 charge and then compare that with just the managed service which doesn't charge you for that data out communication right and so generally you'll see the price that'd be pretty similar right and so with Amazon ElastiCache in addition to us hosting Redis we're very much invested into the engine we have developers who are contributing we actually just last year we contributed a encryption in transit to the Redis community along with other bugs and fixes and things of that nature and we have a lot if you think about you know the our platform we have so many clusters that we manage and so we see this at massive scale and we've really tuned ElastiCache that fit a lot of these needs right so one thing which we're going to talk about is you know how we char data and that's coming up in a next slide but the other thing is really removing the headaches with respects and making sure that you know the data is encrypted the manage the manageability of the data making sure that you're patched and interact and all that stuff sort of just goes away in a managed service so use cases if you're new to Redis you generally start with caching right but as you expose those data structures and you think about the different ways that you can utilize Redis well we typically see as caching becomes you know maybe the second you know most common use case and people start using Redis - whether it be a buffer behind adjusting data a fast and just layer or maybe they want to do a leaderboard and so in this slide it says leaderboard but the reality is is you can have a Sales Leader board you can have the leaderboard that's really just ranking various types of data maybe most popular activities to your your website or products could be Twitter feeds so really the leader board is just leveraging a sort of stuff there's also geospatial capabilities we'll talk about that but essentially Redis allows you to to you know pass in a longitude and latitude and do like a jido radius a command and really identify values or members that belong to a particular geography we see a lot of media media streaming really just whether you're putting portions of the media into you know maybe a byte array into Redis or whether you have links to other media and contents session stores chat applications now just became a lot easier with redish streaming the same with message queues people use lists for that and we're seeing a lot of machine learning use cases as well and so this side of the use cases make a lot of sense anytime that you really need to augment an architecture and make something run a lot faster and so this is this slide is really just giving you an idea the different types of organizations that we have using ElastiCache and if you just think about the verticals you can imagine a different types of use cases that they might be using it forward a lot of them have to do with the previous slide that I was just discussing all right so with Redis I'm gonna level set on a different types of topologies that you have just so you can understand what's available the vertically scale topology some people refer this as kind of the classic mode vertically scaled what it really means is all your data resides into one node right and so you have a primary node and for every primary you can have 0 to 5 replicas right and so the largest size cluster that you can have is whatever data you can fit in the largest instance type now in this type it's apology you would have an application that would would connect to your primary endpoint and then you might have another connection from your applications is that's connected to your replica end points right so you would scale your reads off your replica end points and you have your primary for your rights and you know generally what happens is depending on you the way your client works you know there would be various options with you know how you want to do connection pooling and how you want to discover those replicas because the replica endpoints are not persistent now the other topology and this is where we're gonna spend most of the time because most organizations are actually moving into this topology is the horizontally scale topology and one of the reasons why makes a lot of sense to move into this is because you can have a one shard cluster mode enabled topology which really is the equivalent of having one primary with you know the same amount of replicas in the vertical scale topology the added benefit though is that you can change the amount of shards that you want in this topology with zero downtime and I'm going to talk about that in a in a later later slide but what horizontally scales apologies really mean is that rather than all your data that entire key space belonging on one one node you're dividing that key space into shards right so let's assume that you have four shards on the concept of a slot is essentially there's 16,000 384 slots and then that slot would be used to decide what range of what past lock range would belong to each shard and in the anatomy of a shard essentially is the same thing you have a primary and then you have a replica that that is that corresponds to the same hash slot range for that particular shard now where this is different in addition to the partitioning is also on how you connect to Redis and how you talk to Redis so what happens is if you're using a cluster or where a client the client itself you can map it to talk to the configuration endpoints so we exposed that rather than a primary endpoint and in the configuration and point when you talk through that the client depending on the implementation of the client is essentially gonna do a cluster slots info command behind the scenes it's gonna get a map from Redis what the topology looks like and it's apology is what are all the shards and what are the hash slot ranges associated to each shard and it's also gonna have like a flag on which one would be a primary which one would be a replica now once it has that math that map is stored locally in the client and the client then knows that if I were to do a get set type of operation the first thing that it's going to do is gonna execute a circle 16 mod function and it's gonna dis based on the output of that function it's gonna know where to send the the key right what's shard does that key belong to and if for whatever reason that's apology changed after the after the cached map the the the client would generally refresh its map right so that's an implementation detail of the the client and some clients also will frequently ping Redis in the background so the takeaway here is that when you build a custom mode enabled cluster you're basically deciding how many shards you want how many partitions do you want of your data and then you can also by default it's equally distributing the hash slot ranges on that cluster but you can also do a custom distribution if you wanted to and so you can say I want just this range on shard 1 and maybe that range on start a char - and you might want to do that if you had if you wanted to isolate some keys on to a particular shark and then you can always check when you do this you're essentially you should see and depending on the distribution you pick you can see the current items in each one of those shards to see the using the CloudWatch metric that we expose alright so just kind of a quick recap with this slide in the next the value that you get with cluster mode enabled number one is you have additional nodes right and so additional nodes what that really means is you're able to add more memory you're also able to scale your rights you're also able to scale your reads much more effectively you also have more connections if you really needed to support a lot more connections they both support the open source client there's nothing unique about ElastiCache with respect to connecting to it and the other added benefit is the the failover time so I mentioned earlier that if you were a cluster mode disabled you get a primary endpoint and that's a persistent endpoint with cluster mode enabled you're dealing with a map right and so what that means is there's no dns updates or propagation that needs to take place so failover it takes a lot faster and then the other thing that's great is that if your data is partitioned let's say you had 10 shards and one of the shards had a failure only 10% of your rights are impacted I mean you could still read because if you're assuming you have a replica but then until that failover takes place which could be you know up to 15 and 30 seconds and then you can start writing again but it's much faster than that vertically scale topology which also needs to do the record set changes and then from a price standpoint they could be very similar right so it's kind of you know you have to do the math but essentially what happens is you know where you would normally scale up now here's this choosing the smallest instance type that supports the network bandwidth that makes sense for you I would highly recommend a new are fives and fives and pick the pic maybe a smaller size that makes sense to have multiples right and again you can do the retarding capabilities with online recharging all right so just the kind of breakdown to anatomy even further so a shard I mentioned earlier by default you can have up to 15 this is the assuming you're not white listed so 15 shards and again a shard is made up of a primary and a replica we're doing a synchronous replication for you if you want to see what the replication lag is that's another CloudWatch metric and in the replicas is optional you can have either no replicas or you could have up to five per each shard and so assume that a failure happens we're managing this for you there's no sense at all there's no additional tools that you need will detect the failure and it will well essentially we will elect one of the replicas with the least replication lag to be the new primary and it will send off a few SMS notifications in case you want to consume that and maybe do some reporting and again this takes place in roughly 15 seconds and then you can still read from your replicas right and so that's a another thing to keep in mind and also to look at when you're choosing what client you want to use because some clients make this really easy where you can annotate and say hey you know semi reads they're my rights they're now with online we sharding and basically what this slide to show it is that you started with three and you wanted to say you wanted to move to five this is a simple API call and in this particular case I'm scaling out right and so where this is different with ElastiCache is that we do a slot by slot migration now the open the ownership of the request is still at the source and it's just until the entire slot was been migrated to the destination then we we started sending the traffic essentially to the the new shard now what this gives you because you're doing so we're doing slot by slot is we give you the ability to still execute you know your Lua commands your M gets on that particular shard whereas with open source it might suffer from that because there's that split slop scenario right and so that's one the second advantage is that this is a much easier more reliable way to char data and so it's easier to recover from there's no manual intervention if during the restarting capability something went wrong that could be problematic if you were doing this on your own and then the other added benefit here is and there's no application changes right so this is happening behind the hood you're not seeing this your application is still connecting to and writing and reading from the cluster the cluster is still knows which is the primaries and replicas and once the slot changes take place it's gonna modify its map and then your your application will know where to direct the traffic now the only thing that I will mention is that this might add a little bit of latency up to 20% latency but again we're talking about an in-memory data store so where you're initially at 400 microseconds maybe now you're at 600 microseconds and then the same thing is true for scaling in right so you have the ability to change how many shards that you want whether it's out or in and this is great especially if you don't know how many shards you need or if you know that there's an event happening maybe you're a retailer and you want to prepare for the holidays or you know Black Friday just passed or Cyber Monday you have that ability to plan in advance right very easily with no downtime now because you can do this right there's no reason why you can't automate a process right and said I'm always about automation I always talk about automation especially with CloudWatch metrics that you should be concerned with in this particular case say for example you were watching a particular metric maybe was memory maybe you is engine level CPU utilization if it met your threshold kick out an SNS notification trigger a lambda function and that lambda function can execute a shard recharging command and essentially modify the cluster with whatever additional shards make sense for your topology now this is not going to happen instantaneously because there's a lot of you know data being moved around and you know operations moving around but at least you know it's happening durably and you know just account for that you know account for the timing so you might want to be a little bit more conservative with respect to when that threshold is met all right so let's kind of dive into some architecture patterns that we see at least common ones first one is caching right and so this caching slide is obviously showing you a lot of different ways and back-end data stores that you can cache and I'm showing this because a lot of times people are just so stuck on you know databases well the reality is is you can cache anything that can be deduced or you know brought down to a data structure or even bytes right so right us is binary safe and so what you can do is really put Redis in front of anything that you want to reduce pressure from or anything that you just want to speed up maybe it's a web application maybe it's a you know some kind of experience maybe it's a buffer maybe it's something else that you want to augment and inject performance right and so I'm gonna highlight in the caching portion of this talk s3 and in a relational database but if you have questions about how do you cache any of these other data sources let me know but the most important thing to be aware of is that you know you always want to make sure that your cache and the validity of the data that's in the cache corresponds to you know if a freshness factor that makes sense for your for your workload right so knowing what the frequency of change of the underlying data and in applying TTL that corresponds to that data and make a lot of sense so we'll talk about that as well during the the caching portion of the talk another use case that we see is sentiment analysis so imagine you're and you're consuming a lot of fast moving data whether it's click stream or Twitter feeds or whatever it is you could have a lot of different ways to you know consume so in this case I'm just showing a few different options but at some point you want a data layer that's gonna persist that information or buff but be a buffer to the data that's being ingested a lot of people use Redis for this right because Redis can support incredibly high throughput low latency and you don't pay for throughput costs you don't pay for request rates basically what you're paying for is the instance of pricing and so what you can have is you know consume that information whether it's in a list or reddish streams which we just talked about and then you can have consumers or multiple consumers peeling records off of the of the stream and then sending those records at whatever frequency that you want to other places maybe to do sentiment analysis with comprehend maybe to do a leaderboard as you're seeing these tweets or the click streams taking place you want to see what the activity of the ranking of those items are in the in the in the feed and another option is you know maybe sending that data to your primary database somewhere else right so that's the another that's where DynamoDB is sort of fits in this kind of a strange sound here that's what that is all right I'm just gonna ignore it so somebody shuts it off alright so and then the other one is using IOT data so we get this quite a bit with IOT data you know you you obviously have these sensors you're consuming data you know from various places AWS IOT core makes it incredibly easy to do and as you're consuming this data you might have a rule that's saying saying you know for for a specific type of information I want to send that data to Redis and in maybe the other data the raw data I want to have a historical view of all that information and shove that into my data like may be s3 and so we talked about the different ways you can deal with time-series data with Redis again reddish streams here makes a lot of sense and in sort of sets would be the other option all right I'm gonna ignore that all right so the next one is real-time Kinesis filtering so we get this a bit whether it's Kinesis whether it's Kafka it doesn't really matter so assume that you're grabbing or you're processing a lot of streaming data fast moving data but as you're processing that information you want to see if maybe similar information was already persisted right into into Redis because maybe you want to decorate that information right you see a particular customer or a particular tweet query the cache the in-memory cache oh I see some relevant information there what I'm going to do is decorate the information and presented into a process maybe a cleanse stream another way is deduping information or making sure that you just have unique information there or maybe some other counters that you're taking place Redis makes a lot of sense again from the cost perspective the speed perspective and the variety of data structures that are supported mobile so mobile especially with the geospatial capabilities in a caching as well so you have a mobile application maybe you want to build a recommendation engine so you know I'm a user I'm on my mobile phone you can pass up my longitude latitude that will hit maybe your API API gateway will trigger a lambda integration hit Redis semi eight my longitude latitude a Redis execute a Geo a radius command and then have Redis sends you all the points of interest back to your user right so what are the places within a mile radius of my current position makes a whole lot of sense now in this particular case Redis would be augmenting the backend data store which in this case would be dynamo and then you can have a nice right back pattern where you're constantly refreshing or updating the cache by just triggering off of a dynamo DB streams a lambda function right so it makes a nice little process to keep your cache fresh and so this would be another case where you're showing Redis sort of augment another data store and in rate-limiting so this is one that is kind of counterintuitive at first but then it makes a lot of sense right so especially when you're in a cloud architecture where you could you know scale your back-end or your cloud environments whatever the load is or scale in in some cases you might not want to do that right especially if it's cost prohibitive or maybe you have a product where you're selling you know silver gold platinum depending on the requests per second and so what you would do is Redis also supports accountant a counter is essentially the integer representation stored in a string you could increment or decrement that value and so as the requests are coming in you can check against that counter if you know if your your threshold wasn't met then you can allow the API request to go through if the threshold was met then you would just send a response back to your user hey you exceeded the amount of requests that you can do please upgrade to the next package or whatever makes sense for your environment and we see a lot of other sort of you know integration patterns graph search or just another two common ones I'll talk about graph so with graph you know imagine you have this highly connected data this again I'll use the restaurants use case and you want to figure out you know based on maybe some other person's liking what recommendations I should recommend to to myself me being a target person so you would check my you know mine and that person that's similar to me and see what that person likes and then grab that those vertices or those restaurants and then send them over to me but as you send them over to me you might want to also send out to Redis and do a geospatial look up and say are based on those restaurants and maybe there's attributes associated to those objects that say where the the location of those restaurants are you might want to do another geospatial query another one might be you want to see or you want to aggregate similar likings of of people and you want to aggregate that in one of the collections maybe a hash map or a set in Redis you can easily do that right so they sort of make a lot of sense with respect to use cases and the other thing is you can obviously cache right so if you have a query that you're constantly querying you can always a cache that information as well right and in the other use case you know would be search so again like imagine you have queries that you're hitting your your superior search engine you can cache the actual query so the entire query string that you're you're you're you're hitting the back end with and in the response that came out of the the search engine you can cache that another way that we commonly see people using Redis with is as a buffer so imagine you have a lot of fast moving data you're consuming a lot of data and then you want to process that information and peel those records off and sort of you know whether you're using log stash and just output that information into the search engine so basically having Redis serve as a buffer makes sense there as well so I'm going to dive into some of the caching strategies we get this question a lot from people especially if you're new to write us how do you cache data so the two common patterns basically lazy loading and in an write back so we're lazy loading you always assume from your application that the data is stored in the cache whether it's stored there or not there's a solution on how to put it there but you start with assuming it's there and if it's there great that's called a hit there's a CloudWatch metric that will tell you whether the data was found in Redis you can check the hit the the hits the hit count in cloud watch now in this particular case we're using we're assuming that a result set object was stored in Redis so results that object is what you'd get at a relational database so I'm gonna pass a key the key would be save my query my sequel query or something similar to my sequel query and if I get back some bytes and I'm going to assume that that was the the byte or the serialize interpretation of that results that object now what I'm gonna do is I'm going to convert those bytes into a result set then I'm gonna return that back to the application right and so this is really nice because you can add a nice dowel pattern pattern right in front of the your application if the data is not there you query your back-end database just like you would normally do and then what you do is after that you shove that result set object right back into the cache so you just serialize those bytes back down basically you convert your your cached or your result set object into a byte array and then just shove that into run us and then for the key the key can be the sequel statement or a similar query statement that sense to retrieve the data and optionally you always have the TTL so you can say for that particular query I want it to be true for the next hour or day or 30 seconds it's completely up to you now what other people do while the first approach makes sense if you want to offload pressure from the back end you still have to do the wall RS next and it kind of you know iterate over the iterate over the the object in the first approach and this approach as you're doing the initial iteration of the result set object you can convert that result set object into a hash map now the value with converting it to a hash map is then you can persist the hash map into Redis and and what that gives you is the ability when the when your say your API you want to request particular properties of that row you can say for that particular customer just give me the first name last name and address right because now you've you've reduced that that that row into a map and now you can retrieve individual properties associated to the map so in addition to speeding up the backend now you've you're speeding up your application logic because now you're not dealing with iterating over a result set object each time that you're grabbing the data from cache and so from the s3 perspective same concept right imagine you assume that the data is in cache for this particular example I'm going to assume that the data that's stored in s3 was a string and so I defined a a a naming convention for my key name which is the bucket : the key which is the object name and in s3 if I get a value back great I'm gonna return that back and I'm gonna assume that was the cash value nest 3 the value wasn't there I'm gonna query s3 like I typically do maybe I'll get back a s jekt content input stream and I'll convert that input stream essentially to a buffer a string buffer and then again into a string and then shove that into Redis now obviously there's other data structures in different ways that you can do this and this particular I'm gonna take whatever value was in that file or that object and then make that a string and shove that in my value into Redis now with Redis each value can have a max size of 512 megabytes now you never want to store 512 megabytes as a single value but you could write technically you could but generally you would want to have it in a managed manageable size for each individual value now the other thing that's cool about us 3 is you can define this sort of right back pattern that's gonna constantly keep your data in memory fresh so assume you you're constantly you're caching us 3 and you're writing to s3 on those inserts you can trigger a lambda function which is gonna shove the data right back into Redis so you begin when you query the data in s3 out of the at a Redis or and you're also hydrating proactively hydrating the data back into Redis all right and so one other thing I'll mention about caching is that with lazy loading if you're just doing lazy loading that might make sense much like it might be a good cost strategy because you're only caching data that's been queried but if the data is not there you always have that ended that first hit where the data wasn't there and then you have to sort of cache it a little bit of over overhead that you have to deal with if you are if you're doing this right back and proactively hydrating the cache what you might be doing is you might be putting data that is never going to be queried right but at least you're pro what you're increasing the probability that on the road whenever when the request comes in that it's going to be found right and so in practice what you want to do is you want to do a combination of both so as you're proactively hydrating the cache you want to apply it a conservative TTL that will make sense that if the data is never queried then just expire it and then when if the the request finally comes in then what you do is then just the lazy load will catch it and throw it into the and then you want to play with that and make sure that you understand you know what exactly is an appropriate TTL value and then what is the type of data you want to cache and that would be a good strategy now as we're talking about data and throwing data in some memory you know you always want to make sure that you sized your cluster to a good size that makes sense that we're not over filling memory but if you ever did overfill memory Redis is sort of polite with respect to adhering to one of your policies your eviction policies or max memory policies to the best of its ability and so you have a variety of different policies that choose from and I'll give you a use case where this makes sense so imagine you're putting in a cache data which is cache queries and if for those queries you're applying a TTL value and you're also storing maybe a metadata that you're not you don't have a TTL because that metadata doesn't expire maybe four months or you know some other you know longer duration of time within that particular case if you ever hit an eviction you probably do not want to use like an all keys LRU because in all keys that are you is going to ignore the the key the TTL value associated to akita it's just going to look for the least recently used key to evict and the scenario are described maybe the the volatile LRU would make better sense because when it does evict a key it's going to choose a key that already has a TTL so leave that metadata values alone all right so cluster size best practices when you're first kind of thinking about what is the proper size of this cluster the first thing you think about is storage and when you think about storage like let's assume you know you need a hundred gigs of storage the first thing to keep in mind is that Redis itself needs about 25 gigs of memory sorry 25 percent of memory and so by default what we're gonna do is we're gonna reserve that for you so at that point your total cluster size now you need about a hundred and 25 gigs I'm just giving making up an example and in assuming that you're just guesstimating you're not actually sure how much data you need you might want to add a little buffer there and then on top of that if you're the DevOps person you also want to make sure that people are using TTLs because at the end of the day right you want to make sure that the data is fresh in a cache and doesn't affect you know your your experience or your customer experience and you also want to make sure that your youuuu know how to react to the to a situation where you ever need to scale that memory so that's your first plan then the second plan is identifying what your load is so Redis can support incredibly high throughput for each individual node I mean it's incredibly high amounts of operations per second but even with that you still want to have a good idea of you know what's the balance between reads and writes you really need you know maybe a how many read replicas you actually need what's your percentage of reads versus writes so you want to size that appropriately and with writes as you as you need additional writes you'll you'll want to partition add more partitions to increase or add more shards increase your write throughput and then the other thing is always selecting an instance type that supports the proper network bandwidth that you need and then you know lastly that I'll talk about is you know as we we kind of talked briefly about the blast radius that the the nice.you value that you have with partitioning data because only a portion of your rights are affected this if you kind of extrapolate that that logic and think about the different use cases that you might have in your your system if you wanted to sort of isolate maybe you're caching workloads with your high ingest buffer workloads or key workloads along with you know it may be a different workload that might make sense that further sort of protects your environment doesn't affect the blast radius if a particular failure occurred now the more isolation you do obviously there's more clusters that you have maybe additional cost what we generally see people do is is isolate by purpose so all your caching workload in a cash kind of cluster and so on what I would never recommend is just have one cluster and just do everything on that one cluster right because there's just too much happening in one space now I talked about cloud watch so these are some of the metrics that you want to be aware of I'll highlight the you know the ones that you I would highly recommend putting a alarm on so bytes used for cash make sure you understand what you know what limit or threshold you you want to be aware of how much data is actually being stored in your in and Redis and the cache hits and this missus is incredibly important because at the end of the day it's all about making sure you have a hit in Redis right so you can just check that check your counts and if the ratio is you know anything less and that's 8090 percent and that's a problem that's there's room for optimization there right and that's really making sure that TTL values are appropriate engine level engine CPU utilization so that tells you how much CPU is actually used for Redis and I would make sure that for that one I would have probably multiple alarms kick off maybe one at forty percent you know maybe one at fifteen sixty percent and it had maybe at seventy percent maybe trigger that had that alarm to rashard add more shards to would reduce some of the that pressure or could be replicas depending on what type of requests are causing that problem evictions so again evictions means that you're over filling Redis so in that particular case scale out scale up you definitely don't want to see evictions take place the same thing with swap you shouldn't see any swap I mean this is an in-memory system in memory datastore so so as soon as you see some swap you know that essentially means that you you need more memory and so we have a couple other talks we had a reddish stream talk actually uh I just earlier today we also have a workshop if you want to get your hands on some Redis Friday if you're here on Friday please come by we'll also I have some reddish streams taking place in that workshop and again hey thank you very much I hope you learn something new and have a great day today [Applause]

Channel: Amazon Web Services

Views: 13,066

Keywords: re:Invent 2018, Amazon, AWS re:Invent, Databases, DAT302-R1, Amazon ElastiCache

Id: QxcB53mL_oA

Length: 50min 53sec (3053 seconds)

Published: Thu Nov 29 2018