Can Redis be used as a Primary database?

can reduce take the place of your primary database system can you replace your relational postgres or mysql database with redis or your nosql mongodb or cassandra would reduce can radius be more than just a cache these are questions that you will be able to answer by the end of the video stay tuned this episode of the backend engineering show is sponsored by my friends at redis labs i'm gonna break this video into three sections first of all we're gonna define what really is a primary database what qualifies a primary database the second thing is does redis actually check the boxes of a primary database and then finally we're going to go through features and radius that go beyond just a primary database i was personally surprised by by most of these features to be honest let's just jump into this video so first of all what qualifies a database as a primary database these are few features slash capabilities that a primary database must have in order to be considered your teller data store where you store your information and these are not set on stone features you can argue about these kind of features but in my opinion i think these features are what qualifies the database as to be your primary main database system atomosity individual statements in a transaction should all succeed or fail thus provide a unit of work isolation how isolated a given transaction is from changes made by other concurrent transactions durability rights committed by a transaction must persist on a storage medium that even if a case of a failure we should see these rights and these rights should not be lost consistency it's the spectrum by which a read transaction gets a consistent value from the database system and i call this a spectrum because you can get a consistent value or you can get an eventually consistent value after some certain time so your system might not be consistent at this time but after a while it will eventually converge and get consistent so consistency eventual consistency consistency all of these fall into this category availability the spectrum by which a read or write transaction get a successful execution from your database system again availability is also a spectrum because you can configure this based on the cap theorem that i talked about you can check that video right here and finally concurrency control what happened when a transaction tries to change a value that actually had been changed since the transaction started do you block that change or do you allow it and fail later pessimistic versus optimistic currency control very popular to method there's no right or wrong i talked about this video in detail if you want to learn about concurrency control here check it out now that we know some of the properties of our primary database does radius actually check those boxes let's see atomicity yes radius is atomic you can start a transaction and you can do multiple statements multiple commands and then you can commit these changes and if there is a failure if you're using aof append only file writers will make sure to only commit all of these commands or none of them at all so this is atomic isolation redis uses the highest level of isolation which is called serializable so if you're executing two transaction at the same time they will be serialized to be exactly after each other durability radius have something called append only file durability format which is very similar to the wall right ahead log or the redo files in other databases and any command or transaction that you commit goes into this append only file and and it's very fast because it happened only so you can achieve high durability with append only file and the good thing about redis with durability is that you can configure this durability it's almost like a spectrum unlike other databases you can go with the snapshot durability that means all rights that have been happening to memory which is very fast will be flushed to disk that snapshot of the memory will be flush to disk asynchronously on the background every x amount of seconds the power of that your rights will be super fast because you're always right into memory and asynchronously you're gonna write and persist this to this yeah there is a possibility of failure but this is something you can figure do you want strong durability go with aof if you want fast right you go with snapshot and you can actually combine those two as well just like other databases you can configure it is to be strongly consistent or eventually consistent strong consistency is done by generally synchronous replications that means you issue a right to a master node that right does not commit until it commits to all other worker nodes if you want faster rights you do asynchronous applications so as long as the master rights succeeds commit and tell me it's done asynchronously in the background update your worker nodes and as a result yeah reads might get a little bit of a staler version but the rights are fast it's a choice that you make speaking of consistency it's worth mentioning that redis enterprise offers strong eventual consistency in active active clusters between data centers when we use active active clusters conflicts are inescapable since we are writing to both instances at the same time and when we get conflicts we are bound to be weakly consistent redis enterprise offers conflict-free replication through predefined rules and as a result we get strong eventual consistency between those instances and availability redis is available and you can configure this availability based on the asynchronous replication versus synchronous replication if you chose strong consistency usually you're not that available because if any right from the master to the worker failed then you're really not available right because you're gonna fail the right right and on the other hand if you chose eventual consistency then yeah your rights will always succeed as you are available but you might get a little bit of a stale data what are the mercy of the cap theorem finally concurrency control as i talk about many time in the channel reduce is a simple single threaded database system so there isn't really a true concurrency but you can still prevent other transaction from changing something that you're about to read through something called optimistic locks or optimistic concurrency control and this is achieved by something called the watch command in redis you watch a value you start your transaction and if someone else committed a change to that value your transaction eventually at the end when you try to execute it or to commit it it will fail thus turning you telling you to retry it that's the effectively the optimistic approach right there's obviously pros and cons to each approach but yeah it does support that and starting from redis version 6 io in redis is actually now threaded while the core remains single threaded and we have knew this even before redis 6 redis still uses multi-threading to take the snapshot of the memory and write it to the desk and it seems now they expanded that to almost all ios pretty good now that we know that redis has the properties of a primary database let's go beyond just a storage system and actually i made a few videos talking about some of these features you may know red is for caching but it's actually used for about 20 completely different things here are a few common use cases caching obviously it's a very fast in-memory right cache so you can use it as a right through cache right back cache you can use it as search for session management you can use it for pub sub yes it's very popular i've seen many many many many companies users use redis for pop sub systems so just like that you get features from a q and a pop subsystem so if you're looking for a pop sub system you can replace your kafka or rabbitmq with radius and speaking of kafka you can use redis streams which works like kafka to ingest real-time data for example from an iot devices to connect different microservices together uber actually uses radio streams to replicate their data across data centers they also obviously use redis for other common use cases such as queuing and caching i'll have a link to their video in the cards a lot of other users use it for rate limiting especially if you're building an eye api gateway or ip white listing because you can you you're quickly looking at our ip and because it's a key value store you're going to get it very fast in addition redis is also used for real-time search knowledge graph real-time analytics application metrics and forecasting and real-time artificial intelligence and all these use cases really are even possible because redis provides more than 15 data structures and six modules which work natively in redis to make it all possible this is why redis is not just a primary database but it's actually a multi-model multi-structure database so here are few data types that redis supports strings obviously lists sets hashes sorted sets bitmaps and hyperlog logs another fundamental benefit of having these native data structures is they make it easy for engineers to store data naturally without transformation for example in java you have sets arrays hash maps etc all these types exist in redis natively however if you're using an rdbms you'll need to morph and transform these data structures into tables not only that you'll also need to access this data using sql this is as opposed to regular array set or hash map commands this additional overhead is a result of the mismatch between how you think an object in object-oriented programming versus relational databases tables this is known as the object relational impedance mismatch martin fuller has talked about impedance mismatch and object oriented programming a bit i'm going to reference his video in the description below another plus for red is because it's a very popular database you have many supports for a lot of languages and sdks as well another thing i love about this and i talked about this in my previous videos about reddish is the redis binary protocol which is called resp that that protocol actually supports something called pipelining which i talked about in my http videos pipelining is the ability to send multiple commands at the same time on the same tcp connection and instead of doing a request response like so send a command wait for results in the command wait for a result so you get much much better throughput as a result so during this video we learned what a primary database is we saw that rellis checks these boxes of a primary database we also seen that radius use cases go beyond just a primary database we've seen how red this is a multi-modal database we've seen it reduce the impedance mismatch in programming with native data types and finally redis enterprise provides even more enterprise grade features such as active active conflict-free geo replication now i gotta ask the question to you guys what do you think can rudders be used as a primary database let me know in those comments down below i'm gonna see you on the next one you guys stay awesome goodbye

Channel: Hussein Nasser

Views: 18,899

Keywords: hussein nasser, backend engineering, redis, redis streams, redis primary database, redis inmemory, redis vs postgres, redis vs rdbms, database engineering, redis ACID, redis cap theorom

Id: VLTPqImLapM

Length: 12min 17sec (737 seconds)

Published: Fri Jul 30 2021