Learn Redis in 40 Minutes - Upstash Redis

redus is one of those tools that is used in so many places because it solves so many problems so learning even the basics of redus is a core part of becoming a well-rounded engineer and in this video I'm going to show you just enough redus for you to be productive I want to show you just enough rdus so you can take it and you can start using it in your own projects and eventually use it in a production project so by watching this video what are you going to learn you're going to learn what redus is and you might have an idea of what reddis is you might already know that it's a key value store but I'm going to show you that reddis is actually much more than that you're going to start to realize why redus is so popular you're going to learn how to use redus via the command line and we're going to go through some samples of how to use redus in node.js to solve some really common problems so to get the most out of this video I recommend that you go to the description below and you clone the repository that I've supplied there and then in that repository you're going to find some files that you can use to follow along with this video and to do that you're going to need no. JS installed and I thought to myself what's the easiest way for someone to get up and running with redus and the answer to that is obviously up stash so I reached out to up stash to sponsor this video and they kindly agreed so if you click on the link to up stash in either the read me of this repository or in the description below you going to be able to sign up to a free account and then you're going to be able to follow along inside of the Reds terminal so what is the structure of this video well this video is going to be very interactive and again I really want you to follow along with what it is that we're working on here because we're going to be solving some real problems and I want you to turn co-pilot off and get some hands-on experience with reddis so we're going to get started with up Rus and this is just going to take a few seconds we're going to learn some basic commands and data structures and then we're going to dive in solving some real problems in no. JS so we're going to solve these five problems and if you're just interested in one or two of these problems that's okay I'm going to leave time stamps on this video but firstly what is redus well redus is a key value store and it stores those keys and values in memory this makes redus extremely fast but what you might not know about redus is that it can handle lots of different data structures and it's Reedus's ability to handle lots of different data structures that makes it so powerful so redus actually has a lot of features that you would expect from a normal database for example redus has pubsub and we're going to see how pubsub Works in no. JS later on redus also has transactions which is extremely helpful redus also has persistence so you could use redus as your primary database if you wanted to I wouldn't necessarily recommend that just because storing data in memory can become really costly and then finally redus actually has programmability with lure so we're not going to get into persistence or lure scripting in this video but we are going to have a look at transactions and pubs up so come over to ups.com and click on this login button I'm going to log in with GitHub once you're logged in you're going to see this create database and let's just create this database now I'm going to call this redus I'm going to call this learn redus I'm going to set my database to Global and then I'm going to set my primary to Sydney just because this is the closest database to me and therefore it's just going to be a little bit faster I'm going to enable TLS and this just means that my traffic is going to be encrypted and then I'm also going to enable eviction and this just means that I'm going to evict entries when I've reached a maximum size so let's create that database and this will just take a few seconds so now that my database is ready to go I can start using this straight away so there's a few things that I want to show you about upd rers just to save you a bit of confusion so so firstly there's several different ways that you can connect to your database the first way that you can connect to your database is using this redus protocol here and you can use something like IO redus this is how we're going to connect to our database this is just the typical standard connection to a redus instance however with up- Rus there's a second way that you can connect to your database and this is through htps so if you come over to JavaScript the reason that up stash allows you to do this is because you can also use up stash in a serverless environment and send commands to your up stash instance over htps instead of creating a connection to the instance okay so the confusing part about that is going to be that your credentials so in this video we're going to be using the Rus protocol here and so the credentials that we're going to use are going to be this password here so you can click to copy that password and then you can copy this string down here that has the redus protocol and it also has the port number if you're going to use JavaScript you're going to want to use the htps link so that's going to be this one just here this link here is your actual reddest instance so it's fine to copy that and then your token is not your password your token is down here so you have this rest API and so you can copy the URL from here and you can also copy the token so if we paste the token in the browser you can see that that is a massive token here whereas if we copy the password you can see that that's quite a short string here okay so I hope that that saves you a little bit of confusion about using up rdus so what we're going to do is we're going to go over to node and then we're going to copy this string here we're going to look for a file called EnV and then EnV you're going to see this reddest URL let's paste that in there and then you can see that it's also copied to Stars so let's go get our password as well so we can come up to the top here we can copy our password and then we can paste our password in here here so this is saying that the username is default and then the password is this string here so now we're ready to go so let's have a look at the folder structure here and I can explain what's going on so firstly we have this config folder and this is just going to load your rdus URL for you and then we have the problems and solutions folders so the problems folder is where you're going to work and then the solutions are going to have the final Solutions if you get stuck you can refer to those but I'm going to go through the Solutions in this video okay so let's go back to up stash and we can learn some basic commands or things that we can do with redus so if you click on this CLI here you're going to get a nice CLI in your browser and we can use this to send commands to our redus instance so let's first just set a key with a value so we can say set and set doesn't have to be capitalized and we can say my key and then we can give this a value so our value could be a number or it could just be a string like Tom now we can get this key back out with the get command and we can say my key and then we get our value back out to clear the terminal here you can just type clear and then that is going to clear your terminal okay so let's set this key again and now let's say that we want to remove this key so we can say delete so D and then we can say my key now if we try to get our key back out it's nil so you're going to see some feedback from redis here one is okay and this just means that we've successfully put this key into the redus instance you're going to see one and one also means okay so if you see a zero that often means that the key doesn't exist or something else happened it really depends what it is that you're doing so let's clear that okay so now let's say that we want to set a key that expires in 10 seconds so we can use set X so set with an expiry then we can say my key so the next string is your actual key the next is the seconds that you want it to live for so we can say 10 seconds and now we can give it a value so we can give this a value of Tom does Tech now let's get my key and you can see that my key still exists and if we keep doing this eventually my key is going to return nil because the expiry has elapsed okay so let's set another key and I'm just going to call this key name and I'm going to set to Tom and let's say that we want to see if this key exists so we can use the exists command now we can give this name and we're going to get back one saying that this key does exist let's say that the key doesn't exist so we can say name two we're going to get back a zero and this indicates that the key doesn't exist when you do this in no. JS you're going to get better values like true or false okay so now let's say that we want to see all of the keys that exist in our redus instance we can use the key Keys command with an asteris and this is just going to tell us that name is the only key let's set person and we can set this to Tom now let's do Keys again and you can see that we have two keys name and person now another helpful thing to know about redes is how to flush all of this data so we can say flush all and that's going to remove all of the keys so let's do Keys star again and you can see that we now have no keys Okay so we've seen that we can set keys we can perform different operations on those keys what about if we want to use a different data set like Json Json is an extremely common data set and it's going to be extremely helpful if our database can handle Json and Rus can handle Json so let's set a user and we can set that user to have some other properties as well and those other properties can be in Json so let's say set and I want to give this a key of users and I want to give this user a ID of 100 and then we can use that brackets just like JavaScript object notation and we can give this a property of name now we can give this a value of Tom and let's also give this an age and I want to set the age to 33 now we can get our user back out so get users and 100 and we get back Json as we would expect so one thing that I want to note about this is we want to treat our Rus instance kind of like an object store so that just means that everything is lumped into one big object and so to identify different things we prefix our keys here with the object type or the resource type and so I've used users here but let's say that we had a product we would say products and then let's say this product had an ID of one then we would give it a idea of one here and so inside of the key you have the resource type and then you have the identifier to that one resource so doing that is a really common practice that's going to save you a lot of headache when you're working with lots of different object types or resource types in your redus instance so another really common data type is hashes and hashers are used to store objects represented by their fields and value and to use hashes we can use the H set command to set a hash so I can say h set now I can give this a key so users with an ID of 100 now I want to set a property on this user of name and I want that value to be Tom now we can get a property out of this hash with h get users 100 and then the property that I want to get is name and then this is going to return Tom now we can set another property on this so we can set the age and we can set this to 33 now we can get back our age and it's going to return turn 33 so what is the difference between Json and hashers well if you use Json when you query by the key you're just going to get back the whole Json blob when you use a hash you can query by specific fields in that Json so let's clear this so another useful data type is a set now a set is essentially a list except every item in the list is unique so we can use this with set add so s a a can remember this by sad with a d and now we can say my set and this is just the key and we can set lots of different values in here so we can say one 2 three and then we can say four and four now we can get the set members with s members my set and you can see that we have 1 2 3 four even though we put four in twice we can also add to an existing set so we can say s a d d my set and we can add a six now let's try to get the members back out and you can see that six has been added here if we try to add six again and we get the set back out six will only be there once because it needs to be unique so let's flush all and clear so a similar data type is a sorted set so a sorted set is going to mean that every item inside of that set is sorted and then you're going to be able to get them back out in a sorted order so this is really common for things like a leaderboard where you just want to display all of the users top score down to the lowest score so to do that we can say z a d d and we can say my set now we can give this a position so we can give this a position of one and then we can say one let's add another one and we can give this a position of 10 and we can say 10 and then we can add one more and and we can give this a position of three so let's say that we want to get all of our items back out of the sorted set we can use the Zed range now we can say my set and we can give this a position of zero and minus one this is just saying that we want all of the items out of the set and you can see that we get these back out in sorted order let's say that we want these in a reverse sorted order we can do the same but we can give this a REV command and then this is going to reverse the list for us okay so the last command that I want to show you before we jump into our projects is counters so we can say I NCR and we can say my counter now this is going to give us back a one so now let's try get this so we can say get my counter and then this is going to give us back a one let's increment this again and now we can get my counter and it's back at two we can also decrement this counter so I can say deck my counter and then we get this back it's going to be back at one you can also increment it by a specific value so we can say incre by so increment by my counter and we want increment this by 10 now if we get this back it should be at 11 we can also do the reverse and decrement it by a value so we can say deck R by my counter and I want to decrement by five so if we get that we're back at six so that's the basic commands that you're going to need to solve these problems let's go back to our editor and we can start solving some real problems with redus inside of node.js so let's have a look at the first problem here and this is the problem that everybody is interested in with redus and this is cing because redus can make your slow function calls extremely fast so let's have a look at the problem the fetch start a function so this function down here is really slow and it needs to be really fast reduce the call to the network and make the function faster so essentially what we're being asked to do here is to Cache the response so we have this function here called Fetch data and I don't think that we should touch this function here and then down in the main we have some timers and we're going to call this function twice if you want to solve this problem on your own now is a great time to pause the video Try solve the problem and come back so I think the best solution to this problem is to create a function called Fetch with C that we can wrap our fetch function in that is going to handle the cing for us so let's create an async function called Fetch with C now this function is going to take two arguments one is going to be the fetch function and we can just type this out as type of fetch data now the second argument is going to be our key and a key is just going to be a string so the first thing that we want to do is try to get our data out of the case so you can see that I have a redus instance up here ready for us to go so we can say const CED data is equal to await client and this is a redus client doget and then we can just simply get this by the key now we want to say if C data then we want to return CED data if we don't have the data in our case anymore we want to fetch the data and then set the data in the case and then return the data so we can say const data is equal to fetch data now I want to set this in the case so I can say await client. set and we could set this as Json if we wanted to but I think the simplest version is to stringify the result and just set it as a string so let's do that we're going to set it as the key and then we're going to json. stringify and we're just going to stringify the data and then finally we can return the data okay so let's go use this function here so I can say await fch with C we can pass in our fetch function and then I just want to pass in a key of have a look at what this function does it just fetches a list of Pokémon so I can set a key here of Pokemon now let's do this twice and we can open up our console here and if we have a look at the package.json you can see that we have some scripts for running our problems and for running our Solutions and then I'm going to run this function function so you can see here that our first fetch data call took 300 milliseconds but our second one took 24 Mill seconds this is a huge reduction in time it takes to run this function and we didn't have to go back out to the network so let's copy this down and we can see this running three times and then our second and third call should be just as fast so it's really cool that we can cache data so easily in reddis but I want to give you a little bit of a warning about caching your data so there's several issues that can come with caching it's not just a get out of jail free card and you need to really consider those issues so the first issue is that we're now storing a bunch of data in memory and this could get really expensive memory storage is going to be much more expensive than say database storage the other issue is that we haven't set a time to live on this data so as long as we are not calling this flush all then the data is going to live in a redest instance forever so you need to make a call about how long that data lives in the redest instance and that really depends on your use case and there's several other issues with cing that you need to be aware of before you just dive in and start trying to optimize your application with CES so let's have a look at this second problem here and the second problem is for us to build a rate limiter now if you're familiar at all with up stash you'll know that they have a package to do rate limiting in your serverless functions and that is incredibly good package and I would recommend that you use that package or a similar package before you dive in and start building your own rate limiter but for this example we're going to learn more by doing it ourselves instead of just using a pre-built package so let's have a look at the problem statement here the do some work function is expensive to run and must be protected from abuse so we have this function down here called do some work and we can pretend that this does a lot of work so we need to throw an error if the function is called more than five times in a 5-second window so there's different algorithms that you can use for rate limiting and what algorithm you use is going to depend on what it is that you're trying to achieve we're just going to use a very simple algorithm here where we limit it by five times in a 5-second window so let's have a look down the bottom here and you can see that we're calling this do some work function five times and we're going to do all of these calls all at once so we need to protect this function here so let's first build a function and this is going to be an async function called rate limiter and rate limiter is going to take a key now this key could be anything it could be a user ID it could be an IP address it really could be anything to identify a single user making a request then we're going to have our Max requests so our Max requests here is going to be five and it's going to have our window size in seconds and again our window size in seconds is going to be five but you can adjust this to test different things if you like Okay so let's create a key so let's actually call this user ID and let's create a key inside of here so key and I'm going to give this a prefix of rate limit and then I'm going to give this a value of our user ID so again I've used this prefix here because if you have lots of different things in your reddest instance you want to know What that particular object is for and we're going to know that this one here is for our rate limiter let's say that we had a document for users we would prefix that with users and then we might have a rate limit object and a user object both having the same ID and adding this prefix just saves us from collisions there okay so to do this we're going to need to use a concept called transactions so to do that we can use the multi function so let's say const current is equal to client and we need to await this actually client multi now we need to execute multi and then we can call it dot increment and we can increment our key and now we can set an expiry on this increment so expire and I want to expire this in window size in seconds and now we just need to execute this transaction SU exe okay so what are we doing here well firstly we're just starting a transaction with multi then we're going to increment a specific key then we're going to say hey this key should expire in this many seconds and then we're going to get the result of that so if we hover over this you can see that it returns this really funky type here what it's doing is it's returning an array for each one of the steps inside of the transaction so we just need to test that the current value has a value so we can say if not current or current dot length is equal to zero then I just want to return false so if we return false from this function it means that our rate limit has failed and we should throw an error for the user okay so now let's iterate over our current so let I is equal to z i is less than current do length and then i++ and we just want to check that there's nothing in the one value so if current and we want to get the I value and zero then I want to return false so this is a little bit weird but essentially what we're doing is we're checking in the zeroth value to see if there's an error in there so if we have a look at the type again you can see in the zero value here there's going to be an error in that position if there's an error in any of the positions then we just want to return false and we can actually log out the error so console. error and we can say redus error and I want to console.log current the I value and then the zeroth value okay so now we need to check if the rate limit was exceeded so I can say const request count is equal to current 0 and one so we're getting the one position because remember in the zero position is going to be an error and then this is going to be a number and we need to typ cast that to a number because redus doesn't really know that or at least IO redus doesn't know how to infer that so we can say if request count is greater than Max requests and Max requests remember comes from the function arguments then this means that we have exceeded our rate limit so we can return false otherwise we can just return true so we could simplify this here into one line but I think being nice and explicit is a good idea in this case okay so do some work is going to be responsible for handling its own rate limit you could make a wrapper function say called function with rate limiter like we did in the previous challenge but I think it's okay for do some work to handle its own rate limit here just so I can show you how the rate limiter works okay so the rate limiter is going to return a Boolean value and we're going to call that Boolean value success so if we get a True Value it means that we're good to proceed so we can say const success is equal to a wait rate limiter now we can pass in a key here and this is just going to be some user again you can use whatever key you want it could be an IP address a user ID whatever so we want to give this the maximum number of requests so I'm going to say five and then the window size is going to be five so we're going to say if not success then we're going to throw a new error and then this error is going to say rate limit exceeded otherwise we're just going to return a string that says success so now our function down here to test it what are we going to expect so let's increase this to say 10 so I expect the first half of these requests here to succeed and then I expect the second half of these requests to fail so let's open up a terminal and we can find the command for this this is going to be problems rate limiter so pmpm problems rate limiter and let's have a look at the output here so the first five here succeed and then the second five fail so we can actually run this again by saving the file and you can see that we get the same output so if we run it a few times you can see that we're going to get the same output every single time the reason for that is because at the top of the file here we're going to flush all so we're starting with a clean surface every single time that we run this file so the third challenge is going to involve pubsub now this is one of my favorite features of Reddit it is so powerful especially if you're working in a Microsystems architecture or if you just need to publish messages around different services or even if you're creating a chat application or scaling something horizontally there is so many use cases for rdus pops up and it's extremely simple to get started with as well so let's have a look at the problem statement here when a user sends a message we need to publish that message out to other systems the other systems are all connected to the same Reda server so I actually have a video on this and in that video we create a chat application that scales horizontally and uses websockets and it's all connected via redis pubsub and I'll link to that in the description below and I'll also link to it in the read me that video is also sponsored by up stash and I think that's an incredible intro to using pubsub to scale something horizontal Al so to do this we need to create two instances of redus we need to create one for the subscriber and we also need to create one for the publisher so let's say const publish uh client is equal to new redus and then we're going to have our redus URL say const subscriber client is equal to new reddis and again this is going to take our redus URL so let's also create a constant for our channel name so I'm going say const channel name is just going to be my channel okay because we're using this in multiple places it's a good idea to have a constant so the first thing that we want to do is we want to subscribe to messages on this channel now we're going to do this allinone file but you can pretend that the subscriber lives on one server and then the publisher lives on a different server so let's say subscriber client dot subscribe now the channel that we want to subscribe to is going to be our constant of channel name now we want to have a callback here and inside of this callback we're going to have an error and if we do get an error so if error then I just want to console.log or console. error and I want to log failed to subscribe and then we can just add the error here as well we don't ever expect to see this value here I've been writing go too much so I wrot ER instead of error okay so now we're subscribed to the channel we want to start listening to messages on that channel so I want to say subscriber client Doon now now we can give this a string and we want to say message you can see there's lots of different strings you can give this here the one that we want is message and again this takes a callback inside of that callback we get our channel so what channel did we receive this message on and we also get the message so this channel here is important let's say that we had a chat applications and we're sending messages on a channel we'd want to prefix this with chat and then my channel name so let's say that we had a live leaderboard and we're publishing messages out to a client we would want to use a different prefix here like leaderboard so I'm going to leave this prefix here just to remind you you should always be prefixing these Channel names and prefixing your keys and now we just want to console.log channel and message so console.log got message and we can put in our message on channel and now we can put in our channel name so that's our subscriber done let's start publishing messages to this channel now I want to publish messages on an increment of say every 2,000 seconds so I'm just going to set an interval now this interval takes a callback and then it also takes a number and this is the milliseconds that we want to publish this message so I'm just going to say 2,000 and this means that it's going to publish a new message every 2 milliseconds and let's move our publisher client down so we can see all of this together now I can say publisher client dop publish now I want to publish to my channel actually that's going to be channel name and then I just want to publish a message so I can say hello from publisher and first what are we expecting to happen here so we're subscribing to the channel and then every time we get a message on that channel I'm going to log out that message so every two seconds I expect to see this message here so let's test that pmpm and you can see got message from publisher we got it again let's also add a timestamp here so we can say new date we can save that now we also get our time stamp here and you can see the time stamp is changing by 2 seconds every single time so it might be hard to imagine why this Pub sub is useful so I'll give you an example of how I've used this in a previous company so we had a microservice architecture and in one service we were producing events and we wanted these events to be published out to a user interface so what we did is the user interface talked to a graphql server and that gra server was listening to a reddis instance and when it got that reddis instance it published out to its subscriber and that went out to the client so we published the message from the server that was producing the events the graphql server would listen to those events and then it would publish out to the client so that is just one of many ways that you can use redis pubsub okay so let's move on to our fourth problem and if you've made it this far congratulations you probably know more about Rus now than a lot of other developers so thank you for sticking around so in this problem we're going to store users in Json format with their ID as the key we now need to write a function to fetch a user by the email so let's have a think about what the issue here is so we need to store them in Json format so so we can't use our hash function because when we get the user we need to get the whole user you could do this with a hash function but that's not the solution that I had in mind so we're going to store the user by their ID but we also want to be able to get them by their email address so if we have a think about how a regular database would do this so if you insert a bunch of users into a database you can then query those users by the email address but straight out of the box that's going to be really slow how you can speed that up is by using indexes and if we have a think about what an index is essentially it is just a key value pair so you're going to have a key and the key is going to be the user's email address and then the value is going to be the ID you can then do another look up from there so essentially what we want to do is to create an index so we have this array here and this array is just creating 10 users for us all with an ID a name and an email address now we need to insert these users and then we need to find the user by the email address so we need to fill out this function here and then we're going to call our fine user by email and we want to log how long this took so let's call client. hset so we're going to be setting Json and I want to set a key here of users and I want to use the user. ID and then I just want to set the value of user okay so we have all of our users in the database and nice approach to this would be to get all of those users back out look through that list of users find the one with the email and return it but that's going to be really slow we can be a little bit more clever what we can do is now we can set an index so I can say client. set and because we know we want to look them up by email we can say users and we can call this email idx and this just sign ifers that it's a index of email on the user document now we can give this a key of user. email now the value needs to be the user ID so user. ID it might be tempting to store the whole user object in here but what we want to do is we want to reference back to this user object so if we change this object here we can still use this index okay so now we need to get our index back out so I'm going to say const index is going to be await client. getet and we want to get this value here so let's copy that and we just want to use the email address and let's actually call this user ID so now that we have the user ID we can get the user back out of of the hash so I can say const user is equal to await client. HG all we want to get them by the user ID but we set the user ID like this so let's copy that and now we can replace this string here with user ID and then we can just return the user so let's go to our package.json and we can copy the command here to run this and we can run pmpm problems find by and you can see that we found the user document here by this user one example.com and this took 200 milliseconds so the final thing that I want to show you is transactions in reddis because these work a little bit different to how a regular database would handle transactions so you don't have a roll back for example because redus is focused on performance and that would really hinder performance so we have this example here where we're going to set a value of should fail and we're just going to set that to a string of integer now we're going to use the client. multi so this just means that we can start queuing up some tasks for reddis to execute so reddis is going to guarantee that it executes these tasks in order Al together so we're going to queue up the first task here to set a key and a value of one we're going to set up the second task and this is going to increment our string so this one here is going to fail and then our third task here is just going to set another key then finally we're going to execute this pipeline so let's go to the terminal and have a look at how this is going to work so you can see here we get value one so this is value one and it's actually been set to Value one we get value two and this has been set again to Value two then we get should fail and this is not an integer so did fail didn't execute it didn't increment this value here it just remained as the last value so this is going to work differently to how you might expect regular transactions to work but transactions in redus are really helpful when you want to queue up several tasks to be executed Al together in a synchronous way so that's the basics of redus I really hope you can walk away from this video and be confident that you can recommend where R should be used and how it should be used thank thanks again to up stash for sponsoring this video and if you liked this video please let me know in the comment section below how you plan to use reddest and make sure you leave a thumbs up thank you for watching and I'll see you in the next video thank you for watching and I'll see you in the next video

Channel: TomDoesTech

Views: 3,405

Keywords:

Id: mofZIitXMvU

Length: 41min 47sec (2507 seconds)

Published: Sun Mar 10 2024