How to Create a Cluster in Redis

hey how's it going justin here with redis just want to take a quick moment of your time to show you how to set up a cluster in redis now i'm just using a vanilla version of redis 6 nothing special and i'm just going to set it up in the terminal and i'm not going to do any kind of major testing or anything like that this is supposed to be a quick video just to set you up and get you up and running this is a portion of an exercise that is within ru 301 running runners at scale that's a course that we offer at venice university that covers high availability high durability scalability and extra monitoring that you can run while you're trying to set up a production level redis instance okay okay cool so let's get started so i am here uh in my terminal i'm just creating a little folder called clustering just so i can set up my little sandbox and i'm going to create a redis.conf file that i'm going to use as a template for all these other conf files i'm going to be creating so you'll see what i'm talking about a little bit but here let's start with redis.com and i apologize i have a very clacky keyboard right now because my trusty keyboard is broken so i'm waiting for a replacement so i have a very loud mechanical keyboard so i apologize so i'm going to start with port 7000 that's going to be our first instance of a node and then i'm going to set cluster enabled yes this is telling our redis server instance that it's going to be set up for a cluster configuration uh cluster config file nodes.conf so this is going to be a standard configuration file for the clustering uh we don't actually have to touch this within this video i'm just stating what the file name is going to be and then cluster node timeout oops timeout 5000. that's five thousand milliseconds until this cluster will be considered a fair or this node will be considered a fail um so if it cannot be reached within five thousand milliseconds then we're gonna trigger a failover and basically that's going to replace this node with another one if it was a primary then we'll take another replica and turn it into a primary and so forth so lastly i'm going to set the replication to append only so i'm sorry the backup so we're going to create a pend only file as we enter data into this node cool so save this and we're good to go so now we have a redis.com file that we're going to be copying and slightly altering into s set no eight eight different folders uh because one folder for every single redis node instance 7005 7006 7007 okay cool now um i will speed this up in post but for now but what i want you to do is copy this redistop conf file to every single folder um because we're going to be adding them so there we go so now do that from 7 000 all the way to 7007 and then let's meet back up okay so now i've copied this redis.com file to all of my different folders so each node instance will have their own redis.com file now we need to actually alter every single redis.com file to match the port that it's going to be assigned to so let's do let's see here vim 7001 redis.conf i'm going to be setting this to 7001 and you guessed it i'm going to be doing the exact same thing to every other uh redis.com file within every single folder so i'll speed up and let's meet up when we're done okay so now we have a copy of redis.com in every single folder and within that redis.com file the port number is mapped properly so within the 7007 folder the redis.com port is going to point to 7007. cool so now let's have let's have some even more fun let's go into every single folder and then start the server with the local redis.conf file now watch this okay so you see where it says running in cluster mode that's great that means that our redis.com file is being intercepted and ran also port 7000 i told it to run into port 7000 so it is running in port 7000. all right but there's more we're going to be doing this for every single node that we have so let's go to the 7001 folder and let's start the redis server there now we're going to do this for every single one so i'm just going to speed this up and you can meet me afterwards okay so now we have eight different servers running from port 7000 all the way to port 7007 next we want to actually connect them all together so to do that i'm going to do redis dash cli i'm going to call the cluster sub command suite i'm going to call create so this is going to take all of the different ip addresses and like basically the hosts and the port numbers of every single redis instance that i want to connect and it's going to bring them all together so i'm just going to copy paste every single one that i just happen to have right over here there we go it looks a little nasty but whatever and then recall cluster replicas one now this means that for every single primary that we're going to have i want to actually create one replica so if i wanted say two replicas per primary i would just change that to two so i'm happy with one and there we go all right so let's check out all this information here so the first four lines that begin with m those going to be are those are going to be our primaries now let's break down some of the information that they're giving us for every primary shard first it's going to give us a hash of that specific shard then it's going to give us the hosts and port numbers so the first one is going to be our 7 000 and then after that it's going to say slots and then in brackets 0 to 4095. those are going to be our hash slots that's going to be the location that is going to be assigned to that chart now i'm going to talk about hash slots and hash keys in a later video but this is really cool this allows us to reshard our keys higher or lower without having to worry about the quantity of shards so we're not actually distributing keys via shard number we're doing it with these hash slots that's really cool but i'll talk about that later i'm just getting way ahead so you see after that slots 0 through four thousand ninety five it's giving us an actual quantity of uh slots that are within that chart and we see four thousand and ninety six so that's going on for the top four primary shards after that with the replica shards let's look at the first one it's going to give us its own hash and then it's going to give us its host and port number so the first replica is going to be 7004 that's the port and then it's going to say replicates and it's going to give us a hash of the primary that it's replicating so i see here that it's replicating a hash ending in 68f7a so that's going to be actually replicating the first primary shard so that makes sense so i see it at the bottom it says can i set the above configuration well sure you can buddy all right it's doing some mathematics all right all nodes agree about slots configuration that's fantastic when the nodes agree i agree so it says all 16 384 slots are covered that's the amount of hash slots assigned to any redis cluster instance um let me actually try to jump in and see if i can play with it so redis cli uh dash p i'll connect to the first primary 7000 and i'm going to cluster mode of dash c and there we go redis cloud colon 7000 that's good to know let me just do a ping a pong let me check out this is my favorite command cluster slots this will give me all the information relevant to my cluster so let's see here i can see the first cluster covers 7000 is the primary 7004 is the replica like we just saw earlier and then it goes from 0 to 4095. and then these are all just based off of what hits first so these top numbers one and two and three and four those aren't in any any particular order so let's look up um from zero see here's zero the four thousand ninety five the next one in sequence will be four thousand ninety six to eight thousand one five and that's going to be uh for the primary the port ending in 7001 and it's replicas 7007 so we can see all the different information that's going on uh with this cluster slots command i really like it we can also do the cluster info command and this will just give us some top level information about our cluster so how many messages sent received cluster size no nodes any failures um just making sure that we have enough slots in there and that everything's okay um and you know as always set foo bar it's great it still works it's still a database so that's pretty much it um yeah so we just set up a redis cluster you can do many different things of this it's pretty endless this is a baby cluster so i would never actually create an even number of nodes within a cluster if you see my video on clustering and redis we'll find out why it's basically the split brain situation which is not good um so yeah if you like this if you want to learn more take our ru 301 running redis at scale class it's really cool um i talk a lot on it there's a lot of readings there's a lot of hands-on exercises where you actually build a lot of these out and in the end you'll have a great skill set of how to set up a redis instance for production that includes high durability high scalability uh high compatibility there's so many great things it's a really really great course i highly urge you to check it out um and if you ever have any questions add a comment on the bottom of this video also feel free to go into our discord channel or just send me an email or tweet and just you know let me know what you want to see all right thank you very much have a good day [Music]

Channel: Redis

Views: 24,519

Keywords: redis, redis cluster, configure redis database, redis tutorial, cluster, nodes, shards, database, NOSQL, command-line, distributed network, replication, primary, failover, scalability

Id: N8BkmdZzxDg

Length: 12min 2sec (722 seconds)

Published: Fri Oct 01 2021