Redis : The Basics for beginners

[Music] what is up youtube and welcome to another video in this video we're going to be taking a look at redis what it is it's key features and how to run it and then we're also going to be taking a look at how to write a small application that uses redis as a data store and if you like this video be sure to like and subscribe because in a future video we'll be taking a look at how to run redis in a highly available manner with ha and replication and then in a future video we're going to combine everything we've learned and we're going to run raiders on a service like kubernetes so without further ado let's go so what is reddish redis is an open source in-memory data store it can be used as a database a cache and a message broker it also supports a lot of different data structures but we're going to take a look in this video at just basic key value stores now redis is one of those types of databases that has the most types of use cases i've seen companies use raiders as a database for critical data so they need high availability and they need replication i've also seen companies use raiders purely as a temporary cache so you can run redis as a sidecar process to your microservice and basically allowing your microservice to cache data so it doesn't have to rely much on external api calls i've also seen companies use the replication feature of the cache to move large caches around to different machines and distributed systems radius also has the ability for pub subs so that allows different micro services to publish messages to raiders and then other microservices to subscribe to those messages and topics so if you're new to this channel everything i do is on github so if you go to my docker development youtube series github repo you can go down to a new series i'm doing on storage in that folder i have a redis folder and in here i have a readme this is everything we're going to be doing today so we're talking about docker how to run redis the configuration the security the persistence as well as writing an application and to put messages into raiders so be sure to check the links down below to the github repo so you can follow along so the first step in our readme points to running raiders using a docker image now for those who know me i like to use docker for everything that i run docker allows me to create like this universal run time and makes it really easy for me to run applications so i don't have to worry about the dependencies and what i need to install and it allows me to decouple my application from the host machine so it doesn't matter whether i'm running linux mac or windows so if we head over to the first step i have a link to the docker hub for redis and if we go over there we can see that redis has an official image and it has a bunch of different tags so today we're going to be taking a look at reader 6 and we're going to run a lightweight alpine image they also have very basic high-level documentation which is good to get you going so this talks about the security of readers how to use a redis instance so very simple docker run command how to get persistence working how to supply your own reader's config and a bunch of other interesting information [Music] all right so the first thing i'm going to do is i'm going to create a new network called reader so i'm going to say docker network create reddit that is because we're going to run redis on this network and then we're going to build an application and run it on the same network now usually when you're running raiders on something like kubernetes you need to make sure that your applications can talk to readers that's why it's important to run them on the same network you can see in my docker run commander i have a port that i'm exposing this is the default port in order to talk to redis now there's no reason for me to expose this port publicly i'm just doing it here to show you guys how to expose it but our application will be running in the same network so it doesn't matter if i expose it or not so to run readers we're just going to say docker run interactively we're going to remove the container when it's done its name is going to be redis it's going to be running on the redis network so dash net radius we expose a port and then we run redis image and we run the 6.0 alpine tag so i go ahead and run that and we can see redis is now up and running so when you do this redis will run in standalone mode it'll expose port 6379 it runs as part 1 and a bunch of logs showing it's healthy our radius is able to launch with a default configuration but i always highly recommend to run it with your own custom configuration now the way i'd like to start is head over to the configuration documentation redis has a very good document describing all the configuration and where i usually start is it has a self-documented conflict for each version of raiders so what i do is i go to the redis version 6 config and you can see here that it points us to a guitar brepo with all the the configuration values for version 6. this is the default config that we can use for version 6 and every single key value is self-documented using a bunch of paragraphs you can see it's all self-documented in the config file i highly recommend to read through the configuration file carefully before making any changes to make sure you understand the impacts of changing configs and what it means so what i've done in my git repo is i created a config folder under storage redis where i have a redis.com this is the redis configuration for version six and this is the exact configuration i got from github that i showed earlier so as i said earlier redis has a configuration built in but what we want to do is we want to override the default config using a docker volume and mounting in this redis config that i've showed here so what i'm going to do is i'm going to change directory to storage redis and then i'm going to run this docker run command so i'm going to say docker run it and i'm going to also pass in the name redis run it on the redis network but this is the critical change i've made i'm basically going to mount in the config folder this folder on the left here into the container and i'm going to mount it into etc readers i'm then going to start up the redis docker container and i'm going to run redis server as an entry point and one of the first arguments it takes is the location of the config so i'm just going to say flash etc readersredis.com this is the config file that i'm mounting into that folder so i'm going to go ahead and copy this and i'm going to paste it to the terminal so now you can see redis has started up and this time it started up with our own configuration so now we can go ahead and dial and tune every aspect of readers now that i have a redis up and running the first thing i want to talk about is a little bit around the redder security and raiders should basically not be exposed to public traffic usually by default this is because raiders instance does not have a password to connect so anyone can connect to that instance so what we want to do is we first want to go to our reader's config file that we created and we want to go and change the default password now i've documented this here this is the key that we want to change it's called require pass and then we want to set a password so if you copy that key and you go to redis.config and you go and control f and you go and find the value you'll see that i've already set this now to a super secret secure strong password you want to set this to a really strong password that all your applications can use to connect to readers so other than running readers with a strong password it's also worth noting that you should think twice before exposing readers to the public what most companies do when they run readers as like a database or a cache is run redis instances on a private network so your application can talk to it over a private network raiders does have tls support from version 6 as an optional feature but you would need to enable it during compile time so that means you would need to compile redis yourself using your own docker file and then turn tls on [Music] now the next thing i want to talk about is redis persistence now by default when you're running redis the way we've just run it it stores its data in memory that means when the container dies all of your data and redis will be lost so it's very important to mount your data to a file system or to a volume to persist that data now raiders also has a document on persistence which i've linked down here now the reason i've linked it is because there are two modes for persisting data and readers there's rdb persistence and there's append only file persistent modes it's very important to take a read through this document to understand the advantages and disadvantages of each mode at a very high level rdb mode provides a higher performance mechanism because redis will take a snapshot of point in time snapshot and write it to a dump file that we can configure we can then mount that into a docker mount to persist that data the append only file mode basically will log every write operation that happens to reduce it or append it to a file so that guarantees higher durability because there will be minimal data loss if our redis process dies but that comes with the performance knock now redis also provides the ability to run in both modes so in this demo i'm just going to show you how to run rdb mode how to run aof mode and just run it side by side now to turn on rdb mode basically what you want to do is come over to the raiders conf and search for dump and you'll see there is a key called db file name dump.rdb so you can change the file or location of this file in this config value this will turn on rdb and it'll basically start writing at various intervals to that file we can also customize how often we want rights to occur to turn on append only mode you can search the redis config file for append and you'll see a key called append only so to turn it on just say a pin only yes and then you can also specify a file name to write those right operations too now i would highly recommend to go through the persistence document to understand the pros and cons because there's also a section talking about what you should use and the reddest community i recommend for safety reasons that you try to use both of the persistent methods now if you don't care too much about your data you can just run an rdb mode alone that'll just dump to file every now and again rdb mode is also great to use for backups so if you want to backup your database from time to time you can just basically copy that file and put it into storage so now that we've gone ahead in our config and we've turned on persistence we've turned on aof and rdb let's go ahead and persist that data to a volume so what i'm going to do is i'm going to create a docker volume called redis and then make sure you change directory to storage readers because we're going to mount that configuration file we're going to say docker run we're going to call it redis run it on the redis network and we are going to mount the config file into redis the next bit we're also going to do is we're going to mount a volume so i'm going to mount that reddest volume we've just created and i'm going to mount it into slash data this is the default place where redder stores data and then i'm going to run the redis server image and i'm going to point it to that config so to run this i'm just going to grab this command and i'm going to copy it to the terminal and that's going to go ahead and write the data to that volume so basically if we kill this container and start it back up we won't lose any of our rdb files our rdb file will still be sitting in that folder as well as our append only file so now that we have our redis running here let's see what it takes to write an application that can access this redis instance so what i'm going to do is i'm going to leave this running in the background and i'm going to open up another terminal i also have all the steps listed out in this readme and i'm going to leave the source code in this folder as well so you can follow along so to keep everything nice and clean and separate the first thing i'm going to do is create a folder under the redis folder and i'm going to call it applications and what i'm going to do is i'm going to create a new folder under that called client and our application will be called client since we're going to be creating a reader's client now usually when i write example applications i like to use simple programming languages like go keeps it very small it doesn't matter what programming language you use you should be able to follow along since go is very much like pseudocode so what i'm going to do is i'm going to create a new file here called clientgo and we're going to write up some source code here to interact with reader so basically what i'm going to do is it write a small web application that every time we hit that application in the browser it's going to take a key out of redis and increment the number and write it back so it'll start at 1 and start incrementing it every time we refresh the page so i'm going to very quickly walk you through what that looks like so firstly we're going to define package main then i'm going to import a couple of dependencies that we're going to need for our application i'm going to import http functionality because i'm going to be running a web server i'm also going to import some logging library so we can log and more importantly here i'm going to import the redis sdk and then what i'm going to do is i'm going to expose a couple of environment variables to tell our application how to connect to radar so i have a redis host a redis port and a redis password then i'm going to define a default value for our counter this is the counter we're going to be reading and writing to redis then i'm going to go ahead and create a main function now in this main function there's a couple of things we're going to want to do the first thing is we're going to want to go ahead and establish a reader's client using the sdk and pass in our options so we can pass in the host the port as well as the password we basically tell it to write to the default database in redis and you can see here i've basically created a local variable called r and i'm just going to assign it into a global variable so that i can use this client in other functions and the next thing i'm going to do is define a router for our web server and then i'm going to also define a router method so i'm going to say on get and we're just going to take in the response writer the request details and parameters we're going to pass it down to a function called increment readers key this is a very simple function that's going to be reading the value if it doesn't exist it's going to create it if it exists it's going to read it update it and write it back and then what i'm going to do is just print a bunch of stuff to the terminal i'm just going to say that we are running and then i'm also going to start up my web server on port 80. now let's go ahead and define that increment readers key function so i'm going to create a new function i'm going to pass all those http parameters down and the first thing i'm going to do is go ahead and use that redis client to read the key the key's name is called counter so we're just going to run rdb.get and we're going to pass in the key we want we also have to define a background context which i can go ahead and define globally over here now in go we always have to assume that an error comes back and that we have to handle it so what i'm going to do is i'm going to handle an error the first thing is say if the error is equal to reddest.null meaning that there is no value from this key or this key doesn't exist so we need to define what we're going to do if this key doesn't exist and then we also have to handle the error in case um the error is not null so what what if there is some connectivity error or something that goes wrong we have to then handle that so i'm just gonna say panic if that happens and then in the else block we can basically handle the scenario when the key comes back so the incrementation of that number so we're just handling three possible scenarios one is the key doesn't exist the other one is an error has happened or there has been no error so i'm just going to keep this very basic so if the key does not exist so what i'm going to say is i'm just going to go ahead and say rdb set counter i'm just going to set it to 1 and then i'm going to say counter plus plus and that's going to be all if there's no error comes back and we have the key i'm going to go ahead and convert that value from redis that we've read at the top here and i'm going to convert it into an integer into our counter variable that we've defined i'm then going to go ahead and increment that number and then what i'm going to go ahead and do is run the rdb set command and write that counter key value back to that key so i'm basically writing the updated value back to redis and then finally because it's go we also have to handle that error in case that error is out and after all of that has occurred i'm just going to write two things one i'm going to write the counter to the console so we can see it and the other one is i'm just going to write it to the response http response so we can see it in the browser so now that we have our code how do we go ahead and build this now again i like to use docker as my build environment so that i don't have to install all this dependencies of go so i have a basic step here the first thing i'm going to do is i'm going to change directory to that client folder that we created and you can see our application is defined in here we have client go and what i'm going to do is i'm going to start a development container for golang i'm going to say docker run it i'm going to mount all the source code into the container into a folder called go source i'm going to set go source to the working directory i'm going to run this on the redis network since we want to interact with redis i'm also going to expose port 80 so that we can hit our web server in the browser and i'm going to run golang and i'm going to run 1.14 of that golang version so this is going to easily run my whole dev environment as a docker container so when i go ahead and run that we can see now i'm inside the container and i have my client go application right here so i can go ahead and build it now to build it i'm just going to say go build client.go so you can see how build has failed because it cannot find any of the dependencies so to get the dependencies go users get under the hood so because i'm running an alpine container here i'm going to go ahead and install git and with git installed we can now go and download our dependencies for the application so to start downloading our dependencies i'm just going to define my application as a go module and say go mod init example.com hello i'm just going to give it a name that's going to go ahead and create a go mod folder you can see that over here defining the version of go and the name of our application and then i'm going to go ahead and say go build client dot go that's going to go ahead and use the go dependency manager and go and find all the modules we've defined it it's going to go ahead and download it create a go module go mod file and a go sum file it's very important to then check those two things into git and we can basically pull all of these things into a docker file and create a container image so now that you can see our build has succeeded and we have this little client application that's been defined here so all i need to do now is go to my terminal and say dot slash client and now that's my application up and running now before we test this out let's put this all into a docker file so dockerfile allows us to universally package this up and all the dependencies into a container image which makes it really easy for people to run and it makes it easy to run in a service like kubernetes so what i'm going to do is i'm going to create another file in this client folder called dockerfile and here we're going to define what the dockerfile should look like but the first thing i'm going to say is i'm going to define a build image using the same image we just run for our build environment and we know that go uses git as a dependency manager so i'm going to go ahead and install that i'm going to set my working directory as source and now i'm going to copy in these two go files the go mod and the go sum file i'm going to copy that into the source directory and then i'm going to go ahead and copy the client.go this is our application source code into the source directory as well once we copied that in we can just go ahead and say go build and we can build that application and to make the application lightweight we can switch out to an alpine image very small runtime and we can go ahead and copy out from that build image we can copy our source client application this is the binary that we've built over here and move it to a folder called app client and then we can just start it up with a command entry point now to build that i'm just going to exit out of this container that we had before and i'm going to go ahead and say docker build minus t and i'm going to call this image at redis client v1 so i'm going to go ahead and copy that paste it to the terminal that's going to go ahead and build a container image it's going to compile the source code and put all the dependencies produce a lightweight binary and then put that in an alpine image that we can run now to run our application it's very easy once we've built it we can just say docker run minus it and we can run this on the redis network that we've defined before and then we specify a bunch of environment variables we specify a host of what the hostname of that redis instance is and then we also define a port and a secret password to talk to it and we're going to expose port 80 since our application is a web app so what i'm going to do is i'm going to take this command and i'm going to paste it into the terminal and now we have a docker application running on docker and we can go ahead and access it in the browser so if i go ahead to the browser we can see now we have um accessed our web app which has detected that there's no key in reader so it's gone ahead and added the first key as a value of one and as we increment this every time we refresh we're going to increment the value in raiders so you can see we've gone up to 20. so now let's test out the persistence so if i go to this terminal where we were running raiders before and i go ahead and control c out of it redis will basically terminate and then what i'm going to do is i'm going to run radius again by running the same command and mounting in the same volume and when we do that we can go back to our application and it should continue from the number it's left off so you can see i've refreshed it it's gone back to 21 and we've gone ahead and we can just continue incrementing the numbers so we've persisted our data successfully in reddit so hopefully that helped you guys get the basics of how to run redis how to provide your own configuration and the basics of persistence security and configuration of raiders and also how to develop a small application that accesses redis to write keys in a key value store now if you like this video be sure to like and subscribe because in a future video we'll be taking a look at how to do redis replication and run readers in high availability mode we'll also be taking a look at how to run redis using masters and replicas as well as the sentinel services and in a future video we'll be taking a look at how to run all of that stuff on top of kubernetes so hopefully this video has laid out the foundation and the groundwork for you to learn about the basics of redis so let me know down in the comments below how do you run redis in production and let me know what other sort of videos you'd like me to cover in the future and as always like and subscribe and until next time [Music] peace

Channel: That DevOps Guy

Views: 13,065

Rating: 4.9504132 out of 5

Keywords: devops, infrastructure, as, code, azure, aks, kubernetes, k8s, cloud, training, course, cloudnative, az, github, development, deployment, containers, docker, rabbitmq, messagequeues, messagebroker, messge, broker, queues, servicebus, aws, amazon, web, services, google, gcp

Id: L3zp347cWNw

Length: 21min 27sec (1287 seconds)

Published: Sun Aug 09 2020