Speed Up Your App: NestJS, Redis, and Docker - Here's How!

in this video we're going to set up a redis data store in our micro Services application so there's two real ways you can use redis you can use it as a in-memory data store or a persistent data store so we're going to be using it as a persistent data store so caching in general or in-memory caching is essentially if you have an API request it hits the database and you hit that API request again rather than making that database operation again we save that resource or that Network bandwidth by giving back the information or the data or the Json object that's already cached in memory but for our micro Services application there's some limitations to this approach so the limitations of this approach is that essentially we've only got one server if because we're using a social media application we're expecting huge volumes of messages which we're going to store in the redis data persistence and it means it's not scalable so we're going to use a data store because that can be offloaded to a Zone service its own micro service and then we can obviously scale that um so it's very horizontally scalable and the reason we're using redis as a data store um opposed to just our regular postgres database is a postgres database it's really good for standard information like user information and other relational information but the issue with a social media application is or in theory if people are using your application you don't have millions of people using it you're going to have millions of messages or hundreds of millions or billions of messages and you can't have all that in just one server um so the other thing is we're going to be using the redis in multiple servers so we're going to use it in online presence chat cause so we're also going to set up some Dynamic modules in sjs to be able to use it from a shared Library so some of the other benefits are a key value storage such as redis it's really really fast because we're going to need to do things like we're going to access our previous messages um and redis shines for its quick speed to access that in contrast to something like a relational database which really struggles with very large volume long tail information that we need so for those reasons we're going to be using redis as a data store and we're going to set that up in our microservice application [Music] thank you let's get the redis data store into our micro Services application so we can start off by installing our dependencies by running npmi for install and then we'll get the cache manager redis and the cash manager redis yet now if you look at the nest.js documentation you can see that it uses the redist store from the cache manager Reddit store however in the newest versions of the library there's some typescript issues when you're trying to access the store and then it sort of doesn't let you do it so rather than using a mismatch between the old version of this and the other versions of redis and stuff like that the work around what I've searched is basically using this cache manager redis yet package which allows you to do so so possibly in the future you know this package will get updated with the correct typescript and interfaces and stuff like that so unless you want to degrade to an older version and deal with any mismatch issues for the time being let's just go ahead and use this cache manager redis yet so let's just go back into Visual Studio code here and let's just start to see what we need so in addition to the cache manager redis and cash manager redis yet um we can go ahead and for the cash manager we can install as a Dev dependency the cash manager types there so that's all we need to do for our dependencies so let's think about how we can set this up so we want redis and we potentially want it in multiple services in multiple microservices I'm going to start off by putting it into the online presence service but we want to do this in the shared Library so then we can use it in the other services and for the actual redis because we're using it as a data store like we're storing messages in it and we want to retrieve those at a later date um you know we want to actually have some persistence and not just use this as just cash um but we're using it because it has a faster read write and it's more appropriate for social media chat application in compared to the overhead of something like you know the regular relational database or something like that so that's why we're using redis in addition to postgres and it also allows you to you know do some quick caching for other things so we're going to see that in the next video so basically we come to the docker compose file in the root of our folder structure and we can get the redis service here and we can use the default redis image from Docker hub and the port for redis by default is 6379 so we just go ahead and map that to our local machine from the container uh and then we go ahead and we set our environment variables so we're just using these global.env file which is also in the root so if we just come here we'll see that we have a redis pass which is just says the password Here and then we have the redis URI which we'll use to connect through our node application so it's just redis dot slash slash now they use the default for the default user a user's not specified but you can secure it with the password so I just have this variable here which refers to this password Here of course you would want to make that a little more secure and then at redis and the port number which we got is six three seven nine here so those environment variables are all we need so if we return back to our docker compose file another thing we want to do is because we want to persist the data we want to have a named volume so in a similar way to how we have the postgres database we're going to have the redis stuff in here so we have the dot slash Reddit data folder here that we're mapping from our system oh sorry our local machine and if you look at the documentation it's in the data folder in the container itself so because we have a password um now if you look at the postgres connection um you know it has the environment stuff in the environment file and then you make the connection but with the redis um when you're in the docker container essentially and let's see if there's any other commands here oh yeah so there's this command here to run the start div now you can just run that from the container um but in this case here you might be familiar with using you know variables uh if you're not using if you're just using a regular Docker for example um and you want to enter the shell so that's why we have this slash bin sh here because from the shell rather than the terminal we want to uh you know have this redis server um you know command which we have access to from within the redis container um in which we can have this flag here to require the password so you know when the when this runs the Reddit server it's expecting a password and then in here we can have this dollar Sun dollar sign to access the environment variable so redis pass is what we set up in the environment variables um so that will just mean so when we open our uh you know when the server runs um you need a password to connect to that and it just adds an extra layer of security there um so the final thing we'll do here is we're just going to experiment by using it in the present server um so in that particular service we'll see getting and setting uh some object data and seeing that the cache actually works so that's pretty much the setup for Docker and the environment variables let's take a look at setting up the redis module so again with if I just collapse this and just reopen this one thing I've done is I've just created these folders here so there was some files in the root so there's some modules and services and stuff like that I'll just create some folders here and clicked and dragged them and click the update to update Auto of course you can pull the repo as well but it's pretty trivial but essentially in this modules folder in the shared Library I've created this redis module so the redis module is just the reddest module with the module annotation and what we want to do is we want to have some sort of Dynamic import because we want to and Port this into the presence module but we also want to import into other modules so rather than repeat their coding and similarly to a previous video that we've set up we can just go ahead and set this up so the actual um there's a sort of interaction between the cache and the redis so the cache module is what nest.js provides you and then the that's actually cashing out the box if you just use the standard nsjs um you know stuff but if you want to do more advanced stuff like have persistent storage or um other types of operations like deleting things and stuff like that you know you want to have more control over things so you need to implement uh like a caching or data persistence um you know technology such as redis which sort of sits over the top and works together with that so you will see how they work together um so basically we want to use the cache module and then we want to register and sync and we call that method so when we import this module into another module um you know they'll just register this and then we have this use Factory here it needs the config service because we need to use the configuration the environment variables to connect to the redis database and consequently to use the config service we need to inject the config service so we can go ahead and we can import the Reddit store from Cache manager redis yet and then we've what we're returning here is this object here and it has this store attached as one of the properties and this is an asynchronous operation because you need to await for Loretta store connection so we have the URL and then we can just use the config service to get the redis URI from the environment variables now this TTL in the later versions uh they use milliseconds I think in older versions I used seconds so you just need to look at which package you're using and which one is what you need I think five seconds is the default uh or five thousand milliseconds and of course you can sort of overwrite this or um you know change this on a case-by-case basis um but I'm using five seconds just so I can easily display at the end of the video the caching functionality actually working um so and then we set is global to true because if we don't have is global to True um when we you know import into the module in the presence module we want to access that in the whole uh you know the presence area so or the server so in the controls and all that sort of stuff we don't want to inject it each time we can just reference it um so we set it to Global um we want to get the redis service which we're going to create and that's just going to expose some methods here so we can just have um in this module the stratus module that we're going to import we can include the Reddit service and we can also just go ahead and Export that so we can consume that service in the other modules um so if we take a look at the Reddit service it's actually pretty basic so this is just a regular service you'll see um and there's just some imports here now you have to inject it and you have to use this cache manager um you know constant which comes from sjs common and then you just have this um you know this sort of catching service which is based on the type cache and it's important that you import this from Cache manager because otherwise it will interfere with the regular cache which is a different type and then we can just go ahead and expose the properties so you know redis is a key value store so you can you know get you know so it's sort of like local storage you get something based on a key or you set something based on a key in a value so when you're getting something um you just go ahead and get it and it's an asynchronous operation because you need to get it from it's really quick uh it's typically oil one um but um you know depending on you know how complicated Things Are and your use case uh the swamp question might vary a little um but you know way quicker than regular databases uh and it's pretty much as good as it's going to get um so yeah so we're going to expose this get method um and then based on this Reddit service here for the cache we can access that and then there's a get method and then we can just get something by the key um similarly we've set you set the key to a value we don't know what the value is it could be a number of string object whatever um redis handles setting that and then we just go ahead and get that key oh we set the key and the value that corresponds with that key and then this last operation there's more operations but these are just simple ones uh is deleting the key um and that's just a deal so uh you know pretty basic stuff um and then it's just important to remember that we want to use this in the presence module so we just need to import that redis module um and then if we come to our presence controller one thing that you'll note here is well the first thing you'll note is what we want to do is we just want to test our cachings working so we just have some basic method here so I'm in the get presence command here one edition you'll see here um is that I've got this annotation use interceptors for the cache interceptor and the cash Interceptor is coming from sjs um so basically we've set up our catching and we're using redis so whenever we cache things um the redis will be you know the particular technology used for our caching and that's just going to work and we'll prove that in a second um so essentially yeah we have this cash interceptor so this particular method will cache so that means essentially you know if you if you make a API call multiple times and you have something in the cache you can essentially get that so if you make a second request you don't have to make a database cool and then you just get the information back um so essentially what I'm doing here is well first thing is I'm getting this Foo so Foo is just like some placeholder method here so I just got a service here that we're injecting into the presence controller and it's going to call this service if there's nothing in the cache so I've got this console log to see not cached and just returns a simple object with the key Foo and Bar as the string um as the value so essentially um you know this would be a you know asynchronous operation um but technically it's not at the moment um but we will make it one shortly so this away is redundant at the moment but we'll be quickly adding that in in the next video um so you just inject the redis service so we can access those redis operations um and what we can do is when we get through what we can do is we can just set for the key Foo for example and we can just set that object there um and then what we can do is we can have another variable um so I've just got this full as well a bit of overloading but you know obviously it's just temporary um so this is going to call the Reddit service and then we're going to get Foo now it might not be in there in the first place so if it is there we can say that it's cash and then we can simply return the value um otherwise you know we just go ahead and return things here so let's just go ahead and save that um so let's see if all that works so yeah let's open Postman up um so we can see that I've got Foo and bar back and then I'll click it again and click it again now there's no actual data operations happening so it's all happening really quick anyway um but if we return to visual studio code and we open up our Docker terminal we can see that when we make the API call oh sorry when we make yeah when we make the API call in the first place um we need to get Foo from redis we see that it's not cached um so because it's not cached we need to set get food from redis and then we made another API call so we get it and then it's cash so it's not actually you know hitting that again um and likewise and because I waited five seconds um you get to see this not cached again so let's just see this I'm going to click this three times well two times quick and then when I wait five seconds so one two three four five and then the third time it should be reset so we can see that we get it not cached cash not cashed um so that's great now we've set up the reddest data store and we've got um you know ability to Cache things quickly uh it's going to come in handy for the next video which is going to be about online presence um because where there's going to be lots of data associated with um you know a user and a user's friends um and if they're connected uh it's also going to come in handy for the chat messages um so it's great we set that up um so yeah please subscribe to my YouTube channel and we'll do the hybrid micro services in the next video cheers

Channel: Jon Peppinck

Views: 8,554

Keywords: docker, redis, nestjs, typescript, javascript, nodejs, nest, nest.js, containers, devops, microservices, typeorm, docker compose, app development, redis tutorial, what is redis, nestjs redis docker, redis tutorials, nestjs videos, docker class

Id: N9Ld96ywiW8

Length: 23min 58sec (1438 seconds)

Published: Wed Mar 01 2023