NestJS Microservices with RabbitMQ and Docker

we're going to use Docker to containerize our Facebook Messenger system we're going to explore volumes and how they can be used such that our data can persist and survive container restarts we're also going to take a look at setting up our development environment such that we have hot reloading for our sjs files [Music] we're going to dockerize the sjs API Gateway and also the auth micro service that we created locally and we're going to move them each into their own container we're also going to have rabbitmq in its own container and allow them to communicate through that server rabbitmq is going to have a GUI for it the dashboard admin page and then we're also going to connect our database to our auth server and we're going to use type orm for that and then we're going to have postgres in its own database in its own container and postgres is also going to have its own admin dashboard foreign [Music] let's take a quick look at what we're going to build before we get into it so here we are in Docker desktop and we can see that we've got a bunch of services here in the previous video we created the rabbitmq service but we're going to go ahead and we're going to connect our API Gateway up with our auth service and we're also going to use postgres and there's also a postgres graphical user interface so let's just have a quick look at that so we can see here that we're logged into the postgres database here and this is handy because we're going to be able to see all our tables we can see that rabbit mq is still up and running and we can come here and we can see that our application's still running so we've also we're going to look into volumes and that's going to allow us to not only persist our data so that it provides a container destruction but it also makes development a lot easier because we can have hot reloading so just to demonstrate that if I click on visual share code and I get rid of this exclamation mark and I save the file there's hot reloading there and if I come back here and reload the page we can see that the exclamation Mark's gone so let's build this so let's jump back into Visual Studio code and let's start by looking at the docker compose file so so far we've set up the rabbit mq we've done that in the previous video so what we want to do now is we want to set up the nest.js service and of course this code's all in GitHub so you can link the link is going to be in the description below so you can just copy and paste that if this is going to be your first video I would suggest watching the first one if you're following along with the series but perhaps you just want to you're interested in setting up Docker for an sjs postgres and then using micro Services as well so rapidmq so let's just have a look um we've got the rabbit mq micro service going so let's get the API service going and actually let's start by getting our auth service so here we can see we can have this auth service here this is the auth micro service and what we want to do is we want to refer to a Docker file this Docker file is in the auth folder and that is in relation to this current route directory here so if just to minimize that we can see that the docker compose it's in the root directory here and the docker file is going to be in apps and then it's going to be an auth and then it's going to be in Docker file so let's just take a quick look at the docker file so it's relatively simple and this is for development only um so basically we're just grabbing the node image from Docker hub and then we're going to have a working directory so the working directory is where we're going to be in the docker container so by default there's a user folder and it's customary to put a source app to as a folder where the application is going to be so this Docker container is going to have this Source app folder and then what we want to do is we just want to copy and paste the package.json file over and I've got packaged star.json because it's also going to copy and paste the package lock Json file over and that's going to go into the Container so basically you can imagine the container as a separate server and then there's this folder here we got the package.json file which means we're able to run an npn install they'll create the node modules on the container and then we can go ahead and we can just copy all of our source files over into that so that's going to run here when we reference the docker file and then we're going to have some environment variables now we're going to set that up in a moment um so one thing to note though is that in the environment variables we need to change the rabbitmq host from a local host to rabbitmq colon 5672 and that's because we're no longer having our Nest JS servers running on our local machine they're going to be interacting through containers so we just need to make that adjustment there so if we come back to the Docker compose file um we can see that this service actually is going to depend on rabbitmq of course we need to grab an mq to be up but we also need postgres to be up as well because the auth service as you recall is going to connect to the database to you know have all the information relating to users um so we'll see how we can connect to that in just a moment and then we have this volume here and this first volume this is a bind Mount and a bind Mount what it does is it takes so we got our current directory here so that's everything in here um and if we make any changes we want those to propagate to The Container as you saw at the beginning of the video so we can bind that to this user Source app file and note that's the same as the working directory so what we've done is this project is essentially going to be on a container but we can bind all of the files that are in this directory to that container thus if any changes happen here they'll be reflected in the docker container and then through the ports that we're going to have in the API Gateway that will allow that to be seen on a client so that's the bond Mount um then we have an anonymous um an anonymous volume and an anonymous volume is basically um it's it's going to be on um in the container but in an unknown location and we don't really need to know the location of where it's going to be um ORS we care about is it saves the data so in this case it's not too serious but this becomes more useful when we look at the database because if you have a database and you have data you can imagine if you want to redeploy um you want you won't want your data to be lost but yeah it's handy to have these node modules in here um it's not necessarily for the node modules themselves although the no mulchers files can be quite large um it's more that with volumes in docker if you have a more specific file path so this particular volume or buying map is user Source slash app this is more specific the more specific one takes precedence in docker so what that means is if we make any changes to our node modules in here they won't necessarily be reflected um so we can just re-run things um so if we want to start the container we can see that this Docker file it's done all of the getting the files onto the container and installing the node modules we still need to run the container so it's an sjs server so we can just run npm run start Dev and then we can just reference the auth um and it should be noted that if you're in uh production you might also want to have a command in here which copies um all the guilt files it builds the files for production and then there's a command to run the built one that's relatively trivial um we can add those in at a later stage because we're just focusing on development but the important note there is that this command will actually overrun um a command you might see here oh went into Tire so it's possible to have something like this a command and then you might be running say node and then you might want to point at um some sort of this folder for example um we have Micro services so it might be a little bit more specific so it might be this apps API or auth slash main and then you can run that command um but we can just we don't really need that for what we're doing we're just doing development purposes but just saying if you do want to have production you can just add that extra command in there and then when you're working in um development mode this command will overwrite that command so that's essentially how you set up an sjs server in docker so we can pretty much copy this for the API Gateway as well so if I just paste that in there we can see that we've got the similar sort of thing we've got the API the context is this root folder and then if we go to the API Docker file so let's close this one now if we come to this Docker file here we can see that it's doing essentially the exact same thing um it's one thing to note is that this is actually going to be in a separate in its own container um so if you look into Docker desktop here we've got the auth as a container and then we've got the apis container it is possible just to demonstrate this that if you look at the PWD that might be a little bit small there but um we can just do a quick LS and essentially this container is basically like a server um and we've set the working directory to here and we can see our files in here so that's all I really want to talk mention about that um because just to make sure we're not getting clashes between the two that they are on separate servers there so we can go ahead and close that um now one thing to note is about this first an sjs service this is a micro service um and it doesn't expose a port and we're going to connect to it through the API Gateway which does have a port so if we just take a look over in the auth module here um sorry not the auth module the API Gateway one of the main files here what I've done here is I've just listened to the port 5000 I've just changed that from three thousand to five thousand um that's just to avoid basically I know I'm going to be working with react um and even if you're not um you might I might have some other um Nest Js um micro services that a that are hybrid microservices that both expose a poor end also are a micro service so I'm sort of in anticipation for that I've sort of changed the port um so you know because you don't want to have two ports on 3000 um so that change has been made there and one one interesting thing is when I was doing that I accidentally well not accidentally but I changed the port to 6000. and then I was getting like an error and it turns out that 6000 is actually a reserved Port um so there's a bunch of Reserve ports um and I think you know you need to go into Google and just type in the reserve ports for your operating system um but it's just an interesting side note that um if you change the port to some random numbers um and then something's not working you can just check that it's not a blacklisted port um so back to our Docker compose file we can see that we've got this port here um so I've just mapped the port to 4 000. um from the 5000 to the four thousand um because you know we start at 3 000 and then just added a thousand there you can make it any number you want basically as long as it's not a reserve port um and then here we're referring to the end file now once again this is depending on rabbitmq and rabbitmq has this uh Revit mq host variable here it's also going to have some um well in the case of the auth API or service it's going to have the postgres dependency there um the API Gateway it won't need that um so we don't necessarily need to have that postgres dependency but we do have this auth dependency and the reason for that is we want to start our micro Service First and then after we start our microservice then we want to start our API Gateway and that just ensures that everything is configured properly um so we can have that dependency there now once again we have the same sort of volume with the same justification as before um so we don't really need elaborate on that again uh it's just to make sure that the changes here reflect in the docker container and then they can be accessed and then we're just running this command here to start Dev but for the API this time so one thing to note is we got this Docker ignore file and basically we don't want to necessarily copy the node modules on our local host machine over so we've got the node modules on our host machine and then we've got the node modules in the container um but they're sort of two separate things in a way they should be the same unless you install a new package in which case you want to rebuild but basically we don't want to do that in the content like as we're building it in the docker file because you know that's a bit too long we we run the npm install in in the um file um so in the container so there's no real need for that so we just Blacklist that in the docker ignore file we also don't want the disk folder or any of these environment variables here so we've got this wildcard.env all my files are just dot Envy but uh sometimes well I'm just using this one shared EnV but sometimes um we might have a more specific one so for example we might have a postgres.env or a rabbit mq EnV in the future um so that's the docker ignore um so I guess what we want to do is we want to set up the postgres so let's just uncomment that basically this is quite simple there's just a postgres image that we can use um and then we can refer to some environment variables um and this is just going to be mapped from the port 5432 that's just the default port for postgres and then that's going to be in the container so we're going to map that um to our host machine so then we can access it on our host machine but the environment variables this is where most of the stuff's been set up here so we can come into here our environment variables and again you can split this up into separate files but um for now we might refactor this later but for now this will be good enough so basically we've got the postgres user which I've just set to user we've got the postgres password which I've just set the password and then we've got the postgres database which we're calling messenger um so these three here uh these sort of pertain to this postgres uh environment file because that's what they'll be looking for and there's actually another one um because we've got the GUI as well so the last one here is the postgres admin service now this comes from an image here D page PG admin 4 and this is obviously is going to depend on our database so we need our database to be running before we can view it so we need to be able to see that um so we have this depends on there and once again we're referring to the environment variables now this PG admin is on Port 80. so I've actually made up this random sort of or semi-random number one five four three two um it's semi-random because it's sort of based on the fact that the rabbitmq documentation they have their server on Port 5672 then the GUI They just added the one in front of it so I had an analogous thing here where um this port was 5432 so I just added a one here uh but of course this could be any any report you want that's not reserved um so yeah so that's basically how you get the postgres um container up now of course we haven't actually connected to the container and that's where this variable comes in handy so these variables help with the actual um postgres itself because you're referencing these environment files and it's looking for lows um so you know the postgres service here has these files here uh sorry these variables here then the postgres admin panel it has these PG admin default email and PG admin default password so I just made that password this email to me whatever you want and this is what you'll log into to access the GUI but this postgres URI this is what's going to be used from our um Ness services to connect so basically it still uses the same variables because of course you know you make the database you're going to need a login with the same username and password um but it just has this string here so it's got like the host's name and it's got the particular port number so it's just this URI where it's postgres dot dot slash postgres user which would set here then the postgres password and then the name of the database after the um this thing here so that's essentially how that's essentially all the docker side of things set up and the environmental variables that we need uh I guess the final piece of the puzzle is to look at the type orm so I should probably mention that we are using type horn so type orm is an orm or relational databases so we're using postgres and it's a way to map the database to the an object and it also lets you perform a bunch of um you know so you don't have to write your SQL out manually you can have some predefined methods let's say so we'll see that in the next video but I've just added in this block of code here um and of course you're going to need to install something for this so let me just open up the terminal here um and basically what you need to install so obviously I've already done this but just to follow along we can run npm in store at nest.js slash top arm so that's nessa's implementation of type form and then we just need the typeform itself and then we need PG for postgres so you can just go ahead and enter that command in and of course this code's all on GitHub so you can just copy it over if that's easier um so basically you need to import the type orm module from that there and then in the Imports in the auth module we're going to connect to it so it's the auth module that's connecting it's not the API Gateway um it's just the auth related stuff so we have this type or module then we can have this four root async which takes an object and the object we're going to need the environment variable so we need to import the config module and then we can use this use Factory method which takes that config service and it's going to need the type of the database because type 1 can work with mySQL we can work with all swipes of relational databases so we just need to specify that the type is postgres um and then the URL is that um postgres URI that we've constructed here so that gets referenced there now and then one important thing to do is also you need to inject that because you inject it to be able to utilize it now right here I've got these commented here so essentially basically when you're developing um you let's say um and there's sort of an expectation that you might know a little bit about type orm um but if you're not that's all right it's basically you write a typescript file and it's basically the interface or the class that represents the database and when you type that in and save it if you have these properties on here like especially a synchronized true property here um that will automatically um you know update your database and that's fine for development purposes um but the problem is like let's say you have a table and in production and you have a whole bunch of Records in your table and then you want to add an extra column for example or you know you might want to modify something or or you know you might just want to change something essentially you want to be sure that you're going to save all that data um so in the next video what we're going to look at is we're going to look at type on migrations and ways that you can preserve your data but I just wanted to mention that here because if you're looking for the nest.js documentation when you're getting started you basically see the synchronized true thing and then it mentions um to not do that in production but the implementation details are type orm or it's on the orm side of things um so I thought that was a little confusing when I first read through the nest just documentation it didn't actually say that it's only when you dive into the orm sorry type orm uh that you become aware that you need to be able to do that um but if you're a no typeform that's great I sort of picked it up as I was learning this JS so um we're going to solve that issue in the next video so with all of that we've pretty much done what we've needed to do today so just to recap we've set up reloading we're connected the database to our micro services and we've got rabbit mq still running so thanks so much for watching and I'll see you in the next video cheers

Channel: Jon Peppinck

Views: 12,834

Keywords: webdev, app development, lesson, tutorial, coding, programming, full stack development, programming tutorials, developer, react, nestjs, nextjs, nodejs, microservices, docker, kubernetes, rabbitmq, kafka, containers, mircorservices architecture, node rabbitmq, nest.js, node, microservices training, microservices for beginners, nestjs microservices tutorial, best nestjs tutorial, nestjs tutorial for beginners, what are microservices, NestJS tutorial, learn programming, learn coding

Id: m_1ZoJSi0kI

Length: 29min 32sec (1772 seconds)

Published: Tue Dec 13 2022