Docker + Node.js/express tutorial: Building dev/prod workflow with docker and Node.js

what's going on guys so in this video i'm going to show you guys how we can set up a workflow for developing node.js and express apps within a docker container so the first thing that i want to do is i want to set up a quick and simple express app that we can use for demonstration purposes and the reason why i didn't actually prepare the the express app ahead of time is first of all this is going to be the quickest and simplest of apps we'll be able to knock it out within like a minute um but i want you guys to focus on these steps that we need to actually create this app because they're kind of important because we're going to recreate those steps within the docker container as well so i've got my project directory i called it node docker already here and i've got it opened up in vs code so you know with any other express app the first thing that we have to do is get our package.json file so i'm going to do an npm init and that's going to create our package.json and since we are creating an express app we're going to have to install express i'm going to do an npm install express so now we've got our dependency uh the next thing that i want to do is also focus on the fact that we did uh create our node modules folder so that happened when we installed express and the final thing that we need to do is actually create the our application so i'm going to create an index.js file and i'm going to import our express we'll do const express require express and then we can do const app equals express all right then i'm going to specify the port that i want my express server to live on so i'm going to do const port so this variable is going to represent the port that it's going to be listening on and i'm going to set it equal to process.env.port pipe3000 right so if you don't know what this line is it's basically saying that if the environment variable called port has been set we're going to set this variable to that value however if it's not set we're going to default to a value of 3000. usually common configuration and then we can do an app.listen and we'll listen on that port and when our server comes up we can say console.log listening on port and then we'll pass in that variable all right so now the last thing that i want to do is set up a quick route for testing purposes so i'm going to do app.get and we'll just say the root root path and then we'll do rec res and so if anyone sends a get request to this path uh we're just going to send back a simple response which is going to be we'll just send an h2 that says hi there and i forgot the comma right here all right and so that's our entire express app uh i'm going to start this app real quick we're going to do uh node and then index.js and it looks like i forgot to save probably all right so it looks like i already had another express app listening on port 3000 so i deleted that so now if we do a node index.js it should start it's going to be listening on port 3000 we can go to our web browser real quick and we can just go to localhost colon3000 and when we do that we should see it say hi there i'll zoom in for you guys so that you can see that and so that confirms our express app works so we've got our dummy express app so now we can go ahead and get started with actually integrating our express app into a docker container and setting up a workflow so that we can move to developing our application exclusively within the docker container instead of developing it on our local machine like we just did so now that we got our express application complete or the demo express application uh let's work on setting up our docker container now for this video i'm going to assume that you already have docker installed on your local machine if you don't go ahead and do that now just follow the instructions on their website it's fairly straightforward just make sure you follow the directions for your respective operating system but once you have docker installed on your machine head on over to hub.docker.com and i want you to just search for node within that search bar right there once that's finished loading you'll see that the first result is the official node docker image so this is a public image provided by the node team and it's a fairly lightweight image that basically has node already installed for us so that we don't have to do this ourselves and if you take a look at the documentation you'll see that we've got all of the various different versions that it supports so you can get version 15 version 14 all the way back down to version 10 if you really want and it's got directions for this image or any specific things that are relevant to this image when you want to deploy a container from it now this docker image right here is not going to have everything that we ultimately need for our application because the whole idea behind docker and an image is that the image is going to have every single thing that's needed for your application to work uh and so for our application we obviously need to get our source code into the image and we also need to get all of our dependencies like express and the other dependencies that our application may need so to do that what we're going to do is we're going to create our own custom image and we're going to base this custom image off of this node image so we're going to take this node image we're going to copy all of our source code into that node image and then we're going to install all of our dependencies like express and then that final image is going to have everything that we need to ultimately run our application so let's get started on doing that now okay so let's get started on creating our own custom image to create a custom image we need to create a docker file so the docker file is just going to be a set of instructions that docker's going to run to create our very own customized image so let's create a new file and we're going to call this docker file with a capital d now in our docker file it's going to have a set of commands that docker is going to run to create our own personalized image and the first command that we always have to do is we have to specify a base image so when you create a custom image what you're ultimately doing is you take a base image or a known image of some sort doesn't matter where the image from it can come from your own docker repository it can come from docker hub it can come from anywhere you just need a image that docker has access to and we're just going to tweak it a little bit so that uh it contains all of our source code it contains all of our dependencies and things like that so in this case you know we already found our um our base image it's going to be this node image because that's really the only thing that we need to run right we just need it to have node and anything else to run node so we're going to take this as our base image and the instructions here is going to show you how we can ultimately use that so here you just specify node and then you can do colon and then the specific version you want if you don't specify the version i forget what version it grabs but if you read the instructions it's probably gonna say it grabs like version 14 or something like that or maybe the latest version right so what we do is we do from and we'll capitalize from and then we say node because that's the name of that uh image or our base image and then we can do colon and then the version we want so i'm going to specify version 15 not that we need version 15 we can run 14 or really any version we're not doing anything specific to any new versions i just want to show you guys how to actually specify a version uh so that's something that's absolutely needed uh the next command is technically optional but it's still recommended and that's called the work dirt command and i'm going to set that to slash app so what this command does is it sets our working directory of our container to be the slash app directory within the container so i know this slash app directory exists in the container because i've run the node image and i know it's got a slash app directory but setting the work directory is really helpful because anytime you run a command when you set the work directory it's going to run that command from this directory so we can put all of our application code in slash app and we can run node index.js on slash app uh and it's going to run it uh automatically in slash app without us having to specify uh so the working directory just uh is the directory where you run all your commands and it's also the directory where if you copy any files to your container it's going to by default send it to this directory so this is just recommended but it's not technically necessary now the next thing we want to do is we want to copy our package.json file which is this file which contains all of our dependencies and anything else that we need and we're going to copy it into our docker image so let's go to so we can run the copy command and so we copy the path to the package.json file which is our current directory so we just do package dot json and then we have to specify the directory we want to copy it to in our image and so i'm going to do this dot which means the current directory and the reason why i do that is because we set our working directory up here to be slash app so when you do dot it's going to assume the slash app directory we could technically also just do slash app same result but since we have the working directory set we can just specify it relevant relative to that directory uh the next thing is once we have our package.json remember it's got a list of all of our dependencies we want to actually install our dependencies so we want to run an npm install so in our docker file to run a command we can do run and we can do npm install all right so we now have our package.json file copied over and then we run npm install now we've got express installed for us the next thing we want to do is and this part's going to be a little bit confusing but we're going to copy the rest of our files so all of our source code everything else into our docker image so we'll do a copy again and here instead of specifying a specific file i'm going to say the current directory so it's going to grab every single file every single folder within our current directory and we're going to copy it to just like we did here where we just did a dot which is going to send to the slash app we're going to do the same thing so we can just do dot or dot slash same thing and right here you might be a little confused you might be wondering well why do we copy the package.json first uh and then copy all of the files over do we even need this step before if we're going to copy all of our files this should copy our package.json and it does the reason why i split it up into two different steps is a little bit of an optimization technique so let me explain how docker images actually work when you create an image from a docker file it takes each one of these steps and it it treats it as a layer of the image so you could think of an image as just basically these five steps or five layers so once you build all five layers you have the ultimate image or the final image so this first command creates one layer of the image this second command creates another layer of the image this third command creates the third layer the fourth layer and the fifth layer and they all kind of build on top of each other and what's important is that after each layer the docker docker actually caches the result of each layer and what i mean by that is when we run docker built when we actually build our image for the first time what's going to happen is that it's going to run this first step and it's going to cache the result so it's going to say hey look we downloaded the node image from docker hub and then we cache that result right we then set the working directory to slash app and we cache the result of that we then copy package.json we cop we cache the result uh we then npm install cache that result and then we copy all of our code and we cache that result and this is important because let's say we decide to rebuild the image again right if nothing changes docker is efficient it knows that nothing's changed in any of these layers and it just takes the final cache result of step five which is the last layer and it just gives that to you so if you run docker belt which is the command to actually create the image the first time you run it it's going to take a long time because it's got to run all of these steps and especially the npm install if you have a lot of dependencies it's going to take quite a while but the second time you run it if nothing's changed you'll see you'll be done in less than a second and that's because it's cached all the results and the reason this is important and the reason why i split this up into two different steps is i want you to think about what happens during our development process right when we are actively working on our code what changes our package.json does not change very often right i mean we do occasionally add new dependencies that's normal but it's for the most part your source code changes but your package.json and your dependencies don't change very frequently and so by splitting this up into two different steps what happens is we cache each of these layers right and so we're gonna cache uh the node image we're gonna cache the working directory and in reality those two will never change right we're never really gonna change those two so uh we're always going to essentially cache the result from step two and onwards but the thing about docker is if any layer changes right say that layer three changes and what i mean by layer three changes is that if package.json ultimately changes uh where maybe we add a new dependency uh we have to rerun step three and all of the steps after that because we don't know how that's going to impact step four we don't know how it's gonna impact step five and so by caching um by splitting these up into two different steps uh we can say that listen if package.json never changes then we're going to cache that result and since realistically when we're actually programming package.json is just not going to change so we're going to cache that result and since package.json is cached then we're going to also cache the result of npm install because nothing's really changing so we cache those results and so when we change any of our source code within our application the only step that really changes is layer 5 where we copy the rest of our code and so we only need to rerun step 5. however if we didn't have this split up into two different steps where we have a copy package.json and then copying the rest of our files uh what would happen is that anytime we changed our source code technically it would rerun everything including the npm install because it doesn't know that we only changed our source code it just sees that listen this step where we copy all of our files has changed we're going to have to rerun all the steps after that so by doing this we know that package.json will always be the same so we can cache the result of these two steps and only make and only runs step five which is the changing of our source code so it's a little bit of optimization if it didn't make sense i don't think i did that great of a job explaining it um but i'll show you guys this when we actually build our containers how it actually caches the results of each step and how it's a little bit of an optimization to actually do this and split this up into two different steps all right so the next thing that we want to do is uh we know our application is going to be listening on port 3000 so let's do um expose so we're going to say our container is going to expose port 3000 and then finally when we start our container we want to tell it what command to run right and since this is a node application and our entry point into our node application is this index.js we have to tell it to run index.js so we do cmd brackets and we're going to do node and then index.js all right so when we deploy our container it's going to run in index.js so this is at runtime and this is at build time so hopefully you guys understand this so this is when we're building the image this is the command that's going to be assigned to the container when we actually run the container and so that's all we really need um we're going to go ahead and actually create our image now so let's go to our terminal down here and i'm going to stop our express server because remember we're no longer going to be developing on our local machine we'll be developing on the docker container so make sure you stop your local instance of your application all right and let's build our docker container oh sorry our docker image we're building the image right now not the container itself so we'll do a docker build and then we're going to specify uh the path to our docker file it's actually called the context it's not necessarily the path to the file it has a little bit more of a meaning to it with but i don't want to spend too much time going i just think about it as the path to the docker file so this dockerfile is in the current directory so we're just going to do a dot so let's run that and i realized i forgot to save my file so let's save that i'm going to rerun that and i want you to pay attention to the output notice what's happened and i think this is kind of important when you're trying to understand what's happening uh you could see that on step one of five it says we're going to grab uh the node image from docker diode io so it's pulling the image from the uh from docker hub right and as i said each one of these is a separate step or a separate layer right if we go down we go to step two and it says working directory is set to slash app you can see that it says cached um so you're not going to see it say cached on your machine that's because i ran this as practice before i recorded this video but it's not going to say that it's cached when you ultimately run this so so keep that in mind but if you run this again if you run it again right what should happen is that it should cache all of the results right and so now all the ways down to step five it's cached so if you run it again like i said there's an optimization where it actually caches the results and the second time you run it it's going to be much much faster so now what we're going to do is we're going to do a docker image ls and you can see the new image it created um this is the one without a name because we didn't specify a name you also see the node image that it pulled from docker hub which is node 15. now i don't like the fact that we didn't give this a name so let's do a docker image rm this is going to delete the image that we just created i'm going to pass in that image id and if we do a docker image you can see that it's gone now so now we're going to go back to the docker build command that we ran but this time we're going to pass in a specific flag so we'll pass in the dash t flag so here we can give it a name i'm going to call this node dash app dash image all right so once that's complete we'll do a docker image ls and now you can see we've got our image that we just created so now that we have our image let's go ahead and run it and let's test it out see if everything works so we'll do a docker run and we'll do node dash app image so here we're just specifying the image that we want to create a container from which is this image that we just created but before you hit enter there's a couple of flags that we got to pass so first of all i want to give my docker container a name so that i have some kind of way to identify it so we can pass in the dash dash name option and we'll call this node app so keep in mind that last uh the last entry in my command is the name of the image that we're creating a container from this is the name of the container that we're creating and then finally there's one more flag i want to pass which is dash d so that means it's going to run detach mode because by default when you create a docker container from docker run you're going to be attached to the the cli or the console or whatever it's called but here i can run in detached mode so that my command line's still free and open so let's hit enter and it looks like it successfully created my container um i can do a docker ps and we should see that there's a container open at the moment all right so let's test this out and what i'm going to do is let's just go to my local host colon 3000 hit refresh let's see what happens all right so it doesn't look good it's spinning which most likely means there's something broken and it looks like there is don't worry guys i know exactly what's wrong i perfectly did this so let's tackle exactly what's wrong with our docker container in the next section so we were unable to connect to our docker container on localhost colon 3000 now why exactly were we unable to do that well let's take a look at our docker file and if we go back here we can see that we do have this command expose 3000 and i think naturally most of us would assume hey look we're exposing port 3000 so we should be able to access that well not exactly this line does absolutely nothing this is really more for documentation purposes if you delete this command and create a brand new image it will not impact our image or container at all in any way shape or form it's just so that when you share your docker file with someone else they'll know hey look this image expects you to open up port 3000 for everything to work so this line doesn't actually open up port 3000 and the thing about docker containers is by default they can talk to the outside world so if a docker container wants to reach out to the internet or wants to reach out to any other devices in your host network it can do that however outside devices like the internet or your host machine or any other machine from the outside world by default cannot talk to a doc container this is almost like a built-in security mechanism right you don't want the outside world to be able to access your docker container but your docker container can outside can access them so how do we make it so that the outside world can talk to our docker container and keep in mind when i say outside world i don't just mean the internet i also mean our local host machine right and when i say local host machine i mean my windows machine right here right to actually uh talk to the container from my computer which is kind of seen as like an outside device we have to um poke a hole in our host machine and what i mean by that is that by default right our docker container nothing from the outside wheel can talk to a docker container so we have to basically say on our host machine hey if we receive traffic on a specific port we want to forward that traffic to our docker container and the way we do that is it's very easy first of all let's kill our container we don't need that anymore so i'm going to do a docker rm and then specify the name of our container we'll so we'll do no dash app and then i'm going to pass in the dash f flag so this is stands for force so by default usually you have to stop a container before you can delete it if you pass the dash f flag it'll allow you to delete a running container all right so now if i do a docker ps we should see an empty list so let's do docker let's rerun that command but here i'm going to pass in another uh another flag i'm going to pass in the dash p flag i'm going to specify uh this so let me explain what this is in a second so we've got port 3000 and then colon 3000 so what exactly does this mean we've got two different numbers we've got the three thousand to the left of the colon and three thousand to the right of the colon so the number to the right let's start off with this one is the um basically the port that we're gonna send traffic to on our container and our container remember our application is listening on port 3000 so we want to send this to port 3000 right if this was set to 2000 then we would want to set this to 2000 so whatever our container is expecting traffic on the number to the right of the colon should be set to that value now the number to the left represents um it represents traffic that's going to be basically coming in from the outside world so if another device on your network or even your local host machine right my windows machine if we send traffic to localhost port 3000 right we're going to take traffic that's coming in on port 3000 and sending it to port 3000 on our container but in this case even though the two numbers are the same they don't have to be so let's say we wanted to basically poke a hole so that anybody that sends traffic to our windows host machine here on port 4000 we should then forward it to our docker container in that case we would change this to a 4000 and we keep this to port 3000 because remember it's still listening on port 3000. however if this was 2000 then we would change um this number to be 2000 here right so hopefully that makes sense i've set up a little quick diagram i don't know if you guys understood it at this point but hopefully the diagram makes it a little bit easier to understand so let me pull that up so here in this diagram i've got my host machine right which is this big blue box which is my windows machine and then i've got this node container in green right here uh and so what's going to happen is uh you know we've got those two numbers right basically what we want to do is when our uh host machine right our windows machine receives traffic on port 3000 what we want to do is we want to take that and forward it to port 3000 on our node container so that's why we have 3000 colon 3000 so the first 3000 represents the red arrow and then the second 3000 represents the yellow arrow and this also applies when we send traffic from our host machine to the local host ip so to ourselves so if we send traffic to local host port 3000 it's going to take that traffic and it's going to forward it to port 3000 and once again like i said before we don't have to do 3000 to three thousand we could change this to uh four thousand and then we would send traffic to local host four thousand and it would forward it to traffic on port 3000 on our node container because once again our node container is listening on port 3000 because that's what our express server is listening on all right so hopefully that made sense i'm going to run my container now so i'll hit enter and i changed everything back to 3000 column 3000 because it's just a little bit simpler why not have both of those numbers match there's no need to unnecessarily complicate things let's do a docker ps all right and you'll see that we've got our container but we under this port section you can see that it looks a little different you can see that right here we've got basically 0.0.0.0 on port 3000 with an arrow to 3000 tcp so what this is saying is any traffic destined to your host machine uh for uh you know my windows machine right here on port 3000 is going to get forward to port 3000 on my container so now let's go back to their web page and let's do a refresh and you can see now it says hi there so we have successfully sent a request to our docker container on port 3000. now before proceeding any further i do want to show you one thing real quick let's actually uh log into our docker container and take a look at the files in there so we can do that by typing in docker exec on dash it for interactive mode and then we'll specify the name of our docker container which is node app and then we want to pass in a new command instead of the usual node index.js we're going to passion bash so this is going to allow us to take a look at the file system of our container uh so here by default it's going to drop us in the slash app directory just because uh we set the working directory to be that uh and here i just want to type in ls so this is going to list all the files in that directory and so you're going to see all the files that it copied over so we've got our package.json we've got the package. lock we've got the node modules folder our index.js and we've got the docker file and you know the reason why i wanted to show you the file system is you may be wondering what is our docker file ultimately doing in our docker container right our docker file is there to create an image but we don't actually need it in our docker container and i also wanted to point out one more thing if you take a look at this copy command right here what it does is it copies everything in our current directory so every single file and it copies it into our container or our image and this is a bad thing um because there are going to be files that you don't actually want copied into your container right just like our docker file that i mentioned we may also have an environment file that has a lot of our secrets that we don't actually want copied into our container potentially also on top of that we don't need to copy our known modules folder this is actually a waste of time because a lot of times this folder is actually fairly large and we don't need to do that because remember we're copying our package.json file and then we're doing an npm install so there is zero reason to ever have to copy this node modules folder into our container and on top of that we ultimately want to move away from developing on our local machine so moving forward we won't even have a node modules folder on our local machine it'll only ever exist within our docker container so why are we even copying it in the first place it could be stale we may not even have it so we need to find a way to make it so that docker does not copy over files that we don't ultimately want copied over just like this docker file just like the node modules folder if we have git configured we definitely don't want that copied over uh and so we can do that by creating a docker ignore file right and that probably sounds a little familiar because if you work with git kit has a git ignore file for files that you don't want checked into your git repository same exact concept so let me exit out of my container uh so we can do exit to exit out of the file system and uh i'm going to kill that in that uh container so we'll do docker rm uh node dash app and then we'll do dash f for force and let's create a docker ignore file so i'll do a new file and i'm going to call this dot or period and then docker ignore and within our docker ignore file we're going to list out every single file and folder that we don't ever want copied into our docker container so the first thing is we don't want to copy our node modules folder because we're going to do an npm install from our package.json anyways there's really no need to ever copy it over we don't want to copy over our docker file now that we have a docker ignore file there's really no need to copy that into our docker container uh and then a few other things right if you have git you don't want to copy git you don't want to copy your git ignore uh for now i think that's good um you'll see later on in the video we'll add a couple more files but for now that's okay just do a save and now let's rebuild our image so let's find that build command so docker build dash t the name and then uh the path to the docker file so that's their current directory so i'll run that and just make sure i didn't change anything in my index.js if everything looks good so let's run that all right so that's done let's run our container from the new docker image that we just created so if we hit up for that docker run command let's see if i can find it here we go so docker run dash p so we're going to open up port 3000 run in detached mode we're going to give the container a name of node app and we're going to use the node app image that we just created alright so we did that let me just quickly double check to make sure i didn't break anything so hit refresh it looks like everything's working perfect now let's go into our docker container again so we can run that uh docker exec command and let's just make sure it didn't copy over any unnecessary files so let's do an ls and now it's perfect right we no longer see a docker file there's no docker ignore file i can't verify for you guys that we didn't copy over the node modules folder but just trust me this node modules folder is from running a uh an npm install on the fourth step and not because we copied it over from our local machine all right so let's exit out of that okay so now that we got our application working in a docker container and we can access it from our local machine uh let's see what happens when we change our code i'm gonna go back to my uh my index.js file and i'm going to just tweak something right here it says we're sending an h that says hi there i'm just going to add a couple of exclamation points we'll save that and i'm going to minimize this and let's hit refresh let's see what happens we hit refresh nothing happens we don't see the exclamation points so for some reason our code didn't get updated and i want you to think about why that is it's actually a really simple answer right uh basically what happens is first of all we have to take our docker file we have to build an image so we built an image and then we built a container from it now we changed our code to add the exclamation points but our image that we created from our docker file was run before we made these changes so the code in that image does not have the exclamation points and so basically our image has a stale version of our code which means our container which is running from that image has a stale version of our code and we can prove that if we do a docker exec and go into our code if i do an ls we can see our index.js file and i do a cat which is just going to print the contents of that file if i do an index.js you can see that there's no exclamation point so it's running a stale version of the code so how do we actually update that well it's actually very simple first of all make sure you change save all your changes and uh now what we're going to do is we're going to rebuild that image well first of all let's delete our container we don't need it anymore and let's rebuild the image so where's that build command so build right so now if you take a look you'll see that because our source code changed it had to run step 5 once again so now if we do a docker image ls this new image which we named it is the same name as we had before now has the new exclamation points built into the image so if we deploy this image with a docker run command it's the same exact docker run command hit enter and now if we go back to our our web browser hit refresh we can now see the exclamation points so that explains why um our code didn't automatically get updated however i'm sure you're thinking well this is kind of a strenuous process for when you update your code right every time i make a change i don't want to have to rebuild an image and redeploy a container that's a very slow process that's going to slow down your development time uh and what i'm going to do is i'm going to show you guys how we can work around that because that's obviously not um sustainable you can't be rebuilding an image and re-running a container every time you make one tiny uh change in your code so what we're gonna do is we're gonna make use of something uh called uh we're gonna make use of volumes so within docker we have volumes which allows us to have persistent data in our containers but there's a very specific type of volume called a bind mount in docker and this is a special this is special volume because what it allows us to do is allows us to sync a folder in our local host machine on our in my windows machine in this case it allows me to sync a folder or a file system to a folder within our docker container so what i can do is i can take all of these files and sync them into the slash app directory of our container so that we don't have to continuously rebuild the image and redeploy a container every time we make a change it'll automatically sync those two for us to really speed up the development process so let me show you guys how we can do that so we're going to delete our container and what i'm going to do is we don't need to rebuild our image the image is just fine but i'm going to hit up a few times so we can get to this run command so we're going to use the same exact run command and we're going to pass in a new flag it doesn't matter where you put it and it's going to be the dash of e flag so this stands for a volume there's a couple of different volumes or different types of volumes but remember we're using the bind mount which is a special volume that allows you allows you to sync a folder from your local machine to a folder in your docker container uh and so the syntax for this is you do dash v and then you specify um the local folder or the path to the folder on your local machine so i'll just say path to folder on local machine on local and then we do colon and then we do path to folder on container okay so kind of like pseudo code just uh so you guys understand so this is going to be the location of this folder right because this is the folder that has all my source code and i want to sync that to a folder within our docker container and that's going to be in the slash app directory because that's where we're going to store all of our source code so let's hard code those values uh and so here unfortunately i can't just do a dot right here um it's not going to register that it's not going to work you have to actually pass in the whole path so if you're using vs code like i am you can right click here and then just select copy path and then you can right click and it's gonna grab the entire path uh in my host machine uh to this folder obviously i'm gonna delete the docker file section we just need to get into the node docker which is the name of the folder that houses all of my code and then we do colon and then we do the path to the folder in the container so what folder do we want to copy it to easy we want to copy it to slash app right and that's so that's all you have to do you hit enter and it's going to sync your code however i do want to show you a couple of shortcuts because that does look a little messy having to type all of this out and so what we can do is we can make use of variables and so i'm going to delete this i'm going to show you guys what you can do instead instead of having to type out that whole nonsense and it's going to be different based off your operating system so if you're using windows and you're using the command shell then what we can do is we can type in percent cd percent so that's going to grab the current working directory so that way i don't have to copy that entire path and that only works on windows command shell uh if you're using windows powershell then you're gonna have to type in dollar uh curly braces pwd closing curly braces that's gonna grab the current working directory and if you're on a mac or linux then you can type in dollar parentheses pwd uh and then close parentheses so that's just a shortcut or you could type out the entire path whatever is easiest for you feel free to do that so i'm just going to use my percent cd percent there all right and so let's hit enter and uh you know depending on you know what uh operating system using if you're using windows anytime you're doing file sharing um you might get this warning there's a couple of optimizations that you can make i usually just ignore it for now all right so now theoretically it should be syncing this entire folder with the slash app directory in our container uh and so uh let's minimize this and let me just refresh this so you can see right now we've got the four exclamation points let's go back to our code and i'm going to delete the exclamation points all right and then after we delete it let's save it and let's see if i refresh this page does the changes take effect so let's hit refresh and it looks like it didn't so do you guys know why it didn't take effect well let's take a look right because theoretically this command this flag right here should sync the folder so we should see this file get synced to our container so let's drop into our container and take a look at what the file looks like in the container to see if it actually updated all right because maybe i was lying to you guys and this does absolutely nothing and i wasted your time so we log in let's do an ls so we've got our index.js and let's print out the contents of the file by typing in cat and then index.js and if we take a look at that we can see that hey look it did actually sync our code it deleted the four exclamation points so why exactly did we not see the update in our web browser well this is easy for anyone that's ever worked with express applications i'm sure you might have an idea as to what exactly cause this problem remember anytime we make changes to code in any node or express application we have to restart the node process we did not restart the node process we just changed the code and we hoped that it would automatically work but we had to restart the node process right and um you know we could go in we can kill the node process start it again but obviously that's inefficient right we already have a solution that we already know is going to work and that is we're going to make use of nodemon right nodemon will always look at your code and if any changes take place it's going to restart the node process so that the changes are updated in real time so let's get that set up and installed i'm going to exit out here and we need to update our package.json file so let's get nodemod installed as a dev dependency and i'm going to do this on my local machine just so that we can have it set up in this file so i'm going to do an npm install nodemon save dev so this is going to be a dev dependency just because we don't need it to run when we actually deploy to production and remember i'm just doing this on my local machine right now just so that we can update my package.json file all right so we've got our dev dependency now let's go to my package well i'm already in my package.json but let's set up a few scripts so we'll do a start and this will just be the usual uh node index.js and then we'll have a dev which is going to be whoops which is going to be a nodemon index.js so by running nodemon in dev mode it's going to automatically restart uh index. it's going to restart the node process anytime there's any changes to our source code now as a heads up when i was uh doing a dry run of this demo i did run into some issues on specifically it looks like windows machines so if you're on a windows machine and for some reason later on in this video uh you run into some issues with nodemod not actually restarting you may need to pass in the dash l flag uh so that that kind of fixed most of the issues uh if you do run into the issue just try and try the lflag uh if you want to read up on it just just google the error message that you get or just google no not restarting on windows for docker and they'll give you an explanation for why you need to pass in that flag but i'm going to keep that in there because i'm running a windows machine and i did run into that issue so let's save all of that and let's uh let me kill the the docker container that we have so i think there should be one running so we'll do a docker rm node app dash f all right uh so now that we made changes to our package.json we're going to have to rebuild our image now so we do a docker let me see if i have that command cache someplace here we go build so we'll do the docker build and notice how it's taking a little bit longer this time and that's because our package.json file changed so it had to rerun step three where we copy package.json and then rerun step four and then we run step five because we have to run all the steps after that because we don't know if changes to this file change to any of the cache results so that's why it took a little bit longer this time but hopefully by now you guys get an understanding of how all the caching works in docker all right so now let's redeploy actually before we redeploy there's a couple of things that we have to change i realize i forgot to do this so um back in our docker file right we're not going to run node index.js anymore so in the development mode we're actually going to run if you take a look at our package.json we're going to run npm run dev so we can run nodemon so let's go back to our docker file we'll do npm run and then dev save that sorry about that guys we're gonna have to rebuild our image again so let's uh rebuild it all right uh and so now let's run a container from that image and this time we want to do it with the bind mount again let's hit enter we've got our docker container let's quickly test this out okay so the exclamations are gone which is fine and let's make a quick change to our code i'm going to re-add the exclamation points save it all right and if i hit refresh look at that so it looks like nodemon's doing its job and anytime we make changes to our code it's automatically restarting the node process and our bind mount is successfully syncing the code from our local machine to our docker container now i want to do a quick little test and i'm going to do this because i suspect depending on how you guys followed along to this video so far you may have run into an issue in the most recent step so let me delete my docker container so let's do a docker rm node app dash f and what i'm going to do is i'm going to take this node modules folder on my local machine which i don't need anymore because we're not developing on my local machine and i'm just going to delete it all right so now that it's gone i'm going to redeploy the container i'm going to show you that we're going to break our application so if i it's now running and if i go to my web browser hit refresh you can see it spins and it's eventually going to crash let's give it a second there you go so what exactly happened well let's take a look uh so if i do a docker ps let's see if my container is running and look that's the first issue why is our docker container not even showing up on this well when you do a docker ps it's only going to show you running containers so let's do a docker i think it's ps minus af yep so this is going to show you all containers started or stopped and you could see our most recent container our node app container and it said it exited 30 seconds ago which means usually if it automatically exit it usually means something crashed so if you want to take a look at the logs to see why something crashed we can always do a docker logs and then you specify the name of the container so it's going to give you all the logs for that and this is going to show us some node or nodemon logs and you'll see right here it says nodemon not found so what exactly is happening we know for a fact that nodemon should be installed because we had everything working before the only change we made was we deleted the local node modules folder why should that impact anything because it was working before well let me tell you what happened so what happened was um you know we we create our docker image right where we copy our package.json file and then we run an npm install so at this point when we run mp install it should install nodemon for us and it is in there but then we copy um all of our files which once again not an issue actually i don't know why i kind of made it seem like it was but we copy all our files all of our source code and then the command specified for the container is npm run dev now when we go back to our docker command that we used to actually create our container we created a bind mount so the bind mount syncs this folder with the slash app folder and this is where the issue occurred because it's syncing this folder since there's no more node module folder in this folder it syncs that with the slash app folder so it ends up overwriting the slash app folder and deleting the node modules folder in the slash app folder of our container because it doesn't exist in our local our local directory and since we have a bind mount it's going to sync those two directories together that's the problem right because once we deleted the node modules folder from our local machine it will also delete it from our docker container and then without the node modules folder that container has no idea what the hell nodemon is so how do we get around this issue how do we prevent our local directory our local folder from overwriting the slash app directory and deleting the node modules folder well there's a simple little hack that we can do and that is we're going to create a new volume and we're going to create a volume that's called um there's a couple different volumes like i said in docker bind mounts one we also have an anonymous volume and volumes work based off of specificity so we've got a volume on our container for the slash app directory but we want to make sure that we preserve the slash app slash node modules so we want to make sure this bind mount doesn't override the slash node module folder within the app directory and the way we can do that is we could just specify another um another volume so first of all let me delete our broken container all right and then we'll run that same command but i'm going to pass in another v flag for another volume and i'm going to say this volume which is going to be an anonymous volume is going to be that for the slash app slash node underscore modules this is a little hack that you can do to prevent the bind mount from overwriting the slash app slash node module folder because all volumes in docker containers are are based off of specificity so even though this bind mount should sync with the slash app directory we can see that we have another volume that uh that references the slash app slash node modules folder and you can see that since this is a longer path that's basically more specific so that's going to prevent this bind mount from deleting the node module folder basically this extra line is going to say hey don't touch this folder since it's a more specific more uh it's a longer path it's more specific so it overrides the bind mount the bind will still sync all of the other files it just cannot touch the node modules folder all right so now if we hit enter let's do a docker ps and it should stay running let's just make sure it didn't crash and it looks like it didn't so now if we go to the website let's hit refresh it's there let's make a few changes just to make sure nothing else broke we'll delete the exclamation points hit save and then hit refresh and it looks like everything's working everything looks perfect so far now there's one thing i do want to point out so you'll notice that we do a copy where we copy all of our files into our container uh at the build stage and you might be wondering well do we really need to do that if we have our bind mount and the thing is the answer is yes because the bind mount is really just for our development process we only use the bind mount when we're developing because that's when we're changing our code when we go to actually deploy in production there obviously isn't going to be a mind mount because why are we changing our code in production uh so we do we still need this copy command for when we deploy into production because there is no bind mount and we have to make sure all of our source code gets copied into the image for our production container all right guys so when it comes to docker volumes and bind mods i want to show you one last thing we're going to make one slight change it's not required but it's kind of best practice so what i want to do is i have a i have my container still running if you don't go ahead and just run this command again where you have the bind mount and the anonymous volume and i want to drop into bash so that we can take a look at the file system so docker exec dash it app and then bash all right so we're in our container and let's do an ls and so remember since we have the bind mount this directory is going to sync with the directory in our container however it's a two-way street so if i make changes here it's going to get updated here right if i do a new file and i'll just say my file and i do an ls here right we can see my file however if i create a file within my container it's going to add it to my local machine as well so if you want to create a file just for demonstration purposes you can type in the command touch it's going to create an empty file and i'll just say test file so if i do an ls we created our test file and you can see the file showed up on my local machine i want you to think about a potential issue with that why is our docker container changing our our files do we ever need is there ever a scenario where you want your docker container to actually be making changes to your source code or any of the files associated with the source code probably not right it seems like almost like a security issue for it to even allow it to do that right there may be certain instances where your application actually creates files on the local machine and in that case you might want to allow it but for the most part you don't want your docker container being able to touch any of your files or make any changes to it because there's really no need to and so what we can do is uh we can take our bind mount that we created and we can make it a read only bind around which means the docker container will be able to read any of the files but it can't touch any files and it can't create any of the files right it's read only so let's get started on that we'll exit out of here and i'm going to kill this container and to make this a read-only bind mount it's very easy so let's run this command uh where'd it go there we go let's run this and all we have to do is specify uh colon at the end of slash app we do colon r o and that implies read only so we'll hit enter hopefully it's running uh and uh we're gonna go and drop into bash to our container all right and so now here let's create a file so we'll do touch new file and look at that it says this is a read-only file system it cannot uh make any changes it cannot create any new files it cannot edit any files everything is read-only so this is a little bit of a an enhancement that we can make so that uh it protects our source code and it doesn't allow our docker container to do any kind of funny business with any of our source code all right so let's exit out of here and i'm just going to do a little cleanup i don't need these files all right and now what i want to do is um i want to show you guys how we can start making use of environment variables within our docker container so if you go back to our index.js file and you remember our express app the port it's going to listen on is going to be based off of an environment variable called port that we pass in uh if this variable is not set it's going to default to a value of 3000. so how do we make use of environment variables within docker containers first of all let's go to our docker file and we're going to specify a default value for our part variable so after our copy command it doesn't technically matter where you put this but we can say env so this is referencing environment variables we're going to create one called port and we're going to say the default value for this environment various port 3000 and then we want to expose remember this is just merely for documentation purposes this command doesn't really do anything instead of having to hard code 3000 what we can say is we can reference this port variable by passing in a dollar sign and the name of their variable support so if this gets updated this will automatically get updated as well so let's save that and what we want to do is uh let's see is do we have any containers running yep so let's just kill that all right and let's rebuild our image because we made a change to our docker file and we can do a docker build for that all right so now that those changes are made uh our application shouldn't really fundamentally change because now we're just setting our environment variable to be port 3000 before it was defaulting into a value of 3000. but now since the environment variable is set to 3000 everything should theoretically work the same it's just now going through that environment variable but what i want to do is when we actually deploy the container we can specify what value we want that environment variable to be set because this is just the default value we can always override it so let's deploy a new container with the same exact command with both of our volumes but we can pass in an environment variable by passing in dash dash env or you can just do a single dash and the letter e so whichever you prefer i don't know why i prefer the double dash emv but whatever you prefer uh and then you pass in the name of the environment variable equals and then whatever value you want it to be set to so let's say we want our express server to listen on port 4000 now it's going to change to 4000 however before we hit enter and actually create our container remember since our express application is listening on 4000 we have to change uh the port that we're sending traffic to because right now we're sending traffic to our container on port 3000 but our express app is listening on port 4000 so we have to change this to 4000 or then our application is going to break uh we don't need to change um the port that we uh have to hit on our local machine we could technically change it to 4000 but it doesn't really matter you can pick any value really here so let's hit enter and let me just make sure it's running all right so it's running so now if i go to the web browser hit refresh it looks like it's working just to double check everything's working fine let's just make a few changes to add the exclamation points back save hit refresh it looks like everything's working uh just to double check i do want to make sure that the environment variable did get set so we can drop back into that container again like we always do where's that command docker exec uh so in a linux machine if you want to see the environment variables you just type in print env and so here we can see that the environment variable of port equals 4000 was set and so that confirms that when we ran our docker run command and passed in the dash dash env flag we were successfully able to overwrite the port variable or the port environment variable that was specified in our docker file now when it comes to your application you may have more than one environment variable actually you most certainly will and if you have a lot of environment variables uh you know we you can pass in the dash dash env flag as many times as you want so if you wanted to um you can say uh where where'd it go here we go so if you wanted to pass in another one you can just do env and then whatever that variable is however if you've got like 10 20 environment variables that's kind of a little bit of an exhausting process and it's a little bit of a pain and so what you can do instead is we can actually create a file that stores all of our environment variables so here i'm going to call this um you can call whatever you want but standard standard convention is dot env and here we can just specify port equals 4000 right and so that's going to essentially do the same thing and here you could just provide a list of all of your environment variables and let's save that and i'm going to kill my docker container real quick and so now if you want to load environment variables from a file instead of having to pass each one line by line let's go to that docker run command and we can remove the dash dash env and we can pass in dash dash uh whoops dash dash env dash file and then the path to our environment rival so from our local directory we do dot slash and then dot env and so that's going to grab this environment variable file and then all the environment variables start in here i think i saved it yep and let's hit enter hopefully this works let's go um docker ps all right it's running um i'm going to log into the container and do a print nv let's just make sure that it's set and it looks like it's set so those are the two different ways to specify an environment variable and to set environment variables for your docker container so that your application can get the necessary data that it needs uh now one thing i want to point out is uh as you've been creating docker containers and deleting them if we do a uh docker ps uh you'll see that we just have one container running if you do a docker volume ls this is gonna list all the volumes that you have and you can see that we've kind of built up a couple of different volumes and you might be wondering what are these from and why are they building up as you keep creating containers and deleting containers they're going to slowly build up over and over and you'll eventually end up with hundreds uh and the reason for that is if you take a look at the command that we're running for our docker container this uh this volume that we specified right here for the slash app slash node modules hack this is anonymous volume so every time you delete your container it's going to preserve that node modules folder in here and uh we don't actually need to preserve it right because we're going to be deleting and creating new containers all the time so you can go in and manually delete the volumes right you can always do a docker volume rm and then specify the volume name or you can do a docker volume prune that should remove all unnecessary volumes um but if you want to make sure that these volumes don't build up usually what i like to do is when you do the docker rm command um it's not going to delete the volumes associated with that container so if you want to delete the volumes which in our case we do there's obviously plenty of instances where you don't want to delete the volumes because the whole idea behind the volume is that it's persistent data you want to preserve it so like if you have if you have a postgres database or a sql database right uh you want to preserve all your database records so you would never want to delete that but in our case we just have an anonymous volume so that we can get around that limitation that we ran into before so when you delete a container just pass in the dash v flag when you pass in the dash v flag it'll make sure to delete the volume that's associated with that container so that it doesn't build up so for now i'm just gonna do a docker volume i think prune should work it's gonna say hit yes let's see if i do a docker volume ls uh you can see that's deleted all but one and this is the volume that's associated with the container still running so uh if you want to do the docker that little trick i just showed you with the running container where is that here we go dash fv it's going to delete the container and it's also going to delete the volume as well so moving forward to do the dash f uh and then v so that you can delete the volume as well so now when it comes to um creating our container we've kind of set up a nice little quick workflow for uh developing a node or express application in a docker container however the command to run the docker uh container uh this command right here is kind of long and uh you know i don't want to have to rerun this command every single time i want to get my container up and running although i'm sure you're thinking well you can just hit the up arrow key so it's not a big deal and that is 100 true however keep in mind when you're actually developing uh you know like an express application or some kind of node application you're probably going to have more than one docker container right right now we're just building the uh the express server but in a full-blown application you might have a container for your database you might have an elasticsearch uh container you might have another container for redus you're gonna have multiple containers right and each one of those containers you're going to have to run a command that's this long or potentially as long and at that point it starts to become a real hassle of getting your entire development environment up and running and even getting your production environment up and running because you're gonna have to run five or six docker commands you have to make sure there's no typos or anything like that so what i'm gonna show you guys is a way we can kind of automate all of these steps so that we don't have to run this monstrosity a command and we're gonna use a feature called docker compose where we can create a file that has all the steps and all of the configuration settings we want for each docker container so we can say like hey i want to create a node container using the image that i created and i want to create a volume for the bind mount and i want to create an anonymous volume i want to pass in the environment files and i want to open up these ports so you can pass in all these steps into a file and then you can just run for one very simple command to bring up you know as many containers as you want so if you have like six or seven different containers in your development environment you can bring up all six or seven all at once with one command and bring them all down with one command so let me show you guys how to do that uh so what we're gonna do is we're gonna create a docker compose file and so we're going to do is we're going to do a new file we're going to call this docker dash compose dot yml so that stands for yaml it's a certain type of syntax that you use so what we want to do is with your docker compose file the first thing we have to do is specify the version that we're going to use now if you go to this website right here it's going to show you all the different versions of docker compose and the features that they support now we're not going to do anything crazy so i'm just going to use version 3 but if there's a specific feature that you need then you're going to have to make sure you grab the specific version this is like a this web page has a list of all the features that each version supports so you can just take a look at this page and see what's the minimum version that you need to actually run all the specific features that you want um but i just defaulted version three so we'll say version colon uh and then three now the next thing we want to do is we want to specify all of the containers that we want to create uh and so within our docker compose file each container is referred to as a service so we do services and then we just specify all of our services now this is very important with yaml files uh spacing matters okay so uh we're going under services and we're gonna provide a list of all the different services that we want so i want you to hit tab just once and then here we're to provide our container or our first container in our example we just have one container which is our node app and we want to give this container a specific name you can call it whatever you want we can say like oh my node app or we can call this node project whatever you want just give it a name i'm going to call this node app because that's what it is and then colon and then we're going to add in a specific configuration settings for that container uh so because we're gonna do configurations for that container i want you to hit one tab and then i want you to start adding in the the different um the different configuration settings so the spacing matters unfortunately so if i if i start moving things around you know it's going to break things so you want to do one space for each level and keep in mind if you have multiple containers let's say you have a postfix container you would add a postgres service if you had a redis reader service you'd add one here so this is how you add more containers or more services within a docker compose file we just have one so i'm going to delete these now under node app we have to specify um a build setting so what image are we going to use well we have to build the current docker file in our current directory so we can say build right and then we pass in a path to our docker file so it's going to be our current our current directory or whatever directory this docker compose file is so this is just um automating the the docker build command so that we don't actually have to do it doc compose will do it for us we just have to pass in the path to the docker a file other settings let me actually just hit up the up arrow keys to see all of the different options that we have so uh we do need to expose a port so let's uh open up a port so we can pass in the ports option so under ports one tab remember and here we can provide a list of ports we want to open so for our container we've only been opening up one port but theoretically you can open as many ports as you want so you do a dash because it's going to be a list of ports and then here we're just going to do 3000 colon 3000 you already know what that means i'm not going to rehash that if you wanted to add another port or open up another port you can do that so you can do like four thousand colon four thousand as well but we're just going to open up one port so i'll delete that now i'm gonna hit backspace again so we're lined up with built in ports because once again we're still under the node app service and here uh we want to pass in all of our volumes so we'll do a volumes and we're going to pass in our two volumes and once again this is going to be a list so i want you to hit tab just once and then we're going to pass in a dash so the dash signifies a list so the first one is going to be our our bind mount and the great part about this is that when you use a docker compose file we can just do the dot syntax we don't need to use this variables that i mentioned before so we just do dot slash colon and then slash app and i think you can pass an ro if you want if you want to be read only i don't really care right and then we need to pass in our anonymous volume right which is uh this one right here and that's that little hack that we did to make sure that we didn't override our node modules folder so we'll do slash app slash node underscore modules and then last but not least i'm going to add my environment variable it doesn't really matter where you put it so once again line it up with build ports and volume so we do environment and we say dash and then here you provide a list of environment variables so we can do port equals three thousand all right uh keep in mind in this case this would be like the equivalent of running um the environment variables one at a time uh so let me see if i can find an example so like this however we can pass in a file like we did down here if you want so that we can use the dot env file and the way to do that would be uh let me see you can go down here we can say uh env underscore file colon and then we can say dash you can provide a list of files and then the path to the dot env file so whichever syntax you prefer i'm just going to comment this out i'm just going to use this because we just have one environment variable so there's no need to import it in from an environment variable file all right so let's hit save this has all of the settings that we need for our uh node app service uh and once again you know if you're building a full-blown project you're gonna have multiple services so you're gonna see how this kind of really simplifies the entire process of getting your entire development set up uh and up and running and as well as tearing it down all within one command so make sure you save that and what we want to do is we're going to run the command docker compose so we do docker dash compose this is important right don't do docker space compose it's a different command we want docker dash compose and we want up so that's going to bring up everything that's in our compose file and then one important flag is if we do help let's see what options we have we do have a d flag right if you remember when we run our containers we have to pass in the slash d flag or then we'll automatically attach to our containers which we generally don't want so we can pass in the same thing for our docker compose so we can do docker compose up dash d so let's see if that works so first thing you're seeing is that it's actually building our image and so it's running uh oop looks like i ran into wait up that's just a warning don't worry about that i think that's a yeah new patch so that's just a vulnerability message don't worry about that so it built our image if we do a docker image ls you'll see that this is the image that i built and notice the naming convention what it does is it grabs the project directory which i called node docker and then it passes underscore and then node slash app so the node slash app is from the service name so it took the project folder did underscore and then the service name if we had one for postgres it'd be underscore postgres or whatever we called it all right so it built the image and then it also started our container so if we do a docker ps right you can now see that we have one container running and it called it once again take a look at the name node docker which is the project name underscore uh then the name of the service and then there's an underscore one i think if you have multiple if you spin up multiple node app services i think it goes like one two and three i think that's why the one gets appended um but let's test this out let's just see if everything works um so going back here if i hit refresh looks like it works and uh let's change our code and delete the exclamation points just to make sure it updates hopefully it works and it looks it looks like that guys everything works perfectly so hopefully you guys can see how easy it is now uh to actually bring up our entire docker environment which is one simple command and bringing it up is just as easy as bringing it down so now if you want to tear down everything uh we can do uh let's see we just do docker dash compose instead of up i'm sure you can guess what the command is it's just down and just like when it comes to deleting docker containers by default it will not delete those uh anonymous volumes so if you want to delete the anonymous volumes with the docker compose down we can do a dash v and that's going to delete all the unnecessary volumes that it creates so do dash v and you can see it's stopping the docker container it's then removing it technically when you run docker compose it creates a brand new network uh for all of your services don't worry about that it's kind of outside of the scope of this um of this video but it does automatically create a net us its own separate network so that it doesn't interfere with any of your other docker containers and you do get some extra features and perks when it comes to having a custom network like dns so that you can reference names within your project but don't worry about that for now so now if we do a docker ps you can see that the container is deleted and everything's cleaned up for you that's right one command and you can start and stop theoretically hundreds of containers right now there is one thing i want to tell you guys uh when we run the docker compose up command again what do you think is going to happen right because if you take a look at the steps uh that took place when we ran it the first time you can see it builds the image right it builds the image and then it starts the container now if we run this again i want you to tell me what you think is going to happen right your natural guess is going to be that it's going to build the image again and then it's going to run the container well let's figure that out let's see what actually happens if i run it again look at this the result was much quicker and that's because it skipped the entire build process it didn't build a brand new image all it did was it created the network i did tell you created a network and then it started our container and the reason for that is that what docker compose does is it looks for an image docker so if i do docker image ls it looks for that image based off of that the syntax that i told you which is project directory and then um the name of the service and if it sees that that image already exists it's not going to rebuild it however even if we make a change right let's say we change the default port to be 4000 oops right this theoretically is now a different um is now a different um it's a different image right we've changed something fundamentally in the docker file so it should rebuild the image if we run it again so let me tear it down again let's do a docker compose down right and i think i saved it i can never remember if i do that and let's do a docker compose up so let's see if it rebuilt the image and despite the fact that we made changes it did not rebuild the image so now we're essentially running a stale image right why is it doing that like i said there's just a simple dumb check that docker compose does all it does is just look for an image named this so if we do a docker image ls we can see there's an image there it has no idea that this is a stale image and that there's been an update it doesn't know it's docker compose is pretty dumb it has no idea so you have to basically tell it like listen i've made a change i want you to rebuild the image so how do we do that well first of all let me let me tear things down again and let's go back to the docker compose up command and let's do a dash help and there should be an option that causes it to force a build and let me see if i can find it here we go dash dash build so this when you run the dash dash build flag it's going to tell docker compose to rebuild the image it has docker compose is not smart it does not know when it needs to rebuild the image you need to tell it to all right so let's rerun this again we'll do docker compose up dash d dash dash build this will force a brand new build and there you go so now we built a new image uh now it created everything and everything should be working now all right so we can tear that back down uh we can we can tear that back down uh change the default port to 3000 save that um just so that we don't break anything else and then if you want to you can rebuild it again with the dash build flag and so this will bring everything back up and rebuild it with the new default port and everything should be working so let's just double check and yeah everything's working perfect um but that's the idea behind docker compose there's nothing too fancy about it obviously and obviously you're going to have to take a look at the documentation uh to see other things because eventually when you get a little bit more uh familiar with docker and there's other settings or options or flags that you want to pass in you're going to have to update the docker compose file to take in those parameters so i just covered the basic parameters um but there's obviously plenty more within the docker universe now we're almost done however there's kind of one last thing this is kind of a big thing but you may have noticed that um we're running npm run dev right so how do we actually go into production right because right now when we run docker compose everything is with respect to our development environment because our docker compose it creates that bind mount which we would never want in production deployment right we don't need it to sync with anything it's our production deployment right and our production deployment might actually use a different variable or a different port uh that express listens on or you could use the same one i don't actually know uh it's going to depend on your company or how you set up your project uh but mainly also within our docker file it's not going to run npm run dev instead it's going to run either with my package.json it's going to run an npm start or we can just run nodeindex.js directly so in the next section i'm going to show you guys how we can set up our docker compose file so that we can have a separate set of commands for production and a separate set of commands for development all right guys so this is going to be the last section of the video and this is going to just round out the entire project i'm going to show you guys how you can set it up so that you can deploy your docker containers to both a development environment as well as a production environment because there are going to be some differences between those two environments and the easiest solution is you know you can obviously have a separate you can create multiple docker files there's no rule against it so you can have one docker file for development one docker file for um for production and then you can basically change out what you want so really the main change in our case is just the final command that we run npm run dev for development and npm start or node index.js for production depending on which one you want to use if you read online some people recommend not using the npm command within container because it's just another layer between node and the container so a lot of times they uh for production especially you may want to just run node index.js instead of running npm start um but i think different people have different opinions on that so we can create different docker files and on top of that we can also create different docker compose files so you might you could have one for production and you can have one for development uh there's also different differing opinions on on that some people like to condense as much into a single file and be able to kind of run both of them off of one file but it really comes down to personal preference so what i'm going to do is i'm going to show you guys how to um do everything as much in one file as possible so for we're going to use only one docker file but we are going to split up the code docker compose files into two different files um because um showing you how to do with two different docker files it's pretty easy right you just create two different docker files and then reference them when you actually run the build command uh and then so there's nothing really to that but i do want to show you how to do it with one file um just in case you want to know that because it was a little bit trickier we do create like a custom bash script that actually handles it so let me walk you through that so what we're going to do is i'm going to rename this docker compose file just for uh just so that we have this for reference but we're not going to use this anymore this is just i'm just going to rename it as backup or something docker compose dot backup.yaml and we're going to create a brand new docker file a brand new docker compose file and on top of that i'm going to create two more files so let's do a new file i'm going to call this one docker dash compose dot dev dot yaml so this is going to have the configuration specifically for our development environment and then we're going to create one more and i'm going to call this docker dash compose.prod.yaml uh so this is going to have all the configurations specific to our production environment and remember we're just going to have one docker file all right so we have three uh docker compose files and obviously if you have like a staging environment you can also create another one for that um but sorry these are the three this is the backup just for reference if you guys want to take a look at that later but these are the three so the docker compose file now is going to have any configurations that are shared between both environments uh and what i mean by that is that uh like i said in an actual project you're gonna have you know six seven ten you might have a ton of containers so you'll see that a lot of the configurations for your containers are gonna be the same regardless of environment so there's no point in us copying and repeating all of those configurations across both of these files we're going to create a shared one for all the configs that will be shared between both environments so what i'm going to do is i'm going to do the same thing we're going to set the version like we normally do and then here services we've got our node app and i forgot to add that tab from spaces matter and yaml right and we're going to set the build to be the current directory like we did before and then uh for my production and for my development environment i'm just going to say they're both going to be listening on port 3000 however keep in mind that maybe in your environment they're different so if they're different you don't want to put in this file because this file is only for when things are the same between your production and your development if they're different you're going to want to put them in their respective files this is only for shared configurations where they're the same in both environments so ports we'll set that equal to 3000 colon 3000 like we always have and then we'll also set an environment variable to be um port 3000. all right so this is the only configuration that's shared between both right so our development so in our development our production environment basically we're saying that the the final image is going to be the same ultimately uh that we build from our docker file because both environments are going to use the same dockerfile the ports are going to be shared so we're going to use port 3000 and this environment variable that we're setting is going to be shared between both environments all right so now within our dev and our prod file we can actually go in into our services create a node app section and then overwrite anything that we wanted to so we could technically overwrite the ports we can add in extra configurations and so on so let's go to our dev environment and i'm actually going to go back to this and i'm just going to copy up to node app because it's going to be the same so we go into our dev and our node app and i'm going to set the volumes right because in our development environment we want our bind mounts as well as the extra anonymous volume for making sure our node modules folder doesn't get deleted so i'll do volume and we're going to do dot slash actually i waste my time retyping that we can just copy this from here you just got to make sure the spacing's okay don't mess up the spacings uh and i think i messed it up hold on let's see so yeah one tab and one tab there we go all right and since this is our development environment we're going to pass in an environment variable did i spell that right that does not look right there we go and we'll say node underscore env so this is just common across node applications where in your development environment you're just going to set this to development and then in your production you would set it up you would set it equal to production all right and then the final thing is uh in our docker file i'm going to change this to default to it doesn't really matter what we said here because we're going to overwrite it so i'll just set this to either npmstart or index.nodeindex.js whatever you prefer i'm just going to set this to index.js by default and remember we can override any of this in our docker compose file so in here we can override this that command with the command option and we can say npm we do this on the same line npm run dev right because we have that script that's going to run node mod so that it restarts our code in our development environment let's save that or save everything and now let's move on to our compose prod file but copy the first three lines as usual now in our prod file what do we need to exactly change well let's take a look um we don't in our development environment we had our volumes we don't actually need these the only thing i can think of is we need to change the environment variable to be production and we need to change the command to be either node index.js or npm start depending on what you prefer so let's add those in so we'll do environment and then we'll do node underscore env equals production and then our command is going to be set to node index.js once again you can do npm start whatever you prefer so that's pretty much what we need so let's test this out i'm assuming i've probably put in a couple of typos so we're probably going to have to do a little bit of debugging all right so what we want to do is the command's going to be a little different so first of all uh do i have anything running yeah i do so let's uh let's do a docker compose down dash v to to delete everything so docker ps docker images all right so um now how do we actually run this now that we are going to have to run two docker compose files right because when we want to run in production we're going to do the the base docker compose plus the dev and then when we want to work in production we do the base plus the prod well what we have to do is you do docker dash compose and then you pass in the f flag for file and this is important the order actually does matter so the first file that we want to pass in is the base file so we do docker dash compose compose.yaml then we pass in the dash flag again to specify the second file so if we want to go into development we're going to pass in the docker dash compose.dev.yaml right and so that what's going to happen is it's going to load all the configurations from the base file and then it's going to load all the configuration from the dev file and then if it needs to it'll overwrite any of the configurations that it's been set to uh from the base uh and that's why we pass it in that order right and then we want to say we want to bring it up and then remember we want to run it in detached mode so let's test this out hopefully this works what did i mess up uh dockerdev.yaml what did i mess up uh must be a mapping not an array uh let's see oh yeah look i mean you can see i already messed the indentation the volume environment command need to be under the node app section so that's what broke so let's tab everything up one section and now let's give that a shot alright looks like things are working so hopefully it looks yep everything looks good so let's just do a docker ps and we see our container running let's go to our application let's hit refresh it looks like it's working um let's just uh go to our code make a few changes uh to make sure that things are updating and that nodemon's doing its job so let's hit save let's go back hit refresh look at that so this is our development environment and it looks like everything's working perfectly and we can go ahead and shut that down so we'll do uh remember same thing we do down and we can pass in the dash v flag so we can delete those extra volumes and it should stop it now let's do the same thing uh we'll run that same up command but instead of running dockercompose.dev let's try the prod version now all right and should be the same exact command outside of specifying the prod.yaml file so we hit enter perfect let's hit refresh all right so there's no exclamation that's just because we didn't rebuild the image but we'll go over that but let's just try to change things well i guess there's no exclamation at the moment so let's hit save and let's see if it changes and it looks like it doesn't which is perfect all right we don't want any changes we add anything else here and then hit save remember we're in production so there is no bind mount and we shouldn't see any changes now just to make sure things are working i'm going to bring this down i'm going to do a down dash v and just like before right when we uh make changes to anything like our source code or anything else since we're in production environment and there's no bind mount anytime we make changes to our code we're gonna have to rebuild our image and we have to tell docker compose that we want to rebuild it so we'll do up so right now it's got all this nonsense in there so we do up dash d and then i think it's dash dash build it's going to force a new build right because now we're changing our code there's no bind mount and so there so to actually see the update we have to rebuild the image all right and so now if i hit refresh we see all of that nonsense and just as a double check let's delete the exclamations and let's delete all that nonsense right there hit save do we see any changes absolutely not because remember we are in production environment so this confirms that we now have set up a different docker compose file for our production environment and for our development environment to accommodate our specific needs for each environment now there is one last thing we got to do because uh there's a little bit of an issue and i'll show you why uh right now we what was the last commandment we're running in production mode right so we ran the docker compose.prod and i'm going to do a docker ps let's just quickly get the name of that docker image and i'm going to connect to this docker container so we'll do docker exec it and copy that a name and then we'll do bash all right so here what i'm going to do is well first of all you'll notice that we copied a whole bunch of docker compose files and that's because in our docker ignore they're not in there so we can add those in there as well and we can just say docker dash compose star and that'll just ensure we don't copy anything uh or any file that starts with a docker dash compose so this is kind of a wild card for matching anything after that so we'll hit save on that but that wasn't really the main thing i wanted to address actually so if we go into our node modules folder this is kind of important if i do an ls this is going to show you all of the dependencies that we have all the packages and there's something very important if you look at this you can see that node mod is installed and you might be wondering why is that an issue and i want you to remember we are in production mode right and our in our package.json we can clearly see that nodemon is a dev dependency we don't need this when we run into production mode because we're never going to be using nodemod in production that's only for development so that it automatically restarts the node process when we are developing our project so how do we actually prevent nodemon and any other development dependency from getting installed because right now it's just taking up space and doing absolutely nothing well in our docker file you can see that we run an npm install uh if you want to actually deploy this to production right you would normally run a um you would normally run i think it's a dash dash only equals production right and so that'll prevent any dev dependencies from getting installed um because you're running in production mode so what we have to do now is set up our docker file to be intelligent enough to know uh whether we are in development mode or production mode and then either run an npm install or an npm install dash dash only equals production depending on if we are in our development or if we're in our production environment so how exactly do we do that well we're going to have to basically write an embedded a bash script so here instead of this line right here what we're going to do is i'm just going to go under here and we're going to replace it with this i'm going to say run i'm going to do an if statement and we're gonna say if uh and then we're gonna do brackets and this is important make sure you hit one space all right this this gave me all sorts of issues you want to make sure you put that space uh and then in quotations we'll do dollar sign node underscore env and then that equals development and this is also important hit one space afterwards i don't know why it required that but it did and i was troubleshooting that for a while so make sure you get the spaces just right so what we're saying is if we are in a development environment then we want to run an npm install however we're going to do an else if we're not in development so we're in production we're going to do npm install dash dash only equals production all right and then we end that if statement and we no longer need this run command okay so basically um you know we're referencing some kind of variable here called node env and then when it's set to development we'll do an npm install or else we'll do an npm install with dash dash only equals production now what exactly is this well this is an argument that we have to pass in so here we do arg node underscore env so this is an argument that gets passed into our docker file when it's building the docker image and we have to set this value in our docker compose file so under dockercompose.dev and dot prod we're going to have to pass that in so going to the dot dev file we are going to actually overwrite something in our um in our base docker compose file so instead of doing just build that we're going to have to change this up a little bit so here uh when we do the build command we're going to override that and instead of just doing dot instead of just doing colon dot we can actually break this down into a few more settings we can do build colon and then pass in two properties we've got the context and then we've got the args now the context remember when we did build dot and that's just specifying the location of the docker file that's what the context is so here we just specify the location of the docker file which is in the same directory so we just pass in dot once again and then here under args we pass in all the different arguments that we want to pass so here the only thing we care about right is the one that this docker file is using and it's called node underscore env so we do node underscore env i'm going to set this to development because we're in the development docker compose file and we're going to copy this and i'm going to do the same exact thing in my prod file and we're going to space that there we go i think that should be good yep and we're just going to change this to production and let's save everything and um let's just do i'm gonna bring this down all right and let's just make sure we save everything again all right so now um it should theoretically you know run this if statement um when we want to do an npm install and it should be able to detect depending on what the arguments that's passed in so we're going to run a development fold first just to make sure everything's working and then we're going to run it in production mode after that just to make sure that that actually made the changes that we wanted so let's go back to my up command i'm going to start off with dev all right so it's building that remember i told it i passed in the dash dash build flag because we made changes and docker components is not an intelligent and will not know that we need to actually rebuild it so i always pass in the dash build flag in that scenario and let's test this out so hit refresh just says hi there which is fine when we go to our index.js let's add some exclamation points hit save refresh perfect so just the fact that it restarted automatically and adopted those changes we know that nodemon was successfully installed uh and so we know that that if statement so far right here it ran an npm install because it detected that we were in development mode so so far everything's looking good let's bring everything down with the with the dash v flag let's delete that volume uh and this time we're to bring it back up but i'm going to do it in rod all right and let's just do a docker ps just make sure everything's running perfect uh let's hit refresh perfect uh let me make changes to this code real quick just to make sure it doesn't adopt those changes so we'll hit save let's hit refresh no changes perfect the last thing that we wanted is let's log into the container and let's just make sure it did not install node mod just to ensure that it did successfully run the dash dash only production flag so we'll do docker exec it and then bash right ls let's go into node modules also notice how our docker ignore file worked this time because it did not import any of the docker compose files but let's go into the node modules folder let's do an ls do you guys see anything with known mind it should be alphabetical right so if we go to m and uh see nothing with node mod so it successfully is in production mode and if you do a print nv uh we can see that i would have thought we would have set uh node e and v set to production too so everything looks like it's working it successfully only installed all of our production dependencies and so guys that's all i wanted to show you it's just a matter of two commands uh whoops well looks like we're not actually gonna get out of the container but it's just two commands right so you do docker compose up docker compose down um and then if you're in production you pass in the prod if you're in development you pass in dev that's all you have to do you've got two different environments and we can easily spin up all of our containers and then also spin them back down with just two simple commands so hopefully you guys enjoyed this video if you guys want me to show you guys how to kind of build a actual full ci cd pipeline let me know and i can build i can set up a follow-up video explaining how we can do that using github actions you

Channel: Sanjeev Thiyagarajan

Views: 24,525

Keywords: docker, container, compose, node, node.js, express, app, javascript, development, production, dockerfile, dockerignore, tutorial, programming, web

Id: gm_L69NHuHM

Length: 104min 50sec (6290 seconds)

Published: Mon Mar 01 2021

Reddit Comments

[u/sloppy_networks]

Made a video for people who want to get started dockerizing node.js/express app.

00:00 Intro & demo express app

04:04 Custom Images with Dockerfile

10:20 Docker image layers & caching

20:12 Docker networking opening ports

26:22 Dockerignore file

31:32 Syncing source code with bind mounts

45:16 Anonymous Volumes hack

51:44 Read-Only Bind Mounts

54:44 Environment variables

59:02 loading environment variables from file

01:01:17 Deleting stale volumes

01:03:47 Docker Compose

01:21:22 Development vs Production configs

[u/Neitzches]

What great timing! I've got a branch where I have been trying to dockerize for dev and prod and messed it up. Thanks for sharing!

[u/roofwindow]

Thank you. Very good teaching, I highly recommend!

[u/anatolhiman]

Followed!

[u/javascriptPat]

Very good, thanks!

[u/fungigamer]

Saving with a comment. This is literally what I want thanks!