Docker Networking Tutorial // ALL Network Types explained!

in this video i want to address a topic where i literally get at least one question every single week where people are asking me about so how do i connect my reverse proxy in docker to my other containers or how do i manage docker networks in general so i thought let's make a video where i explain it to you and show you all the different docker network types like bridges mac vlans hosts and so on and i also show you some examples of when and how to use them but first of all if you want to authenticate to your server securely or access your environments with additional two-factor authentication and take a look at the sponsor of this video you probably know it it's teleport because with teleport you can protect and securely authenticate to your server's infrastructure log any users actions and record sessions you can install the free community edition completely self-hosted at no cost so just download and try it out and suppose you want to use teleport within your business environment in that case they also offer an enterprise version with additional 24 7 support and active directory integration and much more so just reach out to the teleport team so if you know how to work with docker you probably should know that you don't necessarily need to configure a network because docker always creates a default network where all the containers you deploy are automatically attached to and for most basic setups this is enough if you just need a single tier application like a web server a dashboard or maybe even a management tool that's totally fine so i guess for very basic setups we probably even don't need this video but hear me out especially when you are planning a more complex setup or multi-tier applications it makes a lot of sense to configure networking in your docker stack and specify how these containers are connected to each other and to the outside network because maybe you need some containers that would need access to a database or you want to isolate them from all the other containers on the host in these situations you need to configure your networks appropriately so let's take a look at the different types of networks okay so i opened two different shells here to my home server where i'm running a docker host in some darker containers and this is actually my production home server setup so i hope i won't break anything here but it's just the best way to demonstrate to you how the different types of docker networks are working and first of all we will start with the first docker network type which is a bridge network a bridge is a virtual interface that connects all the internal docker containers to the docker hosts network so for example when we run a simple container like an nginx container this would be automatically attached to the defaults docker network which is also a bridge network and if we want to access any specific service in that container from outside you would need to expose this with specific port numbers for example for http connections on a simple web server we can expose the port 80. so when we run this container and try to access it from outside we can just simply open a new http connection to the docker's host ip address which is in my case the 1921680.5 and we can now access the nginx web server container which is great so this is nothing new to you hopefully but i also want to show you how this is working under the hood when we inspect the nginx container for example you can see that this container is attached to a network which is called bridge and it also got a completely different ip address here it got an ip address from the range 170 to 1700 16 and the ip address is the 10 at the end so how does it work the docker host will automatically create new interfaces for every bridge or network that you are creating and it will automatically attach those containers to that bridge within its own subnet you could also see that a specific interface when you type in ipa and search for the docker zero network so this should be a docker network that is automatically enabled on the docker host once you install docker and you can see this is our ip network this is our subnet so every container that we are running and not specifically attaching to any different network will be automatically attached to the default bridge network which is using the docker zero interface in the host and all the containers that you're running there get an ip address from the docker's dhcp server in that specific subnet then you can connect all the containers to each other by just referring to the ip address so you can reach container 2 from container 1 by just referring to its ip address within that network and you can also reach any containers from the host but one thing you can do with this default network is to isolate docker containers from each other and do name resolution so when you're running some containers here in that default network you can only refer by ip address and then you can get into problems for example when you want to access a database that's running on container a and you want to connect to that database from container b you can refer to container a's ip address but that might be changed once you redeploy that container or you deploy other containers so therefore it's useful to create custom bridge networks to isolate those containers and to offer dns name resolution within those networks so let's take a closer look at this for example when we enter docker network ls you can see all the different networks that are currently running on this host and you can see there's our default bridge network which is the bridge driver we also have the host network where we will come to later and we also have the non-network so when you want to completely isolate a container from all the networking access just connect it to the non-network and it won't be able to reach anything and i also have a custom bridge network which is called npm default so this is one network i'm using in my docker stack where i'm just connecting one application with one database and i don't want these applications to connect to any other containers within that darker host so i'm creating a custom network where i also can refer to the name of the container by using dns name resolution let me show you how that works you can also create custom networks by entering docker network create and then simply the name of the network just like custom bridge for example and when we now check it you can see that there is another bridge network created with this specific name and it also got a different network id so this will also create a second or virtual bridge interface on the darkest host so when we search for that id here for example let's type in ipa and grab for this id you can see there is another interface that was created with the name br dash and then the id of the network and it got a completely different subnet and when we now start a container and attach it to this custom bridge it should get an ip address from this subnet to do troubleshooting on darker networks i just found a very very nice image which is called netshoot and this docker image is a very lightweight docker image that comes with a bunch of different network troubleshooting tools and you can easily use that on docker or kubernetes to do networking troubleshooting between containers you can see this is an image that comes with some included packages like a bash terminal a curl an iperf an ip tables ip set also an nmap or tcp dump so you can do a lot of things with a basic troubleshooting tools on linux by the way i might do a separate video on some of these tools as well so to run this container you can just go to my cheat sheets repository on github you can find that in the description down below and for the docker cli tool i also created a cheat sheet with a bunch of different commands you will use many many times when you're working with docker and for example the network troubleshooting tool is also including the net shoot docker image so let's run a new container and attach this to the network by using the double dash network so you can do that on all containers that you're running and attach it to the custom bridge network so this will now open a shell inside the netshoot container so we are not on my home server anymore we are now inside the docker container and if we now execute an ipa command you can see that it got an ip address from this new ip address range from our custom bridge network and we can now just access the internet here just like on our default network and we can also ping the host network for example when we ping the ip address of our docker host but what we can do is we can't ping any ip addresses inside other docker networks for example let's try to ping the nginx server it got the 10 at the end so you can see this is not working because docker has isolated this custom bridge from all other networks and what we also can do is we can use name resolution within that custom network so for example if we just run a second netshoot container and give it a different name like net shoot 2 and also attach this to the custom bridge network you can see if i now ping the ip address of container one which i can't remember so this should be the two at the end oh you can see that i can reach net shoot one but i also can resolve the name of the container which is net shoot so when i ping net shoot for example i can also ping this first container and on the first container i can also ping net shoot too by using the dns name and it will resolve to the dot free ip address okay so we covered the default bridge network and also the custom bridges this network type is probably the most used type so i would say that this type i probably use in 90 of all the use cases and it's great because you can isolate containers from the host operating system and you have an automatic dns resolution between them at least in the custom networks but sometimes isolation is not what you want i'll give you an example suppose you want to run containers that should connect to other local services or resources on the host and use the same routing tables if you deploy a wireguard vpn container for example you might not want to isolate this container with a separate virtual ip address instead you want to run this application directly connected to the host network just like every other application that's installed without docker so in these situations i typically use a different network type the host network type and this networks completely removes the isolation layer from the host so it handles the container as it would just be a regular application on the host to demonstrate the host network this is very easy first of all let's stop our nginx container that is still running in the background and let's start creating another nginx container so this time we're also starting an nginx container but instead of exposing the port we are now attaching this to the host network and we don't don't need to expose the port anymore because the host network doesn't have an isolation layer between the container in the host it doesn't have a virtual interface it will just run this container as it would run without docker so let's just hit enter and you can see if i execute the docker ps and search for nginx you can see this container is running here but you can also access this web server by entering the ip address of our host and this is possible because when we check the listening ports on our docker host you can see that this is currently running and listening on all http connections on all ip addresses so this is how the host network works okay so now we got bridges and host networks but i also want to show you two other network types which are very handy in scenarios where you have applications running on the host but you need a separate ip address for them from the physical network and i needed to set up this really just in one case when i was testing pie hole on my docker host and pihole if you don't know that yet it is a network-wide ad blocker by the way if you're actually interested in piehole it's pretty nice to manage dns records and block ads within your whole network and it's doing this by running its own dns and dhcp server for your network you'll find this application is most of my deployments and tools that i'm running in my home lab as a docker compose template on my personal git repository boilerplates so just go to my personal guitar page you will find a link in the description down below of course and go to the boilerplate section and then use that to deploy piehole anyway so back to docker networks if you might want to run such an application that requires a service like dns or dhcp but on the host you already have a dns or dhcp server running and when they then try to start a container you'll get an error message that this part is already allocated so how could you run this container well if you don't think you can just use a bridge network and expose the service on a different external port yeah this might work for some services but it won't work for dns or dhcp because these are standardized protocols and every client really accepts a dns server to run a port 53 for example there's no easy way to tell your clients hey if you want to make a dns request then please use support 8053 instead of the standard one for these situations we have mac vlans and ipv lands in docker and they both are very similar and allow you to assign a mac address to your docker containers and bridge this to your physical network connected with a host so to demonstrate how mac vlans and ipv lens work on docker i want to show you what happens when you try to create a container and expose a port that is already running or used on the host so remember we still have our nginx container that is running here and that's allocating or bound to the port 80 on the docker's host network so when we now try to run an nginx container and try to expose the port 80 you will get an error message because the port 80 is already in use by the host or by another container or an application it doesn't really matter but when we still want to expose the port 80 on this container we can't change it we now could also create a different docker network which is called a mac vlan and bridge this container to the physical network of our docker host and use the same ip address range the same subnet like the docker host is using we still have our networks here we now want to create a new network which is called mac vlan so enter docker network create and now we need to specify a different driver because the mac vlan is a separate driver in docker so with the hd parameter mac vlan we can specify that and now we also want to specify the subnet and the ip address range of this network interface because remember we want to bind this to the physical network of the darker host which is using the ip address range 192 1680.024 so we will need to use that same subnet here so specify the subnet material for subnet mask and we also want to specify the gateway because otherwise the container doesn't know where to connect to this network and now if we would create the network with this configuration and attach a container to it you would probably assume this would get an ip address from the local dhcp server in your physical network so this is unfortunately not the case because this is really dumb let's be honest because docker uses its own dhcp server and when you configure it this way it will try to get the first container the first available ip address in that range which will be the one and then i get an ip address conflict with my gateway the same happens for the second container which will get the iprs2 at the end which is probably in use by another network device within your physical network so to avoid this there are several ways to do it so i do it this way i specify the ip address range of the ip addresses the docker host should assign to the containers and i know limit this to just one ip address not used by any other device within that network so i will use the iprs253 at the end which should be the last ip address that is ever used in a network and i also exclude this from my local dhcp server to avoid any ip address conflict so what i do now is when i create those containers within that specific network i will use a static ip address so i will show you that later let's first of all create this network we also need to specify the parent interface so the parent interface is the interface where this mac vlan will be bridged to in my case it is the ans18 which should be the physical interface of your docker host and we also need to give it a custom name for example custom mac vlan for example let's hit enter and when we check it you can see that there is a new network created with the name custom mac vlan and it got the driver macvlan now you can also search for that interface but you won't find it because this is working differently under the hood in the darker networks so don't worry about it it will be handled differently so now let's start running our net shoot container again but in this example i don't want to connect this to the custom bridge but to the custom mac vlan and if i know would run this container it would get this one ip address this 253 at the end so let's try to do that and let's also see what happens when we try to run a second container on the network you can see this is not working because there are no available ipv4 addresses in the network remember i've just specified within that network that it should have exact one ip address and it can just allocate what it has but we can still run a container by using a static ip address so let's create a new container here on that same network but it's using a static ip address from my physical network which is using 200 so i i know that this ip is not used by any other device and the second container will also use the ip address 201 at the end so let's hit enter here and now i'm running two containers in the mac vlan so i can now just try to ping the gateway which should now work and i could also reach the internet from these containers and of course i can also ping the second container from the first one so it should be bridge to the physical network so now if we try to ping this container from outside you can also see this is working so this is a completely different machine in a different network and i'm trying to ping the 200 you can also see this is working and you also don't need to expose any parts here for example in if we want to run an engine x web server so let's do that again and i want to connect this to the network custom mac vlan with the ip address 202 at the end i don't need to expose the port here let's run this and it's try to access the web server here it now gets a different ip address it gets the 202 at the end you can see i can now access the container so you might now ask so where is the difference between mac vlans and ipvlance well there's not much difference but there's one significant difference which is that mac vlans allocate a different mac address for every container that's attached to the network and ipvlance only use one mac address for all the containers and this is sometimes needed for example if you're running docker hosts in a network where a switch doesn't like that one specific port is running different mac addresses and the switch probably makes problems you could then switch to an ipvlan because then there's only one mac address on the part of your docker host and it basically rolls the connections based on the ip address there's also another difference between ipv lens and mac vlans and this is ipv lens can operate in layer 3 and layer 2 mode so you can change that and it's depending on your use case and setup i've personally not seen any use case where i need this so in a home network mac vlans always solve the problem for me so i never used ipv lands but just in case you have some problems with your network switch because it's rejecting your packets because of the multiple usage of mac addresses on one single port you might switch to ipvlan and see if that works for you okay so i believe we covered all the important network types there's just one more the overlay network so this is a network type that i've personally never used because it's only important when you're running multiple docker hosts in a swarm cluster and you want to connect those containers through one network that's spread across those different hosts but let's be honest guys who uses darker swarm if you need a cluster to run containers i would just go with kubernetes which is probably much more convenient these days but anyway if you still need overlay networks you will find the documentation on the darkest homepage i'll put you a link to that in the description of course so i'm done for today thanks everybody for watching the video and i'll catch you in the next one bye

Channel: The Digital Life

Views: 13,369

Keywords: docker networking tutorial, docker networking, docker networking between containers, docker networking explained, docker bridge, docker bridge network, docker bridge vs host, docker container networking, docker network tutorial, docker netzwerk, docker netzwerk tutorial, docker swarm cluster, docker tutorial for beginners, container networking, devops tutorial, devops tools

Id: 5grbXvV_DSk

Length: 19min 45sec (1185 seconds)

Published: Tue Oct 26 2021