Lesson 4: Whats under the hood - Namespaces, Cgroups and OverlayFS

right so let's look at what makes containers possible what makes it possible to run this process in isolation that we just looked at here it's just one or two processes running in isolation and without an overhead of booting that complete operating system how is it possible to do that you've wondered about it now the answer to that lies in the Linux kernel Magus kernel is where you know which offers us these features which make it possible to run this process in isolation or a sandbox environment and now with the recent version of Windows servers Windows has modified the kernel to support these features that Linux kernel had and that's what makes Windows containers possible now let's look at which are this feature which make it possible to do that and the first feature is the namespaces now what do you mean by namespaces is what you see here the processes are running in isolation they're actually running on a VM but it appears as running in isolation because these are namespace so what you have inside the container is a set of P IDs of process IDs for your own process itself and that gets mapped to certain PID so this actually gets mapped to some P IDs on the host system right here so inside containers it's one two three four the P IDs that you see here they are actually mapped to the process right here on the host so they are running on the host but they're still running in a sandbox isolated environment that's the PID namespace and then there are a bunch of other namespaces which make it look like and feel like a virtual environment just like running a VM and that's what isolates one container from another one or one process rather from another one in a sandbox environment and what are the main spaces there is a file system namespace so each container can have its own operating system other file systems each container can have its own network namespace so it can have its own IP address that you see here and the interface each container can have a UTS namespaces is for the hostname so it can have its own hostname it can have its own PID mainstage that I that I just showed you and recently we can also create user name spaces that's an important security feature because earlier if you were a root user inside the container if you could break out of the containers you could gain the root access on the system but it's no more possible because you can map it to a non root user on the system even if you break out of the container so that there is an isolation there is a namespace around it so that that's interesting one right and that's what makes running processes in isolation possible now when you run this process in isolation and launch multiple containers on the same host there is a possibility that there is one container which has a memory leak for example and then it affects rest of the system or other containers running on top of that right and that's where Google's contribution to the world of container comes into play which is which was called as process groups but now known as C groups or control groups and what control groups allow you to do is control as the name suggests control the resource utilization and keep a tab or a limit on the memory CPUs and so on and so forth so even if you are running multiple containers just like VM you can assign those containers a certain amount of memory and certain shares of CPUs on the on the host system that it is running on apart from controlling the resources C groups are also responsible for collecting the metrics and that's what is used in most of the monitoring systems which are used to monitor the docker containers and the docker environments mainly the stacks UI stats API is apart from namespaces and C groups which are two most important features the file system that docker has chosen is quite interesting and that's what we're going to look at here now talking about the file system let's again compare it with the VMS so when you talk about VM let's say you have an application let's say we are talking about a software delivery so when you deliver your application to the customer if you are delivering it as a VM you're going to have your application along with the runtime environment so your application along with let's say your app app is 50 MB and then you have some libraries which are hundred MB and some binaries and dependencies which form another 100 MB and then you also need a operating system with the kernel and so on which is about 200 so the complete image size is 450 MB now let's assume we upgraded our application from v1 to v2 what happens now is even though the changes that we make are maybe just 50 MB the application layer application part we have to build the complete image of let's say for 50 MB in this example now that's what happens even if you change one single file on that VM imagine that right so every time you make a change you transfer for 50 mb/s now let's compare this with how it works with the docker in specific now when you talk about docker even though you have the image which contains the application and the runtime environment and it could form for 50 M B's as well so it is comparable to VM in the size of the image itself however when you actually transfer it since you have made changes just to one component just the one layer when you transfer it you are not transferring for 50 MB but you are only copying over just the changed layer that is 50 MB and that's what makes it very very lightweight to transfer so containers are not only lightweight to run as we looked at earlier it is also lightweight to transfer because of the file system that docker has chosen and the kind of file system the docker has chosen is called as overlay file system and that's what makes it really really interesting here and let's look at how it is how it is form right now even though you you have multiple different layers logically it will appear as one single image and when you transfer though it works as are saying works similar to arcing where it is just comparing the diff between what you have and what we do not have and just copying over the layers which you are missing there and this is possible because of the overlays file system that docker has chosen and to implement this overlay file system there are multiple different library drivers and overlay file system supports layering concept there is another concept copy-on-write which we're going to look at next but this is implemented using different types of storage drivers and the choice of storage drivers typically depends on the underlying operating system that you have let's say Ubuntu uses a USS core OS uses a ESS Red Hat our Center is generally uses either device mapper or betrassus there is VSS or VSS as well which is possible right and that's another interesting concept that docker has in terms of the file system that it has chosen now it doesn't stop there it gets even more interesting and that's what we're going to look at here let's say you have built an image which is about 500 mb/s and when you actually create an instance out of that and this is for a VM so we am you you may have a template or an image or a MI or whatever you call it as that golden image and when you actually create an instance out of that image what happens first when the one instance is launched is this complete image gets copied over so there is a replica which is created and the running instance will have the cloned version of your original image so it picks up twice amount of space and second is when you launch an VM it takes longer because it is actually cloning that disk basically right so that adds to the runtime or the boot time as well now how does it work with docker again it gets very interesting here so if you look at the way docker works let's say you have an image docker image which is 500 mb/s but when you launch a container that is the running instance of the docker image it does not clone this image but it rather mount set as a read-only image right that saves you one is the time to take so that's the when I launched Jenkins it didn't take really long time it didn't even take seconds it was milliseconds because it did not clone the image it just mounted the original image as read-only and it allows you to make changes though and the way it works is this called as copy-on-write so what it does is it creates a read/write layer or writable layer on top of the existing image that you have so existing image is mounted and any changes that you make the if you are making changes to a file inside that image it gets copied over here and that's what gets written and that's what you write actually so it allows you to write it appears as a virtual machine but internally it's actually copying a file from that read-only file system and that's how it allows you to make changes so this concept called as copy and write makes it extremely efficient to run those containers without taking up a lot of space as well as a lot of time and if you want to learn about copy and write I can point it you a bonded your resource here so that's from the Dockers official images and this is an interesting technology and all of these choices that docker has made in terms of the underlying technology is what makes docker a very interesting and compelling proposition for building and deploying your software

Channel: School of Devops

Views: 17,969

Keywords: Learn Docker, zero to docker, namespaces, cgroups, what is docker, what are containers, overlayfs, layered filesystem, school of devops, gourav shah, docker champion

Id: 2ZdJ_3sBr6A

Length: 10min 12sec (612 seconds)

Published: Tue Jun 20 2017