Docker Internals Session1: Introduction to Containers and Docker (cgroups & Namespaces)

[Music] [Applause] hey guys welcome to my channel and in this video I'll be talking about containerization and docker containers are the new hot topic in the ever-changing world of fighting and I'm sure you hear about it if not already in this video I'll take you through the important concepts you must know to understand containers and will also provide a brief history and introduction to docker now many people address containers as lightweight virtual machines and at first glance it does look and feel a lot like virtual machine however they are anything but that even though both virtual machines and continuous providers the field of abstraction over the underlying hardware they both differ vastly in their working features scope and a level where they function in the stack hence it is not advisable to compare virtual machines and continuous with the same length since that might lead to wrong assumptions understanding containers requires a totally new approach and that is what we are going to do now so for now we will suppress our temptation to start with the comparison and rather understand the containers first then later we can conclude for ourselves on how they are different from virtual machines in simple terms containers are nothing but customized isolated processes running inside your voice they are no different than any other OS process in the way OS treats them as they share the same kernel container processes use existing kernel features to create an isolated runtime environment it is important to note that these features already exist in almost all new Linux kernels and applications like docker only facilitate the creation and management of such isolated processes now the Linux kernel code does not directly provide provisions to implement containers however they can be built using a combination of kernel features which we will discuss next the first idea of containers and process isolation came with the chroot system code that was introduced during the development of version 7 unix in nineteen denying a siege shoot on a UNIX or Linux based operating system is an operation that changes the apparent route territory of a process and it's children this was the first apparent demonstration of primitive process isolation within an operating system now in this demo I have my Linux machine which is running a center 7 OS let me open two terminals to demonstrate how chroot works now on the first terminal I am inside my directory slash temp slash jail on the second terminal I am in the root directory and have listed all the directories in it Here I am planning to make the slash temp slash jail as the root directory for a new batch process that I have launched now under this directory I have created three more directories bin lip 64 and lid I have also created files just to mark our actual position one inside the jail directory and the other in the actual system route directly inside the jail bin directory I have copied the binaries for basic Linux commands like bash LS and CD since once I change the root directory for my process I won't be able to access the actual bin directory I have also copied the supporting libraries required beforehand now let us launch a new batch process using CH 2 CH root the directory path and the process and there you go our new bachelors launch however something is changed if you notice when I do a listing of my root directory I am actually inside the change directory this is not the actual root directory of the system which you can see on the second terminal here using CH dude we have changed the root directory for the new batch process we launched this creates a simulation of a root directory for a process and its children using a system non root directory now see root alone is not enough when it comes to total process isolation as it just modifies the part name lookups for the process and its children however it was one of the first demonstration of the idea of containerization and process isolation as one of the need docker developer famously put it today continuous a basically chroot on steroids now over the years UNIX and Linux systems developed many new features to build upon this idea and created various models of containers until finally in 2008 we had the LXE which is short for Linux containers docker was built upon alexey and released in 2013 which became the most successful model till date to create monitor and manage containers now Alexian docker build containers based on two very important feature of the kernel namely c groups and namespaces which we will be discussing next now the Linux kernel features C groups or control groups allows you to limit resources such as CPU system memory network bandwidth or a combination of these among user-defined hierarchical groups of processes running on a system traditionally all processes in an OS that allocated similar amounts of system resources that can then be modulated by an administrator using the process nice value here in the top output the ni column represents a nice value of each process however in this approach applications that had more processes tend to receive sources than the applications that had less processes irrespective of the application importance now see groups use a different approach with C groups you can create hierarchical group structures where each individual node represents a process separate hierarchy structures are created for individual resource controllers like CPU memory etc these resource controllers are also called as subsystems in Red Hat machines nodes inside each group inherit the attributes of the parent groups and the resources used by each process is charged from the parent group resource Kota with Red Hat 7 the C group hierarchies are binded with the system D unity hence C groups are also called as system units in Red Hat 7 now on my system I am running to docker containers so let us view the sea hoop hierarchy for say the CPU subsystem using the command system D see GLS CPU here you can see docker has created two subgroups for our two containers now all the processes under this container will belong to the same container C route or the system unit my first container is running one process under it while my other container is running two processes under it and it is easy to control resource limits by putting resource restriction to the container subgroup instead of controlling it at individual processes similarly we can also give the C group subsystem for memory the list of subsystems mounted on your system for Red Hat 7 can be found in the file / proc /c groups now cgroups was about limiting what your processes he used where as namespaces are about limiting what your processes see namespaces are used by process to see and identify system resources so by manipulating namespaces you can restrict water process can or cannot see on your system this feature is key to continue ization applications like docker to simulate an isolated runtime environment as of version 3.1 to Linux supports six namespaces namely IPC mnd net PID user and UTS the namespaces for each process to join is specified under the directory slash proc the PID number / NS note that a namespace will get destroyed when the last process in that namespace exists now let us discuss these names faces one by one in a little more detail PID or process ID is used to uniquely identify a process within that namespace multiple PID namespaces can be nested in such a way that the process will have unique PID in each of the namespaces starting from its current namespace now we all know the first PID on any Linux system is reserved for system D or ini T process however nested PID namespaces can have other processes with PID 1 which is separate from the system PID namespace Here I am running a container inside my base center a certain system now in my container I have launched two processes first the bash process and then launch a child process top note that the bash process has a PID one and its child process top has a PID 95 now these pids have their scope limited to this container only let us see these processes from our base system on my base OS if I execute the command PS - EF there you go we can see our container processes however they show PIDs of one 194 and 7/8 to 6 respectively this shows how nested namespaces work just to add to that namespaces enable us to grab a global system resource such that for a process within that namespace it appears that they have their own isolated instance of the same resources let us now briefly discuss the other namespaces net or network namespace is used to virtualize and identify network resources on the system each network namespace that you create will have a private set of IP address its own routing table and other network related resources now you can create virtual network interfaces ve th these virtual interfaces come in pairs which behaves just like a crossover cable so that you can use them to connect a namespace to the outside in default use cases multiple virtual interfaces from different namespaces can be connected to a virtual bridge or the host machine where the physical interface exists on my centrist host machine and currently running to docker containers now if I list all the interfaces here you can see that two virtual interface peers for the two containers now the other side of these peers would be on the container which you cannot see here now these peers are connected to the docker bridge Dockers zero this bridging enables containers to access each other and the outside world through the host physical interfaces now we will leave the details for later videos since this is just to provide you all a brief overview MNT or mount namespace provides isolation of the list of mount points seen by the process processes inside different namespaces see different mount points which also totally alters their view of the filesystem hierarchy just like SIA should note that when using Amenti namespaces commands like mount and human no longer have global consequence but will only affect the particular namespace there you today I PC is used to isolate certain inter-process communication or IPC resources such as system VIP C objects and message queues the UTS namespace is simply used to isolate system identifies for hostname and finally the UID or user ID namespace provides user and privilege segregation across multiple containers just like the P I namespace user name spaces are nested this allows us to map an unprivileged system user to a privileged root user inside your container this ensures security as even if the privileged user inside your container somehow breaks out his privileges won't exist outside the container on my system I have configured my containers to be launched as unprivileged system user as even though on my container I have root access and I can launch processes as route from outside these processes are launched with an unprivileged user ID this however does create some complications especially if your container needs to access certain resources on your hosts and do perform necessary testings before implementing this now to create a container you can launch the process using one or more such namespaces and group them in control groups to get an isolated runtime environment now different configurations of containers can cherry pick one or more of these features that define the level of isolation further usually the more the isolation the more your flexibility is limited for direct access to OS resources outside the containers but then it increases the security now creating such containers from scratch is easier said than done as it requires some rigorous configuration and testing this is where applications like docker come into picture then streamlined the whole process of and save us a nightmare of writing hundreds of lines of codes or shell scripts every time you want to create or modify a container on my sent over seven machine I have already installed docker now to launch her container I can simply execute a one-line command docker container run - IT for interactive terminal will use the center seven image execute the command bin bash and there you go we are inside the container it is that easy similarly you can launch containers for any application you want using its docker image about which we will discuss in some other video on a side note there are many types of container runtimes available not all of which are based on C groups and namespaces some of them use completely different approach now without going into much details I will just list some of the well-known container runtimes classified on whether they are based on C groups and namespaces or not now that we have gained some good insights into the concepts and working of containers I think it is safe for us to not try and compare containers with virtual machines most of us already know the basics of virtualization it starts with a hardware layer on top of which we have the hypervisor the hypervisor is responsible to abstract the underlying hardware resources and provide them as multiplexed units to its upper layers this allows us to install multiple operating system kernels on top of the hypervisor also called as virtual machines next you need to install libraries and application dependencies on the VMS and then finally we run perhaps on the VMS now let us compare the stack using containers again it starts with the same hardware layer now instead of hypervisor we have the Linux kernel now the kernel runs a containerization demon like docker this demon enables creation of containers or isolated runtime environments each container can have its own set of libraries and dependencies and then finally each container can separate apps in isolation with each other now the differences are quite obvious most notably unlike the hypervisor continued ization engines do not form an abstraction layer in the stack but runs just as a teaming on the same kernel as a containers the abstraction of hardware resources is managed by the kernel itself the function of the docker daemon is to only create monitor and manage these containers using kernel features it is more of a container management interface rather than an abstraction layer containerization provides huge flexibility and efficiency in the way you manage your infrastructure and is currently the hottest technology in the industry containers are extremely lightweight and running them is less resource intensive compared to VMs however that doesn't mean containers can completely take over virtualization today both of these technologies have specific use cases especially around the topic of security and data management some experts do suggest that using containers along with virtualization provides even better advantages as we discussed before the concept of containers is not named however it required the emergence of docker to really pull it into the mainstream of application development today almost all big IT companies are investing to develop and integrate their technologies around the container concept now docker is relatively new and a lot of its features are still under development however with these trends one thing is for sure docker and containers are here to stay that is all for today's session stealing from mostly and you don't forget to like share and subscribe see you next time [Music] you

Channel: TheITNoob

Views: 10,465

Keywords: Introduction to Containers and Docker, Understading cgroups & Namespaces, Containers, Containerization, Docker, control groups, cgroups, namespaces, difference between containers and virtual machines, containerization vs Virtualization, What are containers, Isolated runtime environment, isolated process, systemd

Id: 0Xb421-9CTo

Length: 17min 38sec (1058 seconds)

Published: Mon Sep 03 2018