Hey everyone, in this video I want to look at what are containers? What is this Docker thing? Because everywhere we look at technology today we hear about containers and Docker and Kubernetes. So we're to dive into really what is a container? If we go and look at an operating system, do we actually see a container object? Is it a type of resource? Spoiler it isn't. So let's take a step back though and where the containers come from and why do we even need them? And it really comes down to I want to run applications. That's what I care about. My whole goal fundamentally is for my business I have some requirement that I have an application. So I have this application which would be a number of processes that I need to run now this application. Well, it probably has certain dependencies, so I could think about the app itself. Well, it might require certain runtimes, it might use certain frameworks, certain Dlls, whatever that might be. And of course, all of that runs on a certain operating system, which could be Windows, could be Linux. And all of this has a certain amount of resource, because for right now I'm thinking about bare metal. This is early early days, although even today there are some scenarios where I don't have a hypervisor I'm running directly on the hardware of the machine. And that hardware we have things like CPU with certain number of cores as functionality. We have certain amounts of memory. It will have certain storage. Now some of that storage is local, some of it might be remotely attached, and then I'll have some network connectivity. So this box has a certain amount of resource and I run that application. So we have the OS, we have dependencies, we have resource, and we have the app. But this was really, really wasteful. Because if we think about some of the challenges associated with this, I have all of this just for one application. Now I could absolutely also say, well OK, let's run app two on the same box, so I'll run app two on the same machine. But straight away, maybe this is an evil application and we get challenges with this. Now I have multiple apps on the same operating system instance. Now, straight away, if I'm an application administrator, I'm an owner. I need permissions, but often those permissions are scoped at the operating system level. So from a role assignment perspective, role based access control, I have now control over other applications. That is not my application. That's a bad thing. I may require maintenance, we certainly will require maintenance. And so now the challenge is, well each app may have different maintenance windows. How do I make that work? It gets far more complicated but I now have to try and workout well OK, I'm can be down for this number of hours. Well you can be down for this number. Where do they overlap? How do we work together to do that? They may need different versions of dependencies. Maybe one of them needs a V1, one of them needs a V2. Basically I get conflicts in what other dependencies I actually require. These could be noisy, like one of the applications maybe is really greedy and so uses up more than its fair share of the resource, the underlying CPU, the memory which starves the other app, which means it can't perform as it should do. Maybe they're actually conflicting other things? Maybe have you both want to offer port 443? Well then maybe there's other tyes of resource that I could conflict on South. I can conflict on these resources as well, but also, what is the amount of resource I need? The amount of resource I need can change. And what this would lead to is when I have to buy the box that I think I'm going to need for the next 5 years, which is just a staggeringly huge amount of waste and if we guess wrong. We have to throw the box away. Hopefully we can reuse it for something else but hey, this is just where I am that I'm now having to go and buy new boxes so running on bare metal certainly running multiple apps on the same bare metal. Was not a good idea. These conflicts, the noisy neighbor, the permissioning, the maintenance and the organization around that just wasn't practical. So bare metal, not a great thing. So then we came up with the solution of well, OK, what about virtual machines? So virtual machines, great. With a virtual machine we still have that same set of resource. So we still have all of these same resources. I'm not going to draw it all out again, but like the CPU for example, the memory, etc etc. What we now do is we put a hypervisor on top. This could be hyper V, this could be ESX, but now the hypervisor sits directly on the hardware. And if I think of a virtual machine, what we're going to do is virtualize the hardware. So the physical bare metal box, well, it still has a certain amount of resource. So let's just move this actually down. So we have the resources still lives on this bare metal. So there's a bare metal box. It has a set of resources. We're going to run a hypervisor on top of it. So I wrote Hyper V as an example, and then we create these virtual machines. So the virtual machine, let's just draw out the idea here. We'll draw 2 because we add 2 applications. I've got my VM1 and VM2, and we're going to virtualize the hardware. So what that means is each virtual machine gets a certain amount of CPU. Each virtual machine is given a certain amount of memory. Each virtual machine will have its own view of the storage. Typically the way this works is this would actually map down to some kind of virtual hard disk files, vmd, whatever that might be. It have its own network connectivity. Then every single one of these has its own operating system, which then. You install what you want. It has its own sets of runtimes, its own frameworks. They're completely independent. And then at the top, sure, let's just keep the colors the same. We have our applications. So in this VM I have my application one, and in this separate one over here I can have application two, and this solved our challenges. If you think about, well, they're their own operating system instances, so maintenance is independent. I can think about well the permissioning or the RBAC. I could set it at the OS level, but it's still only its app in that. But it's its own resources, so it can't be noisy. Now that does depend on how I configure the box. I might over provision heavily for example to CPU, which means it sees more when you add in the sum of every virtual machine is more than exists. And you assume, hey, they don't all Max out at the same time. So potentially there could still be some challenges. But it's its own resources, it's it's own dependencies. I'm not going to get those conflicts so get this great boundary of administration. I can't conflict with anyone else I shouldn't have noisy neighbor if I don't overprovision too much the resources of the virtual machines. But there's still a challenge with this, because if I think about what I'm doing now, they're each running their complete own operating system with a complete set of resources. So that means a virtual machine is fairly heavy. I have the overhead of the disk space of the OS. I have the overhead of the processes that have to run the OS, the overhead of the memory the OS has to consume when I create a new virtual machine. Even if I have templates that I'm copying, there's generally still some kind of specialization process and things that have to go through, so they're fairly slow to provision. That's a pain point. Also what tends to happen this is very rarely prescriptive. IE there's some runbook that someone follows to kind of try and do these things, but hey, the developer had this installed on their box and they forgot to include it because they just took it for granted it was there. So when I then try and run the application in production it doesn't really work. Which all of these together. They're generally not portable. I can't take a virtual machine, for example, from one development environment for one application and just copy it. There is specialization, maybe around the naming, around the configuration. I can't just take it and copy it as many times as I want. So there's challenges with the virtual machine, it's very heavy. I'm wasting a lot of resource. They are slow to spin up. They're not portable. They don't drive. I can be prescriptive if I use the right technologies, but it's not that common. And so VMS, yes, they work to a certain extent, but they're they're too slow, they're too heavy, they don't drive the right behaviors to ensure consistency between deployments of these things. And if we also look at the advancement in how applications are architected. It's microservices. We have the idea now that again, an application is going to be a set of processes, daemon services running. But more and more what happens now with microservices is, hey, I need some task, a service springs to life, does it, and then terminates. That's not practical to try and do that in a virtual machine. If I need to spin up 10 things to go and do something, I can't wait 3 minutes for the VM to spin up. It's not a practical solution. And it just gets too heavy in terms of maybe the underlying infrastructure that churn that has to go on. So we need something else. And as you would imagine, the solution's going to be containers. So the idea here is if a virtual machine virtualizes the hardware, then what we're going to have here is containers. They're going to virtualize the operating system. And because of that, because they're not going to spin up a complete OS every time, but rather enable me to have some isolation, they're going to be very, very thin and they're going to be very, very fast to provision. But what we're going to see as well is they're going to drive a very prescriptive method of creating them, but also going to be highly, highly portable. So this is what we're going to have. Now, if I think about this, then, OK, great containers, they're going to save the world. Let's think about what would be required to make this really function. Now from the start, we could say, well, it's still going to have to be an operating system. So we can say once again, there's going to be an OS. And once again, that OS is sitting on a set of hardware, the CPU, the memory, now that OS. It's going to be Linux or Windows containers started on Linux. You're going to see the most of the time on Linux, but Windows did add components to enable containers to run. So there is a Windows version for containers. So I can run Windows containers, but what you're going to see most of the time is Linux, and I'm going to demo on Linux. So we have this operating system. Great. We're going to have a set of processes that make up the application. So I'm going to say I've still got my idea of my app one, and I've still got my idea of my app two, and I want to run them as containers. I want to run them on the same underlying OS. So if we're going to use containers, what does that actually look like? So if I think about it, we're going to add in OK containers. And I can think of a container as a sandbox, a bubble. We're going to surround the processes of each application. So obviously I'm going to create two containers. I'm going to create container one. I'm going to create a container two that will surround all of the processes within those particular things. So if I'm running on the same OS, great. It's solving a bunch of problems. I'm not wasteful anymore. I'm not wasting all the processes for multiple Os's. I'm not having the disk space, the memory, the CPU. I'm going to create these very quickly because they just so far seem to be some processes on an existing OS. They're going to spin up very very quickly. I still have problems. I still have to consider a number of challenges. Remember we face because this we're just back to here and we didn't like this. This had a whole set of problems. So how do I solve those problems? What is containers gonna do? So what do I need? What are the things I have to have? So remember the first thing I have to have is I have to have the idea between them and even outside that I need isolation I need. Each container not to be able to see anything about other containers or even other processes that aren't containers. Its view should be limited to what's in its bubble and nothing else. That's the first thing I need. And then I might remember noisy neighbor. So we have the amount of resource. So the next thing I'm going to have to have is an element of control on how much. It's allowed to use. I want to be able to meter it. I want to be able to do accounting to see how much is it using. Then we have the operating system which has a whole set of information. But remember my challenges with conflicts of different runtimes and frameworks. I don't want it to just see what's on the OS. So the other thing I have to be able to do here is somehow it's view. I need to be able to modify. I need to give it a different view of the file system, its own individual view of actually what's happening. And if I'm thinking of containers, remember I want ideally this idea of very prescriptive, prescriptive nature of what's in it, but I also want it to be portable. I want to be able to take a container and based on all of its views and just put it wherever I want. This is a key goal to what I have to have, and luckily This is why it started with Linux. Linux already has a lot of primitives on native capabilities that let me do all of these things. The isolation, the control modifying the view, it just exists in Linux, and that's what we're going to do in this session, is I'm barely going to use the word container, I don't think. Instead I'm going to look at how can I solve these problems with the primitives in Linux, which we'll then see. We can bundle them together and that's what actually makes up a container, so we're going to actually walk through. All of these various components to see exactly this in action. So that's the goal. Now before we get going I just want to go over the setup. So in my environment I've got all the commands set up so I'm using VS code. But what I'm running down here I don't know. If you can see I'm actually running Ubuntu. So I'm running an Ubuntu session and I'm using the Windows subsystem for Linux. That enables me to run Linux distributions directly on Windows, but as far as this is concerned, this is Linux. I'm just running on Linux over here now as part of this. The reason I did this and you might wonder and you might have heard of Docker desktop, which is a very simple way to get up and running and using containers. I did not want to use that because my goal for this session is to actually walk through and show you how these things work, which I cannot do. If I'm using Docker desk or at least it's very very difficult to do so if we go through what I've done. So this is one an Ubuntu Linux distribution. I've installed Docker so if I go and look I'm running Docker version 23.03. If we look at my system information and actually notice there is 2 parts to this, which is kind of interesting. What we can see here is there is a client portion. And there's also a server, a daemon portion of it. So there's two different parts to Docker. And keep this in mind when we actually come back and talk about what Docker is later on. For now we can think of, well, Docker is this great experience I can use to work with containers, work with the images that we're actually going to use throughout this. Now to be able to demonstrate all of the different components, I need a container up and running. So what I'm going to do is I today on this machine I have nothing. If I was to look at the images I have, I wouldn't. I don't have any Docker image list, I don't have anything, there's no images on this box. So the first thing I need is something the container can see as its file system. So right now I'm just going to pull an Ubuntu image that this happens to match. The same operating system that the container host is running, but it doesn't have to. So I pulled the Ubuntu image. If I go and look I can see. OK, this is the image. I see a few interesting things around it. We see these weird terms, merge directory for example, upper directory, work directory. We see this overlay to thing and then we see it's got some layers. And it just has one. The Ubuntu is very neat and tidy, it just has one layer. And when I pull this what I'm actually doing is I'm pulling it from saying called a registry. Think of it as a great big store of different images that I can use. And now I have this image I want to go and create a container. So all I'm going to do is Docker run and what I'm doing here in this command. So I'm using the Docker CLI. I want to create a new container. I want this container's name to be Ubuntu. I'm going to run it detached so it doesn't take over my session, but it's also interactive. This is just something I need because I'm going to launch a shell process that otherwise would just disconnect, and I'm going to limit its resources to a certain amount of memory and a certain number of Cpu's. I'm going to create this new container using the Ubuntu image that we just downloaded. And I'm going to start within this environment a bash process. So we talk about what is an application, it's number of processes. Why? I'm just going to launch bash. So I'm just launching a shell. That's it. So if I run this, it's finished already. So that's the first thing to notice here. I've created a new container pretty much instantly there was. Maybe it's not even a second. Now there's other examples I have in this sample file that you can play around with. I've got an Alpine, I've got a Microsoft Mariner. You will notice the Microsoft Mariner is stored in a different registry. Microsoft's own registry is not stored in the Docker hub, but I can still interact with it. But now I can go and see what containers do I have. So this is a Docker PS A so show me all of the containers I have and I'm only doing this to show you. Well it's running. It's using the Ubuntu image, the process it run bash. It's been running for 40 seconds and it's again name is Ubuntu so you can see that's running. Now what I'm going to do quickly is just grab a few values so I can use it in my demo later on South. What I want to do is grab the Docker ID for this container. There's also a long ID. I need to store as well and I actually want to get the process ID that is looking after the process running within this container and this will all make sense later on, but I'm setting these up so in the script as we go through this it will look a bit neater we could run these commands. So again if I for example just run without storing it to a variable. You can see that's the long ID, and the psaux just shows me all of the processes and I'm searching for the particular docker ID I just executed so I can see I've got this strange process here. This container D Shim run CV2 is actually hosting. This container I created and we're going to look at this in a lot more detail, and I've also got some commands here that you could just go and look at what did it actually do so we can see the storage area of where it's actually storing the configuration of the containers. I can actually see the files it created. I can even go and look at the full configuration, this bundle of things that it's going to use and that's the setup. So that's where I wanted to get started. So now we can actually go now. So I've got a container running that's really all we care about at this point is we have one container running and with that one container, that's going to let us go and view all of the magic things that's going to happen. OK, so let's take a big step back then. So what are the components? Remember we have, remember. We still have the idea of the CPU, the memory and the other things. So we have the idea of there is resource, there's still an operating system, so we still have the idea. On top of this, we have the container. Post OS, which again could be Windows or Linux. And the key part here is this is the kernel, this is the kernel of the operating system and it's running on those resources. This could be bare metal, this could be a virtual machine, it doesn't affect anything. That's the whole point of this. So we have the physical or virtual environment, we have the container host OS which has access to a certain amount of resource. And now we're going to run containers on top of this. Now in this case, we're going to have the idea that once again, we could have two different ones. And I'm going to focus on processes. When we say an application, an application is one or more processes that are running to let it do its job. So I'm going to think about the idea of, well, I've got my process one and then we had that same over here. That's not right. We have the same over here idea of our process too. And remember what are our challenges and let's deal with them one at a time. So my first challenge is someone being greedy, it using up more than its fair share of the resource I need to limit. How much of something it's allowed to use now to limit what a process can use in Linux? Well, we have this idea and I'm just going to put it inside. It's just different colors so it's easy to see. So for this we're going to use the idea of a control group. So this can be in control Group One. I'll put this in Control Group 2. So what we're using right now is this idea of C groups. Now there are many different types of C group related to different types of resource, but the whole point here is it's how much of something I'm allowed to use. Now when we think about this, there are different subsystems to C groups, and I can pick which of those I want to use as part of my limits. But obviously common ones we might think about is CPU. It could be memory. It could be block I/O, could be the number of processes that I'm allowed to have. It could be what devices I can see, maybe even want to be able to freeze. I want to be able to freeze the processes within it. So I have this idea of, OK, I'm going to put it in a C group, and that C group is going to let me control how much can be used. Now everything in Linux lives in a C group. By default, it might all be the same one. And I can then take certain processes and put them in different C groups. And one of the nice things is Linux exposes a virtual file system that lets me go and look and see, well, what C groups there are, what's in each C group so I can learn and understand them. So let's go and actually take a look at our environment so we can see, OK, what is the C group for that container we created. So if I jump over here. So firstly I could just look at. If I look at this process C groups, this shows me all of the different C groups that I have in the environment. I have the hierarchy having enabled, so it gives me some base information about the C groups, but what I want to do is really dig into well what is in my environment. So in Linux, if I do the dollar dollar, this is my process. So if I run this command, what I can see is I'm in just C Group 0. What about the C group of the Docker process that kicked off my container? Well that's got this container D slice OK. What about the C group of the process within the container? So what I'm going to do now is grab the process. Remember I ran bash within that was what I launched. So if I now look at that C group, it's in a different C group. It's in this C group, Docker 166, this great big scope. So the process within my container. Is running in its own C group and I can again. I can look at the processes in Linux over here and although it won't maybe believe me, right now we're really focused on these bottom ones. So if I look here, this bash is the process we're running inside a container, and one of the things you can see when I go and look at this. Is is actually running. This is its parent up here because if you looked at the process ID's, what we can see quite clearly its parent. For bash, its parent process is that container D Shim above. So bash is running as a child of that other process. OK, so we know now bash is running in its own C group. Now, as I mentioned, there are different controllers. So if we just go and look and we dump out what they all are, or we can see, hey yeah, look, there's CPU, CPU, set, I/O, memory, pids, RDMA, miscellaneous. There's a whole bunch of different things that I could restrict if I wanted to, and I can even see which ones are active. Which is going to be the same in this example, but let's view the actual C group in more detail of this particular one. So once again, this is all of the different C groups we have. Running I could see the particular C group for R slice. So this is my C group because I'm looking again at my docker, the long ID of my container which we stored as a variable early on South. I'm looking at what is my C group for my bash process and well OK, how do I know? How do I believe this is actually my C group? One of the things it stores is the process ID's that belong to this C group. And that is stored in this cgroup dot prox file. So if we dump out that file it has one process 11:20 and if we scroll back up again we can see our bash process sure enough process ID 11:20 so we know. That that process we launched is in this particular C group and it's all on its own. We can look at the pids current, we can see there's one process in this C group. Now remember when I launched this container I did 2 limits. If you remember, I said I'm going to limit the number of memory to 256 megabytes and it only to two Cpu's. Now that dash dash CPUs actually uses two other values, it's based on a quota and then what is the slice? But if we look at the CPU Max within this C group, we can see it's two, because what we can see here is a CPU quota of 200,000. Then the actual size is 100,000, so 200,000 / 100,000 which is the period is 2. So that limit I placed on my container is being enforced through this C group, through this CPU Max. I said 256 megabytes of memory. Well, if I look at the memory dot Max, sure enough, there it is. So when I think about those limits, I'm not doing anything magical to make this work. I'm just. Using C groups and this is just another little command that lets you go and look at all of the C groups that exist. And again, we can see our Docker one right here and that's it. Like, it seems maybe like that's too simple, but that's phenomenally powerful. The C group is what enables me to limit the amount of resource it's using, so it's how much? But it does also add accounting. So when I want to know well how much CPU, how much memory, how much IO, it's the C group that I can question to say what is it using, what are the processes in this C group using? So when I think about a container, when I want to control. What it's doing when I want to have accounting to measure how much it's doing of different things, it's C groups powering that. And if there were multiple processes in a particular container, those processes would be in the same control group. So that's how the limiting is done. That's how the accounting, the metering is done. It's just using that C groups feature of Linux. And again, if I think Windows, Windows have added their own functionality to add their own capabilities to enable containers to function on Windows as well to see groups. That's the magic of stopping the noisy neighbor and stealing all of the resource. Great. The next challenge we faced, remember, was what can I see I don't want? One container, one application being able to see processes of others or even things outside of it. So then the next thing I can really think about I need is this idea of isolation. So I want to put them inside really a walled box. I don't want it to have any. Visibility outside Xbox. So I want to control that it shouldn't be able to see other processes on the container host OS. It shouldn't be able to see processes for other containers. I want it to be able to see nothing. Solution to this is called namespaces. Once again a Linux primitive, and once again there are different types of namespace. There are namespaces for processes, that's obviously a big one, so I can't see things outside. There's things for inter process communications, there's things for file system, there's things for host name, domain name, C groups, and there's many different types of namespace. But this is going to solve the problem of saying being able to see other things. I want this ability to partition. The resources visible and every partition is isolated from every other. So let's take a look at this part. So once again we're going to go and look at our container we created to see, well, how is that working? So now we're going to look at this namespace region, and if I look at the namespace I could again look at mine. And once again we're using this virtual file system. Now this is. All the namespaces I'm part of and we can see them all then. So I can see, hey, there's the C group, the IPC, the mounts, the network, the process IDs, process IDs for the children I create. The time, the users UTS, all of these different elements, well, I can see them now. I'm not worried too much about this actual symbolic link value to the nodes in the OS. It doesn't really help me that much. But what I care about is let's look at that Docker. Process that container D Shim, but what you will see is it's actually in the same namespace as I am. The values are the same so it is not partitioned in what it can see at all. But what about the process, that bash process that runs inside the container? So I want to actually go and look at what is essentially within that wall. So if I look at its namespaces. Well, now there's actually some different values. Let's make this a big bigger so I can hopefully try and see the mall. So if we scroll here, well firstly, if we Scroll down again, sorry. So if I look at the C group, 402-653-2484, actually look at some easier ones. If I look at the network, the network is a different value. My network is 374, the parent process is ending in 840, the process ID of the parent is 284, mine is 373 and my amount is 370. My parent is 282 and that that would. Kind of ripple up to all of these. So what I can see here is I'm in a different set of namespaces so that bash process is in a completely different set of namespaces from everything else. Which makes sense. We want to be able to isolate it. So since we know the processes was part of that, if I look at the processes on the host system. So if I do the psef, so let's look at everything, there's a load of processes running on this Linux. Lots of processes. Let's run the same command executed inside the container. So using Docker exec the container we want to run the command on and then the command we want to run. So I'm going to run exactly the same command but inside the container. So if we run this, well it's a bit sad it has. Two processes which we would expect it has bash which is the core process. And then remember when I run a command it's a child of bash, so for the split second it's running, this exists but it will quickly disappear again. But also notice bash's process ID one, it's the root process. Which is not actually its process outside. Outside the container, its process ID is 1120, but because it's in a different namespace for process ID's for example, it thinks it is process ID one. It is the root of the processes within its space South. It's completely isolated from anything else. And again it is present, so it is visible there on that parent list. Now I can prove it is the same one because you may be like, well, how do I know that is the same bash that's running inside the container? So what we're going to do is we're going to waste some CPU. So what I'm doing here is I'm once again going to execute a command in that Ubuntu container. I'm going to execute a shell command that's basically just going to send the yes command to null. It's just going to burn CPU. So if I run that command, which is very wasteful, so that's triggered. But now if I look at the processes on my machine so this is the host, I can see it. I can see very clearly down here that yes is running. And I can also see it's been fairly greedy with the CPU. If I was to run it again, that amount of CPU is actually going up pretty quickly. So we run it again, and then if I run the same command now inside the container, the amount of CPU basically matches, so it is the same process. So the container, well, it only sees its own processes, but outside of it, it's just a process. And this is a key point. Container to the host OS is just still visible. Now what I don't want to do is just leave this running forever. So what we'll do is we'll kill off that process. So we're going to send it say kill 21. So now that has killed off the yes process. That's inside the container. Just because I don't want to waste all my resource and we can always check that again. So just within here, Yep, that yes process is gone. So that's showing that hey, these processes, they're really not anything special, but they're just within that C group to control. Within that namespace, I can control what they're allowed to actually see. And it goes beyond this as well. So now networking is a far more complicated concept. If I was to look at the Docker networks, the big one here is we have a Docker bridge. So that Docker bridge is used to basically map the physical networking I have. So if I look at the bridge, yeah, I've got my bridge running. If I look at my network namespace, I can see that I've got two network namespaces. So I've got the regular namespace, and I've got that network namespace just for my container. So I can see the container is running in its own network namespace. Now if I was to look at the physical adapters, well, the adapters in the host OS, I see a couple. The key ones I really care about. I have my actual IF0 adapter, I have that Docker Bridge adapter. Then I have a virtual Ethernet device. That virtual Ethernet device, that's what's going to be used by the container, and it's mapped to its network namespace. So that virtual Ethernet device is then mapped through to the Docker bridge, which then maps and talks to the ETH 0. But inside that container, I would see none of that. Now what I'm going to quickly do is inside the container. Oh, I've missed off a command. Docker exec Ubuntu. I need to install a tool because the basic image I'm using doesn't actually have the tool I need. So I'm quickly just going to install the IP route and I'm only doing that. And I wouldn't normally run these sorts of things in the container because I wouldn't be debugging it like this. But I'm only doing this because I want to show you what does the container see, the container with its network namespace. Doesn't think it as a virtual. It has if zero, that if zero. Remember, has been mapped to that virtual Ethernet device. And again, my whole point here is to once again just show you that the namespaces are petitioning and isolating everything. It cannot see any of the elements outside of itself, and that's the key point here. So what have we achieved now? So with these two Linux primitives I can now control how much these processes in this bubble. And the bubble is made-up of a C group and now a namespace which is providing the visibility isolation. So I'm partitioning off at the kernel level. Hey, you can only run and use this much stuff. I can see what you're using and namespaces control what you can see. That's great, and that's solving a huge part of the challenge. The next part of the challenge though is, if I think about it, this host OS, well, it has a file system, so there's a file system on the underlying host OS, but I don't want. The processes running in these containers, these bubbles, these sandboxes to just have free range access for multiple reasons. One, I don't want them to be able to access it in the 1st place. Think isolation. But also remember they may have different dependencies. This set of processes may need a certain framework, this set might need a different framework. Also think well if I installed it on the underlying host OS. Think the portability. I couldn't then just easily take it and put it somewhere else. There's things I'd have to install on the container host OS, which I don't want to do. I want its view of its file system. So if I think about its file system, its root of each of these, I need to pivot it. I want to pivot its root to be something else. Now you may have to see this this thing called char root, Ch root, which is a more basic version. But basically I'm going to change the view of these processes to have a different root of what it thinks is the file system. And what I'm actually going to do with this pivot of the root is I'm going to use something called a union file system. So I'm going to have a Union file system, and the whole point of a Union file system is its layers. So I could think about there's going to be certain layers that maybe the base layer would be what its view of the operating system would be. So there's going to be a base layer of the OS, for example, Oh my OS. There might be another layer that its parent is this. This may be a runtime. Maybe there's another layer that may be some application, some daemon, some service, but all of these layers. These and obviously this would have its own set which some of them could be the same, some of them could be completely different. All of these layers are read only, they are immutable. I can't change them well, I need for an OS to work, for an app to work, for things to process. I have to be able to maybe modify certain things. So the other thing that gets added on top is this thin temporary. Read write layer and the point of the Union file system is. When this looks, what it's going to look at is actually a folder called merged on this top level because this is going to have this as its parent. So we're going to redirect its view of the world. Its root is going to point to this folder on the file system of the host that's going to be its world. And this is the key point. Now each of these layers actually has a subfolder called diff, and diff is its actual content. But we can break this down into when we actually use this and we're going to go and see these what we will see the terms. Is this top read write layer. This is going to be called the upper Dir. All of these others, these read only ones, all of them together, are going to be called the lower Dir. There's also like a working area where it functions temporarily, but this is what's going to power. The file system side of this. Now one important point is this read write layer up here will initially be empty and it uses thing called copy on write. So if I'm creating a new file, it will just create it in this layer. If I try to edit a file from any of the underlying layers, it will do a copy. On write, it will copy the file into this layer. The whole file, it's not block based, it will copy the if it's a really big file. Going to copy a really big file up into this, which would maybe take some time. We start to hint at some performance implications of this, but it will copy the whole file and there it can be edited. If I delete a file, see if I say OK I'm going to delete a file from under these layers, but it can't. These are read only. So what it will actually do is if I say I want to delete a file from this layer or this layer, it will create a whiteout file. It will create a file up here which is a special 00 device number character device that basically says I'm hiding this file. If it's a directory, there's a special attribute it can create again on a directory with a special attribute so I can hide things, delete, but it never changes. These layers are read only, it cannot change them because these layers could be used by other containers. And also think about the prescriptive nature. Think about the portability. I don't want to ever modify these things because then, well, hey, if I take this and put it somewhere else so I've got this prescription on how I create it, it wouldn't work consistently anymore. So I never change anything from these layers. That's the key point of this. So I'm using this Union file system and I'm changing this set of processes. Its view of what the root is to basically this subfolder of something on the underlying file system of the host. And this is made-up of multiple layers. You can think of a tar file gets brought down and expanded out which is the content, and the Union file system brings them all together into this merged folder. So if I look at merged, it's the sum of every layer underneath. And there are different solutions that provide these Union file systems. Overlay 2 is the big one that is commonly used, and that's what we'll actually see used in our environment. But before we look at that, the fact that you can't change these layers is why we don't patch. Imagine this was an entire operating system and I said, OK, I'm going to patch it. Remember, I can't change this layer. The patches would copy and write the content, send it to this. Make the changes in here. If I deleted the container and create a new one, it's lost all of that. If there's an update to the OS, we delete and we recreate from the new version of this. And because we're going to have these very prescriptive composition files to create the containers and we typically won't store state, definitely don't store state in this. If we have to store St. Durable, we're going to have other storage volumes. We have a multi tiered architecture where there was a stateful database. We don't store things we care about in here, so we don't patch. When there's a new version of Ubuntu, or a new version of the runtime, or a new version of some Daemon we're running on top of, we just create a new instance of our container built off of the new image that has the patches baked into it already. So that's the whole point of everything we're going to do. So let's look at this. So if we could jump over to our demo again. So let's look at the layers. Now firstly, what storage driver am I actually using? So if I go and look at my Docker info and let's scroll up a little bit, I think it's near the top there we go. So if I look at my storage driver, we are using overlay 2. So that is the Union file system that we're leveraging to provide this view to make the images actually function in our environment. And I can see the file system that we're actually using for my container. So I'm doing a Docker container is the key point. Looking at container, I'm inspecting the particular container called Ubuntu and we can see all the details about the container. The bit I care about here is this area, this graph data section, and then this graph data section. We see that thing I talked about, the Lower DIR, the merged Dir, the Upper Dir, and then this idea of the work directory, which is where those temporary copy on right things are stored. And what you'll notice from these paths is, well, that merged directory is that, as I said, is the merged. Folder. Its Upper Dir is that diff section, so the contents. The Lower Dir, well, that's the content of every layer that sits below. So in this case there's two. There's the base OS layer, which is this one, and then it also creates an init layer, which is just the files that Docker needs to function. And then there's the actual read write layer. Of the container, which is the upper Dir. So they're just folders. They're just folders on the file system, and we can view what Docker thinks the size is. Now notice the size is actually pretty big. This size here is just the size of the container's read write area. The total size this adds in the OS is 125. Why is it so big? Remember I ran that silly apt install to get the IP tools. Normally this would be 0, but it's big because I ran that apt install and we can look at this if I go and look. So all of those layers they're stored in the Var Lib Docker overlay 2 folder. So if I look at this I can see the images. Now two of them have the first name. So this top one this is the Ubuntu that is the Ubuntu image we downloaded. These two got created when we created our container. So again, this init one just contains a few things that Docker needs to function that have to be present. But it's this one without the init that is the main read write layer for this. This is going to have the diff which is all the files we write to it. This is where we actually make those changes and we're going to be able to see this. So firstly let's look at the size of each of these folders. So we use the DU and we can see OK, the Docker image. That top one is 84 megs. That sounds about right. So that's the Ubuntu image. The init folder is 40K, you can see there it's got the init name. At the end there, that's really small because it really doesn't contain much. But my containers read write. Why? That's really big. Why is it 175 megabytes, which really doesn't seem to match the size we saw up here saying even the total is 125? Well, think about what this folder actually has to contain. So let's go and look at it. So I'm just going to have a quick regex expression to steal the actual mount that it's using so we can go and look. So now we're actually looking at that layer and we can see, OK, there's one that's 129 megs, so that's the sum we can see. One is 47 megs that should be the content of justice, this layer. And then there's this 175 megs, which is that grand total. Well, remember. It has two key points. Merged all. That's everything. That's its layer and all of the layers below it. Remember, that's the combined view. So that merged folder contains the contents of the layer, which is diff, plus the contents of all the layers underneath. That's not actually copying it to merged. Merged is just a view. But if you think mathematically well then sure, because the difference my layer is 47 megs. Then I have this merged view which is my content plus the layers underneath. Well then the total size of my folder are those two things added together. So it's an inflated size. It's not actually that big, but it's because. This layer right here, this upper Dir has its own content which is this. So this is its content, which is whatever data I've written or modified or deleted. But the whiteout files are tiny since its creation. But then the merge folder is this plus all of this, and again, it doesn't copy it into the merge. Merge is a view, but when I look at the sizing. It will look like that size is there. So that's why this folder looks really, really big. But it's just a view that combines all of those things together. That's really the key point behind it. Anyway, this is what's important to spell that point out. Let's go back to our demo. OK, so that makes the size make sense. Sure, let's keep going. So now what I actually want to do is. So I mentioned that init folder, and I said it was actually really nothing in it. Nothing in it. Get it? That's a terrible joke. So if I look at the init folder for our container, it has its own content as normal. So it has its diff. It has link, which is just a reference to itself. The symbolic name lower is its parent folder, so that's the link to the image on which it sits. And then work is that, hey, ongoing copy on write operations. So if we actually go and look at the contents of diff, this is all that's in it. This is what I was saying. The content of the in it is just a few things that Docker needs to be present to make sure it can function correctly. So it's 4 directories and five files. It's tiny, but it's just putting a few key things there that are required for it to function correctly. But also to understand, hey, my host name and justice, some basic information. So that's all the in it actually contains and then it's in its lower. So when I look at the lower, this is just a reference to the parent image. So if we go over here when I dumped out the lower, go back here. This is its parent image. Now you'll notice this strange L slash. So the L slash is actually a symbolic link and we can dump out the contents of the L folder. So that's just a folder under overlay 2 and it's just a set of symbolic links. So here we could see hey, its parent was two woe or two woe is this one. Which we can see is the Ubuntu image because it's very different from the other two. So this was its parent which is Ubuntu and then we have our containers 21 in it, one non in it. Now the reason it creates these symbolic links really just comes down to later on it has to be able to mount all of these different folders and there are some limits in how long a mount command can be. So rather than having to deal with these really long names in the mount command, it uses those symbolic links. That's the only reason they're there. But hey, it solves the problem with the mount. Now if we look at the Ubuntu image, it doesn't have a lower, it doesn't have a merged because it's the lowest layer, it just has its content. And it has what its symbolic link is, so it is the operating system and we can look at it. So if we actually go and you could look at the more detail about it, we would see all of that. We can also check that symbolic link is just to itself. So if I dumped out that INIT link reference its link. Jdu 7 is JDU 7UP here. So again, that link file just contains itself, so those are the different components of it. Now I could then go and look at my containers. So so far I've looked at in it. We've looked at the Ubuntu. What about my container itself? My container itself? Well, as we would expect, it has a link to itself. It has its content, it has its parent, it has the merged view. So remember merged is it and everything below it in the single view and its copy on right temporary area. If we actually go and look at its content again in the sizes of those, we can see the temp the user. The var. So this is the content of that image, so we're looking inside what has it created. Well, as part of installing that IP set it's created stuff in what it views as temp user VAR and etc. And I can create a certain amount of content in each of them and I can just do a simpler view etc. Temp user var. Let's prove that is actually the file system of the Ubuntu container. So what I'm going to do now is create a file called Hello World. So I'm executing inside the Ubuntu container the Touch command to say Hello World. OK, let's execute. So now remember from the host OS, let's look at the contents of diff. Sure enough, it showed up. So I created a file in the container on its root, and it's there on the host. I could do it the other way, let's just create a file directly in the container's diff folder called hello from host. Now we wouldn't normally mess around with these. This is useful for debugging purposes, but I want to kind of prove a point. So I'm going to now create the file directly on its diff folder. And now if we do an LS inside the container, let's scroll up file it touched and the file I created from the host. So this kind of proves that point of what we're actually doing is we have mapped to the container folder on the host. That's all we've done. This is looking at a tree of its diff folder. Again, these were all the files that got created. It made it messy because I installed that IP data, but sure enough there's our files. And this is the cool thing. I already showed you the symbolic links here if I look at the mounts on my actual host operating system. Well here we can actually see it. So if I look at the mount, I have this overlay mount I'm mounting on this great big path and I actually have to zoom out here. The key point here is we're mounting the merged folder. That's what the actual mount is. So the mount is to the merged, the type is overlay. This is what's creating it and its content is actually. Let me give it a bit more space. OK, so the content for this the lower Dir. Remember this is all the read only layers. The Lower Dir is made-up of these symbolic links. So I can see I've got link here that short name. And this short name, that would be the init folder and the Ubuntu and then the Upper Dir. Remember that's the read right layer. Well that's just the container's diff folder, it doesn't bother using the symbolic link for that, it's just the full name and then the work directory for the temporary copy on right so I can see the mount created on the host OS, and again, it's using that pivot root. To actually then change these processes views using the namespace to just see that and so the final view is merged and there we can see it. So if I look at merged then I see all of those files. So that's the final view. Now this was actually fairly simple because the Ubuntu image quickly look. I've got the command here the. A bunch of images, one layer which was very neat and tidy of it. Often they're not. So if I was to look for example for httpd, so I look at Apache, and if I pull it, one of the things we'll actually see watch what it's doing. Notice it pulled 5 layers, so it is actually made-up of five layers, not one because there's going to be an OS and then some. Dependencies, then the Apache service itself, all these different things. We can see the various commands it did. Now we won't see all the layers, some of them will just be missing. These were things that were done and then merged, and so it doesn't exist as an actual layer, but it will be summed up in higher layers. But here we can see all of the commands that went into creating this, and then if I inspect the image I can see the five layers. So this is the five layers that make up that image and we'll see them on the file system. If I now go and look at my overlay 2 folder, I have a whole bunch of new ones. So I can see here these five. I looked at the date, basically the time. So these are the five that were created just by downloading that httpd. So those were all created as part of that. Now, those were created through a Docker composition file. So we talked about what is Docker. Docker does a number of things and we'll come back to this. But I've talked about layers and I've talked about, well, they're made-up of different things, how what magic creates this. So the whole point is what we have. Are registries. Now these registries can be public, for example the Docker hub, the Microsoft Container Registry, or I might have private ones. I just have things specific to my business apps that only I have permissions to. They're not publicly exposed, so this has the images stored in them. So when I think about these layers, this is pulled down from a registry. But what I actually have is I create a Docker file, and a Docker file starts with a from. It's built up from a certain image, so it's using a certain image, for example maybe it's Ubuntu or httpd, and then it has tasks that it's going to perform to modify it. Which will then create its own image that I then would push to store in registry and then when I actually wanted to put it in production on environment, I would pull it to make it appear. This is the portability and that's the immutability. So it's very prescriptive. This is the key point. So this is prescriptive. I'm building from this, which itself would have been built from its own composition file. The layers build up. This is what I'm doing. It creates this image, which is then portable, which I then bring down and run on these different environments. So that's how these things fit together. So if I was to look at my environment for a second, let's jump back over here. So what I'm going to do is let me just move over. To a folder I've got a Docker file. Now this Docker file is very basic, but I'm starting from that httpd image that I pull down and then I'm running a command. I want to remove its content for the HTTP folder and then I'm copying from my local environment into user local. So basically I'm replacing its website with my own website. That's it. That's all this super basic Docker file does and to build it I've moved into that folder. Remember I just run Docker build and I'm going to create that image bad Father. In my current location you can see it's doing all the different things and notice it's writing a new image. So I created a new image right here so I have a new image. If I now look at all of the images on my system, well I've got three. I've got that Ubuntu I originally pulled down, I've got the httpd which is the Apache web server, and then my bad father that I have created that was built off of the httpd. If I look at the history of my bad father image I can see remember it contains all of the commands from the lower layers. And then eventually my little copies that I did in my environment O all of the lower ones were done as part of the httd and the OS. Everything else. And then it's got my ones that I did to create my actual file. I could get more detail about the underlying image. Notice it has lots of layers because. It depends on the layers of httd, so I need all of those things to actually function. I could go and look at the differences. So if I get a link to my image reference and then look at the diff, well the diff layer is just the contents of my web page, remember, all I did was delete this content and then copy in the content. So that's it. That's the difference of my layer. That's what I have added to bad Father. And now I could go and run a container using that image, which at that point would then go and create that read write layer on top of that. Now as part of this, because it's exposing a web service, I want to publish a port. Now the format is always container, sorry host to container. So I'm doing the container pool 8080 maps to pool 80 of the container. So host pool is 8080, container pool is 80. So when I look at 8080 on the host, it's actually going to show me what is in port 80 from Bad Father. So if we execute that's created, I can go and get the Docker ID. And at this point it's just running. So if I do my Docker PS A it's there, I can see my bad father is running. It's also showing me the ports. So hey at 80 is mapping to 80 on TCP for IPV 4 and it's doing IPV 6, it's there. So what this should mean is if I went to. My 127.0 dot 0.1 pool 8080 there's my website so let me just oh, I'm on the wrong screen. Sorry, did not fully get that ready. Here we go. All right, so this is my bad father. So I've gone to my local host, but Pool 8080 which is remember that that was the container host port. So this is me and my son on the Disney ride and then if we click it we can see him terrified on a different ride and a close up view of him terrified and me looking quite happy with myself, hence bad father. But that was it. We can see that working and then I can obviously stop that container running. I can remove that container, I can remove my image, I can even then go and remove the httpd image. So I've gone and tied it up. All of the different things that I did. Now that's one approach to that. Remember the challenge though. So imagine within my container I want to do a lot of IO or I want it to be durable. This is not like there comes a point where this is not practical. If I want it durable, this is not the solution. I don't want to write it here, I want to write it somewhere else. If I needed to be highly performant with this this diff idea and this copy on write, it may start to impact certain things I want so I can inject other melts so other folders that aren't going through for example this overlay to system. I just want to add another folder to be available to this container as well so I can do that as well. So let's look at that in action so if we jump back over to RBS code for a second. So I'm going to kill the container, which actually cleans some things up anyway. So if I just stop our main Ubuntu container and I'm just going to kill it completely, the whole bunch. So when I'm stopping it, OK then we'll remove it. So we'll go and delete all those folders. So I'm going to run it again, but this time I'm adding a volume. This is the container host volume. This is the container of where it will be seen. So I'm mapping this Mount C users John OneDrive projects get random stuff to just a folder called slash stuff within the container. So we'll run this again and then we look at within the container stuff. It just works because its view. Slash stuff is just mounted now to a different area. So that's really powerful and useful. When I think about, well hey, I do want some other storage available to me. I use that. I don't have to just use that overlay too. There are other things I have so that that's an important thing I'm going to do. OK, so we have solved it. I can control what it can use. I can control the isolation. I can control its file system and give it the ability to interact. I have this great portability. I could now take this container image I've created and deploy it to Dev, then UAT, then production. I don't have to worry about well, did I forget to install something because it's completely self-contained? My application that I created is built off of another image which itself is built off of other images, but it's completely prescriptive. It's immutable. I have versions of these images as well, so I know even if a newer version was created, I don't have to use it. I can use latest, I don't have to, so I can be super prescriptive in what I'm doing. And I know it doesn't matter where I run it, it's just going to work. So that's the solution. But there are all these other elements involved. So I have namespaces, I have the C groups, I have the file system stuff. So basically I have this idea of a whole bundle of configuration. And yes, I technically could stop at that point. I could go and create the C groups, I could create the namespaces, I can create. It's not even that hard. But it's not very friendly. Remember, part of this is I also want a great developer experience, I want a great administration experience. And there's other factors in this as we start to grow this and other features I need. So I don't want to manually create namespaces and see groups and the mounts and the overlays. That's not really practical. So instead what I want is a container runtime. And that's really what all of this is going to boil down to. So we have all these different elements involved and what I want to do is really bring it all together. I want something to handle the C group management, the namespace, the creating the mount using the overlays tend to help me come and get those images tend to go and help me create those images. I need these various things. I don't want to manually do any of it. Now I do want to stress the point. When I talk about container runtimes, there are many solutions. I'm going to focus on container D and run C there. I would argue the de facto ones today, but there are others. They have various pros and cons. So we're going to start with the idea, then remember, if I think about what I want the container runtime to do, we still have our concept of the group of processes. So I still have my group of processes, my process one which I'm going to use the word container now I think we understand. So I've got my containers which remember are actually made-up of C groups, namespaces and those file system pivots. And I've got multiple of those. So I've got that P1 and I've got my P2. And again, there can be multiple processes, depends what the app is, but they're each within that certain C group namespace shared within there. So something has to go and do all of this stuff for me. And there's something that's going to do. This is container D so we have this mighty powerful container D Now I do want to stress, Docker kicked a lot of this stuff off. We're going to talk about Docker. Docker used to be this big monolithic thing. And then what they really did is they spun off bits of themselves to help drive these open container initiatives, these standards around it. So container D spun off of Docker, something else runs these spun off of Docker. But now it's this open container initiative, these standards to enable other things to work, different components to fit together and be interoperable. So that's a key Dr. about what we're doing. But container D, it's driving a lot of the things we're going to have now. This is considered a high level container runtime because what it's offering me is a number of different things. There's a whole configuration, a bundle of configuration required for these C groups and namespaces. So one of the things container D is going to do is create. This OCI compatible bundle this configuration so it's going to be given hey I want this container in blue with this much memory whatever else it will go and create the right configuration. It will worry about getting the images. So remember somewhere out here we've got images. Oh sorry that's wrong. I have a registry, whatever that is. Well that registry contains the images. When I want to go and create a container, well something has to go and pull the image. Or say is it already cached on our host? Container D does that. I need to be able to interface with it so it provides me a management interface. So other things can actually go and talk to it. It will go and manage the network, the storage. So it's doing that part of the requirements and then it does something else. It's then go and talk because if there's a high level runtime. It stands to reason then there must also be a low level runtime container D itself can't create a container. It doesn't understand the primitives required to create the C group or the namespace. It doesn't do that. The low level runtime is run C. Run C is essentially the lowest level we get to run C is what actually creates the container. It puts the process it creates the namespace, it creates the C group. It creates all of the things we need based on configuration driven and commands from container D so it is move it down the low level runtime however. There's a challenge that if container D directly created the run C process which then created the process is running in the container. If I have updated container D, it would restart, will any child process would maybe shut down? Would it would cause a problem? So what actually happens is there's a container. D Shim process which actually then calls it. So container D creates the container D Shim, it calls run C and likewise it would have its own Shim. I'm not going to do it all container D Shim which calls that, so that solves the number of problems. So the whole point of the Shim is now if this restarted, no big deal, that Shim is still there. So it lets me have this daemon less containers. Essentially it handles the capture of standard in, standard out, standard error. So if things happen, hey I'm still capturing it. Can take that and write it to a log file for example. It helps me maintain connections to the container. But one of the interesting things is this run C actually exits, so it gets called, it sets it up. And then it exits. So then what happens is this process actually gets re parented to have container D as its parent, which is what we saw. Remember when we looked at the processes, it wasn't run C that was the parent of the process in the container, it was actually it's container D Shim. So container D calls container D Shim, container D Shim calls run C, run C creates the bubble the container. Runs, puts the process in it exits, and what the Linux kernel lets you do is rather than. When the parent process exit, it has to default to go into process ID one, it lets you give it a different parent, so it exits, but it changes the parent to the Shim. But again, the Shim is different from container D, so container D had to restart or do something. These keep running so I don't have to bounce on my containers just because there's some maintenance on the container D engine itself. And we see exactly that. So if I was to jump back over again one more time. So if we look at the container runtime section, we if I look at the tree of the process is it's this the process in the container is a child of container D Shim. It is not a child of container D that's the whole key point about this. Now that's really a critical part of all of that. And there's also the Ctr. command. Ctr. actually lets you interface directly with container D so that that's a really, really important point about all of this element. Now, there is actually an important thing I want to stress when we talk about. Pay the containers and its view and it's running on this kernel host OS, this layer. This image does not have to be the same distribution of Linux as the container host OS. One of the key drivers of Linux is it has to be backwards forwards compatible. They all run the same Linux kernel whether I'm running Alpine or rocky or souse or Mariner. They run the same kernel, they're all running the same Linux kernel. They just do different things in the user space. S what that means is my container host is running a certain Linux version. Mine is Ubuntu. But these images, this could be mariner, this could be Alpine, this could be rocky, this could be Ubuntu. I can mix those together. What I can't mix is windows. I couldn't have this being Linux and then run a Windows image because remember, these processes as we saw are running on this OS. I can't run Windows processes on Linux now. One of the things you will see sometimes is it will allow you to do that because what it will do is it will create a managed VM. So basically every container then runs in its own little managed thin VM, but it's still a VM. That's how I can also get kernel mode isolation between containers. I could. I'm still using containers for that portability, that prescriptive nature, but the container actually runs in a really thin VM so it has its own kernel. Could argue that defeats a little bit of the point of it from the multi tenant environment. I want that because I don't trust my neighbors. There still might be certain exploits I could try and do via the kernel, so running this is great in a trusted environment. If I don't trust my neighbor, like in a public cloud, I probably don't want to share a kernel with this person. So that's where they may create this very thin managed VM. So I get that isolation. But in terms of compatibility, I can absolutely run different images from Linux because they were running the same kernel and we can prove this. So if I jump over, we know I'm running Ubuntu in my environment. So what I'm going to do here is I'm going to run a Mariner. So Mariner is Microsoft's own distribution and I'm going to show two things here. I'm not actually going to bother pulling the image first, I'm just going to run it but notice what it's doing for me. And this is container D at work. It's saying I can't find it, but I don't have this. You never did a pull. I'll do it for you. It's pulling the two layers of Mariner and then it started the container. Now if I look at my OS version. I am running Ubuntu. That's the container host OS. If I look at the version of the OS inside that container I just created, it is running CBL Mariner 2 different Linux distributions, but it doesn't matter, they share the same kernel and that's really a super important point about all of this. I don't have to have that same compatibility, the same distribution because it's Linux. Like they run the same Linux kernel. That's a key tenant of Linux. It has to have that compatibility. So yes, these processes the user space because maybe I can offer different features by Sousa or Alpine or Rocky. They can be built off of an image that's Ubuntu or Mariner. It doesn't matter. They're all still one of the container host OS because the kernel of Linux is shared is the same. It's just this user mode space up here that might be a little bit different. OK, great, that's that's the containers, that's how it all works. I barely mentioned poor old Docker. And I did say, I would tell you what Docker is. Remember, Docker kicked off the popularity of containers. It existed before Docker. There were other components that they would use to create these things, lxc, for example. And there are other things that happen here. Docker really made it mainstream. But again, they've worked very closely with the community of the open container initiatives. They've split apart. What was this monolithic Docker thing? To create this componentized elements so I can plug and play. I can put different things in. So if I think about, well OK, what is Docker? So docker. Remember container D at this management interface, so absolutely Docker oops. What is that doing? If you're too slow, the whiteboard does things I don't want it to. So we have Docker here. Docker. It talks this management interface container D also we have things like Kubernetes which would be a different video that solves bigger problems that containers on their own don't really solve. So it has its own container runtime interface, its own API, so via that it goes and talks to container D or other things, it supports other container runtimes. But Docker itself, well, it has a Docker daemon. This service that runs always runs in the background. And then what it also has is this idea of the Docker client, the Docker CLI. Docker CLI is what I can use to say, hey, I want to run various commands which we saw we've been using the Docker CLI this entire time. So Docker CLI was talking to the Docker daemon which went and talked to container D which went and created the bundles of configuration. When we created containers, it went and fetched the images for us. Docker Daemon was telling it what to do and then it told other things and poor old 1C gets created, it does its job and then disappears into the sin now. So Docker provides this fantastic experience for the interaction. But it also remember what did we use? Well we talked about registry. Well we have Docker hub which is its registry. So with Docker CLI I can create images. Remember we saw creating images so I could go and push images. Doesn't have to be it is following the OCI standards. So I can also create images using Docker and I could push it to other. The Microsoft Container registry. There's different options available to me, but it gives me this great developer experience. So it's this registry for images, for extensions, for plug-ins. It gives me this background process to interface with container D it used to have its own Docker Shim thing that sat in between that it's kind of gone away. Now it just works with container D. It gives me the command line tools to help me create, manage, ship images and to work with the containers. So that's where Docker fits into this picture. Now I just used a Linux distribution. I happened to run it on Windows Subsystem for Linux just because it was an easy way to run a Linux. If you just want to play with this, Docker has that called Docker desktop. I installed Docker desktop in whatever OS I have. I can run Windows and Linux containers. It basically will spin up either it will use Windows subsystem for the next one windows. It will create a VM so I can pick what type of containers I want to run. Gives me a nice little graphical interface to interact. Guide me through steps to learn it so highly, highly recommend that. I didn't use it because I wanted to be able to show you the C groups and namespaces and what I could have done it with Docker desktop. It actually runs the containers in its own generally hidden partition. I could have got to it with WSL, but it seemed to be better for me to just run a full Linux distribution and show it to you. But if you just want to play around with it, Docker desktop is phenomenal, so definitely recommend it. But this was it, so hopefully this answered the question of what is a container? It doesn't exist. You're not going to find containers on the operating system. Containers bundle together primitives of the operating system, C groups to control how much of sync I can use and measure account what it's using, namespaces to isolate pivot roots, and a union file system overlay 2 to give it its own view of the file system that it can interact with. We take all that, manage it with a container runtime container D which calls run C which sets all those things up. Is great for the developer experience, for the operations around that management. But yeah, it's just using primitives in the operating system now. There are things it doesn't solve at a bigger scale. I need multiple containers, I need resiliency, I need service discovery, I need integration with lower balances, and I need richer services. That's Kubernetes. That's different video, but I hope that helped. Until the next video, take care.
