Building a container from scratch in Go - Liz Rice (Microscaling Systems)

I appreciate there's quite a specialist audience here and I also appreciate that the chairs of really really comfy and it's kind of middle of the afternoon so everybody stand up go on stand up there is a point there is a point well slightly a point okay we're all amongst friends here now a lot of you probably responsible for writing container runtimes and in that case you can stay standing up but if you have any sort of shadow of a notion that you're not a hundred percent sure what we mean by a container sit down if you're like mmm lightweight VM what does that really mean if you're kind of mmm isolated execution environment well I understand the words but I don't know what that means in practice if any of that is ringing any bells and bear in mind we are amongst friends please just take a seat great okay if you don't programming go you're also allowed to sit down okay anybody who's still standing up basically it's your fault if it goes wrong because you need to shout out if anything goes wrong you are my peer reviewers okay so um I am really massively plagiarizing this talk from a talk that I saw by Julian Friedman I don't think he's here today but if he is well whether he's here or not massive credit to him for basically everything I'm about to show you but when I saw this I thought oh my god it all falls into place and I felt the need to go home and run through the same code myself and put it all together and understand how containers work and I think mark was talking about how containers sort of put together an illusion for the code that's executing in them that they're sort of operating in their own environment and what we're going to do this afternoon is sort of see how that illusionist illusion is put together okay I don't know what all the other speakers were thinking of this morning because I haven't seen a single one of those yet and I thought it was the law that you had to have one of those pictures in every talk about containers and we are about to take one of those apart and see what it's really made of okay so here's some code that I prepared earlier there's not a lot to it at the moment I thought given that we are a specialist audience I don't really need to demonstrate this but you all know that if you want to run a command you in da clay you'd sort of type in doc around the container name and maybe there's a command and some arguments and when I run my code I'm going to do go run main go for those of you who aren't go programmers that basically says kind of compile and run my my file here so that's the equivalent of the docker bit then we're going to have an argument that says run and then we're going to have some commands and some arguments so we want to be able to set up a container our container is going to run whatever we specify and though in that command and the arguments okay so we're going to look at the come on line arguments that have been passed in and first we're going to look at the first one and if it's run hopefully the fact that not all of your go programmers won't you know make this impenetrable it's pretty straightforward so if the first thing is run great and in any other case we are going to fall in it's all over in a big heap okay so great what do we mean by run well I'm a big believer in debugging so let's print out what we're trying to run running this is a bit awkward to type on this sir so we're going to debug printing out everything from argument to and onwards so that's our command and all the arguments that should tell us what we're trying to run and bring that up screen a little bit more we're going to set up something that can actually run something so the command we're going to run is specified by odds - that's the one that's called command and optionally o3 and onwards right this demo would again be very dull if I didn't set up stood instead out Studer need to actually put an equal sign in there in one more okay and I have this little utility function called master which will just panic if anything goes wrong and we must run that command so for the non go programmers we're setting up this command and then by calling run we actually run it okay code reviewers have I missed anything are we happy right nobody's saying I think so I think we're probably happy right right this is a Linux virtual machine running on my Mac it's got a shared directory so that I can get at that main go file I was just editing few things running at the moment not very much let's see what happens if we run my container let's go really wild and echo something okay I think that you know that's a container right we've executed something we said we wanted to echo container camp and we have echo container cam must be a container right let's get really ambitious and see what happens if we run a what it says it's running it can't really tell what's going on or if anything has happened and we can still see the same files we can still see the same processes and if I look at the host name the host name is there's a bun - I could change that okay we could tell from that process list that go is running so I must still be inside my container because that's the the executable I just ran so if I quit out of this I am no longer in my container but lo and behold my container was able to write over the host name because there's no isolation at all at this point that is not there's a terribly secure container let's reset my oops host name otherwise things will get very confusing right so that's just not good enough we can't ship it yet but fortunately this is where we get into the exciting concept of namespaces I'm going to need this book attributes library like that attributes and we're gonna pass in some flags at least one flag and this one is called flown new utx for unix time sharing system because we all know that unix time shown system means hostname okay so i'm going to run this again can't really tell that i'm inside a container at the moment but ok hostname is there's a bun - that's what we expected let's call it hostname c camp inside the container is it is hostname c camp let's exit the container well hey we have protected the hostname inside the ketosis of running container can play with the hostname - its hawks intent and we haven't affected the host machine okay this is making progress what about process IDs we saw before that process ID list was exactly the same as the host machine and fortunately there's another one of these flags called new PID new process ID that sounds pretty promising right so let's run that we're inside the container we run PS and that doesn't look very different okay we're gonna have to debug this thing so it would be really nice if we could just come in here and say we're running as process get the process ID well that's all well and good except where I've got that debug line is before I've run the command I haven't got my new process ID I have got my new your namespace yet for my process IDs so we're going to do a little trick we're gonna have two copies and the first copy it's gonna be run and it's going to create on you namespaces but it's not just going to run the command straightaway we've heard a few people talking this morning and this afternoon about fork and exec and running proc self XE is basically a fork exec and now I have to invoke a bit of go sort of incantation so that I can pass in instead of you know how we passed in run as the first parameter before we're going to Thomas in child when we fall can exec okay and this is going to be child and we don't need to create new namespaces because we've done that already so the first time when we come in here we are we call run and we call run and the second time we come in we call child yeah so that when we run run we're doing this fork and exec with the new namespace I don't think I saved that okay right we can see post this idea of one that sounds pretty promising me as that oh we have created a process namespace array but from inside our container we still have the same list of processes and they start with you know 1500 and what have you so what's going on here well as I'm sure lots of people here know that PS doesn't look directly at the list of running processes it looks in sash proc and there's a whole load of processes that are running on my machine and that's where PS gets its information from so if we want PS to work correctly inside the container there we're going to have to give it its own file system give it its own /proc at least and now fortunately I have well so in my home directory here I've got a host root filesystem and I also happen to have with me right I am a root filesystem so this is a copy of I think it was Anna bunting file system that I just happened to have conveniently lying around so inside our child process we want to use that root file system so I'm going to change the root to FS oh and we want to make sure that that works and we also want to change directory to that so that the root directory is home slash root FS okay let's try that again something has definitely changed here and the reason why it's changed is because myoma root file system we have got that root filesystem looking like it is slash inside our container now this is excellent we can look at well let's let's look at the process list yeah one last thing /proc is special we have to mount it this bit requires a lot of concentration because it's critical that I get these parameters the right way around okay one more I got those right way around we happy with that get okay it's happening again run it again and lo and behold we have a process list just relating to this container that's coming out of our own smash prog inside the container just got those small number of processes in there we've built a container we've got thank you that's what we need so I mean you can you can imagine that you can go further than you know say for example the user name space you you might want to isolate but let's just sort of recap that's what 50 to my codes huge 52 lines of code can somebody remind me what Dockers valuation is um I guess maybe that isn't quite ready to ship not quite production quality but I think it kind of gets across the idea of namespaces so setting up inside your container what you can see so we covered the unix time sharing system which is hostname we covered process IDs we looked at file system just have to do a similar sort of thing for users into process communications and networking and you're done the other thing that people say oh when I first got involved with containers people are always saying oh just namespaces in C groups and everybody not some kind of goes oh yeah I understand what that is okay right this is namespaces and C groups are control groups and it's about limiting resources and I think this doesn't really need to be demonstrated it's an easier concept and you can say well I only want this container to have 10% of the CPU or a certain limitation on how much memory it can use that's a pretty straightforward concept but that's what people are talking about when they kind of get oh it's just namespaces and C groups and the last thing I wanted to touch on briefly is to sort of get towards why I'm wearing a badger on my shirt and so images container images when we copy that file system over if that had been done in a bunch of layers that would basically be a container image that's all we're doing with an image we're creating the file system that the container sees and we're also throwing in maybe some configuration commands setting up things like environment variables but that's all we're doing with a container image and the banja comes in because a little project that I've been working on called micro badger which which lets you look at those container images and inspect the different layers so we saw some kind of diagrammatic explanations of how layers are built up into an image in the previous talk and you can use micromanager to sort of inspect exactly what's inside all the different layers in any public docker host image so I urge you to check that out at your leisure bit of further reading I kind of feel I really have to point you at Julian Friedman's container gist is this very slightly different from mine I also have one on my github which is slightly different from his and I would very much welcome any kind of comments and questions and people using micro badger there's our Twitter handle and and that's pretty much the end of what I'm going to talk about so but I think I've got maybe I can take a question or two while I point at Gareth and Michael I hope they're paying attention because they need to come up because we got a little thing that we've been working on that we want to announce today so that's how you build a container that's how you build a container in 52 lines of code thank you very much

Channel: Container Camp

Views: 126,651

Rating: 4.9132528 out of 5

Keywords:

Id: Utf-A4rODH8

Length: 19min 3sec (1143 seconds)

Published: Thu Oct 13 2016