Docker for novices

thank you very much I think your everybody for coming an expression like say a big thank you to Linux count for inviting me to come and speak and I really want to say a huge thank you to the volunteers I'm sure we all do it's been a fantastic conference they've put a lot of hard work and for you know the COS sitting that they'll do it all no spare time I think that's fantastic so thank you to the conference good job okay so today I'm going to spend an hour and 40 minutes it's working you through material that should probably take two days normally so it's going to be quite a packed agenda I'm going to assume you've got no knowledge of Dockers and I'm talking a very basic level about what docker is why you'd want to use it I'm going to do some basic terminology so this is one of the things I found very confusing was what's the difference really contain an image so we'll get that cleared up straightaway and some other terminology as well and then actually hopefully spend most the time actually sat using docker and getting some practice at using it okay now I've tried to put in here the things I found confusing over the last sort of 18 months or so that I've been using docker so hopefully you won't have the same frustrations that I will so we're in a forest there are many trees and I'm going to hopefully lead you through a path let me say that you won't have your own frustrations because you'll be following your own path after that but hopefully you won't you won't hit the same potholes I did so we'll look at what docker is why it's useful and then just how to use it and I want you to be confident enough when you go away from here you can carry on because there's a whole lot of things we can't do we won't have time to do I've got more material than we can cover so I hope for you'll we want to take away what I've written and just carry on using it and then and then take it further there are things that we can't I can't talk about just because of lack of time and complexity so docker security is quite a big topic we won't be talking about networking if you are from a DevOps environment you're used to terms like you've hearing terms like kubernetes and swarm we won't be talking about this this talk is for docker at the small-scale it will be particularly useful if you are a developer or tester if you're going into a DevOps environment you will still need to understand this material though to build on top of it and I won't be able to go into every single gory detail and every command option there's lots of them so you'll need to have the documentation open in front of you while we're working okay absolutely okay so hopefully everybody can do this from the command line is anybody not able to do this alright so you should be able to just flip over to a shell and go doc version and then go docker container run hello world who can do that if you're on Linux then you could either keep doing sudo which is a bit tedious but the solution is to add yourself to the docker group and then log out and log in again and you'll have the permissions you need all the material here I here is in this github repo you'll probably want to clone this repo now because when we start doing exercises a lot of exercises of pre-written so all the materials and the repo it's going to type in the commands and there's a git er community which I've got open so if you want to up join that and ask any questions either now or in the future please do so why don't just hit escape it does good yeah the discovery of mine yesterday so it was quite good I just met you shut down the Year slideshow okay so a bit about me so I've been using Linux for a long time actually started I actually organized the first Linux conference in the UK back in 98 and I think I was a minute I'm a mini musty rustling that respect I currently work as developer relations that a software house sitting in Melbourne called paper cut software have actually paid for me to come here and also the my colleagues helped me refine and rehearse this material so very many things to them back in Australia I really like technology in all its sort of various forms but sometimes a bit too enthusiastic and I'm a king consumer of coffee and craft beer you can contact me on all the usual places as well if you afterwards so whatever why did I come to docker so a few years ago you know doc docker became very popular he was talking about docker but it was very hard to look at what's going on and I got very frustrated going to what was supposed to the introductory discussions about docker and there was explain to me why I would use it and what problems I could solve and sort of the basic ideas and that's so I've always been conscious that it's important to start from that when talking about docker and sort of what is the difference container an image and until you've got those basic understandings it doesn't make much sense so that's very much my objective in delivering this sort of sort of talk is to try and get people bootstrapped into that so he talked to people about docker you get words like just you know what's dr. wells it's all these things and people use various phrases but if you actually look at that as a phrase on its own it doesn't really mean any much fruit doesn't mean very much so let's talk in a bit of detail about what docker muck what sort of problems docker myself so I've got a simple model of an application which should be familiar to most people so we've got the concept of an application or a service and it's a collection of processes a run on a computer and the resources that they need to actually get useful work done so a process is a software a piece of software it's loaded into memory and running it's a single single process and the process has access to resources so the resources are all the things external that you need to actually do useful work obviously things like files and data in the files also permission to access networks security profiles and so on and so forth yes that should be all pretty simple and there's you know a sort of picture of it of multiple processes accessing different resources to make something that's useful okay so how could you run that well the obvious thing you could have a big machine with an operating system and you just run all these different processes on the machine and the operating system controls access to the resources it's a nice simple model it's got some problems processes can just one process can consume all the memory and starve all the other process of resources or you have to buy a big enough machine with enough Headroom to cope with peak capacity which is a waste and so on and so forth so there's a lot of capacity issues with that sort of thing it's also a complex environment to recreate for development or testing and not practical so there's a variety of issues and of course one process yep I feel early on Linux you have to type sudo unless you make yourself and member the docker group so then in the 1919 90s or so we got virtual machines so in virtual machines were carving up the computer into partly in two different areas and they're protected by having their own copy operating system so that's the difference as multiple copies of operating system running each process is completely isolated from its neighbor that can only get resources given to it to given to its guest operating system that did a lot and the you know VMs are very popular still but they have problems they're actually quite complex to provision and manage and it's hard it's still hard to actually run up the same configuration to test and development as you do to production ok it's slow to start up you know I'll take you if you're having to boot if you've started processing it means booting of VM it takes a finite amount of time several minutes in fact to boot up and of course if you're using licensed software then you've got multiple licensing costs as well now as an aside we're talking to talk about containerization and that does not mean that the M disappear there's still a place for VM technology in the operational environments I don't think that vm's are going away because they're not they're just changing their role it slightly ok so let me come to the concepts for container so a container is a similar model to virtual machines but we remove all the guest operating systems so each of those processes is actually accessing the Linnet underlying Linux operating system here the difference is is that container layer is mitigating or is managing that connection and these controlling the access to resources ok so each of those processes thinks they're actually running the separate machine even though they're not they can't see themselves they can't see one process that go and you see one set of users that given access to they can't consume more resources than they even access to this type of thing so that affected me in a box in a container okay but it's not perfect it's can be harvested once you got it ranked working you're running it's very quick and easy to run generally but it's actually fairly complex to set up and test and that's what you can make sure it's resilient and if you are going to do this at scale then you start to read neat real react you take some of your applications to use things like macro services so typically when people think of docker or containerization I think in macro service architectures but a lot of the applications the world and of course it's not micro service based so having got sort of those ideas out then if you look at the more formal definition from Wikipedia of of containerization then it starts to make sense it got we've got this concept of containers which contain processes that look like real computers to the presses inside them but in fact or not they're just effectively a box in which you can in theory escape out of them which to which you have granted certain resources and and so that they're isolated from each other so does that well that makes sense yeah oh by the way please do free feel free to ask questions I may not be able to able to answer all them bar so I do my best and if not I can take things offline and answer them later okay yes yes I mean that is one of the mechanisms user multiple kernel mechanisms that are used to provide this facility and I'll give you a reference later so you can just server ball so what's the benefits of this scalability it's quick and easy to start these workers you're not so you know saying guest operating system starting one or two or three processes so it takes milliseconds to bring up a new container if your container crashes you can restart it very quickly and easily if a container the container is constrained so it can't in theory run amok and and just take over all everything else or the Machine lower costs you don't have multiple copies of your operating system you can see them a lot less memory so where you could maybe have three virtual machines you can have ten or a hundred containers that's a huge saving and it's now reasonably easy to make your Devon testing ramens look like production depending on your dev environment that's not universally true but for a lot of developers and this is the attraction they they can they can quickly spin up something that's got the same dependencies in the same architecture albeit a smaller scale that's production and it makes it much easier for them okay okay so so docker is a technology and it's a company and they provide and the docker product provides you the containerization so containerization the generic term if you look on the Wikipedia page of referenced earlier they're lists of other Dockett other docker sorry other containerization products and they have been around for some time dock is not the first one there duck is the most well-known them and the one that people are using the most now it specifically worked on Linux kernel back in the day there is now the ability to run docker on Windows I'm not going to talk about that I don't know much about it and this is an its conference anyway okay there's a really good video here that I recommend by Liz Rice which actually shows you how to build the containerization technology from scratching go so it actually explains how the its kernel can run these containers and the facilities that dr. depends upon to make that happen so that's quite good and because docker is so popular there's lots of material out there to help you as lots of tools for orchestration like kubernetes or swarm there's lots of training courses there's talks at conferences there's books there's videos and so on ok so you get lots you can get you can get get up quickly any a question so for us or making sense yes okay so they they work on top of Dockers so what I'm going to show you today is just running docker at the small scale so starting 1/2 or 3/4 single application stuff things like well Orchestra that so kubernetes and schwarber examples of orchestration frameworks so they allow you to run hundreds of thousands of containers so often people talk about pets versus cattle so today we're going to be treating our containers as pets with individual individual ones we care about them we take care of them when you're running a hundred thousand you don't care if one drops off the machine you just start up another one so the ability to start these machines at scale or start you know these containers at scale monitor them and manage them that's why you use an orchestration framework does that make sense ok to very poor piece of terminology docket images so a docker image is the sort of the underlying concept of something you need and all it's got in it is a bunch of files and some metadata and it should be as small as possible people can exercise making it really really really small sometimes you know it's act is unproductive it's like over over optimizing your code but it doesn't contain a Linux kernel it's just the additional files you need besides the kernel to make your system work sort of have things like links packages in it that you might need if you need to copy your cuts for you your application might be in there it'll obviously have your your your compile programs they can run many configuration files that might be needed and then a metadata about what sort of resources it needs what networking connections and so on and so forth and it contains a default startup command so that when the container starts the default command kicks off and the health the whole thing is running the important thing about an image is it's at rest it's on the disk it's not a running process okay compared to a container so container is where you take an image off the disk and you instantiate its contents into it into a running container okay so it's actually running it's it's it's it's in flight it's going and a container will contain at least one process obviously it usually contains more than that but as few as possible is ideal and you can start you can use an image to then start as many containers as you want so if you have a microservices type architecture then you were low balancing architecture you can just keep adding more containers from the same image just scale your your capacity horizontally now an important thing which is kind of surprising to people when they first come across is that a container does not modify the underlying image so if I start up a container from an image and the container rights to its file system and I shut down the container that those changes are gone there's nothing preserved ok the image is guaranteed to always be the same each time you so I'm saying that when the container starts up it has a virtually basically has a virtualized file system it's a virtualized copy of the image and it can update that it can update that that container file system but when you shut down and remove the container that file system disappears and you all you're left with is the original image now I don't talk about later on about how you actually do preserve file system changes but it's outside of the image okay that's that's with stability right so if you have an image it will always be the same your guarantee it's not going to change you start it start container from an image today it's gonna be the same as it was yesterday you know no no change okay and the important thing to note is that the working application will often consist of multiple containers connected to each other in various fashions to make a working system yeah and so the magic phrase to remember is you build images and you run containers if you keep that in mind your life will be much happier yep makes sense okay right let's actually start doing stuff at the command line so let's actually get an image to run so the first thing I wanna do is to actually pull an image from docker hub and the way you do that is to go doc is that font big enough yeah sure out the back you see it's up the rack yes good so docker container if you got if your installed the command completion by the way so if you come that works on Mac and Linux I don't know about Windows but you hit the tab key with enough characters you do get and you've got completion installed it will work so I go doctor to contain a pull alkyne and I did something wrong because it's coarse docker image pulled off it you see I'd be doing this for every year and I still get confused wieners and images because of the way I got my system set up went if I want to fix this cuz I don't always doing it he's wanted me to log into to hub I don't think most of you'll have that problem I already had it pulled down so it didn't actually do anything except check the shot yes John it does yes so you have a you have a local library so I have duck a client installed here not that's all well has it got docker engine but but I'm using a docker client the dr. engine so when I I have a local library of images and he's just pulled that image into that library yeah so it's not anything that it's just sat there and I actually look at it so if I go docket image LS I've actually got quite a few images and I have a laser pointer there is my Alpine image that was and it was built by somebody else four weeks ago I didn't build that okay I'm putting it for a place called hub docker calm which is a website it's the world's biggest website of images of thousands and I have for Apple which you'll be putting you'll be putting images from my docker hub account later on it's a bit like github in that respect it's a place where projects can put images for other people's to use so if I've got an image I can actually look in it so if I go docker image inspect don't worry about this too much because we been doing up you go and do an activity we do all this the minute inspect Alpine latest and that's combined command completion others only back take by LP hit to have any filled in everything else for me that's really quite useful and I get lots and lots of information which I'm not going to take you through but you can actually inspect all sorts of things well you can expect images you can expect containers when they're running you can spec networks which were integrating in strict volumes so they've all got this method this is the method I track talked about earlier an image has metadata this is the metadata from an image and something useful things are for instance the show that the command well as a shadow lets you see the shadow the command it will run is under that CMD heading in the container configuration stands up yeah and it's also the path it uses and so on but I'm not going to look at in detail now okay right let us do some work if we should be able to go to the workbook on github and hopefully you can start doing tasks 1 to 8 yep do you know where I mean if I go yeah doctor for novices so here and then if you go to the wiki section here as a workbook so you two will do you will pull from Dhaka I've been not able to find that No I like the game Kim so that's that's the that's the workbook yes oh yes sorry my my mistake I didn't update that slide my apologies that's that that's the URL you can read that let me just quickly update that we've gone too far I have lots of slides most of which I hope not to show why is it so slow but I can't edit the slides for the moment just try and really I think you can't read up there let me know Oh good tips thank you anybody lost sorry yes yeah if you trust me yeah yeah I certainly can you can just so I did the pool I did the inspect as a better version of the inspect command here so you actually get out so that's the way I'm actually extracting a specific information from the inspect because it's all JSON so you can use a JSON format to extract the information right I can actually run it so that is a container shell prompt so I'm actually inside a container so if I go PS I've only got two processes running one is the shell that's running in the container and the other ones the PS command they're running on top of that if I look okay I get the whole thing it behaves slightly differently under under Mac and Linux what else could anyway that's the one in container you should read the notes about the IT option by the way that's pretty pretty important and actually if you actually look at this so if you go docker contain content unless - a that I hope that containers still running in fact they're but they're all still running so you actually got to type in an extra command which is docker container RM and now if you do a delegate it's gone now I'd actually start hello world a couple of times so I need to get rid of those as well so an alternative to using the name is to use the ID which is this magic char number like 5 5fb so if I do that I've lost both women oh yeah I've got rid of one of them and if I do the other one so I had a to tie ball of containers missing yeah it's okay so it's like in okay first of all here's a tip you often hear people talk about how registries and repositories and and repos in Dhaka are just like gitlab and it's not true or github it's not true they are there are some similarities but they are different enough to be confusing so don't use that model having said that they every image and every land every version they've sort of every object pretty much in docker has one of these unique IDs which is basically a shot I can use a simple technique as long as you type enough to be unique it will know what you're talking about so I've already used the first two or even the first one yep you get nothing that is because glad you're after I've got a demonstration of that sorry what was your question okay I'll demonstrate that right now the same same question okay so what I haven't talked about I don't and it's in the slides but I'm not going to talk about it a startling just demonstrated he's a thing called the container lifecycle okay which is part of what you're doing so I actually have a script that shows some of this off so so what I'm doing now is I'm going through a series of docker commands with some technical explanation of what they mean and I'll also explain limits so the first thing we're going to do is that we're going to run the docket container called alpine and instead of it being interactive I'm going to use the minus D option it's going to go into the background and just do some work so just sit there okay and what that does is it actually runs a little script fragment that just prints out integer every every 5 seconds so it's giving me back again going back to this idea that everything in docker has a unique number there's the full char for that container so it's actually quite long okay now because the container is running I can run docker container LS yeah and it tells me the container ID it could tells me the image it came from it tells me the command it's currently running which is a while loop when it's created and its status okay so this is important this is is up being up to 25 seconds okay it also has a name it generates a ret if you don't give it a name it generates a random name which we can't see cuz it scrolled off to the right okay now do you ever I didn't inspect of an image I can also do an inspector of a container ii see so as well as image having metadata when it's actually just on the disc once the container is running it also has metadata and i can look at that with the inspect command so if i hit and expect to find just get loads of as a json scrolling for the top so if i pipe it through less it's a bit more useful I can see the image it came from I can see the command it's running so if you look at the top there there's the actual while loop that's going on they've got the pass to it and there's a bunch of other information which we won't bother looking at right now that's quite tedious so here's a version of the inspect command that uses another one these format modifiers to actually just extract a single piece of information in this case it's the status so the status is running okay so we went to not running to running I can now stop the container so I issued a stop command so I stopped contain a stop name of name of my container not because I'm a right image the name of my container okay that will take a few seconds because runs a while loop going to has actually got a kill the while loop so I think issues a kill and then eventually get the board and issues a kill nine or something that's documented somewhere okay and so we had to stop now the thing to note is if you do docker container LS again you get nothing the list is empty but if I do the inspect command again so the same inspect command to just get the status the status is exited and if I run docker container LS - a which is what this is about to do then it's actually in the list we just didn't appear unless we put the - a in because it status is now exited so it's still in memory no file system change has been thrown away that logs are still there I could look at the logs I could I could reattach to it and start fiddling with it I can read I can actually start it again which I'll do in a minute so it's still it's to the it's still a chunk of allocated process space albeit one that's not doing anything okay it's in a special of docker State so if I run the start command I can crank it up again so you see what mm-hmm so any minute that's worse yeah I haven't maximized it but I can't I can't pull it back but no because if I'd maximized it I'd do that although sorry you're right ah damn yo thanks that why is that still any closer like right so I just rang container start so that equals exiting I've now ran start it's come back up again and the status is off the bottom the screen right sorry the SEC is now running okay and now it's running I can actually attach a terminal to it so remember it was in demon Road it was in touch mode so if I go attach mcalpine that's right in my Alpine I'm now accepting now I could start seeing these numbers coming out so it's been sat there for twenty two times five seconds running except the timer suspended it's just hitting numbers so I've hit ctrl C because I'm bored it will finish so now because it was attached to terminal and I've killed the process that was running inside it through the terminal it's gone back to exited so I didn't actually have to do stop because I hit ctrl C inside the inside it it's gone too exited why stop why is these notifications coming so now actually so now that if I to actually get rid of the so yeah so now it's stopped I actually need to remove it okay so if you stop it if you stop a container it's not enough you actually have to remove it and you remove it with the RM command so it stuck a container RM and now if I do talk of container LS minus a that lift is truly empty and if I do the status command this is where I can't find I can't find that contained view the status command on so to do the inspect command so it actually just gives an error okay now if you want to if you haven't stopped a good potato and you want to remove it you can use the minus F but like in Linux file systems - f is to be used with care so let's get our life cycle so if you carry on through those tasks that will actually give you practical experience of of users of hopefully understanding that I fear that out of the two questions at this side about about that there's a control PQ option which I've never got to work successfully yeah you can do there's an exact option so you actually see if you have a run in container that's doing something or other you can actually start a new shell on it and inspect it is going on that's with the attached to a good question that's worth remembering attached I remember it's in the workbook or not one thing I didn't say is that we have far more material that we can cover if I mention that so I don't expect you to finish all the tasks and before we have to move on to the next slot because we are short of time sorry not well because it's not the prices running oops I accidentally got the stream going I have such an ego I have to yeah watch my on live stream so everybody happy in it was anybody lost no good this useful information learning stuff good getting some thumbs up at the back yeah honestly I step in the weight room I would have exited really because you I was just yeah because it's in the background so I'm just starting a shell because you have to give it a command to run you've got it that's it oh hang on they haven't so you've started it so you've now got to you can now exec as you can type instead of typing out you sir to be 6d yeah yep oh don't know - won't work that's a no tribe Sh yep okay do a so what's your status you've exited you shit's not running yeah do do start uses document yep you got me the sh that'll cause an error okay so have you started no no how did you start it original you put it on to kill it and start it again so do it dr. RM remove WR rim container ID so go okay so now now now do the original docket contain around come on let's see how you did right that that I she doesn't stay around that's that's that doesn't it just prints out hello world in exit exits you want to be running Alpine yeah yeah Alpine is the image that will give you a shell by default yep yes PS vs. Ellis yeah but you can do at least in thinking if you do dr. PS minus Q a you will get just a lists of identify useful so I use that to feed it into other thing like dokuro ring okay so don't forget to RTFM green.this read the fine manual there's a lots of documentation on so probably a little aside here docker is an evolving product so things change quickly it's one of the reasons you want to be on a breezy modern version and the documentation doesn't always follow or it's a bit patchy the good news is neat buttons if you see something you actually go on the website and edit their docs so if you do see something I encourage you to do that but information can be repeated in multiple places and so and so forth so thanks for the practice but there's lots of information there it's gets a bit tedious typing all these commands so shell alias is your friend there's an example here throughout the rest of this workshop the workshop notes and and I will use these aliases at the bottom they're quite useful but you can create quite sophisticated aliases and I've put a couple of links in the further reading section with with links to people who posted lots and lots of very complex aliases yes you could but you can't you can do you can do yep but it's easier just to add yourself to the docker group log back in log out and log back in again if you're a mac or wind how do people here on linux oh okay my see I thought it'd be alright because would usually do these things people running on Macs and Windows so the problems are slightly different so nobody's running docker machine yes fantastic doctor machines slightly different doctor machine so doctor machine is useful for two people firstly people who have platforms that don't support the current docker deployment on Mac you know that it's a sort of default deployment so if you've got an older version of Windows or an old version of a Mac or so on then use docker machine because that depends on VirtualBox so using different technology the other reason for using docker machine is because I like to deploy to any machine so because it works with things like VirtualBox if you're into provision machine a different machine you can use document in from your workstation however in the new release of docker like 18:09 they've introduced an SSH facility so it's not clear to me still wide use docker machine but I'm not a systems administration guy so I'm not an expert so don't don't take anything I say seriously on that regard no no because you know you're still running binaries yeah so you know you're still making system calls you're still running on hardware from the so on a Linux box because there's nothing you've said between you and the hardware on links box you're effectively in the process that's running inside the container is a normal process so it still gets CPU cycles it's still managed by the kernel but it's just it's just controlled by the containerization layer you can run docker on part by the way there are things to do that sorry yeah there's a lot in the instructions ah bear hedge boy I made the rookie mistake dressed he'll make all the time which is buying a sigh - team sorry yeah it's yeah okay I might I might press on if that's okay so because we've done that we don't need to we don't we don't need to go through a lot of these slides yes I'm sure your police's honest um I am these I will update this fight because all these links are slightly wrong because I moved all stuff as mark I like the geek I moved into doctor or novices we talked about that okay let's talk about network ports so I talked about the fact that all these containers are isolated right which means they can't write the network they can't they can't write to disks they can't do anything well they can't they can do stuff but they can't you do anything the information or they can't get new information so so we have to fix that because otherwise they're useless so we need to find very controlled ways of exposing information sinks and sources to the container and one of the ways to do that is to give them access to a network port okay yeah you use what's called port mapping it's also called publishing so bit a bit of mix up your terminology you'll see it referred to in both in the docs but the idea is is that a specific port on the container is exposed to the outside world on another port on the engine box okay so for this example here I'm running the nginx I'm running nginx web server okay and nginx by default publishes on port 80 so inside the container the Internet's process is listening on port 80 for stuff and what this - - publish option does is say ok so inside the container it's port 80 on the outside to the outside world it's bought eight oh eight nine and anything that comes in on a ti-89 will get sent to the container on port 80 yeah how do you want to prior to picture for this I didn't have time so that makes sense yeah so that means that you can just take this we have it don't don't worry about doing this example because of the reserved as an activity to do this which is different so if I just run this container it's actually on download the the image does that very long I write all these examples to work really fast across poor conference Wi-Fi it turns out our Wi-Fi was really good but means of sylvans really crossed yep all of them zero zero zero zero yes good question right so come on start up Oh all interfaces so you can connect no so you can connect into this machine on put all right oh wait at nine and you'll get you'll get into the engineer's box I didn't run it in daemon mode so it's blocked never mind that is why we have model tips so now I can just hit that end point whoops and there's the default nginx index - real far yeah make sense so here's a tip I used to I could never work out I could never work out which way around those numbers win so what I do is I remember that it's it's outside and then inside when I draw an arrow but you know going from the outside to the inside and that applies to the volume command as well well that will be using melt which doesn't quite work that way but but if you see publish or minus P or you see volumes or things like that and you can't work remember which is which it's always this is the outside on the left and the insides on the right and the informations going from one to the other but your onic helps I'm gonna say no it's so what you do is the on the metadata you say I'm going to publish port 18 but you can only specify which port at runtime so you actually have to include that line yeah I know where you call it nothing I'm not a good enough Network going thank you now if he's not the same as docker networking okay I did so I wouldn't talk about networking so I'm now going to lie one of the things we won't be doing is all won't be sitting at the dock to a container network so you can actually set up a whole networking infrastructure inside the docker engine and across docker engine so you've got multiple machines or running docker containers which you need for application you can set up all these virtual networks between them but that's completely outside the scope and not something I know a huge amount any about anyway in the case but this is this is port mapping and that is really common so it's so common that we've got an exercise to do it task 9 so what I'm this is a completely different example of actually written an API server that you can run and hit and the source code and all the docker build stuff is all in that repo so it's not some I mean when you run it is just going to work but what you can then do later on is go back and look at how I set it up and built it which is the real value you're just cutting and pasting stuff doesn't know she teach you engines yes you can but newsing is pretty different mechanisms so there are various options to do sound and things but you know I've never used them dude yes yes I don't know how but I'd be surprised that didn't work because you know this stuff's used at scale so how would you work in the scale in robbery you couldn't do that I don't bloody know how good question anybody stuck sorry okay so stop it you should really just go RM minus F and killer no not docket kill it's da car and minus RF remove - firsts okay anybody not finished I have a bad knee so excuse me first sit down I wrote to my wife that I was feeling nervous so she's it's not going to be on that YouTube okay I think I killed it okay goodnight yep yep yes you can do you can do that because you the doctor demons running with root privilege it's just people don't usually do it but yes you can and you can map 18 so what you also quite come and see is 80 to 80 so people put 80 on both sides like : well it will do something when you're running on 80 but yeah sorry [Music] okay cut prism so file storage so that's kind of the major missing piece is that had you had you sort of read and write files databases configuration files whatever the correct way to do it is to use volumes so it's not fantastic but the idea here is to show that volumes are made controlled by Daka so what you do is that you say I want to create a volume and docker will create the volume for you and it will mount it into the filesystem of the container and every time the system read and writes so the container read and writes to that value to that mounted filesystem it will be R it will it will go to the volume and then the volume persists beyond the lifetime of the container okay mounts are sort of the same thing but different so amount is a directory that you say to docker here's a directory and I want you should mount it into the process into the into the docker container so it's not it's not a question of saying you're doctor please create me a volume and manage it it's here as a directory just mount it so if you're a developer that's really useful because that you can put your source code and stuff on that bring up your development container and start working and whichever little exercise to do that but a volume is different so if you've got a database file you so write to doctor and again we've got access to docker I want a database directory you manage it and to back that up and to manage it use other docker containers to create your tar files or whatever the backups okay so you it's and it's it's squirreled away somewhere inside the docker file system you don't have to worry about it but with amount you have to worry about it now the reason I'm talking about this is because volumes are very clearly what you should be using in production environments you should not be using male now possibly some use cases were not true but generally using amount in a production environment or even perhaps a QA environment is a bad smell if you're a developer then the only things that should be in your mount points are things at your source code that don't need to move with the container okay they're just things that you need to use whilst you're using container so differents obvious or not obvious but you know does it make sense yeah okay so okay things ism so here's an example so this example is a bind Milt so that diagram is actually slightly wrong it should say a bind mount and dative on you so here we have a containers running its the nginx container again and I'm buying mounting the nginx HTML folder into a local that example is incorrect I apologize it's wrong that should that should say volume not burned so what I'm saying here and yeah they can all that as a byte this is it correct body mount this example should say volume which is bind and this is a volume is managed by Daka so what I'm doing is I'm saying - docker please create a manager volume for me and when this container starts map that space to the HTML directory for nginx okay and that means the contents of the of the HTML documents are preserved but there's a typo in it and of course we've still got the publish this one is a mount now a part of the fact that type is different where it says bind it should have said volume in the previous one the only difference is the way that the names specified so if you notice where as a source is I actually put a directory name not a name put the name the value of a directory path not the name that that will something to be corrected over so you can do two examples ten eleven don't fight any questions about that before we carry on we depend they're both good question it wasn't clear now one thing that I emphasize in the workbook and I'll just do this verbally is you guys here I'm using - - mount so let's - let's do the correct example - - mount when you got on the in the big wide world I'm gonna let you loose all the examples you'll see used - V instead - V doesn't doesn't use that format it just uses source and destination and the only way you can tell the difference is the fact that for a bind for a bi mount it's a full directory path and for a volume it's a name I'm missing and all you do is missing that forward slash at the front to tell you it's the directory path so it's really really easy thing to miss so this is now look for the preferred format but you will undoubtedly see majority of people using - - volume optional the - V option which doesn't specify it explicitly they are explicitly like this yes no it does map you can discover with where the directory is but it's not a directory you have to worry about because like I said when you actually want come to need to back it up or you know or copy it to move it then you say to docker okay please copy this stuff out for me to standard output and I will then put it into a tar archive or whatever no will be hard to do but yeah it's not the way it's meant to work yep any questions let me do user signs of us anything that Dunkirk rates for your three different networks but you don't care about those like repeat the questions the question was was was there any special containers or or images or stuff or volumes that docker crates for you shouldn't delete in the answer is no the only thing is docker crates is three different networks and you can you can't delete those everybody finish this up so yes you can add extra parameter to the Mount come up to the mount option to make it read-only correct correct yeah I'll sit PFS appointment faucet ah because I'm there's actually a small bhujette in the instructions so if you go to file set be so mad I can't your directory like it says you should do in the workbook oh hang on no it doesn't sorry I miss you read my instructions that's that's now running so if I go so in here's my example Coast I go they were about to do that cuz I'm gonna go sir oh dear there's a bug they're having trouble with the exercise every spot the bug I had no idea of skill level people who turned up so I decided not to make it too hot save that it works any jobs for lure Primula anybody had trouble with it does yeah okay yeah that's exactly right yeah because suffering might die so if it's if it's hanging around you can restart it so that's one reason you can start it again you can run Diagnostics on it so you still got access to the lot if it's if it's still hanging arranged through access to the logs you can still look at the file system because remember they a container can change its file system it's just that when the container is is removed those changes get lost so you can still pin it you can still through this file system and so it's if you sort of got a debug your container that's why dude that's why you want it hanging around but once it once it's stable then I always use the minus minus RM option and is two minuses because it's really short for remove it's not two single flag options yep [Music] so the question was if I set up a database server with access to a volume in that set of another however many other database servers and pointing them at the same volume does it stop me I don't think so but I've never tried never looked so it's good question I think it does but but I've never because at the end of the day is still Mott you still just another file system you know still it's still rim it's one of the next kernel so he's not that isolated then it's still being a lock tape my dad's got a separate lock table okay does anybody manage to do the volumes one so I created a database example using SQL like this tea tiny bit still illustrates the point so what's he's doing is downing another sequel another image I created and all this does is run sequel Lite so I'm passing a crate table command to sequel Lite and it straight to the table for me but I haven't got a exam got a volume command in there so we are then try and run an insert on the same table I'm sure you can all guess what's going to happen no such table the tables because you know it's a different container say me of each different container yeah so any change I made in the first container is gone and it can't see the second container so the way you do that is that you pass a well first of all make sure you can actually look at volumes because they're things so there's no volume on there volumes so now I can start running these commands the same exactly the same command except it's got the - - mount option so I created a table and this time if I do the volume and I see it's actually created a DB demo volume and so now I could run insert on that and it worked and of course the select will work as well July make sense yep so three independent containers yeah correct and I can and now you could just delete the volume if you want so you know volumes the thing I can go to curve volume RM DB DB demo he's gone so if I try and run that's lip that's licked again no such table lost all my data we used volume RM with care was the listen okay sorry say that again you get a what okay no why because I make sure my dock my repos with private their public okay can I go through the material then they'll come see it see if anybody else gets the same problem okay okay that's a shame alright let's talk about environment variables so yep are you talking about building your software okay so so I mean so there are various patterns for doing that but yes you could do that but another way would be to actually have a have a separate container you'd get clone in to build the software and then you and then you deploy your built software from one container to the other so you never get kind into production image or container yeah make sense okay right parsing it so we've talked about volumes which is sort of i/o you can read stuff and you can write stuff we've talked about network ports which is which is about reading stuff and passing information back we've got a great environment that environment values so the classic you know something environment strings that you passing it they're obviously read-only it's we're getting information into the container but you can't get information out which should be you know you should use that from the way that sub shells work so there's a minus e option so this works when you're doing a container run command and you pass it a minus e option so there's an example down there and an important point about environment variables is that there is it's tempting to pass secrets and other bits of security information using environment variables but they're not secure so don't do that there are other mechanisms which you'll need to go and discover to actually load and there are newer mechanisms out with a very latest release of docker to actually allow you to securely pass secrets around when you're building images and things like that or running images so don't use don't use environment roles or other things they're quite useful so here's a really simple example so if I run if I run the Alpine image odds are M minus RT Alpine so notice the Winterfest as an aside I didn't type anything off the Alpine so actually got a shell in there so I can do the usual things if I exit from that and I used - thighs RM so I don't have to remove it but if I give it another command like env it will run that come on and exit so that's really useful for little demos like this so the env command that you don't know prints out on my environment values so by default you only get for environment variables path sort of home and also the host name okay but by using the - iakh ssin i can pass in extra stuff so the first example if I just type in the name of an environment variable that will be taken from my environment so the current environment has that just to prove that let's just so I can't yell CI Tama set to Australia if I go by this e LC time then hey presto the kind of gets a copy of that as well okay if I want to pass something different I just say equals and then some other string so in G B D F dot - 8 and suddenly the containers in GB locale so that's how you pass environment variables in yep toss trophies right you had a problem okay that's kind of weird I have tested these where are we the patrols ah yes different or you okay you don't need to fix that I'm not short said what so for the time being could use follow the interest rates for them this is the instructions that they need to posted in the get er rip positive chat thanks for that and my apologies again for the mistake so when you got access to that clip away yeah I'm missing I'm missing a build but I'll fix that later today and it'll but it'll work this example by the way is is these examples gonna get more and more complicated and have more more moving parts so if the actual example doesn't make sense because I'm using a profile login file let me know so just apply Anita's fix some work it works if you make notes for mythix what time's lunch trough city training okay okay container logs start going is everything that gets written to containers standard out or standard error is considered to be a log and docker will save it for you for a certain amount of time it's actually a buffer so eventually it will get overwritten and there's a command called docker container logs and you give it because very reference and it will it will then spew out what it's seen so far and if you use the - - file option it will behave like tail - F okay pretty simple I'm going to skip the next so I got some question are you meaning the same in the so it's whatever's on the containers standard out and standard in so it's whatever whatever is is connected when you do docker run okay very quickly we need to talk about how to create your own images okay so you need to write a an image you need to create a docker file which is a list of instructions to docker and then you run docker build so here's an example docker file it's very simple but it shows some important elements the first thing is the first thing your docker file is the base image in which you are pulling so what will happen is that docker will see this and it will pull Alpine 3.8 down and will then build all the other stuff on top of that image you can get scratch images with basically nothing in them okay you need to provide some metadata so it's always expected that you provide the maintainer and description labels and that just gets embedded to the JSON that we saw earlier on next is where you start ringing run command so this is actually instruction to docker to basically instantiate this image as a container and then run these commands and then store the results away as the new image so in this case it's something very simple it's just running a package manager to install lure and the last thing is the default command so this is the command that will be run when when the when the container starts up in this case is just running bash sorry not bash is running this shell and it's been running the lower - version command okay that's all it does now you will see a lot more in dhaka files but this is the basic simple stuff the other thing that you will see which I haven't got in here is a copy so if you look at the docker file supplied for the last example with with the profile file is actually a copy - copy in the profile file and that's the majority of what you'll see the bill command you can have a default docker file name but I prefer to use something a bit more descriptive and therefore I have to use the minus F option but I always put docker file at the end of it you give it a friendly name so this is called a tag and the tag includes the tag olanta so it's a bit confusing because the bit after the semicolon is also called a tag so I prefer to use the word alias for this this collection of this string is an alias and it consists of a name followed by a tag but to do that you use the Manistee option okay and the thing that you always forget is a build context that little dot at the end you have to give it the name of the directory which contains all the files that you'll need to build this even if there are none okay so if there are notes so in the previous example I'm not copying any files in so there's no external files needed for this example so I have to give the name of an empty well I can give the name of any directory just the files won't be used but you always have to have that directory there so you often see a dot and the dot is actually quite hot easy to miss when you're reading it so be careful yes is the answer you but you've built dock accordions for all sorts of purposes and depends on your particular environment and workflow what you build but you've got the flexibility so you've built what everyone so we're very short of time you can start this exercise but in five minutes we'll be moving on to something else but I would suggest you start with with task 14 and then there are a bunch of other follow-up tiles right where we build more more complex images which you can do afterwards and ask me questions through the keto form it's probably easiest but go for go for 14 to ask 14 to 5 minutes see how you get on yes do not use the soap so you'll find that most images have a latest tag right so you see you know so people will either use the term latest or they will just leave it blank and therefore pick up ladies because that's a default don't do that okay the reason for not doing that is there it there is a convention that latest is the current production version that's the convention but it's just a convention and it may be very out of date it may change so you know you build suffer on a specific version which happens to be latest so you use the latest tank and then next week they update the image well they you know they real able a new image with the latest tag and the old one doesn't have it anymore you rebuild and suddenly got a different version of your different version under under your software and something breaks so I don't elastic present workshop yesterday where they said we don't use latest and I think that's dead right but when you are when you are building dockerfile when you're using docker files always explicitly put the version that you want to use otherwise things will change under you they have used so know if you use latest there and you rerun the build you get a different version from docker hub or you risk getting a different version so don't use latest yes correct yes yes because it's small and convenient so I'll pass a popular version I did put some comments in the workbook that Alpine is smaller convenient habits got a couple of problems one is that it uses a different package management system - the one you're probably used to that's a bit fiddly a second thing is it's so small it comes without things like it uses busybox for the shell and things like there are limitations so so yeah so it's really cut down so it's really useful for demos and for small prototypes and that sort of thing but I'm always using Debian when I when I'm building this stuff tip for you is don't use Debian stretch or Jessie or whatever use always a slim version which is usually about 20 Meg's smaller so it's a bit cut down so that works well for me does that make sense I did say in the abstract that I would show you docker compose running so I feel obligated to fulfill that contractual requirements and I'll do that in a minute yep you got the correct subdirectory look make file set D you're correct default your parent default and today to detention order to that of the Dan Durda now I've done my job correctly okay yeah a copy works for me so if I go docker container run works ok duck a composed so just a quick aside first of all I wouldn't recommend you start using docker compose until you understand a container build and run I'm sorry dr. image build and dr. danee run properly and understand the nuances of docker files and running them and so on but once you've got that then docker compose is a useful tool because it allows you to to to orchestrate multiple containers not a huge number but a few containers to create a working system now I talked about kubernetes and swarm being for the you know hundreds of thousands or even hundreds or dozens composes for the few that you need developer working system so maybe a database and a proxy in that this is and that's the example I'm going to show you so it's quite useful and and you can actually put a lot of the information that you would scatter into docker fast and here and there into a single Yammer file and just running it brings the whole thing up for you so I'm actually I'm actually using somebody else's docker files glycol breakfast who runs what I think it's a very good udemy course on this what I'm going to do is to download one of his composed files and quick quick quick it's such as the a more fun it's specifying the images I want to use the ports are going to expose the environment variables I need and all the other processes and sequels it's a property it's a it's a it's going to bring up a clustered msql environment but from one file so if I go docker compose minus F compose three yellow it doesn't work because I forgot something I guess you need a command and so bring it up so if it needs to will bring anything down it doesn't need to so it's now going away and bring up all this infrastructure for me produces nice logs color-coded different colors different containers sorry yep just from one command it's bringing up four different containers which includes a clustered MySQL instance and the fact that could do that on a laptop is quite impressive so there is so that's it running and you know on a little tiny laptop I brought up a fairly complex environment it's not an orchestration but it is a way of sort of packaging an application a traditional application that just needs a database server and a thingy and in one of those and a couple of those so for that sort of environment it's great yes all it is is a wrapper around all the other stuff it's like running make instead of having to manually type cc and Lincoln that it's just a mess it's a fancy make system for the docker but in the same way that you know you wouldn't start if you have knob viously programmer you wouldn't start from make you'd start from CC it's the same here you need to understand the end up what's going on underneath the hood and I think I'm officially out of time so if you've got any questions just follow up online I knew we wouldn't finish everything because it's just too much to cover in the length of time we got thanks for all the questions and I really enjoyed it you

Channel: linux.conf.au

Views: 4,703

Rating: 4.7460318 out of 5

Keywords: lca, lca2019, #linux.conf.au#linux#foss#opensource, AlecClews

Id: xsjSadjKXns

Length: 99min 46sec (5986 seconds)

Published: Thu Jan 24 2019