Creating Cloud Linux Images for Linode with Packer by Hashicorp

[Music] during my career as a Linux administrator it's just an awesome thing when I find out about an awesome utility or tool that allows me to get my job done easier and more efficiently so I was really pleased when I started reading about packer by a Shi Corp which allows you to basically create cloud images in an automated manner which is awesome because you know configuring the operating system tuning it and getting a set up and creating custom images and things like that can be a very time-consuming thing for anyone that does DevOps or Linux administration so in this video what I'm gonna do is give you guys a quick example of Packer I'm going to use Linode as an example for the platform but Packer is able to use you know many different platforms not just Linode but it's an easy example and I thought that I would show you guys and what I'm gonna do is use Packer to build my very own Linux cloud server so let's go ahead and dive right in so here I am logged in to my Linode dashboard we don't have any Lynne Oates right here but you know we're going to be creating one here very shortly if you don't already have a Linode account I do have a link in the description below and also shown on the screen right now that will give you $20 in credit towards your Linode infrastructure because they are a valued sponsor of my channel what we're going to need to do first is create an API key because we need to have a way for Packer to associate with our Linode account in order to log in and create the image that we need to create creating an API key is very easy you simply go here to your profile you'll have an option here API tokens as you can see I don't have any right now but I'm going to basically add a personal access token which is this first category right here I'm going to click the button here on the right to add one yeah we need to give it a label so I'm going to call mine Packer now here we can optionally choose to have the API token expire if you click on it you have these options right here you can basically have it expire in three months one month or never the option that you choose here just depends on your personal preference and also maybe your company security policy if there is one I'm going to choose never in my case and then here on the bottom we can choose what permissions we will give our API token we don't want to give any API token everything we only want to give it access to the things that it's actually going to need in my case I'm going to give it full access to images also the Lyne notes I have on my account and that should do the job so I'll submit and right here we have our personal access token now it's very important that you don't show this to anybody especially not on a YouTube video where there's gonna be thousands of people watching this because anybody with the personal access token can use it to do all the things that you gave it permission to do so if this key was to fall into the wrong hands virtually everybody could be able to create a Linode on my account so in my case I'm just gonna delete this anyway before I edit this video so this won't even exist by the time you're seeing it so I'm able to show you and I thought it'd be cooler to show you guys an example of what this looks like but it's very important that you don't make this publicly available and a very good example of what not to do and probably the best example of what not to do is don't include your access token in your github repository especially if it is public or anywhere where you're storing your configuration this should stay secret anyway I'm gonna go ahead and copy it and click OK this will not be shown again so you're gonna want to make note of this or you'll have to recreate now the next step for us is to go ahead and download Packer because we have our key right here but in order for us to actually use it we need to have something to use it with and we haven't downloaded Packer yet so let's go ahead and get that finished so here on my browser I am at the official Packer site which is Packard IO and I'm on the download section here so again this is the URL I'll have a link in the description below that you can click on and regardless of your platform you are able to install Packer so if you have a macbook for example or you're running freebsd linux or even windows there's a version here for you now given that this is a Linux Channel and I pretty much only have access to Linux machines anyway I'm going to show you how to do this on Linux but it should be pretty easy to do on any other platform you simply download it for that platform now in my case I'm going to download the 64-bit version which is probably what 90% of you or more will want to use because 64-bit is the way to go nowadays in most Linux installations are 64-bit so if you are at all unsure go with this version but if you do know for sure you're on 32-bit go with that version in case you're curious I'm going to show you real quick how to tell if you are on 32-bit or 64-bit if you didn't already know so I'm just gonna open a terminal and then you type you name - M and you can see here I am on 64-bit I'm using a 64-bit version of Ubuntu this is a boon to 1910 and there actually isn't a 32-bit version so I already knew that this is a 64-bit but regardless of your Linux distribution you should be able to run this command right here which will tell you if you are on 32-bit or 64-bit in which case you would not get the 64 right here I'm going to go ahead and click on the download button right here for the 64-bit version go ahead and save it back here on the terminal I'm in my downloads directory which is the default place where browsers on most Linux distros download their files and if I list the storage should be there and it is there's Packer we did download that so I'll clear the screen we need to unzip it now obviously you're going to need unzip installed for this to work you just do which unzip to see if it's installed my case it is in most distributions that are Debian or Ubuntu based if you didn't get any output here you could simply do sudo apt install unzip just like that put in your password and in my case it's going to tell me that it's already installed but if you're using a different distribution you do need that unzip command refer to your documentation for that distro on how to do that if you're not running Debian or anything based on Debian or a bun to put that out of the way again we have this file right here we need to unzip that which is pretty easy we just type in unzip and then the file name itself which will probably vary by the time you get to watching this video depending on how long it takes me to edit it but press Enter and if we list the storage again we have two files we have Packer on the left and the original zip file on the right we don't need the zip file anymore so we can go ahead and get rid of that just to be tidy and we have this Packer file right here so this is not an installer most of the hashey core utilities don't have installers actually I don't think any of them do so basically you can put this Packer executable anywhere you'd like and run it from there if you are in the current working directory that the Packer binary is downloaded to and unzipped to you could simply do dot and then fork slash and then Packer you can see that it's working I mean we didn't give it anything to do any work to do but it did recognize it and it's giving me some options here that I can use to as a reference to get started but you can keep packer here in downloads that's really not a typical place to keep it or you can move it to somewhere that system-wide that you'd be able to get access to it a lot easier now it's very common to create a bin directory in your home directory that involves editing the path I'm not going to get into that right now so another perfectly valid place is actually I'll just show you so sudo MV for move and we're gonna move Packer to user local bin and press Enter now we could type which Packer and we can see that it's recognizing the fact that Packer is in user local bin this is system-wide everybody will be able to access Packer from this directory if I list the storage of user local bin you can see the Packer binary as well as the permissions but the problem is it's owned by my user we definitely do not want that so what I'm gonna do is su do CH own then root : root user local bin Packer and if we check the permissions now we'll see that it's owned by root why does that matter well the reason why is because if it's owned by our user account and some kind of malware or some kind of you know horrible software it gets on our system it's going to have access to change that file and we definitely don't want anything to change unless we want it to we can use the sudo command to update Packer we simply redownload it and move it over top of it so now that it's owned by root we need root privileges to modify it which means malicious software would not have root permissions and would not be able to change it it's just a security thing I recommend putting it in user local bin with root permissions it's just better so I'm gonna go back to my home directory here and again we have the Packer command now we don't need a dotter forward slash anymore because it's in a recognized place and we are able to use it so how do we actually use it for the purposes of our mission here which is to create a Linode VM and that's what I'm going to show you right now now first of all we need to create a directory to hold our work now how you manage your directories and your your naming convention and things is completely up to you but what I'm going to do just for the sake of simplicity is I'm gonna make a directory I'm gonna call it Packer underscore WD for working directory I'm gonna CD into their course it's empty because we just created that folder and in the real world you would basically have a folder for every project or a thing that you're working on so you might have multiple Packer directories with the configuration files inside in this video we're only going to have this one but you could have a folder for webserver and then have the configuration in there you could have another directory for a file server have some configuration in there and you could even have them all under a master directory that tracks all of them and most people will probably put that into version control might get that's beyond the scope of this video but for now we're just going to work in this one directory now we need to create a few JSON files that is the format that Packer uses for its templates and it doesn't matter what you call it but what I'm gonna do is open up an editor I'll just use Nano to keep it simple and I'm gonna call mine the node example dot JSON this is an empty file because we're creating a fresh one and I'm gonna go ahead and paste in my sample here and then I'll go ahead and walk you through it okay so here we have my example file and in the description below will be a link to the wiki page for this video where you can go to actually download this file so don't worry about typing everything out manually actually Packer is very very touchy when it comes to formatting so even so much as one comma being out of place or any character being out of place is going to be enough to basically make it fail so just to keep it simple go ahead and download that example that you'll find there and then I'll talk you through it right now now here on the first line we have an opening curly brace or bracket which is just the beginning of our file no surprise there and then we have this section right here called variables and what I did was I created a variable called Linode API token as you can see here with underscores in the name and when you setup a variable you can make it equal to something by putting something in the quotes but notice here that I don't have anything in the quotes now when we run this script it's going to fail because it really does need that Linode API token to be declared now it doesn't matter what you call this but I just figured that was a good name but we do need a token we'll fix that later and then here we have the end of the variable section so going down to the next section we actually have a declaration for our builders and actually the Builder type that we're going to use is Linode so essentially this builders section starts with a normal bracket than a curly brace and then we go ahead and we put in the various fields some of these required some of these not and of course we do need a type Linode in our example which is what I have here and then this is required right here this is not something you could just name whatever you want these are actual fields that are expected and it's called Linode underscore token you have to have that again and you can see right here I don't actually have our API token instead I have something in additional brackets so what this is is actually a variable we have user and then we have the name of the variable which we actually declared up here but when we declared it we declared it as empty more on that later but essentially right here I instead of actually putting the API key here I'm actually just gonna reference that variable because if we upload this file to get up we don't want to have this token in the clear we want to withhold that so we're just going to reference that here and then at the end of each of these lines of course we have this comma right here and then image this is required this is the next line here image we need to tell it what image we want I am using Debian 10 so we have Linode slash Debian 10 because this is a little image and the image we want is Debian 10 now just so you know you do have a limit of 2048 megabytes so if the image is actually more than that then you are not able to use it so for example Ubuntu 1804 that one is bigger than the limit so if you were to use Ubuntu 18.04 for the image right here it will fail which may not always be the case if they update that or they fix that or something like that that might not be the case but I'm using Debian 10 because it's smaller and then moving on we have a region right here and I'm going to give you guys a link in the description below for their API you could get the various values for things but this is basically where we want our image to be located geographically and the same with instance type we're setting up a nano instance right here there's different instance types of different prices and configurations for example this is just a small one now SSH underscore username is root so why we need SSH well the fact is when hacker is configuring our image and creating it for us it needs to SSH into that instance while it's running and configure it you'll see the process later when we actually start using this script now instance label since it is going to create an instance in order for it to actually create the image we want to create this is going to give it a name and timestamp is a pretty cool variable because you you know you don't have to declare this because it already knows what it means so if you want a timestamp on when it's created it's just good to have it's not required but basically the label is going to be my server - and then the now the image label is a little bit more important because the whole goal here is to create custom images that we can use to deploy more servers with and it's a good idea to have a descriptive label for that now obviously mine isn't very descriptive it's just gonna be my image and then a timestamp which is okay but maybe yours would be something like you know bata web server or a counting server whatever it might happen to be for you might have a more descriptive name right here scroll down a bit here [Music] and then finally we have the image description my first Packer image that's just what I decided to put there you can put whatever you'd like now here we are marking the end of the builders section that started right here and we should actually be good to go with just this so what I'm gonna do is save it and then exit out back here at the shell we are not quite ready to go yet because we still haven't declared our API key we referenced it but we didn't put anywhere for it to know what it's actually going to be equal to so what we want to do now is create another JSON file we're gonna do Nano let's just call it variables JSON easy enough and press Enter and here we are with another empty file and then I'm gonna paste in the contents here and then I'm going to explain what we're actually doing and here we go now in this case we have the variable that I created earlier that has to match that's a custom name I just called it Linode API token it's in quotes there's a colon here in a space and then it's going to have the entire API key right here now we can go here to this line it's just wrapped actually it's not wrapped it's just going off to the side we can see simply is just in quotes here now if you have other variables you're gonna want to put like you know vakama --there the last one doesn't need a comma there's only one so there is no comma so basically the only purpose of this file right now is to include that token now we could include other variables as well like for example where we have the image name or any of the other fields we can make those variables and we can put them in this file so I'm going to go ahead and do ctrl o + Enter to save and they control X to exit out we now have two files we have Linode example json and variables dot JSON and we actually should be ready to go let's go ahead and see if we can get our custom image built so to go ahead and start the process of building our image there's a very specific command which I'm going to paste in right now and there it is so it wrapped a little bit here but basically what we're doing is we're calling the Packer binary that's of course the one that we saved in our user local bin directory we want to build something and we have our API key and its own file variables JSON so the - var - file option equals right here allows us to basically basically tell it where to get that from and of course if your variables file is located somewhere else you're going to reference it from that location and then finally you are giving it the script or template that you would like it to run for you so I'm gonna go ahead and press ENTER and let's see if this works so far so good I'm gonna go ahead and let this run so let's take a quick look at what exactly happened here so again we executed the command Packer build we gave it a variables file and then we gave it a template file to use that we just created here and once it ran it basically is telling me that it's running the Builder is creating a temporary SSH key because it needs to login to the instance to configure it so it's creating it it's connecting to it and then it has to wait for a while before SSH becomes available once it's connected it then shuts it down creates the image and then it just tells us that it's finished now this image isn't really that valuable because I mean you could just go in the dashboard yourself and just using your mouse say you want a debian 10 instance and get it built and we didn't really do anything custom here that would differentiate ours from you know the standard w10 so what can we do to basically add a little spice to this now one thing we can do is actually run a script to basically have it do some things for us and I'm going to show you that process right now now first of all we need to create a script so what I'm gonna do is just do Nano script SH and again we have an empty file so I'll go ahead and paste in the contents here and here I have a very very simple script so the idea is basically that we want some customizations made to our image you know whatever those customizations might be we're just gonna put those customizations in a script and then when it's building the image it's gonna run everything here now in this case the sudo apt update is required because it's not going to have an up-to-date package index so any package installation will fail if you don't have this and then we'll of course make the entire script fail then on this line right here we are installing Apache 2 which is the name of the Apache package in Debian and Ubuntu and we're just doing a sudo act install - why patchy - if we don't have the - why option then it will fail because it's going to ask are you sure or something like that and the script isn't able to answer a question so it's just going to fail the - why assumes yes so I'm going to go ahead and save the file the ctrl o enter and ctrl X and now we have that file script SH along with the others so I'm going to create another template and show that to you that's going to utilize that script we just created so I'm going to do nano the node example to chase on another blank file I'll paste in the contents and here we have the template so basically it's the same as the previous one I've added a new section here we scroll down you'll see I have the provisioners section that I just added so again we were using only the builders section before which is where we were using to declare that we want to create an instance on the Linode platform and then all the fields associated with that that's the Builder section which is this here and it ends with a bracket we're creating a new section and this one's really simple all we're doing is in this provisioners section we are just creating a provisioner of type we're just creating a provisioner of type shell and we want it to execute script SH so whatever you call your script you want to make sure you have the right name here because if it goes looking for this script it's gonna assume it's in the current working directory and if it can't find it then it's gonna go and fail and we definitely don't want that so I'm gonna go ahead and save this file and then we'll go ahead and run it and here's the command to run our new template I'll press ENTER let's see if this works so far so good I'm gonna go ahead and let this run so as you can see here we were able to build a custom image it did work and if I scroll up a little bit you can see all the things that it did for us so let me go up here a little bit further so for the most part a lot of this is the same as before but we do have this right here that tells us it is using our script script SH to provision we're going to get some warnings this is basically some warnings that show up from within dubbing itself we're gonna ignore that because it doesn't actually apply to us but what its gonna do is show us the output from the script so right now it's running apt update updating the package index and it's also showing we have sixteen packages that can be upgraded so that could be another thing that you could add to your script is to automate that if you'd like some homework but anyway now it's gonna go ahead and run the install of Apache two which is basically what it's doing we see right here it's telling us those packages will be installed it didn't ask us for confirmation because we did - Y which we have to do anyway just went ahead and proceeded to install so that was downloading everything and we're not really concerned with these types of things right here because we don't have a TTY long story it still works fine just keep scrolling and then here we see of course the end of the apt output where it's installing those packages for us now that it installed those packages it's going to shut down the Linode instance then it's going to create the image and now it's telling us that the image has been created is it telling us the truth let's go ahead and take a look so here on my leno dashboard we now have two different images right here you can see the new one here at the bottom so now we have that and then I can go ahead and click on the three dots right here and I can deploy it to a new Linode let me go ahead and click on that and now we are going through the process of creating a Linode which is actually pretty much mostly the same as before but we're using a custom image we're going to choose that image right here region I'm just going to go ahead and use the first one right here that should be fine I'm gonna go ahead and create a now note here on the Nano tab because it's the cheapest way to go and for the label I'm going to just put lltv type in test so we know it's the right one root password we haven't put in that and of course I hope yours are stronger than mine but I'm gonna go ahead and delete this anyway so it really doesn't matter and then we can go ahead and create it and here we see the progress up at the top so I'm basically just waiting for that to finish and if this worked properly when I visit the IP address which you see right here I'm gonna go ahead and copy that in a new browser tab I should have the Apache default page since we did actually tell it to install Apache for us so I think enough time has passed let's see open a new tab here used in that IP address moment of truth let's see what happens we have the Apache to default page Apache 2 does not come default on Linode instances we were able to basically automate that installation now Apache 2 is not a very exciting example to be fair because it only takes 30 seconds to just run sudo apt install Apache 2 whenever you want to install that but it was just an example of a you know basically a proof of concept of what you can do here because you could put whatever you'd like in the script for example let's just say there's some files that you want to have present on the system that is you know something your company wants to be the case maybe some company specific customizations or any kind of thing that you often find yourself manually doing would be a good candidate for this particular script but the whole idea is you can use packer to create your own custom cloud image and Linode is just one example you can use it against AWS for example and you could customize that there's just all kinds of cool things that you can do with Packer it and you know as you were able to see it's really easy to get started you just give it a script you give it an API key and you're good to go so hopefully that was fun for you guys and go ahead and try to extend it maybe try to use it with a different platform or maybe put some different things in the automation script if you'd like but just play around with it and have some fun because that's how you learn so I'm again I hope that was helpful let me know in the comments below what you guys think and what you guys were able to build and I will see you in the neck video thanks for checking out my video I really appreciate it if you found it useful click that like button and if you haven't already done so make sure you subscribe so you'll see the latest content as soon as it becomes available if you want to help me out there's links down below for my patreon page as well as links for purchasing my Linux books and also my affiliate store which has a listing of Linux compatible hardware that I've actually tested personally thanks again for watching and I'll see you in the next video

Channel: LearnLinuxTV

Views: 2,390

Keywords: LearnLinux, Linux, Tutorial, Review, Howto, Guide, Distribution, Distro, Learn Linux, operating system, os, open-source, open source, gnu/linux, devops tutorial, hashicorp packer, linode, cloud, deploy, image, automation, automate, virtual machine, droplet, infrastructure, hashicorp, packer, hashicorp packer tutorial, devops tutorial for beginners, linux tutorial, linux commands, learn linux for beginners, devops, script, scripting, custom software, custom install, ubuntu, debian, cli, deploy server

Id: wpEu2iFu7oI

Length: 30min 2sec (1802 seconds)

Published: Tue Nov 26 2019