Getting Started with Ansible (Part 1)

hello everyone i'm jacob kauff and i'm the nerd of the stream and today i'm giving you an introduction to ansible [Music] okay everyone so ansible is an i.t automation platform that was first released in 2013 and was acquired by red hat in 2015. today ansible is a major part of red hat's products lineup but until just about a month ago i had never used ansible however as of this week i am a red hat certified engineer and a lot of the objectives on the exam for that certification have to do with ansible so i've gotten quite a bit of hands-on experience with it over the last few weeks as i was studying for that exam today i'd like to take a few minutes to explain what ansible is and show you some real world examples of how you can use it we're not going to be covering things as in-depth as you would need to pass the red hat certified engineer exam for instance but this video is intended for people who like me just sort of ignored ansible for a while maybe you know that it exists you've heard the name but you're not sure what it is or why people use it like i said ansible is an automation platform and at its core it's really just a way that you can control lots of machines at once so if you've got a fleet of a dozen servers or hundreds or even thousands of servers you can manage all of those servers from a single control node one of the key features of ansible is that it doesn't require any client-side software running on those fleets of servers that you're controlling you can actually control windows devices various cisco products lots of different types of equipment with ansible but when it comes to controlling linux servers ansible does all of its work over ssh and it uses common system tools to do what you tell it to do so the only requirement before you start using ansible is that your control node has ssh access to all of the machines you want to control i would say that everything you can do with ansible you could theoretically just do with the bash script but the advantage to using ansible is that you're using a program designed to apply a consistent configuration to a large group of servers it's designed to be item potent so you specify the state you want the servers to end up in and then you can run ansible with your set of instructions called plays against your servers as many times as you want and running those instructions more than once isn't going to mess anything up because the instructions aren't just specifying what ansible's going to do they're specifying the end result you want that server to end up in so if the server is already in that state nothing's going to be changed even though ansible is a red hat product it does work with other linux distributions as well so today what i'm going to do is set up five servers i'm going to have an ansible control node that's going to be running centos 8 and i'm going to have four servers that are managed by ansible two of those servers are going to be running centos 8 and 2 of them are going to be running debbie and 10 just to demonstrate that you can control many different servers with different operating systems on them using the same ansible control node like i said ansible instructions are written in plays the plays contain different tasks that ansible's going to perform and you can make a playbook that contains multiple different plays that ansible's going to run through and all of the configuration is stored in a format called yaml it's a little bit particular about usage of white space in your files but it's quite human readable especially once you've started working with it for a little while my intention with this video is really to demystify ansible for the new user and show you what it's capable of from a server administrator's perspective so without further ado let's cut to the desktop and get started alright guys and here we are on the desktop i'm going to go ahead and create our five servers first and then we'll talk about the setup we need to do before we can start using ansible so our first server is going to be a cintos 8 server i'm going to use the cheapest option available here on digitalocean and i'm using digitalocean for my example just because i've got a lot of credit built up on here but of course you can use ansible to manage physical servers virtual machines vps's any type of linux server i'm going to add my ssh key so i can log into the server right away without needing a root password generated and this server i'm going to call ansible controller so we'll create that while that's provisioning we'll create two of our manage nodes and these are going to be the two centos manage nodes so also running centos 8.3 they're also going to have my ssh key installed just so i can get in to do the initial setup and the host names will be sent os one and two we'll spin those up and finally we will create two debian servers to be our other two managed nodes those will be running debian 10 and these are going to be called debian 1 and 2. all right so it looks like our ansible controller has an ip address already now before i ssh into that i am just going to show you what we need to do before we can start using ansible there aren't very many requirements like i said our managed nodes will not need any software installed however we do need to set up some permissions so the ansible controller will be able to control all of our managed nodes so for one thing we're going to start by making an ansible user on every node both the controller and all of our managed servers now you don't need this user to be called ansible and you don't even necessarily need to have a dedicated ansible user you can use ansible as the root user and you can just run everything as root on all of your servers obviously that's not a great security practice and the advantage to having either an ansible user or an automation user or some sort of dedicated user for your ansible automation platform is that when you're looking at logs if your ansible user performs an action and that action is logged you'll be able to very easily identify okay that action was being done from the ansible controller node so if you want to know why it happened you can go back and look at what you have set up on the ansible controller node as opposed to if you were running ansible as root you wouldn't really know in a log file whether an action was performed by ansible or by a system administrator doing something manually either using sudo or while logged in as root now just creating this dedicated user isn't going to really secure things in and of itself there are identity and access management capabilities built into ansible tower and other red hat products that you can use along with ansible to control who's running tasks as this ansible user but we're not going to get into that today now on all of our manage nodes ansible needs to be able to do its job and a lot of those things are going to require root access so ansible needs to be a sudoer and it needs to be able to use sudo without entering in a password on every manage node your automation system won't be very automatic if you have to enter in a password for every single one of your servers every time you try and run an automated task the idea here is that you click one button or type one command and it's going to do a lot of things for you automatically and so to be able to do that the ansible program and the user that it's using needs to be able to perform root operations now ansible does not just support sudo it does support do as it supports regular old su and a few other options we're going to use sudo since it's a default option that's available on both red hat and debian based systems finally the ansible user once again it needs to be able to ssh from the control node to manage nodes without a password and ssh is how ansible is actually going to connect when you run tasks and then once we've got all of that set up we can install the ansible package on our controller node and then we can finally get started using ansible a lot of this stuff that's done on all the managed nodes i'm about to do it manually really if you were doing this at scale you would want this to be built into your imaging process or something like that but i'm going to open up a terminal here and i'm going to ssh into our ansible controller node and i am logged in here so we are going to start by adding our ansible user user add dash m ansible the dash m is just going to create a home directory for us and now i'm going to log in as that ansible user and to do this at the terminal here i'm going to just use s u dash ansible that's going to log me in as ansible and if i say who am i you can see i'm logged in as the ansible user if i print my working directory i'm in the ansible user's home directory and i'm going to go ahead and generate an ssh public-private key pair with ssh-keygen and i'm not going to change any settings i'm going to leave this at the default location and most importantly i'm going to not enter a passphrase if you do set up a passphrase you can set ansible up to have that saved that way if somebody steals your key from the ansible controller node they won't just be able to use it to connect all your servers that's an option that's available but just for simplicity we're creating it without a password for now so we've got our ansible user on the controller node and i'm going to go ahead and cat out our dot ssh slash id rsa.pub this is our public key that needs to be installed on all of our manage nodes so i'll open up a new terminal and in this one i'm going to ssh into our first centos server so ssh root at that centos server and we'll just get connected here so once again i need to add a user user ad dash m ansible now in order to give that ansible user sudo privileges you might think that we just need to add it to the wheel group however that's not what we're going to do today and let me show you why if i run user mod a capital g wheel ansible that's going to add the ansible user to the wheel group as a supplementary group the capital g means we're not reassigning our primary group we're just adding this additional one to the ansible user and now if i log in as ansible if i do sue dash ansible i'm logged in and if i run sudo test i'm going to get this prompt here this is asking me for my password now there are a couple problems with this for one thing we have not set a password for the ansible user so i don't have a password to enter for another thing like i said before we don't want our ansible administrator to have to sit there entering in passwords manually for every single server that ansible is going and touching when we're running a playbook now the reason sudo is set up to ask for a password separately from just logging in in the first place is so that if you log into a server and then you walk away from the server while you're still logged in somebody else who sits down a few minutes later won't be able to just run any old administrative command with your user however in this case this is an automation user we're just going to go ahead and say that nobody's logging into our endpoints with this user manually at least after the setup's done it would be very hard for them to do so since we're not setting a password and password authentication isn't even enabled on the server now it is possible to configure ansible to provide these passwords manually but that's just outside the scope of this video so we want to be able to run sudo commands without this password prompt so i'll hit ctrl c to get out of this password prompt i will exit back out to my root user and we're going to run vsudu to edit our sutures file now i'm going to type in slash wheel to search for wheel and this is going to bring us down to the line that allows the wheel group to run all commands with a password prompt by default now if i were to comment this line out and then uncomment this line down here this lower line would allow any user in the wheel group to run any command as root without entering a password that's not actually what i want to do maybe there are other sudoers on this system and i want sudo to behave as expected for those users i want them to be prompted for a password it's just our ansible user that i want to exclude from this so i'm going to come down to this line i'm going to hit y y to copy the line and then p to paste it in and then i'll hit i to edit this line here and we are going to replace the percent sign wheel which is signifying the wheel group with just ansible and if we specify a word without a percent sign at the beginning this is specifying the actual user so we're saying the ansible user in particular is allowed to run on any system any command that it wants to without entering a password so i'll hit escape colon write quit enter and now if i log back in as ansible i can run sudo test and i'm not prompted for a password which is exactly what we want the last thing i need to do on this node here is allow ssh access using this ssh key from our controller so i'm going to copy the public key over here if i do an ls here on our directory we can see we don't currently have a dot ssh file over here now if you wanted to you could set a password for your ansible user since we're logged in as ansible we could just run password then i'd have to go and enable password-based authentication on this server since it's disabled by default from digitalocean's imaging process then we could come back over to our controller node and we can run ssh copy id and then type in the ip address of our manage node over here that would prompt us for the password and then once it logs in with the password it would automatically copy this public key over to the manage node that sounds like a lot of work because it is we don't need to do that we're not going to need password authentication once we set up key based authentication so i'm just going to do it manually and it will actually be quicker at least if i hadn't explained it i'm going to run makedirectory.ssh and then i'm going to v into dot ssh authorized underscore keys and in here i'm going to go into insert mode i'll paste this i've got a video about using vim it's actually about text editing on linux in general that you can go and check out if you're not familiar with vim or v but i've saved that public key in my authorized keys file and now if i come back over to my controller i can ssh into ansible at 104.248.8.122 and i should not be asked for a password when i do this um it's asking me to verify since this is the first time i've connected that this fingerprint is correct i'll say yes and i've got permission denied okay let's take a look at our permissions because i'm guessing this is a problem with permissions if we take a look at our ssh directory this would be seven seven five which is not what we want um and this would be six six four which is also not what we want so sshd is ignoring this file right now so when we're doing this manually and we're creating our directory and our authorized keys file manually we need to remember to set our permissions and to do that i'll run chmod 700 on our dot ssh directory this is going to set it so that our user the owner of this directory can read write and execute but nobody else can go into that directory nobody else can traverse it and nobody else can read or write to or from it so i'll run that we'll run chmod 600 on our ssh authorized keys file and this is the same thing except we don't need our user to be able to execute authorized keys this is not a script it's just a text file so 600 is going to give the user read and write permissions and everybody else once again should not be able to look in here and see who's allowed to connect as this user i also have a video about linux permissions that you can go and check out if you want to learn more about how those numbers work but now if i come back over here and try that again i am able to log in to our manage node from the controller and i was not prompted for a password so that's set up exactly how we wanted to set it up i can run sudo test again and i'm not asked for a password so that's perfect now one last piece of this puzzle i do want to be able to ssh in using a host name rather than the ip address so that's going to be centos 1 in this case you can see that's not a fully qualified domain name and i don't have any dns records for this ip address so that's not going to work by default so in order to do that my ansible user for demonstration purposes i didn't add it to the studios group over on this controller because we're not going to be managing our controller with ansible you can if you want to i'm just not in this example but i am going to exit back out to my root user so that i can edit my etsy hosts file and in here there's a scary comment at the top that says this file is not persistent that will not be here on a default installation of centos this is an artifact of digitalocean's imaging process and it is incorrect this file is persistent uh despite the comments so we are going to add in our ip address followed by the host name of our manage node i'll write and quit that so now i can ping sintos1 and you can see that it's resolving to that ip address if i try an ssh in from my root user it's not going to work if i ssh into syntos one it's going to fail because we don't have any ssh key from our root user configured on the manage node but if i log back in to the ansible user now i can ssh into synth os 1 by name it's going to ask me to confirm the fingerprint again since we're connecting using a hostname for the first time but it is going to let me in if i clear out of that i can do it one more time you can see i'm logged in this is the controller node i'm going to ssh into centos one and it's just going to let me write in to centos one so i need to do that on all of these servers i'm going to cut out my ssh id rsa.pub one more time i will go ahead and get our setup document out of the way here and i'm going to grab the ip address for our second centos node and i might just speed up the video here while i perform those exact same steps on the second manage node running centos all right unless i forgot anything i should now be able to ssh into centos 2 and after confirming the fingerprint i can get in there and i can run sudo test i'm not prompted for a password perfect so now we just need to do it two more times for our two debian servers we'll exit back out over here now in our debian system you can see here that we don't have as many examples in this file the syntax will still be the same we just don't have the example there so i'll add in ansible all equals all no password all now you might notice that we are currently not in nano hitting ctrl x doesn't really do anything to get us out of this document we're also not in v hitting escape doesn't do anything and the colon does not get me any sort of command prompt fun fact this v sudo command the top of the sudoers file says this file must be edited with the v sudo command that's actually a lie you can open up etsy sudoers in nano or vim or any editor that you want to as long as you're doing it with root privileges the reason we use the v sudo command is because if you make a typo or if you mess up your syntax v sudo will warn you it won't let you save invalid configuration and so that prevents a system administrator from locking themselves out of the system by destroying their ability to use sudo commands but i do often just edit this with nano directly instead of going through vsudu just because i never really take the time to set up my editor environment variable and so it just throws you into whatever editor is set up by default we're currently in an editor called joe and it's a recursive acronym for joe's own editor i've never been in this editor before i don't think i've ever used it once but to get out of here we're going to hit ctrl k and then control q and it's going to ask us if we want to save this modification to the file we'll hit y and it will save that and exit out so now that we've done that i can go ahead and log in as our ansible user it looks like we're in sh here instead of bash so another way that debian is different from centos is when you create a new user without specifying what shell you want it's going to give them the sh shell by default instead of bash that's not a problem for us if we do print working directory you can see we are in home ansible if we do an ls alh we've got we've got bash files here we just aren't logged into bash um i am going to just make a directory.ssh and we'll do the same thing we did before nano dot ssh authorized keys copy and paste that key into here chmod 700 on dot ssh chmod 600 on dot ssh authorized keys we don't have tab completion in sh which is annoying but i think we've done everything we need to do if we come back over here let's add this to our etsy hosts file 134 209 38.107. that is debian 1 and so we'll log in as ansible we will ssh into debian one and we are logged in we've got an sh prompt and we can do sudo test and we're not prompted for a password nice i am just going to go ahead and set our shell that was not intentional having it default to sh i don't think it would mess anything up but to change a user's shell we can run user mod dash s for shell bin bash ansible and now if we ssh in there again we should get bash which we do which is good we're going to be using the shell module with ansible later that's the only reason why i want to make sure we have the same shell on all these machines just so we have consistent output so now we will set up our final server here final managed node which is going to be debian2 so we're going to run user add dash m ansible i'm going to specify dash s for the shell we'll make that bin bash so we don't have to worry about that again this time when we run v sudo i'm going to append editor equals nano to the front of the command that will set our editor environment variable to nano for this particular command so that studio puts us into nano instead of that weird joe editor that it used before we'll enter in ansible all equals all no password all will save that login as ansible authorized keys all right and we will log in as ansible on our controller again ssh into debian2 and we'll run sudo test and we're not prompted for a password all right i am currently 43 minutes into our recording for this ansible video and we are now ready almost ready to start using ansible uh we do of course need to install ansible there is an ansible command and if we just run ansible you can see it's not found because we haven't installed it yet now ironically enough ansible is in the default repositories for debian so if your controller node is a debian machine you can run sudo apt-install ansible however here on centos ansible is not in the default repositories it's not in the default repositories for red hat either if you're running a red hat system you can get ansible through subscription manager since we're here on centos and i do not have a subscription to red hat enterprise linux we are going to get ansible from the apple repository i'll open my web browser back up this is a page on the fedora wiki about apple it stands for extra packages for enterprise linux and there's a command right here that we can run on centos 8 that is going to download a package and installed it that package is going to add the apple repository to our system and i need to do that with sudo which for my demonstration purposes here my ansible user on the controller node does not have re-privileges so i'll just run it here logged in as root okay and we will run dnf install dash y i'm going to install ansible i'm also going to install nano at this point on our controller here cintos does not have nano installed by default and that's the text editor that i'm most comfortable with so that's what i'd like to use while i'm building these ansible configuration files with you guys so i'll run this and you can see it's pulling an ansible it's also pulling in some python dependencies and spell modules are written in python so if you ever find yourself needing to write a custom ansible module which is very unlikely but if you ever want to do that it's done in python that's how you would do it the reason i say it's unlikely is because there are a ton of ansible modules out there already that can do most of the things that you could think of automating and whatever you can't do with existing modules you can probably do with the shell command without having to write an ansible module just for what you're trying to do but in any case that is finished running so now that that's been installed i can access my ansible user again and as the ansible user i can just run the ansible command and we're going to get a little bit of usage information we're not going to go through all of this i am going to bring your attention to the default ansible configuration files which are located at etsy ansible if we take a look inside of there you can see we've got etsy ansible ansible cfg this is the default ansible configuration file and then we've got our default ansible inventory file here at etsy ansible hosts now an ansible inventory file is telling ansible what hosts are available to perform actions against it's not necessarily saying what you're going to perform each action against you specify that with the actions but this inventory file is going to be a registry of hosts that are available to be managed with ansible we've also got a roles directory and roles are a more modular way to specify tasks that you're performing as opposed to a regular old ansible playbook which is what we're working with in this video i'll probably make a follow-up video about roles but for now let's take a look inside of this default ansible configuration file just to see what's in there now remember our ansible user on our controller node here does not have root privileges so we won't be able to edit this file but we can take a look inside of it because all these commented values are the default values so the default inventory file like i said is etsy ansible hosts down here you can see the default sudo user that ansible will attempt to escalate privileges to on our manage nodes is root and by default it will ask us for the password for sudo and just for ssh into the machine if we don't have password lists set up for those things of course we did it properly so we're not going to be prompted for a password when we run ansible against our managed nodes down here we've got an option for when we're going to gather facts now by default whenever you perform an action against an ansible node the first time that ansible touches that node in your playbook it's going to gather a bunch of information about the system and that information is going to be stored in variables that you can then reference later on in your playbook so by default ansible is going to gather facts by default but you can turn that behavior off if you want your playbook to run a little bit quicker and you don't need that information to be gathered if we keep scrolling down here we can see our default path for ansible roles can also be changed here we've got an option for remote user that's pretty important so you can see it says user bin ansible will use the current user as default so if you just use the ansible command to run a one-off task it's going to use the same user that you're currently logged in as it's going to try and log in with that username on your remote servers however when you actually run an ansible playbook which uses the ansible playbook command it's going to use this remote user that's configured unless you set a different user in your playbook that means it's going to try an ssh in with this user which isn't going to work since we set up our ssh key using the ansible user on our remote nodes and not using the root user we've got other things that we can configure here for instance where we're logging to a lot farther down here we've got a section for privilege escalation like i mentioned earlier this is where you can change the method that it uses to escalate privileges if you're not using sudo on your systems and you can really read through here there are way too many options to go through in a video one by one however we need to change some of these values and we don't have right access to that file because it's owned by root we're logged in as ansible so not root and other people do not have write access to this ansible.cfg file so how are we going to configure ansible if we can't touch that file well we're actually going to do it just in our current directory here if i do an ls there are no regular files in my current directory when you run ansible it will actually check your current directory for an ansible.cfg file and if it finds one it will use that file instead of the global file so we're going to copy etsyansible ansible.cfg we're going to copy it right here to our home directory for the ansible user and we're going to nano into it so that we can edit a couple of these settings for one thing our inventory file is another thing we're going to want to edit so for that we're going to use home ansible inventory and the other main option we're going to want to edit is remote underscore user like i pointed out before we're going to change that to ansible since that's the name of the user that is configured with the proper privileges on all of our remote nodes next we need to create our inventory file if we take a look at our default etsy ansible hosts file that is going to be in any format dot ini format which is a configuration format you may be accustomed to if you're used to using windows and it uses configuration like this you put group names inside of square brackets and then underneath those group names you list out your servers you can list either host names or ip addresses as you can see here you can list ranges and you can also within these hosts files specify variables for your hosts there are no examples of that here but for instance if we wanted to specify variables for the web servers group we would specify web servers colon vars and then we put our variables underneath that now any files can get a little bit messy to read especially when you've got a hierarchy of groups that are inside of other groups and variables inside of your groups because everything is going to be right at the start of a line while i'm glad that my terminal is helpfully telling me what color blue is when i hover over the color names that's an interesting new feature in console when you're configuring using the dot ini format all of your hosts are going to be at the beginning of the line so there's no indentation and it's a little bit difficult to read the other way that you can specify your hosts is using a yaml format that's what we're going to do so i'm going to nano into inventory and i could have made this called inventory.yml or inventory.yaml i just called it inventory to keep it simple uh but we're going to start with all and then a colon that's going to say this is our all group and this group is not optional all of your servers need to be in the all group in ansible if i wanted to at this point i could specify my four nodes and i would do that like this i would say cent os one centos two debian one debian two and in fact if i just exit out of that file right now let's check that things are working so far um so we've got our inventory file and our ansible.cfg those are in our current working directory so if what i said about ansible using the current directories ansible.cfg is true when we run ansible it should pick up on this local configuration file which is telling us that our inventory is this inventory file and this command that i'm about to run should work i'm hoping it works so if we run ansible we're going to specify which servers we want to run this task against so i'll say all and this is the reason why you need that all group is because ansible expects you to be able to specify the all group to use all of your servers we're going to specify a module that we're running uh with dash m and the module is going to be ping and this module does not require any arguments if it did we would specify dash a and put the arguments inside of quotes but this is a very simple module the paying module is just going to go out ping all of our servers and it will tell us whether or not we're able to reach all of the servers specified in our inventory so let's try running this all right and that didn't work and the reason is because i forgot to put these hosts inside of the hosts label inside of our inventory file you know this is the fourth time i'm recording this video today i'm over an hour into the recording i really don't know if this is going to turn out a useful video at all but this is what your yaml inventory file should look like so the yaml format for inventory files is a little bit more complicated once again we wouldn't have had that problem if we had used the any format but we're going to get into more complicated stuff later on in this video and that's why i'm using the yamo format because it will be easier when we're doing the complicated stuff um so we're going to put those inside of hosts let's try this one more time and would you look at that okay as you can see i ran ansible all dash m ping i did not have to specify which configuration file we were using or which inventory file we were using but ansible knew it read that configuration file that's in our current directory which points to the inventory file um and now that we've got that inventory file written properly we've got our four servers and it went out and pinged all of them you can see the response pong from all those pings now you can see here we're getting some warnings for our debian hosts and i've seen some other warnings for cintos hosts before it's warning us that our python interpreter is being automatically discovered which basically means that if somebody installed a different version of python on the debian systems that could break ansible if it's a version of python that ansible doesn't work with so to fix that warning we're just going to come into our ansible.cfg file we're going to specify an option here under the default section and this option is not present anywhere else the option is interpreter underscore python i'll copy that if we search for it you'll see it's nowhere to be found we're going to set that to auto underscore silent and that is going to disable that warning so now if we run ansible all dash m ping again we don't get the warning and this is what ansible output typically looks like for each of our servers we're going to have whether we succeeded or failed in the operation then we're going to have a list of ansible facts that were discovered we'll have whether or not anything was changed on the server so in this case we're just pinging that's not going to change anything on the manage nodes so that's false for all of these and then we get the output from the module that we ran for an example of something else that you can run easily against all of your hosts we'll run the setup module this is going to print out setup information and this is going to show you all of the ansible facts that are actually being discovered so by default once again unless you disable fact gathering all of these ansible facts are being collected by default every time you connect to a server and a lot of it is hardware information you see the amount of ram the disk space we've got our networking configuration farther up but some of this stuff that's hardware specific especially say these servers had spinning hard drives those hard drives need to spin up before you can see how much free space they have or what their total size is at least it might take significant time to gather these facts and so that's one reason you might want to disable fact gathering by default in your ansible configuration we're going to leave it enabled though because we're going to use some of these variables later but we now have our ansible inventory file set up so let's actually start writing a playbook like i said this ansible command this is going to allow you to run a single module and there are many many many ansible modules available if you go to the ansible documentation you can scroll through them all you can see there there are modules for cisco devices modules for checkpoint uh networking devices there are azure modules near the top since it's in alphabetical order f5 devices dell emc devices vmware i saw windows in there somewhere there are just a ton of different modules that are in here and so no matter what you're trying to automate um it would behoove you to go and search and see if there are modules available for it once again if there aren't you can probably hack something together with just the command module and the shell module or you can develop your own modules if you want to so what are we going to do here with our ansible setup now that we've got this configured what kind of stuff can this do for us besides pinging all of our servers and telling us if they're online or not we're going to start with something fairly simple we're going to start by gathering log files so let's say that you're running servers and you want your log files to be periodically pulled into your control node so that you can maybe back them up centrally put them into some sort of archiving system for logs there are better ways to do this with snmp but let's say that you want to do it at the file system level you want to create a file that's going to have journalctl for the current boot which is going to include a lot of useful information if i just do a journal ctl dash b0 and i run that through cat here i'm logged in as ansible so that's not going to come up with anything if i do it here as my root user you can see journalctl that includes a lot of information that also shows up in d message and that information from d message is going to be prefaced with kernel uh but journalctl also includes logging from all of our other services on the system you can see we've got logging from systemd itself when it's starting service is up then later on we've got logging from network manager that's a service that was running we had cloud in it when the server was being provisioned we've got sshd when we were sshinging in so a lot of information that can be useful to have for security reasons or just for troubleshooting reasons let's say that we want to have that pulled in from all of our servers periodically and we want to be able to do it with a single command pull in journalctl from all of our servers let's take a look at how we would do that we're going to do it with something called a playbook and ansible like i said it kind of goes like this the way that you execute tasks in ansible is with modules but the module itself it's just a tool that you can use um typically it's going to be part of a task and so the module i'm going to indent it here to signify that the module is part of a task the task is going to be part of a play and a play can contain multiple tasks a task can only typically contain one module so i'll go ahead and put another task here just to make that more clear and then the play is inside of something called a playbook which is just a text file that is defining all of our plays and so this is the ultimate hierarchy that you're going to end up with you've got a playbook file that playbook is going to contain your plays one or more of them and then each play can contain one or more tasks each task will contain a single module the reason why it's broken up into a hierarchy like this instead of just being able to specify modules directly in your playbook without going through all the play and the task these layers is because these different levels of indentation in this list here are different places where you can figure different things for instance each play plays are where you specify which hosts you want to run something against that's one of the things you can configure at the play level a task is where you can configure other conditionals such as i only want to run this task if this host that i'm currently operating on is running xyz operating system if it's running debian or you can say i want to run this task and i want to loop through it and do it several times with different pieces of input information i want to run the same module multiple times using a list you can do that with a single task so we're going to get into more complicated examples i think the really the best way to understand ansible is just to do it so we're going to create a playbook here this playbook is going to be called logs.yml playbooks are also specified in the yaml format uh you do not have the option of specifying them in any other format unlike the inventory file where you can use any if you want to instead of email playbooks have to be on yaml we're going to come into here typically it's good form to start a yaml file with three dashes it's not technically required if we're not putting anything on top of the three dashes you can put basically environment variable type stuff at the top but we're not going to get into that so at the top of our file we're going to start off by defining a play and this play is going to have a name the name is going to be collect logs so this is a play to collect logs from all of our servers we're going to specify which hosts we want to perform this against we're going to perform it against all hosts we also want to specify what user we're going to do this as and in particular whether or not we need to become the root user to perform this action and that's just going to use an option called become and we're going to say yes now ansible uses shorthand since this is such a common option to use because so many of the server administration tasks you're going to be automating will need to run with root access so they don't call it become root user they just say become yes if you see this line it typically means it's being done as the root user the only time that would be different is if your ansible.cfg or configuration elsewhere is specifying a different user that become will operate with but for us it's going to be root and then down here we specify these are going to be the tasks inside of this play now notice that i had a single bullet point or a dash at the beginning starting off this play because a playbook is a list of plays now the toughest thing when i was learning to write these yaml documents these playbooks the toughest thing for me personally was getting a handle on the formatting when do i have indentations when do i have bullet points and when do i not and really after you've written a dozen playbooks or so it starts to become second nature but it helps to recognize what exactly you're listing out anytime you've got a dash which i think of as a bullet point you're making a list and so our playbook is a list of plays all of this information is part of the same play it all needs to be indented the same so that the yaml parser knows that it's part of this list item but it is all a single list item it's all a single play that's why we only have a single bullet point at this level of indentation now down here we're starting a new list for tasks because you can have multiple tasks in a play so this first task is going to be called we'll give it a name we're going to call it copy current boots logs to file now you might be wondering what this name actually does it's going to be printed out to the terminal while ansible is doing this operation it's also helpful to have in here in your playbook so that when you're editing the file you know what it's supposed to do you can have comments in yaml for instance i could have just said collect logs with a hash mark at the beginning here and that's not going to be parsed by yaml at all so that would work for editing but once again if we specify a name parameter ansible's typically going to print that out while it's performing the operation which can be helpful now part of this current task that we're editing is going to be running a module this first module we're going to use is called shell and this module is going to run a command in a shell environment there's also a module you can use called commands it's sort of better practice to use command if you know that you don't need to use the shell module shell is going to give you your environment variables and things like that whereas the command module won't in this case we actually do want to use the shell module because we're using shell redirection which is a feature of the shell that you wouldn't get with the command itself so the command that we're going to run in a shell is journalctl-b0 just like i showed you before that's going to collect all of the logs for the current boot and we are going to redirect that to timp journal ctl so the temp directory we can usually edit that without being root or we can create new files in it without being root uh the reason we need to become the root user for this play is because journalctl in order to view those logs we need to be a privileged user so that right there is a task it's extremely simple we're saying i want to run this command on every single server in our inventory we're going to specify another task now and by the way once again there was no bullet point before the module because it was part of the same task that we gave a name to and you can't have more than one module that's part of the same task anyway so there's no point in bullet pointing those the next task is going to have a name compressed log file and for this one we're going to use a module called archive now archive is a module that takes more parameters than just a single parameter so it's going to have parameters specified another indentation level in so another two spaces this is actually the more standard way to do things most modules will look like this where you're specifying multiple parameters the shell module is one of the few that only takes a single parameter and so it's easier to just put it on the same line for that one so the archive module is going to take some parameters how do you know what those parameters are before i type them in here obviously when you're using ansible yourself you won't have me doing everything for you so let's say that you know you want to archive something but you want to know what the parameters are to do that well ansible comes with a program called ansible doc and that's ansible dash doc ansible dash doc is going to give you documentation on your modules so if we run ansible dash dock archive it's going to give you basically a man page just for this module gives you a description of what it does it gives you all of the options that it can take and then toward the bottom it gives you some examples for common functions below that it can even give you return values these are things that you can then store in variables and use in later plays in your playbook and later tasks in your play now the other place that you can get that information is at docs.ansible.com if you have a web browser you're on a computer with a gui and it's easier for you you can just come in here and search archive and even if you're not sure what the archive module is called just searching what you want to do archive a file the first result here is the page for the archive module so this will give you the exact same information it gives you a synopsis of what it does all the parameters you can pass in examples and return values the exact same stuff now personally what i use even more than the regular ansible dot command is ansible doc dash s and you can put the dash s at the end if you want to it can go at the end here instead of in the middle but ansible dock dash s is going to give you the short version and it's just going to give you the options that you can pass in and it's also going to give you the options that you can pass in in the format that you would need to use in your playbook you can see we've got an example name here it's in a little fake bullet point and then just like we did in our playbook we've got the archive module specified at the same indentation with no bullet points and then further indented they use four spaces where we're going to use two but further indented we've got all the different options that we can pass in and it notes at the beginning of each description which options are required this is extremely useful when you don't want to have to search through a ton of information you know what you're trying to do you just need to know how to spell a certain parameter or you know if it's dst or d-e-s-t for the destination for example if you pipe that through cat it gets rid of the paginator and then you can just scroll through you can see okay this is the est for a destination it's a nice quick way to find out what your different options for your modules are so we're going to nano back into logs.yml and continue here and so i'm going to give the archive module the path to the file that we want to compress or archive and that's going to be temp journalctl so the file that we just created up here in our previous task next we're going to give it the destination of where we want to save this archive to that's going to be temp journalctl.tar.bz2 we're going to use bzip tube compression for the best possible compression slightly better than gzip we're going to tell it the format that we want to use that's going to be bz2 and we need to specify that as a parameter because the archive module it doesn't know you know we could call this file whatever we wanted we could just call it journalctl.tar and it could be formatted as bz2 the file name is not a reliable way to determine the actual format so that's why that's a separate option down here next we are going to specify whether or not to remove the original file it defaults to no we're going to say yes or we're actually going to say true personally i like to say true or false for a lot of these options the ansible documentation typically says yes or no the yaml parser in ansible knows how to handle it either way i tend to go with true or false just because it seems more correct more what i'm used to from programming languages the last option we're going to pass into the archive module is force archive and that's going to be set to true and what this does if you don't set this it defaults to false and by default if you archive only a single file it's going to archive the file with bzip2 but it's not going to put it into atar archive it's it's just compressing it it's not actually archiving it we're trying to create a tarball i don't care that it's only one file i want the tar metadata so that i can extract it with tar and not have to remember whatever weird bzip2 specific command i would need to use to extract it so we're going to use force archive true to make that happen and finally we will specify our third and final task for this play that task is going to be to copy the log archive to the controller right now this is going to happen on every single node that we're managing it's going to happen individually on each node so all four of our servers will end up with a file at temp journalctl.tar.ez2 that's great but of course we would need to go and access each server individually to actually read it we want to copy them all to our current server so to do that we are going to use the fetch module now there's a copy module that only works from the controller to the manage nodes fetch is the module you use to go from the manage nodes to your controller so two separate modules for the two directions the source is going to be timp journalctl.tar.bz2 which is now the only file left over from this operation because archive is going to remove the original tamp journal ctl file the destination this is going to be the destination on the ansible controller here that destination is going to be home ansible logs and i'm going to make a logs directory in our ansible home directory in just a moment and it's going to be journalctl dash and we need to give each of these different files a different file name there's going to be four files because we have four servers in our inventory that we're running this against if we just specify a single name like journalctl.tar.bz2 it's going to do this it's going to copy from our first remote server to journalctl.tar.ez2 then it's going to copy from the second remote server and it's going to overwrite the first one it'll overwrite with a third file and then it'll go to the fourth server and overwrite that file and so we're going to end up with only one file from one of our four servers if we try to name them all the same thing the way we're going to get around that is with an ansible variable and this is a default variable so we don't need to define it anywhere to use a variable in our playbook i'm going to use two curly braces a space i'm going to close the curly braces i'll come inside of here we're going to keep a space in between the curly braces and the name of the variable which isn't really required but it's the proper format to write this in the variable we're going to use is ansible underscore hostname and that's what it's going to look like this is what the line is going to look like so ansible underscore hostname is one of those ansible facts and you might have noticed that i put this line in quotation marks the reason i did that is because we're using a variable here and the rules for this are i don't actually need quotation marks for this i could get away with not having these quotes and that would work in this case however if you have an option let's say that we had dest let's say that we had a variable that contained the full path of the file that we want to save to if i tried to do this log dest path if i had a variable called logdestpath that i was defining somewhere else you can do that in your playbook here i'll show you how to do that if i tried to define this here we would get an error and the reason is because if a variable starts a line that's supposed to be a string that variable has to be enclosed in quotation marks if there is no variable in your line you don't need quotation marks at all you don't even need them if your variable is not starting the line however i don't like to go with the bare minimum formatting required i like to give myself a little bit more safety than that and so i've just gotten in the habit personally of if i'm using any sort of variable anywhere in the line i include quotes that way if i go and change this line later and this variable ends up at the beginning i won't have any problems if you wanted to if you wanted to be super strict with your formatting you could totally put quotes around every string that you're passing in anywhere and that would be perfectly fine personally like i said this is just what i've landed on is if you're starting with a variable quotes are required if you don't have a variable quotes aren't required so if i'm using a variable anywhere i just say okay that's the line that i'll draw in the sand and i'll say i need quotes or i want to include quotes just for consistency's sake anytime that i have variables the last option we're going to specify for our fetch module here is flat true and by default if you don't specify this ansible would create inside of your destination it would create a directory with the name of the host then it would stick the file inside of that directory so what i said before about how if you don't specify a hostname for your destination for the file it would just get overwritten that was a little bit of a lie because flat defaults to false and so if you ran it then you would actually get four directories with a journalctl inside of each of them but since we're saying flat is true i don't want all those directories cluttering up my system i just want inside of this home ansible logs directory i'm specifying this is where i want the files to go this is what i want it to be called flat true is going to basically say the destination name is the destination name for the file not a directory in which to put the file alright so we're going to save this and let's see if that worked ansible dash playbook so this is how we're going to run an ansible playbook the command instead of ansible it's ansible dash playbook we don't need to specify our hosts unlike with the regular ansible command for running one-off tasks you need to specify hosts there uh we don't need to do that when we're running an ansible playbook because we specify which host to run against in the playbook we just need to specify which playbook we want to run and in this case oh before i do though let's go ahead and make a directory logs so if we do an ls we can see we've got our logs directory there otherwise we probably would have gotten an error when the fetch module was not able to put that file into a non-existent directory uh but we'll go ahead and run ansible dash playbook dot slash logs dot yml take a deep breath we'll run this okay and we got through it we did not see any errors which is good let's take a look at what we saw there so as you can see we had a play at the top here that said collect logs and the first thing that we did as part of that play is we said here let me cat out my logs dot yml um so inside of this play we said hosts all and so the first thing we're going to do is gather facts about any hosts that we're touching in this play now it's important to know that the facts are not gathered at the beginning of your playbook they are gathered at the beginning of the play where the server is being interacted with the reason it's important is because there are variables that are defined when these facts are gathered and i have had situations where due to the order that my plays were written in i tried to access a variable and it didn't exist yet because i didn't touch that server until later on in the playbook the implications of that might not be entirely clear right now to you if you don't understand what i'm saying at least i wanted to mention it explicitly because i've found it to be important so we've got our play we're gathering facts and all of these just say okay because nothing was changed when we gathered those facts now down here you can see when we copy current boots logs to file in that task all four of our servers were changed when we did that and you can see by the way that these are in different orders for these different tasks by default ansible is going to go one task at a time but it's going to connect to up to five servers at a time by default you can change that using the forks option in ansible.cfg or by specifying that as a variable but yeah we've got four managed nodes where it's going to operate on up to five at a time so these are going to be in different orders because it's just going to report as they come in and respond to the commands so we copied our current boots logs to file obviously you know none of these servers had that file before so it was created successfully all four of the servers we compressed that log file and that went okay once again no failures and then finally we copied the log archive to our controller node and it says changed on all of these and so we had three things that were changed total three tasks where each of these servers was changed and we had four successful tasks because that first implicit task is to gather facts if anything goes wrong you're typically going to see the failed counter increment unless you specified ignore errors in which case it will say ignored or if you specified a block with a rescue block which is basically like a try-catch in programming where you say try to do this but if it doesn't work just move on but do this instead if you do that it will increment the rescued counter unreachable is when your server's offline and skipped is when you operate using a conditional and a server doesn't meet that conditional but in this case everything was okay so if we take a look inside of our logs directory you can see we now have a log file for each one of those servers we've got journal ctl centos one two debian one and two if i do tar x v j f on logs journal ctl sent to us one for instance that is going to put a journal ctl file in our current directory here which we saw since i specified the verbose option for tar if i cut that out you can see the last thing that was reported in journalctl was ansible connecting to the server to run that playbook if we go back up to the top though you can see that this goes all the way back to our current boot and in fact let me remove that journal ctl and that metar xv f so if i take a look at the top of that cintos machine's log file you can see the kernel we were running was a red hat kernel red hat 8.3.1 because we're running centos and it's running kernel 4.18.0-240 so if we come back down to the bottom here and we run tar xv jf logs journalctl debian1 and i'm just going to pipe that through head because i only want to see the top of it oh actually i'm getting ahead of myself i extracted that file i'm going to cap that file out and run that through head which will show us the first 10 lines of output you can see the kernel for this one says debian and so we know that these are actually different files i'm not pulling any smoke and mirrors stuff here it went to all four of our servers and gathered those those logs so our debian log is going to have a debian kernel 4.19.0-10. so that is the very first playbook that we have run this video is an hour and 47 minutes long already it's going to be shorter after i edit it down there are two more playbooks that i want to show you but they get progressively more complicated and i think it would be a good idea to go ahead and split the video here i did not intend for this video to get so long but i think it's good we're getting into detail that setup took up a lot of time i know so stay tuned for my next ansible video um if you think it's a little dry and simple so far like i said we're going to get into some more complicated stuff if you have any questions about ansible feel free to ask in the comments i am going to go ahead and finish my second and third playbook that i planned to make but i can make more ansible videos after that if anybody has questions that aren't covered in those videos if this video was helpful or interesting to you you can go to nerdclub.notts.com and join the nerd club for just three dollars a month to help me make more tech videos like this one for now though i'm jacob cough and i'm the nerd in the street and i'll see you guys in ansible part two see ya [Music] you

Channel: Nerd on the Street

Views: 6,112

Keywords: nerdonthestreet, jacob, kauffmann, jacobgkau, Ansible, how to, getting started, for beginners, beginner, tutorial, guide, walkthrough, playbook, plays, tasks, modules, archive module, shell module, inventory file, YAML, formatting, fetch module, setup module, ping module, write a playbook, write playbooks, setup, beginners

Id: 0CRiWGV57WE

Length: 59min 23sec (3563 seconds)

Published: Mon May 17 2021