Ansible 101 - Episode 2 - Ad-hoc tasks and Inventory

let's see good morning I need to make sure that the stream is actually working I believe it is but if you can see me or hear me please say hi and chat really quick also might as well say say where you're from last week we had a lot of people from all over the world and it was it was inspiring to see everybody kind of coming together and learning together so it was it was definitely a fun and exciting stream for me especially after the Drupal live stream which is a little more humdrum and not as exciting and interesting because I'm fighting with Drupal's migration system that's a lot fewer people are on that so it's not not as fun to see where everybody's from it is fun but anyway it's exciting to see everybody that Cape Town huh very nice Colorado up in the air about a mile high Albany so looks like people from all over the place as as I promised last week I'm wearing my Netherlands shirt today this is one of my another Lynne's shirts the Goering family hails from the southern part of the Netherlands and been over there one time in my life I almost had a chance to go there again this year but of course coronavirus Cove at 19 came into existence and that squashed all hope so maybe I'll get there in here too we'll see but I'm glad to be here with you today and so far knock on wood haven't had too many ill effects due to coronavirus though it's kind of sad I'm hearing a couple more stories of people being furloughed or experiencing layoffs and that's part of the reason why I'm doing this this particular livestream series and why I have the books free and originally I announced that I was going to make my books free until the end of March last week the CTO of device 42 actually contacted me and said hey Jeff I like what you're doing I want to make it another month how can we do that and I said I'm I'm open to any suggestions and they basically are sponsoring me for this month to keep giving away my books for free which is awesome so if you know anybody that would benefit from ansible for DevOps which is this book or ansible for kubernetes please send them to my my website Jeff Garlin calm it has a link to those books and tell them about this video streaming series and let them watch it as a reminder all these episodes will be recorded and available afterwards as well but I also told the vice 42 that I would I would give them a plug because that's it's extremely generous what they're doing and I'm very grateful I've written so many times about the importance of supporting open-source development because the tools we use ansible and kubernetes and drupal i use a lot all these different tools that i use so many of them are open-source based on community development work and most of the people that build these tools you know some people are sponsored by a company or they work at a company full time and get to work on these things but that's a very small percentage of the people that contribute to open source and just like me most people that do open-source development have families we have full-time jobs we have a lot of things to do and the open-source work usually doesn't contribute back much to our bottom line so I told device 42 that I would give them a plug and this is it ansible is great for driving on IT automation but to make the automation work you need to make sure that you have an accurate real-time picture of all your IT infrastructure and that's where device 42 comes in they provide comprehensive discovery of the entire IT estate from mainframes to kubernetes and just like ansible and this is the reason I like them more than a lot of other systems like theirs is that it's agentless now you can try it for free download a trial at device 42 comm and see how I can take your ansible automation to the next level so thank you again to device 42 and if you if you are if you're benefiting from this and getting the free books and things please reach out and give them a shout out on Twitter there at device 42 so thank you to them and last week in this up in Episode one we started off in the beginning of the ends well for DevOps DevOps book in the preface and introduction introducing why ansible exists what it's for and we started in chapter 1 and 2 doing very basic things describing to ansible an inventory of a server that we had running India in Amazon ec2 and then we had how we had a very very basic playbook that installed NTP and got it running it looks like there's a lot of let's hope you don't support the Netherlands football team I actually do and the few times that it's ever happened that the Netherlands played the USA I was always cheering for the Netherlands don't tell the US men's team that I did that but yeah I'm very happy and someone saying my patreon link is a 504 I'll try to fix that later and and again this is both being recorded and a live stream so if you have a question in here I try to go back through and look at the chat and answer anything that I didn't answer in the video sometimes a stream sometimes the chat kind of gets away from me and I can't check it during the stream so don't worry I usually see anything that you're talking about but I'm going to in this episode get more into ad-hoc commands and between last video and today I actually looked it up so this is the word and hoc ad-hoc and it sounds like an American English it's pronounced ad hoc and in British English is France and hawk so I I'm an American so I'll say ad hoc but if I say ad hoc you know don't kill me so anyway these commands are a way for ansible to quickly do a specific task or run a specific module on any servers in your ansible inventory and I put in here the title conducting an orchestra because that's kind of a lot of things in ansible you're kind of you're kind of being the conductor of the orchestra of all your servers and the cool thing is you can operate on one server you can operate on 10 servers you can operate on hundreds of servers and we're gonna explore how ansible treats all those different scenarios in this lesson and Oliver is also mentioning with patreon and all that it might be loading ok you might be having other connection issues but yes you can sponsor me on github which is very helpful I have something like 20 people who are sponsoring me there's you know sponsorship arranges for any kind of amount somebody says New York English is an hack yeah that's an in Boston but I can't hock something like that I actually have relatives in all these different places so it would be interesting to have them all read this and see what they say but there's a lot of different things you can do with these kind of commands and usually it's it's more useful to use a playbook which we'll get into in the next chapter but I often use ad hoc commands to check on resources or to get log files or things like that just when I need to debug a situation there's a lot of times you know when you have a running server that it's doing something that you don't expect or if you need the emergency do a command on them restart a service on all your servers or something like that these can be helpful for that but it's I've seen people do things like applying patches and updates with the UH more apt checking resource you should checking log files like I said managing users and groups you could you could have a situation where you need to quickly add or remove access for a particular user on your systems you can use these for those managing host files and DNS settings quickly copying a file to her from a server deploying an application this is something where it starts getting in the realm if he probably should be using playbooks for this stuff rebooting servers that's something I do a lot if I if there's an if there's a security update and it's really hot like I remember for heartbleed that was the first one where it was like I need to get all my servers upgraded immediately I just went in and and ran the command to run up upgrades and reboot immediately and that was extremely helpful for me because I didn't have to write a playbook and test the playbook and then do all that and managing cron jobs something as mundane as that so most of these tasks like I said it's it's better to run them in a playbook if you can but but sometimes I sometimes it's it's more important just to run the command and get the data or results right away and also if you have a monitoring monitoring system in place usually it's better to just use that monitoring system to see things like resource usage or log patterns and things like that but there there are times when the monitoring system has problems and you still got to go into the server's individually and using ansible for it it's a lot easier so the first thing that is useful for this to demonstrate how it works is again I'm going to rely on vagrant and VirtualBox you can use vagrant with other systems too but it's nice to be able to use vagrant to build a server to test on vagrant can also manage multiple servers locally or in the cloud or anything through the vagrant file so I'm going to show you how to do that today we're going to set up and I don't actually have the graphics on my computer with me here but I'll show you in the picture here if I can get it get it aligned oops I need to move my mouse so I can see so we're gonna set up a system like this where there's one database server and two application servers and we're gonna do that using a vagrant file so I'm gonna go in here and open up let me let me to change my windows here on my other screen so I can actually see see what I'm doing and make sure I'm not messing things up or putting somewhere something somewhere you can't see it so I'm gonna go into my downloads folder and I'm gonna make a directory for this project called ad hoc and go into it and then I'm going to create a vagrant file so again that the to do that you can just say vagrant in it and then the box that I'm going to use his Deerling guy sent to a7 and someone I think asked where can I find boxes to use and that is you can go to vagrant I think it's vagrant cloud I maintain a number of boxes but vagrant cloud has a listing of all the boxes that are available I have boxes for bun 220 sent to s 8 sent to a 7wn 10 I usually I tend to work with sent to us Debian Ubuntu and sometimes Fedora for my projects in vagrant I maintain those boxes I also maintain images and docker that have ansible installed on them and so you can use those as well but let me go back here and vagrant cloud you can search through it and find boxes for almost any platform in the world mark says long live vagrant yeah it's it's kind of you know it's been around for a decade or so just like it's it's a little older than ansible in fact and it's still extremely useful a lot of people have moved on to other systems and tools but in the end we're managing servers and that's what vagrant does too so I like to use it along with ansible for a lot of my testing and exploration work so I've created this this vagrant file so I'm going to open I'm going to open this folder ups su BL is a shortcut for sublime and in on the Mac you can give it a directory and it will pass that directory into that command and sublime lets me open up this project in sublime so here's the vagrant file that was just created I'm gonna delete a lot of this extra junk that I don't need all these comments and we're gonna set up the configuration for this so that it will work a little bit more easily for our testing purposes one thing I'm going to do is I'm going to say config dot SSH dot insert key equals false usually what vagrant will do is it will insert a created SSH key into the virtual machines for a little bit of security for my testing purposes a lot of times I I ignore that and just use the insecure key that comes with vagrant just because they're gonna be local machines that nobody's going to access outside of my my own computer anyway and then I'm also going to turn off the synced folder vagrant by default synced folder vagrant by default has a it sets up a synced folder that's the current working directory into the folder vagrant inside the VM and sometimes that can cause issues and you know if you don't need I wouldn't enable it so I often do this and my my test plate looks as well and then I'm going to set up some VirtualBox options since I'm gonna have multiple machines I don't want them to have a ton of memory because then my computer which is already straining from doing this live stream as you can see from the CPU gauge up here it's gonna run out of memory and be even worse so I'm gonna set config that VM that provider VirtualBox so this is setting setting up things for VirtualBox I do V V that memory equals 256 megabytes and V that linked clone equals true this is this linked clone thing is kind of a micro optimization it lets you build machines a little more quickly instead of creating a separate virtual machine from scratch it can create one and then it can kind of link a clone of that so it's a little faster to start them up if you don't do that it's not the end of the world it's just it's a little bit slower so now we've we've told vagrant about all the things that we want our machines to do now we need to define three different servers so I'm going to say app server one and I'm going to define that with config dot VM that define that divine define app one I'll call it do app one oops I can't type today there we go I have one and I'm gonna give it a hostname and give it an IP address so I'm gonna say app actually I'm going to just call it happens to have app one app dot VM that host name equals work for orchestration app app one that tests and then I'm gonna give it a network an IP address so I'm gonna say app that VM that network private network I can't type again IP is 192.168.0.1 networking that vagrant sets up with VirtualBox so I have app server one and I'm gonna copy this out and make two more servers one is app server 2 and 1 will be the database so I'll say episode - yep - for cap - and you might also wonder why I use the test domain I used to use dot local back a long time ago but there were some issues with that sometimes Mac OS versions would use the dot local root domain - to do some special things and that could screw things up some applications and VPNs and things would use that so I stopped using that and I started using dot dev well a few years ago Google actually bought the rights to that dev and so that dev started having issues in certain scenarios so when I used that test because that test was officially set up to be like a local testing environment root domain so most of my examples nowadays use dot test in them because that's it's just easier to work with that and you're not gonna run into weird DNS issues like you could in other cases so I have f1 f2 and this is going to have the IP address dot five vagrant can actually automatically set IP addresses but I'm doing this because I want instable to interact with them a little more easily so I'm gonna say DB so this will be a database server DB server and this will be DP and that's six all right so I have these I have this vagrant file set up it's going to set up VMS using the sent to s7 box I turned off the synced folder that comes with vagrant by default set the VMS to use 256 Meg's of RAM just to save my computer from dying because it's about ready to die and I'm setting up three three different VMs app one app - and DB and I'm gonna run a vagrant up and that's going to start these three vm's so it takes a few minutes especially if you've never downloaded the the sent to a seven box yet it has to download that and it looks like I have an error app what do I have here I'm 30 Oh dB because I called it D be right here so do that let's run vagrant up again and it's gonna just like last week it's gonna download the box if you don't have it and then it will start up three VMs in this case last week we just started up one but this week we're gonna set up three later on in the book there's a few examples that set up like five or six different things and yeah and some Aaron just mentioned that the hard thing with these streams is that because my computer's a bit slower well there's a couple reasons for it OBS wants to the software I used to stream these things once as much CPU and GPU as possible I have a MacBook Pro 13 inch and it's a 2016 and it doesn't have the fastest of everything I even bought the non-touch bar version because I hate the touch bar and and so OBS kind of takes up all that that CPU and stuff and it adds on maybe five to ten seconds of delay in the stream and then YouTube it's little thing that runs in the background it's also using up some of the some of the CPU and it seems to cause another fifteen to twenty seconds of delay so everything I say it's 30 to 45 seconds before someone on the stream sees it and then someone might react to it and leave a chat message and then I'm sitting here and you know and in this case Aaron found the issue after I fixed it but I hadn't really fixed it yet in real time so good job Aaron you get a thumbs up or something yeah actually Oliver just gave you thumbs up so good job while it's bringing up these machines I'll talk a little bit more about how we're going to connect to them with with the example from earlier we had that I think it was just one server and we called it what was it that was installing ansible we did a lot in the last video in a short amount of time but in the last one we just set up one server and we let we let vagrant connect to it using vagrants generated inventory which you can do you can run we could have an ansible playbook that runs in here and vagrant will intelligently create a temporary in a file and pass that into ansible through the background but we're not going to do that in this case we're going to create our own inventory describing all these servers so I'm going to create a new file and call it inventory and even saving a file is slow right now inventory and note I've never tried doing this during a live stream so I don't even know if the live stream is gonna continue working throughout all this but I'm gonna create an inventory file and any line in the inventory file that begins with a hash or pound is a comment line so I'm gonna comment that these are going to be the application servers and we'll call them app that'll be the group again inventory groups are in the square brackets and last last week I know some people were discussing inventory file formats typically I start out with this format which is the ini style inventory format it's the oldest format that an Sable's had and the basic style is you have a group in brackets and then you have servers listed so like 192 168 that's 64 and the other one is dot 5 and then you have database server and that's BB so the group is going to be DB and it's dot six so this is like the the ini style you can also do yeah Mille and there's other ways as well that you can set up inventory Fran scible a couple of those ways we'll get into in later chapters I don't want to we don't want to get too bogged down with inventory right now the main thing that I'm trying to show is how you can just tell ansible hey I have these different servers and they're in different groups so I can work with in different ways so the next thing we're going to do though is setting up two groups like this lets us operate on app servers or DB servers but what if we also want to be able to operate on all the servers like let's say heartbleed happens and we need to upgrade all the servers to the latest version as soon as possible and thanks however that's crowd-sourced me a new macbook pro that would be pretty awesome I've I've been considering that so the new MacBook Air just came out and it's faster than this current MacBook Pro CPU at least but I have been thinking about getting a 15 inch at some point just because the it has a dedicated GPU that might work a little bit better with OBS and it's just faster for everything video editing all that kind of stuff but anyway this I can't complain too much this computer's been great the worst part was when the battery started expanding because it this was the recalled MacBook Pro one morning I came to my computer and I grabbed it and the entire keyboard was kind of bowed about like this and the spacebar was like this and the screen was also starting to bend a little bit and I found out after the fact I had brought it in the Apple and they replaced it it was like five days out of service after the fact there was a recall a few weeks later on the MacBook Pro for expanding batteries and that's always fun they said it can expand so much that it explodes luckily mine did not do that but we want to be able to operate on all these servers and there's there's not a way in the ansible ad hoc command to say like operate on these two groups that's that's easy and not clumsy so I can actually make another group I'm gonna call it multi and in this Intex I can use a colon and say children and that's a special thing that tells ansible that this group is going to be a group of groups instead of just just a group of servers I'm gonna say this group has all the servers and I'm gonna say app and DB so that when you use children like this you can give it different groups and it will put those groups together and then also to connect to the servers if you have if you're using just the username that's your normal logged in user so - like Jake yearling I think it's Jake yearling and you have your SSH key and your SSH key chain or if you have have it in your default location that's great but for this case I'm going to I'm going to define the SSH user that ansible is going to use as well as the SSH key that it's going to use because it's the ansible can't auto discover them and there the key is not in my a my SSH keychain right now so I'm going to do that by giving the variables to this multi group and to do that I'm gonna say variables for all the server's and I'm gonna put them into multi VARs and the first one is ansible ssh user and i'm going to set that to vagrant so this variable tells ansible what user to use and similarly there's ansible ssh private key file and vagrant by default creates as I said earlier I use this option to turn off the insert key thing if you have this on then then vagrants going to create a random key in a place on your system you need to have to find where that is using ansible SSH or vagrants SSH config command but I'm just going to use the default key that comes with vagrant and that's in home folder that vagrant that the slash insecure private key so this is my inventory file and I can use this with ansible I can use it with ad-hoc commands and I can use it with playbooks if I want to for now I'm just gonna use it for ad-hoc commands because that's what's in Chapter three and we're going to start working on these three VMs and if I open up VirtualBox you'll see that vagrant created these three different running machines and configured the networking and all that kind of stuff for me so here's the three ad-hoc app one app two and dB so I'll get back out of VirtualBox and I'll move on to running some of these commands so you know as I mentioned earlier in the book I have a lot of tips tips are things with a little key and info is things little I up here and warnings are things that are important to remember to protect yourself for security purposes that kind of thing and I have a tip here that that says a lot of things that we're about to do you know you might be like well I monitor my servers with Nagios or with prometheus or whatever tool that you use mewn and Perik all these different systems you should be monitoring things that way that this is more to illustrate what ansible ad hoc commands can do and and also another thing that I think is interesting is a lot of organizations a lot of groups don't monitor their servers externally too so sometimes you might you might have issues with your monitoring system internally especially if it's on a private network or inside your V PC and Amazon but there's actually problems in the outside world accessing your servers so I always make sure that if I'm running an application that's accessible over the Internet I have something outside of my own tools monitoring it so I use Pingdom a lot i run a service called server check-in it's called server check pn there there's other things too that are they're either free or low costs that can just monitor a server make sure it's up or make sure that webpage loads and has certain content so I always like to to mention that to people because some people don't think about that and they have these great monitoring tools in place but it doesn't help if your site's down to the outside world so the first thing we're gonna do here it looks like Oh Devon is saying that ansible 2.0 is deprecated the ansible the ssh part so yeah yeah I I forgot to update the book here and that I'm basically going through the book at version 1.2 - by the time that you're watching this video if you're not watching this live this book might be at version 1.4 or 1.8 or whatever it'll be so some of these things will change over time and one of those things that has changed is these variables can be ansible user instead of ansible ssh user so I'm gonna leave this for now just because it's following the book at version 1 to 2 but if you do find any issues like Devon has found here please feel free to go to the the books repository and mention them and if it's during the video you can just mention which episode it's in if you if you see what page it's on in the book please feel free to pop the page in there I usually try to get all these little bug fixes and buy the next book version and I do that every couple months so again a reminder the books are free right now go go grab them free while you can for lene pub and every time I update the book you'll get those updates for you forever so let's go ahead and start using this inventory that we just created the first thing I'm going to do is I'm gonna say ansible multi so again explaining these parts of this this is ansible is used for running these ad hoc commands multi is the group name or the server name and I'm gonna have to I'm gonna pass in the inventory file so I'm gonna say - I inventory and the argument positions don't matter here I could say ansible - I inventory and then multi and then I'm gonna say - a for the argument again going on last week's episode if you if you don't provide a module that's - M for a module ansible defaults to the command module and so you can just pass arguments straight to the command module which is a command to run so I'm gonna run hostname and we'll see what it gives us I'm also going to grab a quick drink of water here okay so it gave us the host names for these app one app - and DB and this is also a point where some people might start getting errors you might see error messages about not accepting the host key or you might see like no host match that kind of thing usually that indicates either you have a problem in the inventory file or if you've never logged into these servers before you do need to make sure that you accept the host key or you set an option to ignore the host key or like you can do what I do in my SSH config file which is in your home folder SSH folder and it's config you can tell it to ignore a host key for a particular server or you can you can accept the host key when you setup the server all these different things these are just general ssh issues that sometimes can surface when you're managing lots of servers with ansible so another thing that you might notice so that this first one we did we got kind of lucky it goes one and then two and then DB just like we define them in the end taury right here but if I keep running this I'm gonna run it again and see what happens of course it's a lot slower with the CPU being maxed out if I run it again you'll notice that the second time the database actually returned first and then 1 & 2 if I keep running it again and again every time the order is going to be a little different and the reason for that is that ansible by default runs in a parallel parallel nature it by default uses 5 forks so it Forks itself five times to be able to run the command as quickly as possible on a group of servers and you can actually change that behavior to see how ansible can manage the servers in a different way using the dash F flag for forks and so if I say dash F 1 instead of sending out the command to five servers at a time and getting back the results five servers at a time it's going to send it to one server wait for that server to respond and then it's going to send it to the next one wait for that to respond and so on so in this case you can see it it went in the right order according to our inventory file it went to the app 1 app 2 and test because we're just using one fork this is useful because with a lot of people you might have you might have a hundred servers or a thousand servers that you're managing and if you want to do it do something on them really fast let's say it's emergency time and you have heartbleed or something else that you can't quickly get a playbook up and run it for it although you know the end goal of all this is that you would be able to get a playbook up quickly get in into your CI system run it against your hosts because play books are a little more auditable but if you didn't need to do this really quick you can set Forks to a hundred and assuming that your control system has enough memory and CPU available to fork the Python processes that ansible uses that many times to be able to run that command on that many servers at the same time you can run commands on a lot of servers in a very little amount of time and again this is all just using SSH you don't have to have anything installed on those servers to be able to do all this stuff so that's that's how ansible runs tasks in parallel by default and and again just to just to also prove that you can you can put these arguments anywhere else in the ordering here you can pass the inventory first and say multi and that's going to do the exact same thing is up here and you can even put the the group on the end of the thing it doesn't really matter what position these arguments are in and sometimes I try to always do the inventory first in the group and then the module and arguments but sometimes I kind of mess that up please don't don't hate me for it you know I'll try my best to do it consistently but don't be confused if I if I go in different orders so some other things that I've done in the past just to see some information really quick is sometimes I use like D F dot H or - H to see how much space is available I am reminded of last time I was managing an elasticsearch cluster elasticsearch for anybody who hasn't ever run it before can very quickly eat up your disk space especially if you have a kubernetes cluster with hundreds or thousands of pods running all of them dumping all of their logs into your elasticsearch cluster and we hadn't set up the the elasticsearch curator job correctly so the disks kept running out so it was important to be able to see which servers the disk space was was being used up like and it and the other thing is that elasticsearch kind of uses up so much memory or so much this space but then it's not it kills the process and makes the index stale if it gets up to like 80% by default so we were having that issue and I was quickly able to see which servers we had to quickly remediate in our search cluster because of that so DF dead H is helpful command free free - H gives you the available memory and these are a lot of different commands you can run to kind of get a quick idea of of how your systems are doing outside of your other monitoring tools another thing that I've had to check up on a lot is the date on systems so I am mentioned earlier the service server check-in that I use for server monitoring and that's one system where the actual timestamps of everything so it's a distributed system where there's some servers that do the checks around the globe running in different hosting providers so that I have some redundancy in case like Amazon Goes Down or Google cloud goes down that kind of thing and early on in the process I was having a lot of weird issues that I couldn't really figure out what was going on and this also happens with databases and search systems and all this kind of stuff it's important to always have the same date on all of your different servers and make sure that the timezone is the same some of the hosting providers I was using for server checkin would change the the timezone on my servers for some reason even though I had set it manually and forced it to UTC they would change it to the local time of that server and that was throwing off my scripts now the scripts should be probably a little more foolproof of course and always use UTC by default anyways but this this threw them off and that that alerted me to the problem I need to fix the timezone handling of my of my application so you know date is another command you might want to do might want to use when you're managing your servers another thing that's important to note is that anytime I run any command ansible is always a reporting changed and that's because ansible can't know when you run just a random command like date or free - H it's not actually changing our system but ansible can't know that because you're not using one of in Sable's modules to do this stuff when you run arbitrary commands ansible is always going to say you know something changed even if something might not have changed it'll find ways that we can use an tools modules and see you if it changed or didn't change in just a minute there's there's another command that you can use it's the setup module that can give you back all of the information that ansible can see about the server every once in while I do this just that I can see see some of the data that ansible can see about the server and that you can do I'm just going to do it on one server so DB - M setup and that's going to return pretty much everything ansible discovers automatically about the server of course it's taken forever and so you can see it gives you like the IP address the Python version on the server it gives you information about the the processors this is useful sometimes when you're actually building a playbook and you need to know something about the server that you're going to use to key on in the PlayBook or to template a file so it's helpful to know that the setup module if you do - M setup gives you back all the information ansible can figure out about the server let's see Darko mentions can I post my instable recipes for managing elasticsearch and that answer is a definite yes there's I already have a few rolls on ansible galaxy galaxy Don ansible com4 elasticsearch and later in the book we're actually going to do an example building an elastic search elastic search logs - and Cabana using file B to send search there's to send the log data back to elastic search so we'll get to that a little bit later in the book and that's an example I'm pretty excited about because it's one of the one of those things where there's a lot of complex systems at play and ansible does a great job of orchestrating putting them all together using play books so we'll get to that soon don't worry so another thing that I'm going to show now instead of just the commands which it's you'll also note that when I ran this this command here the setup module it's all in green text because it says success it knows that it didn't change anything because when you use the setup module it's just gathering information it's not necessarily going to change something so let me go back down and Aaron s is is the setup module the same thing as gather facts yes and in fact there are some examples that I'll get to later we're in a playbook you might turn off fact gathering but later on in the playbook you might need those facts so you can actually use the setup module to get those facts in a playbook even if you turn off gather facts we'll get to that later so the next thing I'm going to do is I'm going to run a command to install a package on these systems using ansible yum module and to do that I'm gonna say multi do this on all the servers and I'm gonna use - B - B again is become that means become a different user and by default that users root using sudo so when I install packages I need to be sudo and I need to run them with sudo because otherwise my user who's a lower-level user is not going to have the privileges to be able to install packages it's going to say - B to become the sudo root user and I'm gonna use the yum module last week somebody asked why use the yum module and there's also the package module there's actually a bunch of different packaging modules there's yum there's a pacman there's apt there's package which kind of isn't the umbrella over all them there's DNF all these different package modules work with different packaging systems but in the general case if you know that the package is the same on whatever systems you're managing you could use the package module which is common alias - Yaman have done all these other modules but I'm I like being explicit if I'm only gonna work on sent to US servers I use yum if I'm only going to work on Fedora I might use DNF if I'm only going to work on a bun - I might use apt if I need something that works on all of them I'll definitely use package if I can swing it that way but anyway I'm going to use the yum module here and I'm gonna give it the argument a name equals NTP state equals present and that's going to tell it to make sure that the NTP package is installed via yum on all these servers and I don't think it is by default so when this comes back hopefully it does come back someday looks like the CPU is now 100% on the two cores so we'll see if this is this survives once it comes back it should report that there's a change on each of the server's since it installed a new package let me glance at the chat here and see if there's anything else someone says why macbook yeah if I didn't do video and photo work I probably would be using Linux full-time I've tried a couple times the problem is that like what is it light table and so many other tools that are available on Linux for media work are just not not even close to as productive as I can be on the Mac with Lightroom and Photoshop and Final Cut Pro and all these different tools that that are only available either on Mac or Windows so and Windows is kind of out of the question because it's just not fun to work an open-source development for all the Linux tooling that I use and Windows even with when the subsystem for Linux - anyway yeah Aaron Aaron mentions Windows 10 plus subsystem for Linux is pretty nice it is pretty nice but the problem is that you know if I'm going that far it's it's a big change from doing all the tools that I'm used to on my Mac for example like transmit there's no FTP client I found on Windows and you might be laughing like FTP who still uses that I work with a lot of organizations a lot of nonprofits especially that still have little old fashioned FTP type servers and I still I use it for SFTP - but I still have to work with those things and transmit is just this like there's a lot of little gems like transmit on the Mac that don't exist on Windows and believe me I've tried a couple times there are some good apps on Windows for sure but you know is switching platforms from Mac to Windows is not as easy as some people would think anyway we're still waiting for this package to install I wonder if it's going to install in the next 15 minutes or we're just gonna keep talking until the end of this this live stream yeah and someone also mentions for Windows that vagrant vagrant in vagrant interacting with VirtualBox through the windows subsystem for Linux can be challenging wsl - which is still not in a publicly release it's in the Insider Channel right now which I have access to and I've been testing it wsl - is a little better for sure but it's still not as seamless as you know working in on a linux machine or on on Mac OS for me there's other good suggestions in the comments on some different Windows apps and things good rebuttals against whatever I say for sure but that's not gonna convince me right now anyway so it looks like it finished on one server it's still going on the other - this is definitely a lot slower so I guess it's a good time to plug if you do want to sponsor me on github or if if you work for Apple and you have access to MacBook Pro inventory go ahead and ship me one that'll be awesome or hack it I'd take a Mac one of the new Mac Pro's you know the $54,000 model that I think Marcus Brownlee or whoever had gotten the the high-end highest model that would probably do a little bit better job than this MacBook Pro so it looks like it finished and like I said it says it's changed because it it's it performed a change on each of the systems and it gives us the output from YUM with the installation something that I haven't talked about and I might later on in the series is the output here that we're getting you can see there's a bunch of newline characters and then the output is kind of messy this is the default output of ansible and the reason that it's done this way is this is this is like easily easily formatted universally on all the different systems and all the different clients there's actually different ways to get output from ansible and you run these commands and one of them that I like is the Y Amal the mo output format and I have a blog post on Jeff Gatling that come about that if you want to search for Y Amal ansible output that makes this a little cleaner and the new lines are formatted nicely so that you can read this output but for most purposes it's not that important and you can set that in an ansible configuration file which I'll get into later so if I run this command again on all these different things and thank you Erin for sponsoring me it's extremely helpful if I run it again it should it should be a lot quicker because it's not actually installing anything we'll see if that's the case or not but it should come back and report that there were no changes yes so it's saying success and this is the message it got back from yum when it tried when it checked on NTP and so again it's it's a quick way using ad hoc commands to see if something's in they're not to install things and another note that I have in the book is you can actually use the full name for these things so - become is that become parameter and that says as I as I mentioned earlier it's it becomes this the root user using sudo by default and you can actually set a different method so if you have a different system that doesn't use sudo to elevate privileges you can actually use different become methods to become a user that has higher privileges in different ways but again I'm working on Linux here so it's pretty much always sudo also if you don't if you don't actually have an SSH key in your commands you can specify - Kay and that's a shortcut for - - ask become pass and that will let you act sorry I'm confusing myself here so if you're sudo if using sudo requires a password so in vagrant the vagrant user always can become sudo password lists and the settings for that and are in the slash Etsy slash sudoers file if you if you actually need to enter a password to become the root user because you're not using password the sudo you can you can set this command flag which is ask become pass or dash k and what ansible will do is it should ask for a password and this is the basically if you're gonna run a sudo command what password do enter for that in my case I don't need that but if I did I could put in I think it's vagrant and it would still work using vagrant because vagrant is the password for the vagrant user and but it's not necessary with these vagrant boxes because the way that vagrant has you set them up so another thing that you might do after you install a service is make sure that it is actually running because a lot of times you know if you're gonna install something you might not actually want to use it so I'm gonna in this case use the service module the service module intelligently determines whether the underlying system for managing services system D or an it V or whatever else it is and most things are automatically detected really easily and the service name for NTP is ntpd friend daemon and I'm gonna want to make sure that it's actually running so I'm gonna say state equals started and enabled equals yes to make sure that when I reboot it's also going to start up when it reboots so I'm gonna do that and I'm also going to show you so if you're interested in how do you find out what all these parameters are there's a couple ways to do that one way is this is the way I usually do it as I say instable service module and trusty Google usually gets you to the documentation for it which is nicely formatted easy to see easy to read and if you ever find a problem on the documentation you can click on this little link and get to the page for the the module and and make the changes and somewhat contribute back to ansible but it gives you all the options that are available here and we just use name state these are the different choices for state and enabled yes or no and so it's oh I I don't want to use the become password anymore because that just trips me up so I'm going to do that and it'll probably take a few seconds and that's the service module you can also say I believe it's interval doc service I could be wrong here but the ansible doc command will give you back documentation for any module that you have and here you can see the service module control control services on remote hosts this is basically the same thing that you're seeing on this page but it's formatted a little bit differently for the command line so you can see options if it has an equal sign it's mandatory so you have to give it a name and the the the formatting here is a little bit a little bit I I generally like reading it on the web interface just cuz it's a little easier I can also link to things if I'm documenting what I'm doing that kind of thing but either way it works and if you're on a computer that's disconnected from the Internet this this will still work the ansible dot command whereas the Internet of course will not be working this is fun Interactive authentication required well that's annoying I thought it didn't need that but maybe it does let's do dash K and say vagrant oh it's because I didn't it's because I didn't use B so I need to take out this K and this doobie because I need to use the pseudo user and I'm sure that somebody's gonna comment while I'm talking right now on the fact that I forgot to use dash B and that's just the way it is when you have this delay in the live stream so now it's saying that the services were enabled so it changed it and again if I run the command again due to an Sable's item potent meaning if you run the command once or a million times it's going to result in the same end state it should say that there's no changes this time and the text should be green highlighting the fact that it didn't change anything yep and so there's a lot of other things that we could do another thing that you could do it with ntp in particular as you can say - a service and TP stop that's gonna whoop s-- that's gonna stop the service if you want to manually force an update of the of the NTP time and date you can do this you can say service that's the NTP date is the command to control the time - queue and we'll give it a server so your that red Red Hat Enterprise Linux that pooled that NTP org so that's going to force an update on these servers and if that ever finishes up when I when I tested these commands a week or two ago when I was preparing for this they ran in about 3 to 5 seconds so obviously the live stream kind of makes for a lot of a lot of delay here so you can see the offsets not that far off but sometimes your servers can drift by a second or two and that can start causing issues with databases and and clustered systems so after you do that you'd want to make sure that you start the service again and DPD start and again you this when I run the command it's always going to report changed and it's probably better if you in and simply even warns about this it's better to use ansible zone modules for these things so ansible warns you hey you're using the service command we have a module that does this item potently it's easier to use it's easier to parameterize so for things like this it's better to use that kind of command than to then to run the command like this fan symbol has a module for it you should probably use the module and similar things for like curl for downloading things ansible has modules for downloading files from the internet and instable has modules for interacting with web services so use those so there's there's also in this chapter a longer example that i'm not going to go all the way into for setting up a django application on these two app servers and the basic setup is you could use ad hoc commands to install my sequel Python so you have bindings so that ansible controller my sequel server it installs django using easy install or you can use pip to install it as well using ad hoc commands ansible has modules for all these things so you could use the easy install module or the pip install module to install python dependencies and you can even run you can run Python through an Sable's command module to test that Django is working all the stuff is in the book in chapter 3 I'm looking at page 29 of the 1.22 version paperback when you manage there's seven different versions of the book so it's it's sometimes hard to say exactly where something is but all these different things that I'm doing are a little bit better served in a playbook and I'm not going to spend all the time especially since it takes like five minutes per install to do them all but another thing that that it's illustrating is the fact that we've been working on all the servers but I can just work on the Divi server by using the Divi Group and I can say free - um and it's just gonna it's just gonna give me the the memory statistics for that DB server and similarly if I wanted to manage applications or services or things I can use any of the models just on the DB server or I can do it just on the app servers and it would just run it against the - app servers that if that are defined and I I won't install my sequel and everything but on the DB server assuming that we had my sequel or Maria DB installed we could use ansible z-- my sequel module - M my sequel user my sequel user and give it the arguments like name equals Django host equals % for any host password equals one two three four five there's different ways to manage passwords for this particular module but I'm just showing this option here proof equals all databases and all commands state equals present and this particular command will fail on the database server because my sequel is not installed but assuming it were this would set up a user for my sequel and then you can also set up a database using my sequel so in the book I also mentioned like this is again this is probably better served with a playbook but it's just illustrating that you could do all of the kind of setup that you need to do using ansible and all your different servers more easily than running shell scripts and trying to manage writing it on the different servers and all that kind of stuff so another thing that that's important to note with ad-hoc commands as you can limit so if earlier I ran this command on the the app servers and I don't need to use sudo for that you can also limit it to only run on one of the servers like you can use limit and then give it a server name so 192 and this can be a hostname or an IP address one six eight dot sixty dot four and I'm really bad about finishing off my quotes so I can just run it on one server you can also give it a list with comma suck of servers and you can use different arguments I believe it's something like : not something not a different not a particular server there's different arguments that you use and you can look that up in an tools documentation for the specifics that's getting a little bit more into the detailed esoteric usage of inventory and typically if you're starting to do a lot of that stuff you might want to look into changing the way that you use inventory because you shouldn't be doing advanced things that are deeper and in Sable's documentation that often I think the first time I ever actually had to do that was a month or so ago and I had an inventory with 70 something servers for the service I run called hosted Apache Solr one of the servers was having really weird DNS issues it's always DNS and I was trying to make it so that I could run a playbook on all the servers but that one until I could figure out what was going on with that server because it was kind of a pet server since it was the only server in that region that had a certain capability that I was trying to target so anyway I had to exclude that one server from my whole inventory and I didn't want to like change the inventory because it's using a dynamic inventory which I'll get to in a later chapter and so I had to use all servers except that one so anyway that's that's one way to do things you can also use regular expressions in here so I can say some forms of regular expressions I can say start at 4 and that's going to do the same thing as the above command did unless you had certain like a bunch of different IP addresses some of them that had that 4 and basically you can do almost anything to target different groups of servers using the inventories the Dansville that you set up an instable it looks like we're running up on time at this point so I'm gonna stop there but in the rest of that chapter chapter 3 there's examples for managing users and groups there's a group module that ansible has there's a user module that ansible has that manages Linux users and groups so look in the book for more detailed examples you can use the package package yum and apt and those kind of things you can get information about files using the stat module and you can copy files to servers using the copy copy module you can retrieve files from servers using the fetch module again to get documentation you can use ansible doc or look it on look it up on google you can create files and directories using the file module you can delete directories using the file module using state absent you can even run commands in the background and they might actually get into that leading off the next next episode of this because that's something that's a little bit more interesting and sometimes you might want to quickly run a job that could take hours on all your servers so we'll leave it at that and I wanted to remind everybody again that all these episodes are recorded so if you missed the beginning of this please go back it'll be on my youtube channel it'll still be up at the same URL so if you're watching it right now you'll be able to see it you can scroll back scrub back in the timeline and watch the rest of the video and again if if you like what you see please click the subscribe button which is right below me I am always it's hard to see in the mirror image here it's right below me the subscribe button you'll be able to see all the videos on my channel and they come up and if you hit the notification icon which I don't like doing you'll be able to get a notification when a new live stream like this one starts and you know as I've been saying throughout the past few weeks it's it's been a kind of tough time for everyone I know I'm getting extremely tired of the Little Mermaid I've started memorizing most of the songs and I'm even I had a dream about one of the songs because my youngest daughter who's potty training has been requesting that every time that she goes to the potty she's like oh and now I get to watch this song from from she calls it Ariel not a little mermaid and I know that I'm not alone in that I think a lot of parents and a lot of a lot of people that have to manage kids around the house are starting to get to their wit's end so we're all in it together keep talking to each other keep reaching out like I said at the beginning of the video we're starting to see some personal effects some people getting laid off or furloughed please reach out to those people they might be feeling isolated and alone I mean in some cases because they are especially you know if you don't have a family at your house if you're alone or if you have a roommate or something who's you know if there's two introverted roommates they they might need somebody to say hello and just welcome them so please please reach out please connect to each other thank you for joining this livestream I hope you'll be back next week again and we'll get back into a couple more ad hoc commands and then jump into our first major play books that we run so I will see you later and I will try to find where YouTube's endstream button is and see what happens sometimes it cuts me off right in the middle of a sentence other times it doesn't and I just look like a fool you'll see

Channel: Jeff Geerling

Views: 51,139

Rating: 4.9725609 out of 5

Keywords: ansible, devops, ad-hoc, tasks, inventory, automation

Id: 7kVfqmGtDL8

Length: 62min 45sec (3765 seconds)

Published: Wed Apr 01 2020

Reddit Comments

[u/geerlingguy]

Just wanted to post this here in case you wanted to follow along with the 2nd episode of 'Ansible 101' (all episodes are archived here). We had a great first episode, and I have some great news about my books to share tomorrow at the beginning of the 2nd episode!

[u/unserfa]

Thank you for your work, you are amazing!

[u/highexplosive]

Thanks Jeff! Looking forward to this one and the rest of the series!

[u/Ironicbadger]

Really appreciate all your work over the years Jeff. Lost count of how many times I've used a geerlingguy role.

[u/sysera]

Thanks for doing these. I lurked while you were live during a conference call but I'll go back and re-watch later. Cheers!

[u/DrAreg12]

Thanks Jeff first episode was awesome! Looking forward to completing this book.