Ansible 101 - on a Cluster of Raspberry Pi 2s

my name is Jeff Garlin and you might know me around the web as gilling guy this is a presentation called ansible 101 and it's a brief quick introduction to ansible presentation is made possible by my book ansible for devops server and configuration management for humans you can buy the book right now it's just about finished from lean pub com and you can also find out more at ansible for DevOps comm instable is what I like to call configuration management for humans it's really simple to use it's really simple to get started with and it does everything you need to do to automate your infrastructure ansible uses SSH for its transport so you don't have to install extra software on your servers you don't have to open up extra ports or have extra demons running on your servers SSH is secure fast and simple just like ansible and that's why ansible uses it you probably already have it set up for all your servers ansible also has 300 plus built-in modules meaning you can do anything from managing to users to installing software to managing configuration files and even doing things like notifications and managing provisioning of new servers using ansible directly without having to install any extra tools the best thing about ansible is that you don't need any extra configuration management any manual process or any extra tools to manage your configuration management system ansible does everything out-of-the-box and you don't have to install anything extra on your servers just to use it it's really easy to get started using instable I usually use pip which installs the same way on all platforms and lets you get the latest versions of ansible right as they're released but you can also install ansible using your favorite package manager on the Mac or on sent to us or a bun to whatever kind of platform that you're using and there's four main parts to ansible we're going to go through inventory ad-hoc commands play books and roles and to do that we're going to introduce ansible using the DRAM bowl the reason I call it the DRAM bowl is because it's actually a cluster of Raspberry Pi two computers running Drupal 8 and I use this for testing and infrastructure deployment and things like that it's a lot more fun to use these I've bare metal servers than it is to use VMs on my laptop even if they're a little bit slower you can find out a lot more information about the DRAM Bowl at the link at the bottom of this video or in the description below so first we're going to go through inventory inventory is just a basic way to tell ansible here are my servers and inventories use ini syntax you can also use other formats and even use scripts to build your inventory but we're just going to cover the basics and ansible will look for an inventory file in etsy slash ansible slash hosts you can also override this and use your own custom file using the - I command in the command line the format for an inventory file is pretty simple you just have a group name in brackets and then the hosts in that group and these hosts can be in US it can also be IP addresses now whatever you prefer as long as your computer can see the hosts ansible will put them in the group and connect to them so I'm going to give a quick demonstration of the inventory file that I'm using with the DRAM Bowl so I'm going to switch over to my terminal and say cat Etsy ansible hosts and this is the file now so first at the top I have a load balancer that's going to take encumber your incoming requests and send them to the web servers there's only one so I have a group name and then the hosts and then again I have web servers that's the group name and then the hosts that are in that group and again these can be domain names or these can be IP addresses as long as the ansible can reach as long as your computer can reach them ansible can communicate with them then I have a group called database with my database server which is the bottom server and the in the rack and then I also have another group called DRAM Bowl that I can define that has these other groups that I defined earlier inside of it as children and that way I can also say add use this variable for all these different servers in all these groups so in scible lets you do a lot of flexible things like defining variables in this case the ansible ssh user to use for all of the servers inside the tramble group which includes the balancer the web servers in the database your inventory might be a lot simpler or a lot more complex it really doesn't matter as long as it's describing your servers to ansible so let's switch back over to the presentation and I'll show you an ad hoc command ad hoc commands are basic ways of running commands or ansible modules on your hosts on your servers and the format's pretty simple you have the ansible command and then you pass it a group name and then you pass it a module and then you pass some arguments for that module by default ansible the command will use the command module to run a given command on your servers so in in the basic use case you can say ansible group bash a and then the command to run it will run that command on your servers so in my case I usually run one command in particular just to make sure that I can connect my servers and I'll run that now we're going to demonstrate some ad hoc commands and I'll first demonstrate that using ansible x' veritable pinging module so if I say ansible all where I can say DRAM ball not to connect to all the different servers bash em ping that's telling ansible to run the ping module on all the server's ansible will connect to all the server's and ping them and see if they return a pong and it looks like they all did so that's wonderful ansible also helpfully tells us that there was no change on the server as a result of this all that happened was the server pawned if we did something like change the user account or installed new software and so would say changed equals true but in this case there were no changes and that first ping took a little time because it was the first time ents what was connecting but now that ansible has a connection to the server's future connections are much quicker you can see we're communicating with all the server's very quickly so another thing that I have with this DRAM bowl cluster is an RGB LED on the front of each DRAM on the front of each raspberry pi right now they're all green but I can actually change the colors to show you how in schools connecting so in the first case I will change all the web servers to blue I have a script on the server called RGB that does this for me and I can say ansible web servers that's the group name - a RGB blue and then I have to use pseudo so it's - s or you can say - - pseudo and it should turn the LEDs on the front of all the web servers blue and we can change them back to green if I say ansible all - a RGB green - s and now all the all the LEDs are back to green now you'll notice that ansible is running this command very quickly on all the servers at once we can actually control an Sable's connection behavior we can tell ansible that instead of connecting it all at once as quickly as possible to connect to one at a time using forks there's a few different ways that ansible can connect to your servers and this is the simplest when you're running ad hoc commands you can override this behavior by saying - - forks equals 1 to connect - one at a time so instead of ansible splitting off and trying to connect to all of them at once it'll connect to one at a time so if we do that it should start at the top and change the LED on each pie individually and to show you again let's switch them back to green and there it goes and you can do two forks at a time we're going to change two servers at a time and ansible is very quick to connect to these servers and you can you can even say six Forks and that will just basically run the command on all the servers as quickly as possible by default ansible is set to have five Forks but you can override that default in your ansible configuration file using the inventory that we set up we can also just target the database server ansible database RGB red and we can switch it's led to red of course for some of the OCD people watching this you might not be very happy with having one led that's different so we'll switch it back to green for you and I was thinking to myself that since I had this cluster with six RGB LEDs on it and since they're pretty bright if I ever need an extra flashlight I set up a command can set any color that I want and then if I need to I can carry the cluster of raspberry pies around my house and see what I need to see of course that would also require that I bring my network and my power with me but I have long extension cords so I can do that let's set all the all the PI's back to green and whoops ha all and I can show you a couple commands that might be a little bit more helpful in real-world usage so if I want to see quickly how much memory my servers are using I can say free - M oops - a for an argument and it'll tell me how much memory is in use on all my servers ansible also has a cool module called the setup module and the setup module will take everything that ansible knows about your server and output it so that you can get information about your servers like how much memory is available what kind of networking it's using what kind of operating system it's on pretty much anything that you can figure out about a server will be in this listing if you use the setup module it's a quick way to get information about your servers that you can react to when you're building playbooks and doing things with your servers so that's the setup module and you can also do things like pipe output from one command into another if you're going to do that you have to actually specify the shell module so if I say it's a little bit all M dish M shell then you can do things like pipe the command of if config over to grep and then you can get the IP addresses on your servers that kind of thing if you don't use the shell module then you'll get an error because by default ants will use the command module and the command module can't handle pipes and redirection correctly so make sure use the shell module if you need to do that ad-hoc commands so helpful if you need to quickly restart services or shut down servers that kind of thing so in this case if our web servers are running engine X then I can run the service module which is an Sable's module for controlling starting and stopping services and I can say name equals engine X of the service to handle state equals restarted dash S and then in this case I want to make sure that it doesn't restart on all four servers at once since I want my load balancer to be able to get some live traffic through so I'm going to say Forks equals two so it does it on the top two servers and then the bottom two servers so answer will do that does it on the first two and then the second two and tells me that it changed the state of the server because it restarted the service so that's helpful so let's get back to playbooks because it's it's all great and good to be able to run these commands on servers but it would be even better if you don't have to run them at all because you have playbooks to do it for you so the problem that a lot of people have is they have these snowflake servers or some people refer to them as pets servers where you log into the individual servers and do things on them or even if you're managing them in groups you have to log into them and do things to them and you might not have everything documented and you might not have every configuration file codified so that you can reproduce the server from scratch in a heartbeat if you need to and so what we want to move to is having all of your infrastructure in code and then eventually also want to move to an idea of having immutable infrastructure where you bring up new servers and then replace your old servers with the new servers instead of having to update a server you can do that kind of stuff once you have everything codified in a playbook and a playbook is a simple UML file it's very easy to write and read and you can run play books with the ansible playbook command so I'm going to show you a quick demo of a playbook and in this case the playbook will be running we'll be building a web server with nginx on it and here's the playbook so for a llamó file you start off by using three dashes that tells that tells the amyl interpreter that the rest of this document is going to be a Molson and then you give ansible a list and this this list is just a list of plays in this case we're just going to do one play and that play is going to run on the web servers so we tell it what hosts it's going to run on in this case the web servers we also tell ansible that we want to use sudo because we're going to need to use sudo to run certain commands like installing software and managing users and then we tell ansible much like the forks setting on ad hoc commands we tell ansible to run this in serial on sets of two servers at a time and what this will do is it'll run this whole playbook on two servers and then it will run it on the next two servers if those first two are successful that kind of thing so that you can do zero downtime deployments and rolling deployments that kind of stuff the rest of this playbook defines some of the information that ansible uses to run the playbook so first we have a section called VARs that defines any variables that we're going to use in the playbook in this case we just have one variable and a lot of times you would want to define this variable in your inventory or you might define it in an external variables file in this case we're just putting it in line here and this variable will be used later in the nginx template then there's two sections pre tasks here and post tasks after the main tasks section and these sections are for things like putting your server in and out of load balancer notifying your team when a deployments beginning and ending those kind of things that you can do in this case we're going to use the RGB LEDs on the on the PI to get indicate when the servers being deployed to so that we can see when ansible is communicating with it and then there's a set of tasks and these three tasks will first make sure that the my app user accounts present then it will make sure that nginx is installed and then it will make sure that there's a configuration file on the server for that server for nginx and then at the end we have the section called handlers and these are the same thing as tasks but these can be called from tasks when there's a change so in this case we see that there's a handler called reload nginx and what will happen is if this tasks up here which says notify reload nginx if this task results in a change it will call this handler at the end of the playbook and reload Engine X it'll use the service module to do that in each of these tasks so the pre tasks here there's one of those and then the tasks there's three of those and post tasks there's one of those and even the handler is just a task each one has a few components first is the name this is just to document what the task is doing when ansible is running it will take this text and put it into your output in the terminal so you can see what's going on then you give it the module name that you're going to use in this case we're using the command module to run a command on the server and then you give the command you give the module the arguments for that module in this case there's a command down here for user there's a bunch of key value pairs not to tell the user module what to do and then you can also give other options like in this case we're saying changed when this never results in actual change in the configuration just in the LED status on the on the Raspberry Pi and then down here there's also notified to notify that handler when there's a change and then we have the main part of our playbook which does all the grunt work so there's first we use the user module to make sure that the my app user is present on the server with this shell and in this group and then we have a task that uses apt one of an Sable's package managing modules to make sure that the nginx package is installed on the server then we have a template module that that will take a template that's in this folder in this file which is right here and it will copy that template after it replaces all the variables inside that template to this destination using this file mode now here's this template file it's it's in Jinja - syntax and if you look it's pretty simple a variable is just printed with two handlebars two curly brackets and that's this variable that we defined earlier nginx listen port that you can see is right here and when ansible copies that template to the server it will replace that variable and set the file mode and then if that results in a change so if we're running this for the first time it will actually restart nginx after that happens so let's take this playbook and let's run it on our on our web server so let's make sure we're in the right directory and then run an instable playbook playbook that animal if I run it on the servers it'll connect to them it will first do the first two and then we'll do the second two you can see it starts on the top two web servers it runs all the tasks on them and it'll switch to the second two servers and run all the tests on them and the cool thing about using ansible modules for all this we could have run a bunch of commands like a shell script or we could even even run a shell script to do all this but the cool thing about using these modules is that they ensure that you can do item potent deployments so the first time we run it results in a bunch of changes in ansible tells us there were three changes on each server the second time we run it though ansible is intelligent enough to know that it doesn't need to change these things and it doesn't need to restart nginx so if we run it again the ansible will still go through the full process it will show us at the end that nothing resulted in a change and so you can run this playbook as many times as you want you can even have it running as a periodic job through a continuous integration tool and it will just tell you that your infrastructures in the correct state as long as you're using ansible modules and doing everything with an eye towards idempotence so that's our playbook and that's how a template works and how some of the basic tasks work but one thing that I noticed was that a lot of our configuration was for nginx and it would be nice if we could compartmentalize that and put it in its own place so that we can make our playbook a little simpler to read so roles are an Sable's answer for that you can encapsulate your configuration in small reusable chunks and there are actually more than 4,000 rolls on ansible galaxy that can help you configure pretty much any application that's out there you can build your own rules or you can use these contributed rules and if you want to build your own roll like in this case we might want one for nginx you run the command ansible galaxy in it then the roll name so let me switch over to here and you can see in my roles folder there's no roles typically I'll install roles in a roles director with your play book so if I CD into the roles directory and then I say ansible galaxy in it and X ansible galaxy will create that role for me and if I switch back over here you'll see that there's now an engine X directory in galaxy creates all these different folders for me but there's only a few that I'm typically interested in when I'm building out a new role for example there's the defaults and VARs directory defaults is where you want to put most of your variables now the VARs directory is good for variables that your role needs but they might not want to be overridden by other playbooks defaults or for variables that you might override in your playbook so it makes it a lot easier if you put them in here so we're going to take this engine Excellus import and put it into our defaults file and save that now I can take this section out of my playbook then we have a couple tasks that make sure nginx is installed and configured so I'm going to copy those tasks out and I'm going to put them into the tasks file for my role save that and then it looks like there's one Handler reload Engine X and I can take that Handler out and put it in the handlers file save that and then the last thing that I need to do to make sure that my role works correctly I can delete that section on my playbook is go to the mate a folder and look in this file this file by default has all this galaxy information if you want to contribute roll back to ansible galaxy you can fill this in now but it's not needed if you're using a private roller a one-off role for something but you do need to define dependency as if your role is doing something like installing Apache Solr you might require Java so you might have a dependency of a Java role that has to run before your solar roll in this case we're just installing and configuring nginx so we have no dependencies so we'll just set an empty array here and now that we have this Rolle filled out I still need to copy this template file into it so let me grab this file and put it into the roles templates directory and then one more thing to make sure is that my task doesn't have the templates folder because ansible knows when you're using a template in a role it'll pull it right out of the templates directory and to call that role from here in my main playbook now all I have to do is add a new section to the playbook roles and then give it a list of roles so in this case nginx and if I run the PlayBook again then it will use that role whoops I'm in the wrong directory if I run that playbook again it'll use that role instead of the inline tasks to do the nginx setup so this will take a second but now it also prefixes all the tasks that are in that role with the roles name and a pipe so you can see what role tasks are coming from it's very helpful and of course there are no changes because we didn't do anything that resulted in a change and the cool thing about roles is you can actually make an entire server set up using just a few roles so for a lot of my servers I have this kind of playbook with a security role and that might set up your firewall and some security settings an accounts role it sets up different accounts on the server and in this case it's a web server so it's running nginx and it would do some deployment steps like maybe pull down a git repository and put some files in place for my app so this could literally be your entire playbook for a server and you can use a few different roles to compartmentalize the configuration for these different things so roles are very powerful and simple ways to do that so there's a whole lot more to ansible and I hope that this this presentation showed you just how easy it is to get started we're already managing this cluster of raspberry pies and we're well on our way to getting our application running on them in just a few minutes and there's a whole lot more to ansible that I can't cover in this presentation but I do cover it in my book and there's tons of information about it on ansible documentation but things like using ansible tower to automate the automation steps that you do and make it team based so you can have different people control different environments ansible also has deep Jenkins integration and other CI tools you can do continuous deployment very easily ansible has docker integration for building and deploying images it has integration with all the major cloud providers and many smaller ones as well and it's easy enough to write your own configuration so that you can provision servers and control your infrastructure that way ansible also has tons of notifications modules you can do rolling updates and zero downtime deployments ansible has a heck of a lot of stuff built into it that I can't cover in this presentation but I would recommend that you go and read an Sable's documentation and also see some of my repositories on github I like my ansible vagrant examples repository and the DRAM Bowl repository has a lot of ansible play books in it and examples of how you can build different kinds of servers and again I have a book called ansible for DevOps and that book is what's basically covering the first three chapters in this presentation so I hope that this has inspired you to automate your life and automate your infrastructure and hopefully maybe even buy my book thank you

Channel: Jeff Geerling

Views: 116,973

Rating: 4.9541159 out of 5

Keywords: ansible, Raspberry Pi (Computer), devops, infrastructure, tutorial, dramble, geerlingguy, ansible for devops, demo, software, technology, ssh, remote control, computer cluster, system, configuration management, cloud, computing

Id: ZNB1at8mJWY

Length: 25min 48sec (1548 seconds)

Published: Tue Jul 14 2015

Reddit Comments

[u/pydood]

very good video. I'm still getting my feet wet with puppet, this helps me understand the ideal of infrastructure as code a bit more. Ansible seems pretty awesome.

[u/hav_]

Good work.

[u/evemanufacturetool]

That was very helpful. Thank you.

[u/awox]

Hey, I borrow a heap of your Ansible shit. You rock! :)

[u/jayemuno]

Been wanting to dive into Ansible and this was very helpful.

[u/[deleted]]

that seems.... really really slow for running a single command. rPis are not that slow