Ansible 101 - Episode 6 - Ansible Vault and Roles

good morning good afternoon good evening everybody welcome to Episode six of the ansible 101 livestream series if as we do with other episodes if you if you'd like to and are able to post who you are and where you're from down in the live chat it's always good to see little global community we are automating things around the world today I wanted to talk about ansible vault which last last episode from episode 5 we just about got to it but it had to remain secret for another week so we'll get to ansible vault today we will talk about playbook includes and imports and playbook rolls which will set us up for a very interesting episode next week which will be on ansible testing with ansible lint with check mode and with syntax check and also molecule which is probably the most requested thing I've had throughout all of this series so if you're interested in that and you're not already subscribed click subscribe below here and that way you'll be notified you can also hit the notification bell to make sure that you get a message when the light when the next episode is posted and thank you for everybody who has subscribed I've I mentioned this a few weeks or I think a few weeks ago that my goal this year is to have 5,000 subscribers on the YouTube channel and I'm almost to 9000 so kind of crazy how that went once the once all the craziness happened in the past couple months but another thing I wanted to remind everybody of is that device 42 sponsored one extra month of giving away my two books one of them is ansible for devops which is complete and has had 22 revisions and I'm working on a big 23rd revision right now as part of this series and also ansible for kubernetes which I'm not finished with yet but I'm working on both of those you can get for free from lean pub right now if you go to lean pub comm and search for either one both of them are free until the end of this month which I believe is tomorrow so if you haven't gotten them yet go ahead and get them if you're watching this live stream later after the fact and it's not April 2020 anymore I'm sorry unfortunately you'll have to pay for the books but there hopefully well worth it to you I think a lot of people are gonna slide String would agree that they are there are probably worth it let me get my windows set up here and make sure that I'm also talking yes because yesterday is livestream I was talking to myself for about a minute before someone mentioned in chat that I was was actually talking to myself and not to the livestream another thing is somebody mentioned that a nice shirt the Red Hat shirt this week is Red Hat summit and as I do some work with red hat and ansible I decided to wear my red hat shirt in honor of red hat summit which is I believe this is the first ever virtual summit and there's something like 70 80 90 thought I don't know how many thousand people are watching the the sessions but it's it's cool to see that it became a free summit it's virtual and there's so many people learning so much stuff that I think out of all the things that have happened over the past few weeks one of the things that I really like is that people especially in the tech community where it is easier to transfer your skills and learn new things so many people are able to spend time learning new skills and in expanding their careers it's it's a good time to be able to do that especially if you're furloughed or stuck at home and don't have much else going on for myself I have some kids and we have a lot of family things to do and so it's been a little tough and crazy but we survive we figure things out um I wanted to especially thank a few people who are now sponsoring me on patreon and github and if you like this content if you want to see more things like it on YouTube if you want to make sure that I have time in my day to work on all my ansible content which you can see at ansible Jeff Garlin com please consider supporting me on patreon and github obviously if you're not able to due to extenuating circumstances that's perfectly fine I understand but if you do take my books for free take me up on that offer which I encourage you to do and someday end up in a job or in a situation where you're able to support me after that that'd be awesome too but no expectations there but thank you especially to Logan Marshall Daniel Linder and Nico wonder who have all started supporting me this week and one of the thing that helps is I can justify doing other projects that might be interesting to you for instance I have received this Turing PI Raspberry Pi cluster board and I'm going to be talking about how to get kubernetes running on this and why you might want to do this why you might not want to and the history of myself with different Raspberry Pi clusters is my pied ramble so the first video for that should be coming out this week and I should be doing a few videos on that but again I'm be able to do things like that because with the support that I get through sponsorship so thank you very much and Erin says pay for the books if you can do it yes that'd be it would be nice but again it's there's no expectation for that I would much rather if you were thinking about getting them or not getting them just get them free and figure out a way to support me later if you want to or just pay it forward to someone else let's see so last week's episode oh and I do need to give a device for ready to shout out this is the last the last episode that we'll have in April and they they basically sponsored the giveaway for the books and all they asked for was a mention in the videos that I do and a couple shoutouts on Twitter so I'm gonna do that too and mention how how good they've been for the ansible community they don't even they're not like primarily an ansible company they're an infrastructure tracking device tracking and that kind of thing company and they happen to use ansible and support ansible but it's awesome that they find ways as many organizations have to support open source contributions and things like that so ansible is a good tool for driving IT automation but to make the automation work you need to make sure you have an accurate real-time picture of your IT infrastructure and that's where device 42 helps device 42 provides comprehensive discovery of your entire IT estate from mainframes to kubernetes and just like ansible it's agentless you can try it for free download a trial today at device 42 comm and see how you can take ansible automation to the next level and hopefully these live streams are helping you take your ansible automation to the next level as well a couple things that were from live chat and comments from last week's episode one thing that I haven't mentioned yet but some people might be interested in Oliver Davies mentioned on the stream that my new MacBook Pro which is able to keep up with this live stream and do it in 30 frames per second at 1080p which my old one could not do he said it was provisioned with ansible and if you've never thought about that you can actually manage your local machine whether it's Mac or Windows or Linux using ansible and I have been doing that for years now I have a project called Mac dev playbook that has all of my Mac's configuration in it I obviously have some extra configuration in a local configuration file and I use my own dot files and things like that but this this project even if you don't want to use this exact configuration and have my exact Mac setup it might be inspiration for you to do something similar but it lets you do things like install App Store apps and things like that all with automation and it's especially helpful if you're like me and you have more than one Mac that you manage I don't anymore but I used to have two Macs and keeping them in sync was a lot easier when I did it all with ansible Nathan black said the space in the handler named from last week makes his eyes twitch it's there's a lot of different conventions and ansible some of them are best practices some of them are widely used some of them are different depending on the person for handlers I usually have a name like restart Apache with a space between it you could have restart underscore Apache you could have a full sentence restart with a capital R and Apache with a period but then it's harder to remember that when you're when you're calling your your handlers so there's a lot of different ways you can do it that's just the way I do it that doesn't mean it's right it's right for me but it might not be right for you and Ted Lily mentioned last week he wants a refund on his free book after I made that mistake last week and had a had a section with a bug in it I'm sorry unfortunately I can't give you any anything back for your free book but you can return it through lean pub if you want to return your free book it's possible I don't know why you'd want to but if you wanted to you can someone asked what's the sublime text theme that I use so if I open up sublime text this theme that has the syntax highlighting and all that the theme is called Monica Pro and someone also mentioned that I believe this was on the episode with the solar example I always unpack things in the temp directory and he's like I don't like doing that because the temp directory can be blown away and then the next time you're in they run the ansible playbook it has to download it again the reason I use the temp directory most of the time in my examples is because it's guaranteed that I can always use it and in other people's situations the temp directory is there too but it is a better idea to use a more persistent location so a lot of times I'll download software to the OP directories or to var or something like that so that ansible can contract where that is I just use temp because in training people it's always better to use a directory that you know is going to exist and you know is going to work whereas there could be permissions issues or it might not exist in certain systems that kind of thing if you use a different directory so sometimes when I'm training even the style that I use sometimes it's not necessarily a hundred percent best practice it's not a bad thing but it's not the best practice sometimes I do that mostly to make it easier to show how something works so I might also do that with fault today when I'm talking about how to encrypt things it might be one way to do it it might not be the best way in your situation but as you grow with ansible you'll start to see some of those patterns emerge all right so anything else somebody found out about me from Sean powers thanks Sean powers then but thanks for everybody for joining I see there's Oklahoma India and famous in India that's that's fun probably not as big as a Bollywood star though North Carolina India again Morocco Croatia so awesome to see so much representation I ansible is truly a global community and you know there's a lot of software it's popular in one region or another but it's it's been neat to see all the people who email me and tweet and things from around the world so ansible is truly a global phenomenon at this point and I'm glad that you're on the stream and let's dive right into ansible vault so I am in the book in chapter 5 and I'm on the section ansible vault keeping secrets secret and some people may ask why would you use something like ansible vault because there's a lot of other secret management systems out there there's there's vault by Hoshi Corp and it's a funny aside ansible vault was added as a feature I believe in 2013 or 2014 and Hoshi Corp vault was announced very shortly after that maybe in 2014 or 2015 the first version of it and it was kind of an interesting thing because at that time it hashey Corp vault was not the big behemoth of secret management that it is today it was still a pretty new on the scene kind of thing that was back when hashey Corp was also announcing ATO which most people don't even know what that is it was not a very popular product so they were announcing some other products along with fault that were not as popular and didn't really catch fire so it's an interesting thing in history that ansible has vault and Hoshi core filt vault but they were kind of two different products but attacking a similar space but both coexists a lot of people use Kashi Corp vault and still use ansible vault for certain things too but if you do need a central secret management service and you integrated with everything in your infrastructure something like Kashi Corp vault or AWS key management or a jurors key vault those kind of things might make sense for you especially if you're having other teams work on them and integrate but for a lot of use cases ansible vault is is completely adequate for storing things like sensitive data encrypted passwords API keys that kind of thing and the ansible vault works just like a real world vault where you take something sensitive like a password or API key that you would have in a variable and you've put it in a vault and ansible encrypts the vault and gives you a key which is your password and then with that key you can unlock the vault anytime you run an ansible playbook so there's a few a few technicalities to that and I have a playbook here to demonstrate how the basics of it work so this playbook uses an API key and this is one thing that whenever I'm doing code review if I ever see something like this in code whether it's an application that's PHP or Ruby or whatever or it's an ansible playbook if I ever see an API key or a password or anything in plain text alarm bells start going off in my head because that's not a good idea I think I put in here you know some people do this and they put it in their private repository somewhere and but I always say you might follow best practices for physical and operating system security but can you guarantee that every developer and every sysadmin and every person that has access to that code repository does the same and can you guarantee that even if you know if your computer stolen or something and somebody does get access to those files that you'd be able to change those keys in quick enough time to not cause damage it's better to be safe than to be sorry so we want to make sure that this is encrypted we don't want to have this this variable floating around in our PlayBook that might be in a good repository or stored centrally somewhere so ansible vault can do that for us pretty easily right now I can I can run this playbook if I say ansible playbook I mean DML it what this does is this is just a demonstration but you would be using this API key maybe to connect to an API and and run some command or change some setting but in this case it's just printing the API key and it's stored in plain text and we've seen this before it's a variables file the variables files included by ansible and it echos the result it registers that result in this task so it just runs echo API key it registers the result and then it prints it out using ansible debug module and you can see that this is the API key and I'm just doing this to prove how ansible vault works by decrypting something anything that it encrypts and it's pretty simple to use it's just ansible vault and then you can encrypt a file and then I'll give it the path to the file what is it it's VARs slash API whoops API key it would be helpful if I could type so I'm gonna do that and it asked me for a password and this is the key to that file basically so I'm just going to type in the password test not very secure but at least memorable and now if I look at this variable file you notice that it's not it's not human readable it's not it's an encrypted file now and it's encrypted using ansible vault it drops in this little header up here to say it's encrypted with aes-256 and here's the data that's encrypted and good luck decrypting this I mean you saw on the live stream earlier what it was and it's right here in the command line but if he didn't have that information it would probably take you thousands or millions of years to get this into this form again so this is secure enough now that I can I can commit it to my code repository I could share it with other people and in fact for some of my insula projects I actually have a vault encrypted file in a public repository so far I haven't been hacked so I'm guessing that the NSA hasn't turned their quantum computers on me and I probably just got demonetized for mentioning the NSA but whatever so let's see so it to run this PlayBook now I have to do something special though because if I run it ansible PlayBook may not a mole it says it's in tempting to decrypt but there's no vault secret found so to be able to run this PlayBook I need to give it the password so I can say ask vault pass and it's going to ask me for the password before it starts running the PlayBook and it'll use that password to decrypt so I'll say test and now it decrypts the password or the API key and I can still get that value out of here so it's a little inconvenient though because especially if you're running this in automation like let's say you're running this on a Jenkins server ansible tower or something like that you're not going to be able to interactively enter the vault password every time you run the PlayBook or on the job so you can also supply a password file and typically you'd put that in your home directory dot ansible folder and you'd call it whatever you want like maybe API key vault past txt and the file would just have that that password in it like test and then you can pass the vault file with what is it it's - - vault password file so you'd say vault password file and then give it give it a file path like that ansible oops API key past that txt something like that and then it'll use that file as the password and then it will decrypt there's a few other ways to do this but I'm not gonna get too deep into it today because this is a one-on-one series now but I did want to mention that ansible vault can also do some other tricks I ansible vault decrypt VARs / API key Gamal and that basically decrypt the file so that you can edit it like this but you typically wouldn't want to decrypt the file even for a second because you might accidentally you might accidentally share that file in your repo or something so I'm gonna leave it encrypted and then you can actually edit it through ansible vault and never decrypt the file in the file system so I can say in sablevulpe edit and give it the password and now it opens it up in my editor and I could I could change this if I wanted to but I'm not going to change it you can also say ansible vault what are some other options here ari key if you want to change the password so if I want to change it from test to something else I can say re key and then it asks for the current password which is test and I'm going to give it a new one which is test 2 test 2 I could have done hunter 7 or whatever that one is and you can also just view a view a file or even create a file encrypted without creating it like this and then encrypting it afterwards there's some other things you can do to which I'm not going to get into today but you might want to look into if you do start using into a vault a lot and that is you can have a variables file so I'm going to decrypt this file really quick test to I think it is now so I decrypted the file and I let's say you have a lot of different variables my app get repo you are all here and some other variables too you can actually have vault vault encrypted variables in line and a file - which can make it a little bit easier to manage in some cases but it can also get a little messy that way I typically have all my vault encrypted variables in one separate file and then my non-encrypted variables in US and another file that way if somebody's not as familiar with how to use vault with inline encrypted variables and things that they don't accidentally sup that file I've seen that happen before but some things to keep in mind another thing that you can do for the vault password is you can actually have an executable script as your vault password file in case you need to get a password out of another system but using a script you can do that the script would just need to output one line of the password basically just like the file would that's about it for vault and in chapter 5 in chat and and some people also asked about how you use vault and tower and awx things like that I'm not going to cover that today we might cover that coming up in a session talking about ansible tower but there's a few peculiarities with how you set up secrets and things in tower itself and someone also asked if vault has versioning it doesn't like I said vault is vault is similar it's in a similar space but it's not the same thing as like Hoshi corpse fault or key management thing it's more meant for a password or API key type thing that you store to get repository and you track that way it's not it's not meant to be a full-on replacement for a system like a hash e-court fault and and that's where you might consider using that kind of system if you need more requirements for that kind of thing and someone also asked if you can only use one password for a single vault yes that is true but I wanted to mention just briefly before I get through the rest of chapter 5 in the book there are some other things there's there's some extra extra work in here talking about when and conditionals on tasks I think I don't have a win in here at all but I might have one in one of the other examples today but when you use when there's different ways that you can have a task run or not run depending on different different variables or different Jinja conditions and I talk about that more in the book I'm not going to go deeply into it today now but it's similar to most programming languages you can compare strings you can use functions to test things about all that kind of stuff it also talks about registering variables and there's also a couple other things that you can use sometimes changed when and failed when when you're using the shell modular command module especially you might want to have custom a custom condition for when it's change versus not change like based on the output of of this response if it has certain text in it you want to say yes it's changed if it doesn't it's not changed even if ansible doesn't know whether or not it changed you can control that with these two variables just something to keep in mind when you're trying to build your playbooks with idempotence there's also a the ability to do ignore errors this particular task is not going to have an error but if you have a test that could error out sometimes and you don't care about that but you do want the test to run anyways you can set ignore errors true and just just a few other things there's there's also the ability to wait for a certain thing to happen before the playbook continues you can also pause a playbook for a certain amount of time things like that it's again this is in chapter 5 if you want to go through that prompts tags is something that is probably good to cover really quick at least for any tasks in the playbook you can give it a list of tags so this would be like API and echo or something like that these are just ways to classify tests and then when you run your PlayBook if I save this I can actually run the playbook and I can give it - - tags equals API whoops that's not an equal API and it'll just run the tasks that are tagged with API so that's a helpful way to be able to trim down how much stuff runs in a playbook if you want to have a playbook split up that way I would give you caution though I use tags very sparingly you can tag roles you can tag tasks you can tag I believe includes and things like that but it can get really complicated and so I avoid using tags unless there's a really good reason - some people are like oh I can tag things if they tag every task and every role in every single little thing I don't do that because a it adds more space in your clay book that that is just visual distraction and B it doesn't really offer a lot of value if everything is tagged because it's kind of like when somebody writes a document and they bold and italicize every single line of text at that point the whole thing is bold and italic and you're not gonna see like oh there's value in just running this tag or that tag so I usually build the PlayBook first and then if I find a path of of tasks or roles that I need to tag and use separately I might do that sometimes they even react attack the playbook so those things are in a separate playbook or in a separate include file or something which I'll talk about in a minute and then I can run those first and then run the other stuff and then I could just run that one include file or run one play that has the specific things I need to do so that's my caution about tags some people go a little overboard with tags be careful of that and they may I asked about vault the Hashi vault module I haven't used that much but yeah you you are able to use some modules France well to integrate with these other key management and vault systems to be able to decrypt things and find passwords and stuff but the specifics of that I'm not gonna cover in this particular ansible 101 series just because it can get really deep and complex very quickly and and there's definitely better places to discuss that so tags another thing is blocks I'm not going to cover it in this in this episode but blocks are a way to take a bunch of tasks and put them into like one chunk of a playbook without having a task include file and another thing that gives you is is the try except always type pattern in ansible it's block rescue always you can have a block of tasks and if they fail it'll run the rescue tasks that are after it and then if you have always it'll always run those tests after the rest of the block runs even if it fails so that's another thing to keep in mind that's something I don't use often if you do use it off and you might be doing a little too much programming in your playbooks it's handy sometimes but I I think some people use it all the time and it starts making your playbooks really complicated and the I don't know I I just I have the philosophy in my ansible playbooks if it looks like you're doing programming work you're probably not automating in the most efficient manner and it's not gonna be as maintainable because ansible playbooks are not Python they're not ruby they're not a programming language and if they start looking like that it's not very not very fun to deal with and as with all chapters I will try to remember to read the wonderful quotes I have in a cow say at the end this one is men have become the tools of their tools by Henry David Thoreau take that for what it's worth so moving on the main topic for today which I want to cover is playbook organization so a lot of these play books that we've done so far pretty simple this is 17 lines you know you can memorize this playbook pretty quickly if you wanted to and they've been pretty simple not a big deal to have it all in one play book but as time goes on your playbooks will start getting a little more complicated so I'm going to go back to Oh to Apache so we we've met this playbook before this one let me make sure you can see it whoops that's not the right one there we go so this PlayBook in installs Apache and it's it's cross-platform so it works with the default variables are for a bunt to and Debian and it has a set of variables for Red Hat I think this is the one that we talked about last week but you can see that even this this is a very simple playbook to but it's hard to get into 35 30 37 lines and a playbook a real-world playbook for handling Apache server is probably going to be more like 80 to 100 lines that's around the threshold where I start thinking how can I break this up and organize it better so that I can kind of deal with the different parts of it a little easier and I have my font zoomed in quite a bit so I can only see a little bit of this but usually it's it's once there's two or three screens worth of text in a playbook because when I start getting to that point and that's around 100 lines but it's it's not a hard and fast rule but I this playbook is still running on the two I have two ec2 instances so I'll say instable playbook I mean that amel and I hope that those two instances are running I didn't check on him this morning but it looks like they are so this playbook installs Apache it installs Apache here and then it copies a config file in place if it did have to copy the file it'll notify the handler that's up here restart Apache and then it makes sure that the Apache service is enabled and running so that you know web request can be served and I believe if you go to one of these one of these IP addresses you can do this right now if you wanted to you can see the Apache page so here's 54.2 o 8.20 1.1 86 and here's the other one bun tutus and our bun two's default page so both of those are serving public web traffic and I'm sure somebody on the stream is probably going to try to ddos one of them so maybe it won't work for you if that happens but this this task file can actually be shortened up quite a bit and and this would be the first the first level of organizing a playbook you can include both handlers and tasks and pretest and anything anything that's a task you can include it in a playbook so that you don't have to have everything in line in one big playbook so I'm gonna take these handlers and I'm going to use an instable plug-in called include tasks I believe let me or in import tasks sorry about that Allah so I'm going to take this task out of here and I'm going to use something called import tasks and then I'm gonna give it a file name so I'm gonna say I'm gonna create a tasks folder and say or actually I'll create it in the handlers folder handlers slash Apache animal so I'll save this change and I'm going to create a new folder called handlers and a new file and I'm gonna paste that task in there and save it as apache camel and if I run this playbook it won't make a change right now because that file doesn't have a change but let me make a change that file so it does run the handler just to prove that it's there I'll take out that comment and save it and what this does is import tests we'll we'll take it we'll take the contents of this file handlers Apache thamel which is right here it'll basically take those contents and copy and paste them in place of this so this playbook when ansible starts running would become that so ansible handle he does that for you and use import tasks and the same thing with these tasks if I take them here and I say import tasks and I'm going to make a different folder called tasks and say Apache gamal then I can do the same thing make the new folder tasks whoops that's text rename tasks you file Apache whoops I'll paste those in there name it Apache Emmel save it and now if I run the playbook I'll see if it works this is something I didn't test when I was going over this part so hopefully it works it should and it looks like it did so now the the playbook is a lot more readable because you can see I mean this playbook is just installing Apache if it did other things this would be nicer because you could have the Apache handlers here and you could also maybe have if you had an app deploy thing you could say handlers app and you'd have tasks for app camels something like that but it's already a lot more readable when you come into this main playbook you see so there's some Apache handlers here here's some Apache TAS and you can start working from the playbook and start doing other things too like you know name clone website repository you can use the git module and say repos whatever and you could have you could have this playbook do the rest of its stuff but but this stuff is kind of noisy because it's not really that important and it's not going to change much and it's not going to affect the way you deploy your application so it's it's nice to pull things out in two separate tasks files this way let's see nothing nothing in the chat that I need to talk about with with this particular setup but that's that's a pretty simple example and that can work for simpler playbooks but again there's there starts to be instances where you might have a playbook that's 200 lines or 300 lines and you might have you might have a list of tasks that's like you know this long and it starts getting a little bit crazy because you'd probably also have a bunch of handlers for those tasks and then that you start getting back in the same problem of your brain just can't can't cope with all that complexity so ansible has another mechanism for organizing your playbooks called roles and I'm going to pop out of here one other thing that I did want to note before I do that you can actually pass variables directly to the imported tasks and you can override variables using VARs like this so for instance this tests Apache Amal it uses a patchy package or a patchy config drought or whatever you can actually override that variable here and call it Apache 3 when Apache 3 comes out something like that if you wanted to that's just something to keep in mind another thing that can trip you up is import tasks let me make sure that I'm right because I often forget this import tasks statically includes the task file so that's literally taking these these tasks which are right here it's literally just taking this when the PlayBook is about to run and pasting them there it's not a mean if you're looking at the engine that's not exactly what happens but in in in actuality that's what would happen it pasted them here and that's kind of how it runs however there are some times when you have a dynamically defined task like a task that that does something a little different I have one example here and I'll type it out here so you can see what about and I have a task on one of my log servers that does a name check for existing log files in dynamic log file paths variable find paths item patterns start at log register found log file that's so a task like this what I do is in in a different part of the playbook for this particular task this task is not working in this play because it's just a demonstration but this variable might be dynamically to find in another task that does something and I have a typo here it might be dynamically defined and this what happens is an T'Pol's engine when it kind of copies and pastes the the tasks and this is a task that changes the way that it does things dynamically based on something else that's in the playbook so instead of using if I pull this into an tasks file instead of using import tasks I would use include tasks tasks slash log amel or whatever it is include lets you have dynamically defined information going into tasks or dynamically changing tasks and what I typically do for that is sometimes it's hard to know just based on even for me who's written lots and lots of ansible code it's hard it's hard for me to always know exactly when I need an include versus import but I always default to importing and then if things break I'll switch it to and include and I'm like ah that was the problem so the book goes into a little more detail on that another thing that's important to know is Play Books can also be included just like tests can so or let's say you have something like a new file I will call it install app and this won't have a handler it won't have any pre tasks they'll just have a task clone app into web root and it'll just have git and blah it'll have the task for forget to clone the app into the VAR dub dub dub HTML directory something like that and what you could do is in this main that animal I can just call this other play straight from here by saying and what does it include sorry I I always have to look this up just because of the fact that it's import tasks include tasks and and handlers and all that and then for play books it is import playbook that's sort of this import playbook and then you give it a path to a playbook so this one would be I didn't save it so let me say this as a panel so I could say import playbook aap amel and let me actually make this work so that we can show it I'll just let's do command date just so that we have something that runs this would obviously be something that deploys your app into a directory and and run some commands or does whatever but just for demonstration purposes here's how you include another play or import another play into a playbook so we have this first install Apache playbook and then it will import this other playbook at the end or at least it should unless my book is wrong like it was last week so there it is install app so it's getting to this this play and it includes it it imports it the same as if you had taken this play and copied it and pasted it where the import command is like that whoops that's the wrong spacing there come on sublime you're better than that there we go so it's the same thing as doing that so that is simple includes and imports and sometimes for a lot of play books that might be adequate if especially if they're smaller play books and they only have 10 20 30 tasks something like that but as time goes on things get a little more complicated let me check my time I'm good Oh 3 No so here's a note j/s application and this one is it's one of the best node.js applications I've ever ittan and the reason for that is it only has one dependency which means there's only about 7,000 packages that get installed when you install this application another one that I had had like three dependencies and that required about 70,000 packages anyway node.js jokes if you've ever worked with node GS one of the packages I'm sure is there just to check if one plus one equals two or something along those lines but here's that here's the app it's one of the simplest it probably is about as simple as you can get for an express based web server it just has a function that returns hello world to every request no matter what and it listens on port 80 so it's just a little web server that returns hello world and when you deploy a node.js app you need to deploy the app and then you need to install any packages in the package JSON using NPM and that's about it you just need to make sure that nodejs is installed on your server then you put your app into the server and then you run npm install and then you start the app in this case like node app KS and it'll start running so that's what this PlayBook does this is going to run so here's my vagrant file it's going to run on sent to us eight and it's going to have an IP address of 191 685 55 and it's going to automatically run this ansvil playbook so I'm going to kick this off while I'm talking about the playbook just because it might take a minute or two it's gonna say vagrant up and with my new MacBook Pro I'm hoping that this doesn't crash my machine I don't think I've run a vagrant VM while I've been doing one of these live streams for a little while so I'll see if it works it should let's see somebody mentions yon mentioned why isn't found log files path in that previous example that would probably be a better way to name it another another note on best practices with ansible I usually like to name variables for Bhosle so instead of having like little abbreviations and things I typically like having the full word spelled out unless it's like 25 characters long so yeah if it was a dynamic variable that you're using like found log files path the found log files path would probably better than log files path anyway so this playbook it has one variable where where do I want to install the node apps that I have in this case this one little app so it's going to install an user local opt node it needs the Apple repo the the I forget what that stands for some of somebody can post it in chat what Apple stands for anyway I need that to install nodejs because this this playbook actually could be run on sent to s7 as well and then I want to turn off the firewall for now just because I don't want it to interfere with my little Express app server but in the real world you probably would not want to do something like this you'd probably want to make sure the right ports are open and then it's connected to the right IPS and things like that and then I want to make sure that NPM is installed I'm using the yum module but you could also use package as we've mentioned before or if you're on sent to us 8 or Fedora you could even use DNF yum is basically an alias to be enough but for backwards compatibility I typically stick with yum or package when I'm dealing with sent to us so that I can have the same playbook work on sent to us 7 which will be supported for another 4 or 5 years and then after I install NPM I'm going to install a little node.js utility called forever forever is just a little a little nodejs utility that lets you run apps like services instead of instead of running them like with no hop or something like that you can use forever to start it and forever will manage that that node.js app there's other ways to do this this is just one of the simplest ones that I found that has worked for years and years and I like simple things that work and then it's going to do some app deployment stuff so this stuff up here was basically getting nodejs and forever installed and then down here it's creating a directory using the file module making sure that that directory exists and that's this one up here user local op node and then it copies that app to the server using a copy module the copy module is OK for copying folders if there's just a few things in it if you have a folder with thousands of things I wouldn't use the copy module find it better way usually a tarball or using git or something like that and then after it copies the app up it uses NPM the npm module and sibyl's npm module to install the dependencies this is a quick short way to do that using instable if you're using nodejs and then it's gonna see if the app is running using the forever list command there's no ansible module for forever I could have written one for this but that's a little bit beyond the scope of this example but there's I'm just gonna run forever list and check if there's any if my app is already running and that's what this wind condition right here is doing it's saying if it if it doesn't see app KS if it doesn't see the full path user local ops node slash app slash app yes in the output of forever list then it's going to start the start the app using forever so there's two different things going on here and here here's the thing running unfortunately you can't see this one from the public Internet but if I go to I could use the host name because I have a vagrant hosts updater installed but I'm just going to use the IP address here and I will curl that and there it is hello world and I can also do it in my browser - hello world one of the best node.js apps ever because it it doesn't have a memory leak in it let's see so that works but this is as with the other playbook it's starting to get a little bit complicated and and you have two different things going on in here you have the setup stuff that gets the software that I need to run the app and then there's also the app deployment stuff so I want to make this a little bit simpler and let me also make sure make sure that I glance in the book because last week I forgot a couple things when I wasn't glancing at my notes in the book you could you could take two things like you could take all this and put it into and include like include nodejs setup include tasks and then this one could be include tasks app setup or something like that but ansible also lets you have roles and roles are like a way to package up a bundle of stuff that could be used for just one playbook or could be used for multiple play books and if I'm building nodejs apps there's there's a good likelihood that I'm going to need to install nodejs on all my servers and in all my play books so it'd be nicer to have this stuff be in its own little thing that I could reuse on different play books instead of having it tied to a specific playbook so I'm going to create an antelope or that and there's a lot of different ways to create roles the way that I'm going to demonstrate here is just the simplest possible way to create an ansible role and that is to create a roles folder so I'm going to create roles and then inside the roles folder you're going to create a folder for the name of the role that you're doing so I'm going to call this role a node Jas because it's going to be no js' set up stuff and a role has at minimum it has two things that it requires for every role and that is a file in the meta directory that gives some description to ansible of what the role is doing and a folder called tasks and the tasks are basically the tasks that are going to run in the in the role and I'm going to get into some of the more complicated things that you can do with roles in a minute but at a basic level a role is just a grouping of tasks that you can reuse in different playbooks more easily so in the test file I'm going to create a main main that amyl file and this will be automatically called by instable when I when I ask for ansible to run this role so in the main that animal I'm gonna take these nodejs tasks I'm gonna take them out of here and save that and then I'm gonna paste them into this test main file and then in the meta directory I need to have a main channel file and this file at minimum has to give ansible a list of dependencies for this role this particular case we don't have any dependencies like the node.js role doesn't depend on a Java role and that doesn't depend on a Fedora role or you know that kind of thing so in this case I'm going to say I say dependencies and give it an empty list of dependencies because I don't this role doesn't have to have other roles called before it runs so at that point I can add in my playbook a list of roles and call it node J s and if I run this it's gonna do the same thing that it just did but it's going to call this node J s role so I'm gonna say if a current provision will kick off this playbook inside that VM because vagrant does that magically for me by the way this compatibility mode message I actually filed a issue report for that integrant and the next version to effect next version of vagrant will not have that annoying warning because ansible 2.0 has been out for a very long time so one thing that you might notice that happened here since I'm calling this role every task in the role is prefixed with the name of that role so that's one one easy way to be able to see quickly where something's coming from if you're using rolls it prefixes every task in the role with that roll name and you can see here it's just install apple repo but it has it it has it with nodejs install apple repo so that's one way to organize your PlayBook in the output to it you can see where things are coming from so now I have this role and then I have my app deployment stuff and for this particular playbook you know a role this role doesn't need any handlers it doesn't need any special templates or anything like that but but roles can have a lot more complexity and I could also break this out into its own role like app deploy and the app deploy role could use variables for all this stuff and if I use the same kind of pattern for all I know DJ s apps then these two roles could be reused in other play books for other apps so I might have 10 different node.js apps and they all use the same role in same playbook just with a few different variables like where the where the app is installed what the name of the app is that thing so that's that's a the simplest introduction to roles and one other note is that roles are automatically picked up in a roles directory in a playbook and they're also picked up from the global roles path which is like Etsy slash ansible slash roles I think and you can also override that if you have in ansible that config file and all the settings for that are in ansible configuration down here roles path no is it configuration maybe I'm on the wrong Doc's page configuration settings so this page has all the different settings you can override roles default roles path and you can see that that is you can also stick roles in your user directory your users dot ansible directory and those are the paths that it searches by default as well as a roles path alongside your PlayBook another way that you can put this role in here so if you use this method then the order of the playbook goes pre tasks and then any task that would be in here would run then went on roles and then it would run tasks but what if you need to do something in your tasks then run a role then do other things you can actually include roles the same way as you would tap tasks files by saying include role I believe it's include role somebody mentioned in the chat if I'm wrong there include role no DJ s and that would include the role at that point instead of up here so that that's helpful sometimes if you need to do some other things any you know pre tasks tasks and post-tax tasks aren't enough for you you can do it that way so that is the simplest possible example of an instable role they have one more example here and this example shows you the power of roles I so I have a lot of different roles on ansible galaxy so if I go to galaxy ansible comm and I can search for we're just going to filter roles here I'm just gonna look at all the roles and I'm gonna go download count so I have lots of different roles here and one of them is Java which is a dependency of Apache Solr and another one which is probably way down the list because not as many people use solar as many of the other things but there's another one for solar that's shared on ansible galaxy and these roles are fairly simple so if I go to instable role Java where is it the roles are pretty simple they're very simple just like the ones I've been showing so far but they do have some extra options so since this role also works on FreeBSD I have an include for that but it does the same kind of things that we're doing here it installs some packages the main differences are it does things for multiple OSS like this where it gives you variables files for each OS the different packages that are available and lets you override the variables more easily I'm not going to talk about that today because we only have a few minutes left but there's a lot of different things you can do in a role to make it very flexible and work on different OS Azure in different use cases and two to talk about that a tiny bit I'll just create a quick roll in this directory rolls and I'm going to use the ansible galaxy command-line tool to create the roll which is going to create a kind of a roll scaffold or skeleton that has all of the options that are available to you as I'm going to CD into rolls and say ansible galaxy knit actually I'll use roll in it test and that's going to create a roll for me and you can see there's a lot more files in here than what I had in that demo roll because at a basic level all you need is a meta slash main dot Hamill and at a slash main that Nimal and it'll work fine but you have a lot more options in your role including VARs that are basically variables that are particular to that roll and should not be overwritten you have defaults which are the same thing as VARs but they're able to be easily overridden from other play books that are using your role and ansible scaffolds out all these things there's a files directory for files that you would maybe copy up to a server and there's a separate templates directory for templates you would copy up and use ansible template module is it also has handlers which doesn't have any handlers but you could put any role handlers in there and they're automatically included as well without having to separately include them in your PlayBook and the meta file here has a lot more options so we just used we just put in dependencies but there's options for if you want to contribute a rollback to ansible galaxy so if you want your role to be on galaxy that ansible comm you can put some some information in here a license the author name and all that kind of stuff and then it will show up on ansible galaxy in one of these sections so that is that is like all that different options for a while but that what that starts to give you the power to do I'm gonna delete this folder it gives you the power to have a playbook that could be used in production that's as simple as this and this this playbook actually does run just to prove that I'll say now I'm in the trash oops in scible 101 no for solar rolls not rolls okay so I'll say vagrant up and make sure it provisions this playbook is going to use the two rolls that I have and I have same defaults so Java is gonna install whatever the latest version of Java is on that system so this is sent to a7 I think it's Java 8 and then Solar is going to install Apache Solr at the default latest version that it's set up to do and if you use saying the false like this you can have basic servers that that all you need is this and if you needed a firewall on it you could have a role like caroling guy that firewall and if you needed to have NTP running you could have your role that does ntp caroling guy that ntp and as long as you use defaults that are set up well you might not even have to have any variables but you could override those easily so I have instable roll firewall oops this is Travis let's go over to the repository you might want to change a few settings like you might want to say you might want to right now I think the default is to log dropped packet so you can look at your logs and see what what kind of connections are being dropped and what traffic is being blocked you could just set that in here Fars copy that paste it and now you can override it in your PlayBook so the having roles separated out from your playbooks makes it a lot easier to manage a lot of different use cases very quickly and easily with a large breadth of applications and things and I can still add if I wanted to tasks and put in whatever things I want down here but I like to organize things in roles if it's something that can be generic and flexible because the cool thing is this Java role I can also use it so let's say I'm not setting up a solar server but I'm setting up a Jenkins server Jenkins runs on Java so it needs Java so that's all I need to do and now I have a Jenkins server configuration and and that's I mean in my opinion as somebody who writes lots of ansible roles and probably maintains about two or three hundred of them at least this is this is the way to make ansible based automation a lot easier to manage and a lot more sane because your playbooks end up being about a few rolls putting them together and then adding some task files there's a few different things too Yogesh is talking about how to maintain the spaces in an ansible playbook one note on that is using a good code editor is essential for that and like sublime text which I'm using here I think this is under my face you can't see it but sublime text has a nice little menu down here where you can set your tablet which is behind my face again hold on a second so it has this little it's still behind my face okay it has this little thing here where you can set the tab width for yanil you want that to be two and you wanted to use spaces I also have a setting in sublime so that every whenever I highlight things it shows it shows the spacing and you have two spaces here and you have four spaces here it also shows you extra spaces at the end of a line I have a thing called I think trailing spaces a plugin for that but I'm going to get into next week some ways that you can prevent problems like yellow white space issues and things like that when we're going to talk about ansible testing so here's that playbook that ran I cheated beforehand and I ran the playbook once just to make sure it ran and it did so that playbook was that just two rolls but that that is that is the power and flexibility that you have when you use ansible rolls in the same way to organize your playbooks anything else in this chat that I see here as usual I'll read through the chat afterwards and I'll try to answer some more questions that I've missed in the livestream itself but we had pretty a pretty dense session today and I didn't want to miss anything important and to that end I'm gonna glance at any other notes that I haven't hear I didn't talk too much about variables files and templates but we can get into that as we come across them in later times another thing that that I might talk about next time is getting roles from ansible galaxy so you saw that there were some roles on galaxy and I just kind of magically included them here and they're not located anywhere here I'll show you how to get those maybe next time before we get into testing just because there's a few different ways to manage that and to conclude things today this is chapter 6 again the end of chapter 6 when the only tool you own as a hammer every problem begins to resemble a nail and that's from Abraham Maslow or otherwise known as Maslow's law of the instrument and so with me I have Maslow's hammer right here is my golden hammer and sometimes I can fall into that trap of using ansible for things that ansible is not really that great for and I've done the same thing with many other things in life so I used Drupal for CMS's and I run I've run static websites basically on Drupal which is a horrible idea but that's what happens when you let your golden hammer take over your life and you don't think about things like that but let's see so that's that's all for today we're going to get next week into testing instable playbooks and linting ansible playbooks making sure that you don't have any of those little errors that can slip into your playbooks when you have whitespace issues or yeah malicious things like that and I'll talk a little bit about how I do continuous integration for all my roles and all my play books to make sure that they are always running well but thank you for watching today I'm I'm always so happy to see everybody from around the world coming together for these things and for other for other things that I follow there's a lot of a lot of people who are doing live coding sessions and things and it's fun to see that but please again if you're interested in this stuff if you're interested in learning more about this Raspberry Pi Turing Pi cluster which is using seven Raspberry Pi 3 plus compute modules please click Subscribe below because I'll be having a few videos on that coming out this week and next week and also thanks to the the company Turing PI who sent me this board to test with its it's been very fun I always enjoy learning new things with the Raspberry Pi with ansible with kubernetes with with cluster computing and all that kind of stuff I hope you're having a good day and I will see you again next week same time same place for ansible 101 episode 7

Channel: Jeff Geerling

Views: 26,189

Rating: 4.9875193 out of 5

Keywords: ansible, ansible 101, devops, playbook, automation

Id: JFweg2dUvqM

Length: 62min 30sec (3750 seconds)

Published: Wed Apr 29 2020

Reddit Comments

[u/chetan-97]

Heyyy when Ur planing to go for next ansible live streaming yiur lectures really help me to learn ansible deeply

[u/geerlingguy]

Paging u/nixfu :)

This week's episode will pick up where we left off last week, learning about Ansible Vault—how to keep things secret in Ansible.

Then we'll start looking at includes and roles, and how you can organize playbooks for easier maintenance.

[u/Housecleaner]

Looking forward to this!

[u/Nighteyez07]

Can I just auto upvote your posts? You've been giving so much to the community in this time that frankly it's insane the amount of content you've put out.

[u/chetan-97]

Heyyy how much time took to complete whole ansible series?