Full Ansible Tutorial for Beginners 2021 | Playbooks | Modules | Variables | Start Here

hello everyone and welcome back to the channel with the growing demand for automation i felt there was a need for a comprehensive master class which touches all the sweet spots of the important concepts and don't worry even if you are a beginner or even if you have some kind of experience working in it this series will help you at every step in your goal towards pure automation with ansible if you wish to learn things differently and if you love visualizations then what are we waiting for join me and let's travel through the world of automation with ansible for 2021 so if you're ready let's begin [Music] welcome to the first part of ansible the introduction i know this may not excite many of you but if you wish to have a strong house you need to have a strong foundation and that is the reason why i would advise you to not skip this section and in this session we will be learning about the whats and whys of automation why automation is considered to be a breakthrough in the current technological paradigm why is ansible special and what can we do with ansible so when we think of automation there is an instant idea that pops up about how the things that were previously done manually has been turned into a cycle of events or steps to achieve the same outcome but with less human intervention and as this is rightly put here automation is the technique of making an apparatus a process or a system operate automatically but before we jump into ansible let's meet jenny and let's understand her side of the story so this is jenning she is an independent consultant and jenny starts her day with a strong cup of coffee and she likes to have her coffee especially brewed and prepared by herself every morning so now let's understand how jenny prepares her perfect cup of coffee so she starts with she first measures the amount of coffee beans she needs then she grinds it to a perfect texture then she moves on to have the coffee pressed and has her water exactly boiled at 92 degrees and keeps it for filtering then she adds a dash of milk and pours it to her favorite cup and then enjoys her perfect cup of coffee with her daily newspaper and the preparation time for this is around 40 minutes approximately and due to the time constraints and work pressure jenny wasn't able to prepare her perfect cup of coffee every day even though she was feeling the heat she wasn't willing to give up and for that jenny decided that she will automate the whole process and let's see how did she do that she combined the steps the exact measurement and the steps that she did manually and created a small automation machine or automated machine which would take the list of raw ingredients and would deliver the final product which would give her the taste of the coffee she always had every morning and now she was able to replicate the same steps in just six minutes saving more than 30 minutes of her precious time so what did we understand here so with automation we are not necessarily reinventing the wheel when jenny felt the need for automation she didn't just save time but she was able to achieve much more than what we might have taken into consideration at first with automation she was able to increase the speed at which she was able to process the product now she is saving up to 75 percent of the overall time she is able to make the process more efficient she can change the flavors of the coffee on the go she can change the amount of milk or sugar or how strong the coffee should be and she has much more control over the risk factors that involve where if she has a problem with any individual component she can change it and it will be up and running in no time and it will deliver the same taste every day if we can take this analogy of coffee and translate it to it infrastructure or application deployment just imagine how much impact it would make to the overall process but how can we do that yes of course with ansible so what is ansible so ansible is a radically simple idea automation engine that automates cloud provisioning configuration management application deployment intra service orchestration and many other it needs i realize that these terms may be very intimidating now but don't worry we will together cover these topics in detail as and when we move forward so the most important thing for us is to be clear that that ansible is an open source community project sponsored by red hat and it's tagged to be the simplest way to automate it and you will be glad to know that ansible is written with python powershell shell and rupee thanks to mr michael dehan who was the one who invented or developed the ansible tool which later on went to become ansible works and then ansible incorporation and which went on to be acquired by red hat in october 2015. we spoke a lot about automation but we also need to understand why we need to learn ansible and why ansible has been the technology of choice for a lot of companies so let's meet the devops team of the company and let's understand their use case so sophie john david and kevin work as a part of the devops team they have a task to deploy the application on more than 50 servers and they have to follow a set of protocols to achieve the desired sign-off for the product team or the product deployment team and the steps that they need to perform are first they need to stop the monitoring activity or the service they have to disassociate the instance from the load balancer again they have to stop the service after which they need to deploy the application and check the service status and if that works then they have to associate it back to the load balancer and all of them have to repeat the same steps across all the servers the team was using cell scripts to achieve this but they were unable to achieve the level of automation they needed and that's where ansible could make a difference with ansible the team didn't have to write the custom code to set up all the deployment activities and they could easily write or convert each step into working modules called tasks and using a single playbook they were able to perform the same deployment activity with much more efficiency not only they were able to save time but they were also able to collaborate on tasks that could be optimized or changed later on by just a single or a simple code commit yes these tasks that we have here are composed in what we call as ansible playbook and they act as the description of the desired state of your system which means you can have control of the state of which you want to bring your instances or servers using the playbook with my personal experience i would tell you that in a very short time i am sure you will be able to be very comfortable writing playbooks it's that easy all the repeated steps that you use to perform during the deployment process can be achieved by making them a part of the task in the ansible playbooks and the best part about ansible is it's agentless you don't need to pre-install an ansible agent on your servers in order to use ansible because ansible works with such connections to the servers to perform its actions that's what makes it reliable and secure and equally efficient and ansible doesn't stop at application deployment it also can help us automate our infrastructure the network configuration and also it works with container deployments and also helps us with security and yes it also works with cloud with ansible you can manage physical devices such as bare metal servers with cobbler stack and red hat satellite and network devices such as cisco juniper or aresta and storage devices like netapp and servers such as hp enterprise and for virtualization ansible supports all the leading virtualization platforms such as vmware red hat virtualization or what we call as rhv or zen servers and vagrant and on the operating system side you will be surely happy to know that ansible supports all the widely used operating systems such as linux rel centos fedora ubuntu windows and windows servers unix osx ibm nginx and ibm power systems as well yes i totally understand that network automation is very complex when it comes to device configurations because you need to ensure that you have deployed the current configuration and there will be hundreds of services that could get affected with a single mistake and that is one of the most challenging things with the network engineers and devops engineers as well with ansible you can automate and reduce the complexity of long detailed and complex method of procedures or what we call as mops and the mops are basically the steps that are used as a part of the network deployment and with ansible you can easily test and validate the existing network state and services and have continuous compliance and ansible has a very good setup of network automation with with red hat and simple automation platform which provides the capability of ansible towers which provides all these features that can be used to ensure that you have a proper automation footprint moving on to the containers yes if you are working with docker and container based application deployment ansible works with dockers as well so let's see the example here where we use ansible for total application deployment with ci cd in place so david that we have here commits his code for the new changes he has made to github along with the playbook for docker which is being fed as a part of the ci cd to jenkins post which we have initialized ansible to push the docker image to the docker hub for redundancy and we create the containers using the latest build swansible makes it very easy for us to manage the steps involved in the container deployment as well so this is one of the very basic ideas of using ansible when it comes to dockers and containers so let's come to something that people are really interested nowadays yes it's ansible for the cloud ansible has its core libraries that provide a base that can help us to easily provision instances networks and complete cloud infrastructure wherever you need with ansible you can effectively deploy and manage your infrastructure such as managing servers it helps with cloud native routing and networking with creating virtual private networks access policies and permissions load balancers and auto scaling policies and much more we get support for the most widely used public cloud platforms like aws gcp azure vmware digitalocean and rackspace and for private cloud platforms we have support for we have support for technologies like web faction cloud stack and openstack as well and as we close down on the question at hand that is why ansible let's check these points or these four points that it really boils down to so as we already discussed before ansible lets us automate complex repeatable tasks or steps and deploys applications at ease when you know that you're going to save time with automation you can share the configuration output with your team members making it usable by others and with playbooks it becomes easy to collaborate with other team members working on the different parts of the task which can be combined together and executed at once later on and it's very easy to automate because you don't need to change your existing technology stack that you're using right now and that's a very big relief i think for most of us in today's session we will do some interesting stuff in our journey towards automation with ansible i know the last episode was all about the background and mostly theoretical but in this session there is a lot for you to set up as well along with me so first off we will create some virtual machines in aws and set up the workbench we will learn how we can install ansible and test some basic features and we'll also do some environment setup using ansible inventory so if you're ready let's begin in the previous session we discussed how we were able to minimize the workload of our devops team by using playbooks and ansible and they were able to install and deploy applications across multiple servers using the playbook so to do this what we need is we need to have an ansible master which obviously will be the working station of our devops team members and from which we will try to control the servers where we need to perform the changes or install or deploy the application and that is the biggest advantage of using ansible that is because you don't have to install any agents on the target machines in order to make changes in them you can just use ssh for unix-based machines and remote powershell for windows machines so here our testing environment will be setting up master node and two or three target nodes so that we can simulate a real-time deployment scenario and at the end of this series we will deploy an actual application as well so get ready for that now let's go ahead and create some virtual machines and install ansible and let's have some fun but before that please make sure you have your aws free tier account i have given a link in the description on how you can create one for yourself if you don't have it so that you can make use of it and you can create your own ac2 instances so if you don't have a free tier account please go to the link in the description below and create your own free tier account and then you can create your own ec2 instances so if you are new to aws and if you haven't launched any instances before then what you can do is you can just go to the console here you will find services drop down so you can just click on this and you can click on ec2 or you can just type your ec2 you will get ec2 virtual servers in the cloud so you can just click on this as well so once you have clicked this you will get a dashboard for ec2 so this is the ec2 dashboard so this is the place where all the magic happens when we consider the ec2 instances and if you see here we have the service health that is mentioned for the particular region all servers are operating normally so you're good to go so you can just click on launch instance and what we are going to do is we are going to launch three instances so one will be the master node and the other two will be our target nodes isn't it so we'll choose amazon linux 2 that is free to eligible and i'll select the x86 version just select that click on configure instance details so we can just choose the d2.micro that is because it is a free tier eligible one and then just click on configure instance details here you can just choose a default vpc and you can choose one of the subnets that you want and here you can just enable the public ip assignment that's it you don't have to do anything else just click on storage this is also fine just add tags so we are going to add a tag here so that will be name and this will be our ansible master then click on next so either you can create a separate security group so i've already created a separate security group for myself but that is simple it just allows port 22 so it's not a problem so we can just create one or you can just select an existing one so if you want to create one you can just give a name here and give a description here and you can just allow ssh for 0.0.0.0 0 and you can give a description as well so i'll go ahead and select the existing security group that i have and i'll just click on review and launch that is fine this is fine for me and just click on launch so here it is asking for us to select a key pair or we can create a new keypad as well so what we're going to do is we're going to just if you are new to this then you have to create a new keypair so if you click on create a new keypad you will be allowed to download that keypad but as i already have the keypad with me i will be choosing the existing one so this is the one that i already have ec2 dot key and you have to acknowledge this so what it says is i acknowledge that i have the access to the selected private key ec2 key hyphen pem and that without this file i won't be able to log into my instance so please don't lose this key because this is really important because without this you will not be able to connect to the instance using ssh and this is one of the keys that is really important for us please don't miss out on this or please don't lose this otherwise you have to create all the instances anew isn't it so i will just acknowledge this and i'll just click on launch instance it's still not running so that is why it is not showing me in the list just give it a few minutes then it will come up so a new instance ansible master is ready and it is running now so you can click on this and you will see the public ip address that we have here and that is what we'll use to connect to this instance so once we have this we need two more instances isn't it so what we can do we can just right click on this and if i want to create a similar instance of this one then i can just go to images and templates and i can launch more like this just click on that that is fine just launch use the same key pair just click on launcher type and i'll do this once again because i need to so i'll just launch another instance so i'll change the names of this one so this will be my target one there's target one and this is target two okay so this is my master node and this is my target one and this is my target too now what we need to do is we need to connect to this instance isn't it so remember the place that you have stored your ec2 keypad okay we'll be needing that and before that to connect to this instance i want to install another tool that i wanted to use basically i've already installed that but i'll show you like how i have installed it so so this is the tool that i want you guys to use because i have been using it you can also use mobike's term or putty it depends on you but actually this this i have been using it for a while now so and i felt like this is a very good tool so i just wanted to suggest you that if you feel like mobaxterm works fine for you then you can use that as well to install this you can just go to the site https terminus.com i'll be providing the link also in the description and once you go there you can choose one of the platforms that you are currently working on and just click on the let's suppose i'm working on windows i'll just click on windows and i'll download the exe okay so once you've installed this so once you've installed terminus you will see this interface that you will be greeted with basically what we can do here is we can create multiple connections or multiple ssh connections and we can save them as well so that is one of the best parts that i like so let's suppose i want to connect to instance or ssh to a particular instance what i can just do is i can just click on new host and i'll just add a name here so it'll be my ansible master and the address is so the ansible master just copy the public and just paste it so here next what you can do is you can just mention the username so username will be ec2 hyphen user and password is basically my key so i'll just i treat it if you want to add a key you can just click on plus key and you can just give it uh ec2 different key okay so there is no need to give a passphrase because we haven't provided any pass phrases while creating the key itself so don't need that and just you can browse for the file so this is my key easy to key hyphen pam i can just open that and i'll just i can just save it so you can do the same as well but i have already saved it so i won't do that once again so once you save it and you browse it you will get this option to select the key so you can just click on this and it will be chosen automatically and the next thing that you want to do is if you don't want to use a dark terminal theme i will just choose a basic one because it's very dark for you to see okay so no need to worry about this that's it you can just click on save so once you have saved it you can just double click on this one and add and connect that's it i'll just zoom it so this is our ansible master okay so here is the place where we are going to install our ansible isn't it so what is the thing that we need to do to install ansible we need to run a command okay so what it is sudo yum install and sybil okay so what it is saying for amazon linux 2 we have a separate command that you need to run so that is sudo amazon linux extras because it is present in the extras library so you can just copy this you can just hold ctrl shift c and you can just do a ctrl shift v if you want to copy paste and now just press enter this is very simple to install ansible isn't it everything starts simple and it ends up being a mess but that's what we are trying to avoid isn't it so if you want to check whether the ansible has been installed or not then you can just do a ansible iphone hyphen version so that you can see here we have already installed ansible 2.9.13 but now the main intention for us is to be able to connect to our target nodes using the ansible master that is what we are trying to achieve isn't it so for that as you already know that we have created ec2 instances and we connect to the ec2 instances using ssh using the ec2 key we have to have a set of keys here that we will use to connect to the ec2 instances okay so for that what you need to do is you need to copy the key that you already have with yourself in the desktop that you have and you have to create a copy here in the machine as well on the master node as well so that we can make use of it so what i will do is i will go to the home folder and what i'll do i'll just create a ec2 key here easy to key dot and what i'm going to do is i'm going to copy the contents of my pam file and i'm going to paste it here okay so if you want to create file you need to use a vim command and bim space the file name that you want to create just press i once you have copied the contents of the key then just do a ctrl shift v and once you have pasted it just press escape and colon wq that's it and now you have to change the permission of the file so chmod 400 ec2 hyphen key dot them that's it so just now we saw how we can create our ansible master and the target nodes but in real time you may not be working with only two or three servers isn't it you will be dealing with different services and a huge number of instances at once so for that we won't sit and type each command and wait for it to happen we will have to improvise isn't it so if you see here we have so if you see here we might have instances which belong to the web service deployment and a few other for the database the configurations will be different for both in most of the cases but we need to automate them in such a way that all the tasks that are targeted to be performed in the web servers should be a part of the list and the same goes with the database servers as well so imagine that your inventory looks like this so this is how the devops inventory might look like if you're using excel sheets to keep track of the instances so if you see here we have target 1 target 2 and target 5 with web servers and other two as a part of the db with different connection types and some using passwords and others using ssh key and the benefit of using an inventory file is that you can keep all the targets that you have under one roof that is under one file and execute them as per your need same way we can see here if you see the inventory file that we have inside which we have categorized the targets with a definition within the square brackets that you see if you have used ini files before you might be aware of this syntax which we use to write the configuration files so here we have the web servers where we have target a.hello.com and targetd.hello.com and for db servers we have targetc.hello.com and targetf.hello.com and if you want to perform any operation or ping to targets at web server then what you can do is you can just type this command by giving the tag name let's suppose we have here ansible space web servers that is the tag name hyphen m that is the module name so that is ping that we are going to use to ping the devices hyphen i the inventory file name or the path of the inventory file and the same goes for the dv servers as well and if suppose you want to perform any tasks that are common for both then what can i do i can just keep the separate tag with a different name called common or anything that you want and add all the servers to that list that we have done with common here isn't it and i can execute the same command like ansible space common which is my tag name hyphen m ping hyphen i inventory dot txt but along with this ping what i can do i can execute a playbook as well isn't it so i can perform each operation based on my tag and my inventory file which will point to specific servers intended in the tag name so let's suppose i want to perform any thing specific to web servers i can give the tag name as web servers and if i specify the inventory file it will go to that particular inventory file and check all the servers name listed under that tag name and it will execute the operations on those particular servers itself and not on others i hope you got the point so once we have this what we have to do is we have to create an inventory file so just like the way i showed you here on the way we can create the inventory files and we can store the data as a part of the inventory system that we have so the same thing we will do for the targets that we have created so we will make sure we have the host name we have the ip address we have the user name the connection type that we want and the key from which we can connect to okay so that is what we will create in the part of the inventory file and i'll show you how you can create one for yourself and if you're doing this along with me make sure you pause the video whenever you need and you can catch up with me whenever you want so let's go back and create the inventory file so go to the terminal once again and just to ls here so here we have easy to hyphen key dot pam file but if i want to create an inventory file i just need to do a vim and inventory so this is the file name that i want to give then just create a text file inventory dot pxt and press enter and this is the blank slate that we have here is the place where we can create the inventory file so i'll just press i which will give me the insert mode and here i'll write my inventory details so the first one that i have is enhancible hyphen target f11 so this is my target one the host name space and simple hyphen host is equal to so here i need to paste the ip address of my host name that is for the target one so i'll go back and i'll copy the target one hostname so this is the public ip address that i have i'll just copy this and i'll just paste it and the same way i'll just type the connection method so ansible hyphen or console connection which is equal to ssh and space and symbol underscore user is equal to easy to hyphen user and the same way what i can do is i can just copy this and i can just paste it again and i can change the ip address that i have for the target too i'll just copy the public ip address of the target too and i'll just paste it here and i'll change the name of the hostname as well and you can just click on escape okay so once we have created the inventory what we can do is we can just type ansible ansible space and symbol hyphen target target iphone 1 hyphen m ping so that is the ping module that we have so iphone m we can provide the parameter for the module that we want so basically we want to ping to the particular instance so i can just type hyphen m space ping hyphen i this is basically the extension to provide the inventory file name so my inventory file name is inventory dot txt and i can just press enter so here it is asking that are you sure you want to continue connecting so this is a manual authentication so here there is a problem that we are not able to connect to the instance because the permission is denied so you remember that we added the key here but we have not assigned it to the search agent so that is one of the things that you need to remember only copying the key here is not sufficient you have to add it to the ssh agent as well so for that your command is ssh hyphen agent space bash and what you need to do here you need to just copy this file that you have copy ec2 to this path ssh dot ssh and then what you need to do is ssh hyphen add you need to add it the path that we have dot ssh slash ec2 hyphen key so we need to add that to the sh agent then just press enter so once you have successfully added it you will get the message here identity added and that is basically your key okay so once you have done this you can just again type the command isn't it i'll just copy this once again and i'll just paste it and just press enter so now you are able to connect to this one so here we have resolved one issue that is for the permission denied ratio that is basically for your ec2 key or the connection key or the sh key and next one what we need to do is we need to connect to other instances isn't it so target 2 so let's connect to that just change this to 2 from 1 to 2 then just press enter so the problem here is it is always trying to connect using a manual intervention so if i just press no it will not be able to connect because the host key verification has failed so here the problem is that there is a setting that you need to make either you can do a export of export of ansible underscore host underscore key underscore checking equal to false you can make this as false so this is one thing that you can do or else what you can do is you can just go to you can do a cat i'll just show you one path that is that you have for the ansible so slash etc slash ansible slash and sybil dot cfg so you can just go there and here you will have an entry as well this entry uncommenting this will disable sh host key checking so this you can disable it as well so what we're going to do we're just going to disable this in the configuration file so i'll just do a whim but as you need to do a pseudo game here don't worry this needs some root level permission so just do sudo vim slash etc slash ansible slash ansible.cfg then just press enter just come down here and what you can do is you can just press i and go to the insert mode and you can just delete this hash now just click on escape and cool in the blue queue that's it try running your command once again yes it is successfully connected but the most important thing that we wanted to do was having to work on multiple machines at the same time isn't it and just now as we already discussed we'll do some changes to the inventory file and we'll see the same being done here as well so vim inventory.ext what i can do is i can just give i can just give a name here so this will be my servers okay i'll just give this a name of servers let's see if you are able to execute the ping command on both of them by using this tag name just press on escape and just click on wq and just save it make some changes to the command so here what i'm going to do i'm just going to specify the tag name so what is the tag name that we have servers isn't it so we'll just type servers and we'll just hit enter yes you can see we are able to connect to both the instances at once and the same way even if you mention like 50 servers or anything it depends on you based on your requirement you will be able to connect to these or you will be able to make changes to these servers or the machines that you have and that is pretty interesting and pretty amazing isn't it the way we are able to automate things and that is what we have learnt now and i want you to practice on this one and try as many hidden trials as you want and if you have any doubts or if you are facing any problems then you can put them in the comment section below as well so mostly the problem i think people might face is due to the connection issues so if you face that or if you are not able to install ansible or anything that you have so any problems regarding that as well you can post them in the comments section below and we can discuss further on them and i'm sure that others who have found a resolution for your issue they will be also able to help you and there is a lot in today's session for us to cover so sit back relax and let's begin now and today's session we will be learning what is yaml and how we can simplify the learning process so that we can understand this concept better and we'll also look at what are the better options that we have for the ides available for us for writing yaml files or yaml scripts so the playbooks that we write in ansible are a combination of tasks that we want to accomplish or execute on the servers or the virtual machines but the playbook itself is a written set of instructions isn't it and we write a playbook with yaml so yaml is the whole identity of your playbook as and as every language or script has a file name or an extension yaml also has its extension as yml what do you say as yml or yaml and yaml that is playbook.yml we pronounce it as yaml even though it is spelled as yml or yaml and the steps that we write in our playbook can be executed across multiple servers with the help of the inventory file like what we saw in the last episode as well so remember that the extension for the yaml files will be yml or yaml and we pronounce it as yaml so whenever you talk to someone or you want to communicate with someone about sharing the file remember that you have to say it in a way like playbook.cml file you're referring to a yaml file isn't it so let's move on then so now let's understand how we can use yaml for that we need to understand jenny's perfect cup of coffee once again so to make her coffee she needs the raw ingredients and the predefined steps like grinding boiling filtering etc and all these have a perfect sense of becoming a type of data structure like a list or a dictionary so for now keep your mind totally away from all the programming languages and computational terms just imagine now that we have come here for shopping and for that we need a list of ingredients isn't it so here we have a raw materials list our list we have coffee flavoring milk water and sugar and we have a key and a value mapping of our ingredients and the brands that we need to purchase so these type of data structures are called dictionaries isn't it so in a dictionary remember that you will have a key that will hold a specific value here we have a specific brand attached to each of the ingredients as you can see here like coffee has beets we are going for a flavor that is hazelnut and we are going for milk that is a brand of beans and we are going to have a normal water and the sugar brand that we have here is domino and then once we have the list of ingredients we have another list with the procedure or steps or what we call as processes so that is a process list so we have grinding pressing boiling filtering mixing etc so these form a list of process and when we combine all these data sets we now get the actual pseudo steps for making the coffee and now we have a combined data set of a single dictionary of process that becomes a list of process so when you think of computation we have to remember that a list is a combination of heterogeneous elements so it could be a combination of strings or integers or it could be both of them and a dictionary is a key value pair where we have an index field which we call as key which holds a specific value and it could be a string or it could be a list or a dictionary as well so remember these points very carefully i know most of you are well aware of these concepts but we have to discuss this because we might have some of our friends who may not be aware of these data types so remember this very carefully so here you can see the raw materials is a type of list and the process is also a list but here the raw materials and brand become a dictionary and the key that you have for coffee will be holding a value with the brand name and similarly this will be a key and this will be a list of values and this process will be another key and these will be at the list of values and similar to that when it comes to grinding it will be a dictionary attached to a particular key which will have a nested dictionary where coffee will be one of the keys and pizza will be one of the values and similar to that we have boiling and mixing so if you see we have the list which is raw materials and raw materials is our key so as you can see the raw materials that you see here is our key and the values that you see here become the list of values and if we have used and if you have already used json before which is javascript object notation and it's more like a dictionary and a list in json is basically represented or enclosed with square brackets so if you see here we have the square brackets within which within which we have the list of values like coffee flavor milk water sugar so in adjacent as you can remember that we have a little key and if the value is a list then it will be enclosed within square brackets and the way we represent it in yaml is basically you will have the key but instead of a square bracket here the list will be represented by a hyphen so any item that has a hyphen will be enclosed within the key and will be formed as a list and i can show you an example of this as well where we can watch the real-time conversion of json to yaml and you will see how a dictionary type is represented in yaml as well so there are a lot of sites that you can go and visit check how we can convert the json structure to yaml and you can experiment them yourself as well so this is one of the sites called json2yaml.com here if i just paste the raw material dictionary that i had and when it gets converted to yaml you can see this is the key and this becomes the value and this is basically your list of values and if suppose i want to add a new value in the list of the yaml data structure then i can just have add a hyphen and i can just add another ingredient suppose i want to add cream so now the cream actually gets added as a value in the list of values that we already have if i want to remove these two i can just remove them and they will get removed in the json as well so this is how you represent a yaml structure okay and remember that the spacing and indentation is very important and that is what we will discuss next and similarly we have another list called process where we have the grinding pressing boiling filtering and mixing and same way we have a dictionary where we have a key as a process and the list of processes and here as well we can see the yaml representation states that as process as a key and these are the values which are represented with hyphen next off if we want to have a bigger and better data structure here so as we already know the example that we have seen before like we have the ingredient process grinding boiling mixing and these have the nested dictionaries as well so these become a whole data structure and this is the data structure that we have so we have the ingredients key which has the list of values process which has the list of values for processes and then we have the grinding which i already told you will be a dictionary which has coffee as the key and beats as a value so this becomes a nested dictionary for us and here as well you have mixing as a key and which has a dictionary and milk becomes the key and true is a value suppose i want to have milk but i don't want to have sugar then i can disable the field by using a boolean value such as false and what is the number of cups that i want i want one so i can make a coffee based on the desired value that i have this is just a structural representation of how a json looks like and the same way when it comes to yaml as you can see here we have the list which is represented by hyphen but when it comes to nested dictionaries you have to remember that you have the key here and the next key also will start with a proper indentation but there will be no hyphen because this is not a list this is the nested dictionary so once you have added the first key then you just need to go to the next line add a space then you can just give the key name as coffee same way we have colon here you can add the colon and add the value for yourself and we can see this in the yaml converter as well so let's go there and check that out so i've just copied the values we had and i'll just paste it so as you can see here this dictionary has been converted where the list of values have been represented by hyphen and here where we had the next dictionary it has been represented by the key and the key again starts here and this is the value and this whole thing becomes a next dictionary okay so if suppose i want to add another key here i can just add it like hello colon anything any value okay so this is how you add it so if suppose i want to remove anything then i can just delete it and it'll get removed but remember one thing very carefully the best way to validate your yaml is to convert it to a json and it should be a perfect json then only you will get to know that yes your yaml is properly validated you already have some websites that can help you validate your yaml but your yaml should be a perfect representation of the json structure but that doesn't guarantee that it will be perfect in the sense of execution because there may be indentation errors which might hamper and which might cause a rift in the way the json structure is represented in which the json structure is represented so we will check that next as i told you before that we will be understanding why the spacing and indentation is very important for us in yaml so let's suppose we have this ingredient and we want these three to be the list of values isn't it so i'll just expand this so these three are my list of values so coffee flavor and milk are the ingredients and here as you can see milk milk has three independent values which makes milk the dictionary or another dictionary within the list and which has domino's that's a brand and the low fat category and it is toned okay so these are three properties of milk not the ingredient itself so that is why we have ingredients as the main key and these three are the list of values within which we have another milk which becomes the key here and basically which has three values in its own list so let's suppose i will show you this in the yaml converter as well and you can see that very easily as well so let's suppose i paste it here so as you can see here this ingredient has three values one two and three so this is one of the values there's a second value and this is the third value which becomes the key for us because this becomes a nested dictionary within itself so milk has additional properties or the metadata it has so milk has three additional properties and let's suppose there may be chances where you have messed up with the indentation so here there can be chances where you have messed up with the indentation here so this blank space that you see should be common for three of them otherwise there will be a lot of problem we'll see why so let's suppose you messed up with the indentation and you did not provide any spaces here so what happened is the toned value that was already present here got shifted to ingredients but toned is not a value that should be present in ingredients isn't it that is the main reason why indentation is very important for us see if we missed out on giving a space here the value that should be present under milk went directly under the ingredients and this is just a funny way of representing an example but if suppose that is a very huge configuration then you will be cracking your head because you will not be able to find the exact error because that will not show as a part of the error itself because that is a perfectly fine yaml but it does not represent the actual configuration isn't it so let's take this example and we'll paste it here so if you see here now what happened is the value that was toned that was inside milk in the part of the list of milk the metadata for the milk is now outside of that and has become a part of ingredients just because i did not add a space here so let's suppose i add the space here it will get inside the list and this is the dictionary that i want now let's suppose i just remove these two spaces as well so now what happened is these two values that were a part of the list inside milk are now a part of the ingredients list which is not the perfect configuration for us so we have to be very clear and very careful about the spacing and inundation isn't it that is what we saw right now and similarly you can have boolean values as i already showed you before so you can have yes or no and you can have a true or all caps true or false and these are the boolean values that you can use and there are two ways that yaml tells us that we can use the multiple lines so the first one is literal block scalar so in little box scale what happens is it preserves the integrity of the backslash n or the new line but in folded block scalar you don't have it it will just consider this as a single line if you don't believe me let's go ahead and see that example as well so here actually it is trying to preserve the slashing this is called literal block scale and when it comes to folded block scalar here it does not preserve the backslash n or the new line as you can see there are no new lines that have been preserved here so that is the main difference so the literal block scalar will be represented by the pipe symbol as you can see here and the folded block scalar will be represented by the greater than symbol the next one is also very important the comment section so just like python you can place a hash and you can write the comments as well if i go back and if i have to write a comment here i can just add a comment like by giving us hash that space hello world comment so this is the way i can add a comment as well and just like strings we can make use of quotes as you know that there are two types of codes that we use so we have single quoted strings and we have double quoted strings and the difference between the single quotes and the double quotes is that in the double quotes you can use escapes or escape characters like slash s slash d and there is a list of allowed escapes that can be found in the yaml specification for 1.1 and 1.2 so as you can see we have the sample text hello slash t hello within the single quotes and this is within the double quotes so if you want to see the difference so as you can see once it is converted to json you can see it is carefully accepting the escape character so so whenever you are planning to use a escape character then make sure that you have your strings enclosed within double quotes and the very next example that we have here is also very important as part of the concept that is called expressions in yaml so here in yaml we can replace a value in a variable enclosed within double curly braces in the runtime so if you have used angular or javascript or any templating languages you might be very well aware of this so the example that you see here the description the file belongs to variable which is enclosed within double curly braces and user so this value that variable that you see here will be replaced in the runtime so let's suppose you want to assign a username to this so when you're printing this uh when you're printing this line what happens here is you will get like this file belongs to xyz user so that will be replaced in the runtime itself so this variable that you see it you can assign the value of a username and it will be dynamically populated during the run time of your playbook handy isn't it and you can also create a dictionary within yaml with this syntax as well so this syntax as well enamel if the value actually starts with a single curly bracket or a single curly brace and has a key and ends with a curly brace it actually represents itself as a dictionary so if you want to test it out also we can do that so if you see if i printed it here if i pasted it here so this becomes the key for the ml code and here this becomes the key and this becomes a value so whenever it starts with a simple curly brace anything after this which has a colon becomes the key and this becomes a value and you might think what if there is a space in between the characters so if i do a world space hello then the world space hello actually becomes the key of the dictionary and what if we have blank spaces or what we what if we have special characters so it gets rejected basically because we don't have any quotes here so let's suppose if i want to give codes here can i give it yes you can give and that actually will support the escape character so just now we saw how we can actually make this and that does not affect whether you have spaces in between or not it will consider it as a single string so remember that very carefully so don't get confused when you have anything regarding this okay so i hope you got a very brief idea on how we can use yaml and this is not the session where we are going to discuss about how we can create playbooks i just wanted to make a short session on how we can use yaml and that is what we will be using to create the playbook so it will be easy for users who don't have that much of exposure to yaml so that they can also learn it they can also practice it and when we move on to the critical stuff like creating playbooks it will be way easier for them as well and for myself as well i'll also get a refresher isn't it and there are multiple ides that are very useful for us there are already available to create playbooks one of my favorite ideas are mentioned here so atom sublime text pie charm or visual studio code you can use any one of them so i will be using atom so if i want to install atom i just need to go to them i can just google atom so the first thing that you will see here is basically item.io so this is the site that we need to go and we can download the atom ide from here so i am currently using windows so if you are using any other operating system then you will be able to install it for your operating system as well so here you can see we have the windows 64 bit for windows 7 or later and there's the download button so what we do when we see the download button we click it isn't it so just click on this and just save it so once you've downloaded this just click on this and open it it's as simple as it can get so just once you've opened it let's wait for some time for it to open yeah so item is currently being installed it'll launch once it is done so once you've installed atom this is the basic first screen that you will see you can just close the unnecessary guidelines that you have welcome guide so there's a welcome page i don't need anything so you have add folders and reopen a project so currently i'm not going to do anything because i don't have any current projects right now so this is the basic id that we have if you want to create any file you can just click on new file and so let's suppose i have this ingredients with me or the text with me that i want to add as a part of the yaml script so i can just copy this and i can paste it and i can just save it i can save it to my folder here so this will be my test dot yaml and i can just save it so this is my yaml file if you want to install additional packages what you can do is you can just go to file then go to settings here you have install you can just click on this and you can search for packages so if i want to install anything specific to yaml what i can just do is i can search for that so so here it actually gives me a lot of options to install plugins for yaml as well so let's suppose this is one of the very popular yaml ide plugins so i can use it so item id support for yaml language so i can just click on install here and i can install this as well so now it has installed right now so there is one more thing that i wanted to install that was linda jcml so for the indentation and validation so what i can do is i can just install this it's very simple to install plugins in atom because you have to just click one button to install a plugin so i don't think so there will be a problem here for you linda jcml needs to install dependencies linter so install dependency yes okay i'll disable the diagnostic and i'll go back to the script once again what is saying unexpected error so yaml lint is not installed so if you don't have a yaml lint what you can do is you can just install pip install yaml lint that's it see now it is perfectly running right now so if suppose you get that error what you need to do is you just need to install yaml lint using a pip command for your python build that you have so here what you can see here that this actually gives us a lot of flexibility when you're writing playbooks that may be a gruesome task for you to actually remember all the indentation and things so this will actually help you to correct your indentations and actually perform your coding actually really better so the document actually needs to start with hyphen knife and hyphen what i can do is i can just add hyphen hyphen hyphen and just save it and yeah great column starts greater than line okay indentation is two but we found zero so what we can do is we can just provide two indentations for everything i just save it yeah so now this is corrected and this also needs a indentation so yeah now the only error that we have is wrong new line character expected slash n yeah this should be fine right now i think there was some error so i think that's fine now so we have corrected all the errors that we have so that is the basic idea of using an ide and using the plugins that are very helpful for us to minimize the workload that we already have that we are already going to do so just go to settings and click on install and install the plugin that you want and if suppose you want to revisit all the plugins that you have installed you can just go to packages and you can see what all you have installed so i have just installed ideml and atom id ui and linter.js so this linterjs actually will tell me what are the mistakes that i'm making so that will be very handy for you as well so i thought this will be a very good session for us to actually reflect upon what is the id that we are going to use for our series so atom is the id that i'm going to use so let's suppose you want to use it then you can use it as well or else if you want to use sublime text your wish you can use it or else if you want to use pycharm or if you want to use visual studio code it's up to you whatever floats your boat you can use them okay and in today's session we are going to discuss about a very important concept that is ansible modules so we will talk about what is an ansible module why do we need to use ansible modules what are the important modules that we must learn and understanding how to use an existing module all the timelines are given in the description below so before telling you what is an ansible module i just want to tell you that the main use of ansible list is to automate execution steps installing applications copying scripts executing commands and a lot more so in order to do this you need a mechanism which can help you with the configuration that you can use to achieve the desired automation that you want for that ansible provides you with plugins or libraries that you can use to get your job done and that is what is called your modules you can make use of the existing modules that are already available enhanceable and yes you can create your own modules or own custom modules as well using ansible so modules that are referred to as task plugins or library plugins are discrete unit of code remember this discrete units of code that can be used from the command line or in a playbook task discrete in the sense each piece of code is able to perform a function for you so that you get your automation in place and ansible executes each module on the remote target node it may be one or hundreds of nodes or thousands of nodes it depends on your requirement but it executes each module on the remote target and it collects return values which is the standard output or error let's take this example here so let's take this example here ansible pattern goes here hyphen m module name hyphenate the arguments so the pattern could be your host details that you add in etc and sybil hosts and hyphen m is where you specify the module name that you want to use and then for the arguments you use the hyphen a and then you pass the arguments so a pattern usually refers to a set of groups which are set of force in the above case that we see here it's the collection of hosts that we have as a part of the web servers group so let's suppose you want to restart the apache httpd service in all cases for that our ansible command or cli command will be ansible space web servers that is our patent hyphen m service so service is a module name and hyphen a the argument which is name equals httpd states equal to restarted or state equal to restarted and what this does is that it executes this cli operation on all the host present in the web server's host list as you can see i have already mentioned a patent usually refers to a set of groups which are sets of hosts in our case the machines that you see here are a part of the group of web servers let's see another example that is point b ansible web servers hyphen m space ping here we are doing a ping operation on the hosts in web servers host group so we are using the ping module here let's see another example that is point c so ansible space web servers hyphen m command hyphen a the operation that we want to perform that is rebooting the particular instance so here as you can see we have the web servers at the pattern that is the host group and hyphen m command so command is a module name and hyphen a is where we pass the argument so here we are executing a command on our remote host to reboot that host so to execute a command on the host machine we are making use of the command module there is another module called shell which you can also use to execute your commands but we will discuss on that in a bit talking about arguments the arguments remember these points very carefully so most modules take key equals to value arguments or space delimited and some modules take no argument and command or shell module simply take the string of the command you want to run so as you can see in the point c the command module takes a string that is basically a command to execute it on the host so imagine this use case right now if you had to deploy the service or web application server on 50 hosts and you are using ansible to do that this becomes very easy because you can do that using a single playbook you make the list of host as a part of your host group and you write your playbook or else you can execute simple commands as we saw to all the hosts at once so the same steps can be replicated over several servers with the same configuration remember one thing ansible actually executes each task on all the hosts and then moves to the next task so you have to be precise on the way your playbook is written so if you see here the stop monitoring task will be executed on all the host then it will move to the remove from load balancer and it will execute it all the host and then it will move to the third step then to the fourth step then it will go to the last step so it is not that it will execute all the step on one host and that will move to the next one it executes the operation sequentially on all the hosts now let's see some of the conversions from the cli to the playbook style so playbook style it means that gamble format so forgive me for saying all these junky stuff but yeah the first point a that you see here so ansible web servers hyphen m service hyphen a name equals httpd state equals restarted so if you want to write it on the playbook it will be written using yaml so the syntax would be so you give hyphen name and you give the name of the task that you want to perform so that is restart web server that is not by ansible you can give any name to this one but you have to be able to recognize this once the playbook runs so please make sure that you give a very recognizable string there and then you give the command name that is service that we already know and then you provide your arguments that are name and state so name and state are the default properties that you have to give and you have to provide the name of the service and the state of the particular service that you want to see service is the module name the service name that you want to restart is httpd or it can be nginx or any service that you want and state is a variable that you want the state to change to so if it restarts it will return 2. the second point be ansible web servers hyphen m command hyphen a and the string to reboot it so there are multiple ways to do this the first one is giving the name reboot the servers then you can provide action colon the module name and you can provide the string of the command that you want to execute here action that you see works in conjunction with the module so it has action colon module options format but it is recommended that you use the more conventional module colon option format so remember that very carefully so like the one we have below here the name equals the name of the string that you want to give and the command that is the module coolant the string that you want to provide so this is also very simple way to do this and there are two ways so you can choose whatever you want but it depends on the recommendation that ansible provides so the second one is the option that ansible actually recommends so make sure that you use that so if you were thinking oh what modules will ansible provide just imagine the quantity of ansible modules that we have here so these are the modules that ansible provides and these are just and these are just high level modules and within each modules we have an individual module that you can make use of imagine having most of the plugins available to you for your automation that's just awesome isn't it and that is why ansible is so popular because it has a very good documentation and it has all the modules that you need for the most highly most highly used ci cd operations so it is a very good tool to use so we have the cloud modules we have the crypto module we have the very famous clustering module as well where you can use kubernetes and we have the command module which most of the people use in general and their daily work as well and we have the network module we have the messaging module and we have the web infrastructure module as well you have the utilities module you have the notification module and there are lot many more of these modules that you can use so let's see some of the modules first let's start with the cloud modules so ansible provides cloud modules that can help you to create infrastructure like provisioning hosts installing applications posting cloud services on multiple cloud platforms for example our very favorite aws so here we have a playbook which helps us to create the ec2 instance another one is to integrate ec2 with cloudwatch so we can enable monitoring as well using this and we have a few cloud platforms that i have mentioned that ansible provides modules for that you can use so there is alibaba cloud you have aws you have atomic you have heroku you have azure you have google your rackspace and much more and you can go to the link that i provided in the documentation below like docs.ansible.com and you can just search modules by category and you can see all these documentations as well the next one is if you wish to use ansible for container deployment using kubernetes you can do that as well so you can perform all operations like creating namespace creating a deployment file service file persistent volume provisioning in kubernetes using ansible now let's take a module and discuss how you can make use of it and how you should read the documentation each module that you have seen have their own individual modules that you can make use of in your ansible command or configuration that you write as a part of your playbook one of those modules that is most frequently used is the command module if you want to learn more about modules please visit the link that i have provided below so that you get more information about the different modules that we have used here here we have seven modules within the command modules and these are the ones that we might be using more frequently in our daily work so understand these very carefully or very clearly the first one that you have is the command module so command module takes the command name followed by a list of space delimited arguments that command will be executed on all the selected nodes so next one is expect module that executes a command and it will wait for the prompt to execute the information provided so it's like let's suppose the prompt might ask you for the password so you can respond to that using the expect module the next one is p s e x e c or partial execution so it runs a remote command from a line sourced to a windows host without bin rm being set up so you can execute remote commands or partial commands using the linux host to your windows host the fourth one is raw you can run dirty ssh commands using the module and let's suppose the host isn't having python installed you can write a raw command to install that using a single cli command the fifth one is script so this is very important let's suppose you want to execute shell script or python file or any script you can do that using the script module just that you need to provide the path of the script the sixth one that we have here is shell so shell is similar to command module but runs the command through a shell or that is slash pin slash sh on the remote node whereas the command module actually bypasses the shell for remote targets we use the win underscore shell module instead the last one is telnet it executes a low down and dirty telnet command not going through the module subsystem so what does that mean is that when you want to log into routers and if you want to run show commands you can make use of this telnet module you can provide the username and password and wait for the prompt as well so that is what we generally do using the telnet commands isn't it if you don't have sh you do telnet and once you have the telnet access you just install ssh onto that and then you make use of ssh so these are some other things that you can do with the telnet command as well but before that we can just take a step back and i just want to tell you that if you want to perform any operation you need modules but how will you use them it's not that all the modules are the same so they will have different parameters different syntax to be configured isn't it so when you go through the documentation for the modules you must understand three things first the example set or the example playbook the second thing is command parameters the third one is return values so the command parameters are the scope defined which you will use to perform operations within a particular module so let's suppose you want to change the directory you can use a chdir and if you want to see next to chdir it has mentioned the type of the input that it needs that is the path so you need to provide a path here so let's suppose you want to execute a command you can make use of cmd which takes a string as a parameter the same way we have the return values so command parameter here returns the list of commands executed so let's suppose you provide um install nginx so it will return the list yum comma install comma and genex so it will return a list to you so for you the most important thing is you need to remember or you need to read them in the documentation before using them in your code so remember command parameters and return values if you see the parameters here the same will be available for you in the documentation and you can use one of these parameters along with the module that you have so let's suppose we have command command here we are trying to execute the command here so this is basically our cmd it will take so here we are not using cmd but if but if i want to use a parameter i can type the command or the module here then i can use the parameter that will be my cmd and i can provide the string that i want to execute and the second thing that you see here creates and it has given the path so what it does is the file name or global path if it already exists this step won't be run so let's suppose you have a condition where when you execute this make underscore database.sh this creates the path to database so if the condition is that if this path has already been created you don't need to execute it once again because sometimes because sometimes when you execute operation multiple times it can have negative impact on the environment that you have so you don't want to do that isn't it so to make this clear for the command that it should not execute this once again you can just provide a creates here and if suppose this path is being created when it executes the command for the first time you don't need to run it so similarly you have to read through all these things all the parameters that you want to have and you can read the comments here to get more information about what it exactly does and you have to make the choice of whether you're going to use it or not so now let's take a look at the command module so remember the command module takes the command name followed by a list of space delimited arguments and you can use it in the cli or in the playbook and the command will be executed on all the nodes that are part of the host group and there are two ways here to execute the commands you can use either a freeform command or you can make use of the command parameter or cmd parameter that i told you before so if you want to execute on windows targets we make use of the win underscore command module instead so rather than command we will use win underscore command so as you can see if i have to execute this i can write it in a playbook myplaybook.yaml and based on all the host groups that i have like system a system b system c systemd if i am trying to execute it from my master ansible master it will get executed on all the nodes the next one is expect module so the expect module executes a command and responds to prompts as we discussed before so as you know ansible is built on python we will need to have additional packages like p expect as the prerequisite so you have to have python that is greater than 2.6 and we expect module that has a version 3.3 or more so here we are dealing with the prompt by using the response parameter so here you are expecting the response to be a password as per the string that is mentioned here or else you can use the same to respond to multiple choices of the prompt using just like a switch case the next one third one is when you might have a requirement for executing commands remotely on a windows machine so mostly we might make use of smb or winrm to do this but let's suppose the host does not have these installed we can make use of the ps exec module that is more like partial execution module so it is dependent on the modules pyps exec and smb protocol to be installed on the master not on the host remember that it should be installed on the master so that execute so that ansible can perform the execution so to connect to a windows host what do we need so for us to connect to the remote machine we need the connection parameters isn't it so it will be the host name it will be the connection username it will be the connection password it will be executable on what we are trying to execute it so far as it will be partial.exe and what are the arguments that we want to execute so write host hello world so that is what we are going to perform here so here once you run the partial.dxe you pass the argument that is right host hello world which is to be executed and if you wish to run it asynchronously then you can use the asynchronous value as true or yes so here if you see asynchronous is one of the parameters connection password is one of the parameters connection timeout is there connection username is there executable is there that we have used here and the hostname as well so what does asynchronous do so we'll run the command as a detach process and the module returns immediately after starting the process while the process continues to run in the background so if suppose that is going to take a long amount of time or it is going to take a huge amount of time then you can just make it asynchronous so that your callback just returns back once it has executed the command and the process that you want to execute will still continue running in the background so next up is raw so it executes a low down and dirty sh command not going through the module subsystem so what ansible tells us is that this should only be done in two cases so first case is when you're trying to install simple json on older hosts that it needs as a dependency or to run modules so as all the modules actually as all the core models actually need it you have to install it so if that is one of the requirements then you can use this raw module and the second one is that if you are speaking to any device such as routers that do not have bison install that do not have python installed then you can do that as well in any other case using the shell or command module is more appropriate so let's suppose you are bootstrapping your machine with python you can just write the command as a string and raw and it will run directly through the configured remote shell and you can change the shell used to execute the command as well so you can provide an absolute path to the executable as a parameter like slash pin slash sh or slash bin slash bash and run the commands using that the next one is also very important which is script so script module helps you to run a local script on a remote node after transferring it so what it does is that it will copy it to a remote template location and it will execute the cell script on the host machine the script module takes the script name followed by a list of space delimited arguments so if your script needs some parameters you can provide them using the argument list that you have so this module is also supported for windows target so you can use it for windows machines as well so if i have to put it in very simple terms if you wish to execute a shell script just use this module so you can define the script module and we have the command parameter as well so you can assign the command parameter and you can ascend the path of the shell script and you can provide some arguments as well so if you see the parameter actually creates the created file.txt so what it means is that once the cell script executes it will create this file and the next time it executes the same command in the playbook and if that file already exists on that path it will not rerun this command so as i told you before about the database example so this is the same way that happens when you use creates the last but not the least telnet so as i already told you before if you wish to tell it to a device and you want to simulate the prompt and execute commands based on the response behavior you can make use of the telnet module so it executes a low down and dirty telnet command not going through the module subsystem that we already know so this is mostly to be used for enabling ssh on devices that only have telnet enabled by default so let's see an example here for sending commands to cisco ios devices this takes me back to my old cisco days but let's just come back to 2021 once again so name equals to the send configuration command so ios so this is the string so we have a telnet command that we are using so telnet module so within which we have parameters like user password login prompt and password prompt so you can pass the username here in the user argument or parameter you can pass the password here in the password parameter login prompt is basically that is what you are expecting from the login prompt where it will ask for you for the username so this is what you are expecting from the device so it will ask you like this username colon it will be the string and for password it will ask you like this pass colon so it can be different based on the device that you have so once you've logged in you are expecting this prompt and this is the most important thing for us to get because if you don't get this prompt you will not be able to execute ios commands or show commands so once you have done this you have one more argument called or one more parameter sorry you can use command and that you can pass the list of commands that you want so terminal length 0 i'm not sure how many of you know this but this is something that will extend the terminal length so that you get all the output at once so it will not give you that more option below so that you have to click enter to get more data so it will give you the whole list of output that you want and you have configure terminal where you can configure the terminal and host name ios this is one of the command so you can execute these three things or depending on your use case you can execute hundreds of commands onto that machine so make sure that you read the parameters list here and you should also read the comments here so that you know what exactly is the use for this one so that it'll be very easy for you to consume this hello everyone and in today's session we'll do a small hands-on demo for ansible modules so imagine you get the task to install nginx or setup nginx on 50 machines so as we are working with ansible we are quite sure this can be automated isn't it so let's see our test bench here first things first we have our ansible master and we have two hosts in our host group and these machines are basically your amazon linux two instances and if you haven't seen the session where i've explained about how you can set up ansible or how you can install ansible and set up your testing environment click on the link in the description or else you have a link on the top right corner now so watch that video before moving forward so having said that the next step is to write the playbook obviously which will have the following tasks so install the latest version of nginx start nginx enable nginx ensure that nginx is at the latest version by gripping the value of the version right now and get status of the nginx service installed and all these steps will be executed on the two hosts which are a part of the host group so let's start with our playbook so when you think of installing nginx on amazon linux2 instances if i'm going to use a basic shell script it would be looking like yum update hyphen y i'll use amazon linux extras install nginx 1.2 this is one of the version that i'll install and enginex iphone v is the command to grab the value of the current installed version and system ctl start nginx which will start the enginex service and with this will enable it so this is the basic ssh or shell script so if i have to do this in ansible i have to use modules and i have to write a playbook for this isn't it so that is what we are going to do today so this is the atom ide that i'm going to use as i've already explained to before that uh why is it very good i have already shown you the setup of atom for using ansible so you can check that video that's the fourth video in this series so please make sure that you check that out the first thing that we want is we need to write a comment so it will be installed and gen x engine x using ansible then the first thing that i want to do is i want to give a name for this task so name is equal to our colon setup and genex power on my server list so this is my host group so the next step is to give the host details so host will be hosts will be my server so this will be my patent name and this will be the host group id as well but in order to execute the commands we need to escalate the privileges elevate the privileges as well so we'll type become equal to true and then we'll define the tasks so here what i'm going to do i'm giving i'm going to start off with my first task i'll give it a name so that will be installed the latest version of nginx and this will take a command module so the command will be the one that i already showed you amazon hyphen [Music] linux hyphen extras install nginx 1.12. equals latest hyphen by so hyphen why what it does is it allows the prompt to download or install the components that you have without giving the prompt so whether it will ask you to download or install it will ask you some prompt whether you have to type in why or no so that will be done automatically for you so once you are given the command i will just pass an argument value where i will be telling ansible that whenever you install nginx it is going to create a path on the machine that i am going to install nginx on and if that already exists then don't reiterate or don't rerun this command once again so this is the first task that we have completed so the next task is to start the service service so same way i'll use not command this time there is one more module service that we can use so this deals with all the services you just have to provide the name of the service as i already told you before so name will be nginx and the state should be started okay so this is the second task so third task will be so the third task will enable the service and here as well i will use the service module again name will be nginx and state will be yes or enabled will be yes and the next one next task for us was to actually grab the value of the nginx version so name i'll provide it name like get nginx installed version and this can be achieved by using just a simple command module the command will be nginx space hyphen b the next thing we wanted to grab the status of the engineering service so what we can do is we can just provide one more name here and we will just type in the name as get status of nginx service and i'll give another command that is system ctl status nginx so you might be thinking that these things are very basic that i'm doing right now but imagine a person who doesn't know anything if we start providing them details with so much complications they won't be able to understand this so i hope you will understand that this session is not just for experienced people this is also for people who just are starting to learn ansible so i just want to keep things very simple we can do fancy things here but i don't think that's the way to learn and that's not the way to actually learn things okay so just be with me on this one once you have done this this is complete for your playbook and all these steps that you see here all these tasks will be run on these particular hosts that are part of my servers group and don't worry i will be providing this the code as well on my git server so you can just use the same thing for your testing and you can do more experiments on the tasks and the commands that you want to use so that will be fun but we have a playbook right now but the most important thing for us is to basically run this isn't it so let's go and run this playbook i just want to show you the way that i have created these institute instances so these three are basically my ec2 instances the amazon linux two instances so there's manageable master this mansible one this is my answer to these are the two hosts so what happens is if you want to use these instances you can basically search them or else you can use something that or something that i use like the tool that i use that is at terminus so you can use this as well this is very easy you can create new hosts similarly i've created my three hosts like ansible master host one and host two and if suppose you want to add new host you can just click on this you can provide a name here you can provide the ip address you can provide the username the key that you want the sh key and you can log into your host so this is the ansible master i'll just double click on this and i'll straightaway connect me to that particular instance and just delete all the things that i have so the first thing for us is to create the ansible playbook so what i'll do i'll just use vim install nginx will be my playbook name and i'll type the extension as yaml just hit enter press i to go into the insert mode copy the content and just paste it here click on escape or hit escape type colon wq to save it and hit enter you can just cat once again to see the content so this is the playbook that we have and if you wish to run it what you need to do is first of all you need to configure the host isn't it so so one thing that we need to do is we need to do sudo vi or vm slash etc ansible and host we have to modify this file so i have already written my servers here but i have changed the ip address so what i can do is i'll just modify this file once again i'll just delete these entries i'll just show you once again so what i can do is the one thing that i wanted to mention here once again is see the host name that you see here the pattern name should be same with the one i'm giving here so it should be within the braces and we'll talk about variables so don't worry about this for now just remember that when the braces inside the ec edc ansible host file you can just modify this or you can create your own host file as well but for now i'm going to use this and already provided host file for me so i'll just enter the ip addresses here so i'll just copy the ip address of my first sensible host and i'll paste it here so for now let us work on the first instance then i will add the another one and the other one or is what i can do is i can just copy the second one and i'll just comment it i just first want to show you how it basically works so so let it run on the one machine that we have here and then we'll see what are the execution steps so just save it so add the entry for the hostname that you want and just save it using colonwq so that's it uh so what you can do if you want to run you have to basically configure it so ansible hyphen playbook space the file name that is install and if you want to have a verbose you can have three or four verbose arguments but i'll just keep it at hyphen 1 or hyphen b for me and just hit enter that will run but basically if suppose you face permission issues then what you can do is you can configure the ssh bash and you can provide the keys for these two hosts that we have i have shown this in the previous video and that link is already there in the top right corner so you can watch that video if you face permission issues so once you have set the keys and the permission levels are correct then just run this playbook so now as you can see it is trying to install the latest version of nginx it is on the first step yes it has installed it has started the nginx server and it has enabled the service and then it has ensured that it is on the latest version and then it gets the status of the nginx installed so the first thing that you have to notice here is just will go up so there's the first step that we have written play nginx server on my servers list so if you go back you will see the same thing here and this is the first step install the latest version of nginx so it has come to the second step where it has tried installing the latest version of nginx and it is on the same ip that is of our host one and it is showing you the command list that it's trying to install i told you the return type for command will be the list of command that we are running right now and it has installed the package nginx package which are without any interruption and the next step that it has it has started the service engineering server and the change is status has changed from uh to true and the name of the service nginx state is equal to started so once it has started it has returned true for this one and then the next step is enable so the status has changed to true and enabled equal to true and the third thing it is returning us the latest version so if you see and this is the version that we are seeing here nginx version colon nginx slash 1.12.2 so this is the version that we are trying to grip here and get status of the nginx installed so this is the system ctl uh status engine x command and this is the output for this one so it is active and running right now and if you see ok equal to 6 changed equal to 5 not reachable failed skipped rescued ignored is 0. so our playbook has successfully run so now what happens is this is for only one and what i will do is i'll just enable the other instance that i had the host 2 so that we'll see what happens if it is trying to execute the same commands once again and what happens if we have more than one host and we are trying to run the playbook on the host group i'll just save it and i'll run the same command once again let me just clear this so let's see what happens when we run this playbook on both the hosts so let's go to the starting point so here what you can see it is trying to gather information about both the instances first and this is the instance which already had nginx already installed so if you see here in this step nginx the install nginx here you can see for this one it has already told that the changed will be false because this step already exists and slash nginx already exists so it will not execute this step again so it is okay for us now so it does not want to install nginx once again but when it comes to the second host 13 127 23 231.21 the status has actually changed to true and it has executed this command so what happens is for every instance that you have in the host group it will go to the task it will execute them on both of or all of the hosts that it has in the host group and then it moves to the next step as i told you before so it has installed the nginx on the second host and once we come down it is going to start the service so changed equal to false because the service is already started for this host the host one but for the new one the change will be true because it has enabled or it has started the service for this the same way the enable nginx will be same or constant for the first host but it will be enabled on the second one and ensure nginx is at the latest level so both of them the change will be true because it is just trying to grab the values and here also the status will be true for both of them because the service is already running so that is why you will see five changes and here only two changes so let's see i'll just run this command once again and let's see what happens see for everything the change will only be two because it is trying to grab the values for only these two ensure nginx is at the latest version and get status of engineering installed these two only will be having a status changed otherwise everything will be green and it will be false because it does not need to execute these steps so let's suppose what happens if the nginx version is not there so 2 dot i'll just edit this to 2.12 let's see what happens okay this does not work because the path already exists let's clear this i'll edit this once again i'll just make sure it has engine x12 like this okay we'll modify something we just want to fail this let's suppose that version only does not exist so what it happens so see what happens is if suppose it fails here it is not moving forward again and if you see the topic nginx 2.12 is not found so you get the error message here that is the standard error and error line will be this one let us just modify the playbook once again and we'll just uh keep it the way it was and there is one more thing that i wanted to show you guys this was to basically copying one of the files from the ansible master to the hosts so that also we can do an ansible by using a configuration command so here i will just type one more command or one more task i'll just have keep a name here name colon ansible copy file to remote server so whatever file that i provide here will be copied to a destination on the host server so the command will be or the module will be copy which takes arguments as a source so that will be home slash ec2 hyphen user that is the path of the home folder and sample.txt that will be my file and here destination folder will be flash home ec to hyphen user that's it so if you want to copy this file from source to destination you can do that this is mainly useful when you're trying to execute a particular shell script or something you can do this by copying it or if suppose you want to keep any configuration files also you can do that i know that you can do that using shell as well but let's suppose you have a requirement where you want to copy this like log files or zip files or something like that you can also do this so this is the same thing that i will do again i'll just copy this whole thing once again or i'll just okay now i'll just copy this i'll just remove this file i'll just modify this once again [Music] just paste it there's a new entry that i have to copy the files but ensure one thing that you have this file already present so i have sample.txt what will happen if i execute it without that file let's suppose okay so we'll connect to the one of the host that we have here host one and we'll i'll show you that it does not have that file already existing on its folder structure so the home slash ec2 hyphen user i'll do ls so it is completely clean it does not have i'm not faking this it really works so the first thing that i want to do here is i just run this without that particular text file let it run the last step it should fail because could not find or access home easy to sample.txt on ansible controller so this is our controller and it is not finding that file here so what we have to do we have to provide that file isn't it so what i'm going to do i'm going to create a sample.txt and i'm going to type hello this is the sample text that we want to share just save it and then do a ls to see whether you have the file or not yes you have it just execute the command once again yay this file is copied now what happens is we can go back to the host and do ls see we have the file here so these are some of the few simple things that you can do using ansible and this is just a starting point for you and there's so many great things that you can achieve using ansible and the ci cd and the automation that you want but everything should start with a small step so take this small step try to learn more read the documentation and so i think that's it for today's demo and i hope you will do this experiment as well and you will also try out by installing and setting up your test environment for ansible hello everyone and welcome back to yet another session of ansible and in today's session we'll discuss a very important concept that is variables and we'll also have some fun as well as we'll do a small hands-on demo for your understanding so we'll understand what are ansible variables how to create variables and what are the constraints in creating variables and how to use variables in a playbook i know most of you are very well versed with programming and you might be aware of what a variable is but let's start with this statement here so when you think of what is a variable our mind tells us that a variable is a symbol that works as a placeholder for expressions or quantities that may vary or change so all throughout our programming journey we have used variables and in initial episodes we have used variables to store data that we might want to use later on in the program this is the same here in ansible as well but the most important thing for us is to have some kind of knowledge with jinja 2 template because ansible uses ginger to template expressions to delegate the variables i have mentioned two links here you can visit them for better understanding but make sure that you have some kind of ginger to templating expression knowledge in order to use the variables and so that you can handle them very diligently in ansible now let's start off with how to work with variables in ansible so ansible uses variables to manage differences between systems so differences between system means that let's suppose you're working with multiple infrastructures but the template remains same so in order to avoid using hard-coded inputs every time you can make different set of inputs for managing different deployments and you don't have to change your template as is and that's the biggest advantage of using ansible and variables in general so with ansible you can execute tasks and playbooks on different or multiple different systems with a single command and this is possible with having a configuration that varies with the desired environment so you can define these variables in your playbook in your inventory in reusable files or roles or at command lines as well so we can define the variables like simple string like foo we can give it like four underscore env for underscore port and we can have digit after the string and we can also and we can also have an underscore as a prefix here so as i can show you who is there for underscore env is there for underscore port is there who which is what you say which is having a five or number after the string is allowed and you can have a preceding underscore for a string as well for your variables but there are certain restrictions for variable naming you cannot use something like star foo which resembles the pointer notation you cannot use variables same as the python keywords such as async and lambda and others and you cannot use playbook keywords such as environment as your variable name and you also cannot use something like hyphen port whose space port or food dot port so these are not allowed so hyphen space dot are not allowed and you cannot and also you cannot use numbers or variable names that start with a number itself so you cannot use numbers and you cannot name a variable which starts with a number and here below you can see i have mentioned the ways to create the valid variable name so a valid variable name or a variable name can only include letters numbers and underscores python keywords or playbook keywords are not a valid variable name a variable name cannot begin with a number but variable names can begin with an underscore so remember these points very carefully before writing your scripts or assigning variables so let's see how we can define the variables so we can assign variables with key value pair so for example we have install underscore path colon the string of the path name here we have the installer under path as a variable and the value as a path so once we have assigned the value to the variable then we can make use of the expression with double curly braces as you can see here this is basically a ginger to templating expression and we can provide the variable name and that will become the placeholder for your data so install underscore path becomes the placeholder of your data and within the double curly braces it becomes the expression so as you can see here we have the source string the destination i can just provide the install path and this install path can change and i don't have to change the template here below in the next example if you see if you want to use expressions you need to apply it with quotes else you will get the error like syntax error while loading yaml so here you can see i have not provided code so i am getting the error so make sure that you provide proper syntax here you can also define variables with multiple values for example we can define a list so using a list of values in a variable so i have assigned a variable name aws underscore region and this is basically your li structure or list index structure so you can provide the values for this and if you want to access the same values you can use the indexing like we do in python so like if suppose i have to use the first value of aws region from the list i can do an indexing of the value 0 so it'll give me the value of a b south 1. you can use dictionaries as well in the variables so there are two ways to grab the values in this one as well so let's suppose i have a variable aws underscore region and it has a key value pair like ap south 1 john usbs 2 colon jesse these are key value pairs so same with like we access in python you can do the same way or the way that we have here see aws underscore region in square brackets if i provide the key name i'll get the value so by using the square brackets that's the first option and we can also make use of the dot operator so aws underscore region dot the key so by using the dot operator also you will get the same value john or jesse based on the key that you have and there is also a way where you can create variables from the output of an ansible task with the keyword the register so remember this keyword very carefully like this is very important register so this is very important because you can make use of the output in a variable and use that variable in the coming steps in the playbook so you can use the variable register and put the output of that variable and here if you see we are executing shell command nginx hyphen v which gives us the value of the version of nginx and we put that inside the nginx underscore version variable which is being dumped using register so the way we are defining it is basically register colon the variable name and the same variable can be used in the other code as the input variable like like we have used it in the debug statement as you can see so message argument actually takes the string as an value so here the current version of ansible is within this i have put the variable that i am grabbing the value from or from the first task that i have executed here and this nginx version variable can be used here as an expression so when you print the debug message it will print this value and there is one more way to grab these values as i already told you the dictionary way that we have here like let's suppose aws underscore region asia ap south 1 mumbai they are all within square brackets so if you want you can directly access them as well within the double quotes aws region dot asia dot ap southbound dot mumbai so you can use either the square brackets or the dot operator as i already told you so let's suppose you want to separate out variables and put them into a variable file you can do that as well so here we have created a variable file which is inside the slash vars folder so if you see we have the same way we have discussed before the key value pair and let's suppose i have a variable that has a list of values so if you have a file how are you going to access it so let's suppose i have my underscore variable.yaml so how i will use it on the right hand side if you see we have defined the variables in the vars module or wires argument so that's module what it does is it provides us with mywear which is the variable name for me and the hello world is the value in device i can define the variables as per my requirement the same way where centroscope file gives me option to provide the path of the variable file so that is very clear here as also we have used vars underscore file module and provided the path of the variable file and that is a way you can use it so you can use your values here inside that particular file by providing wires underscore file and the file name as well and you can use it in your playbook and there are a few ways to enhance your automation by using host and group variables so don't worry about this this is very simple i'll explain you in a very simple term so that you get to understand what are host and group variables the basic difference between host and group variables is that if you are assigning a variable to one machine you can make use of the host variable that's it and if you want to assign a variable to many machines or multiple hosts at once then go for the group variables so let's start with the grouping of variables so sometimes there are resources or variables that could be applied to multiple hosts at once so why would you define the same values again and again isn't it just define once and by using group variables and then it applies to all the hosts that are a part of the group here if you see we have variable host inventory my web servers within which we have four host names and then using the vars keyword the virus keyword that you see here my web service colon wares what we are doing is we are applying these variables to all the hosts so this is called grouping of variables so within which we have like the host group and by using colon wires we are able to define all the variables and these all variables will apply to all the hosts that are present in that particular host group so these will be applied to all the host which are under my web servers so this is called grouping of variables and you can as well inherit group variables to create a sequence of inputs that get applied to a group itself so here if we see we have two group names asia and europe so if you're working with two host groups let's suppose ap.pythonic.com and ap2.pythonic.com are a part of asia and eu1.python.com and youtubepython.com are part of europe so we can create a child group and add the preceding groups inside it so that it becomes a collection of groups so these are the two main groups that we have and within which we will create a emea underscore region colon children by using children we can add these two groups to that particular entity and make it a collection of groups so we can do that by using the colon children scope as emea underscore region colon children within that we can add asia and europe so the main purpose of colon children suffix is that you can make groups of groups using the colon children suffix in inf files or the children entity or entry in the yaml file as well to all the groups that we have within eme underscore region we can assign the same vars suffix to this or the argument that we have here the module so we can assign this to eme underscore region and assign this particular variables to that so these variables will be applied to each of the host or the host groups here and all the hosts present in each of the host groups will have these variables applied to them and if you want to go one step ahead as well similarly you can create a nested group by adding all these groups under a single group by using the same colon children suffix so here if you see i have a region colon children so it encapsulate all these things so let's suppose i am under emea so asia and europe may be coming under that so let's suppose i have pacific region and australian region and emea as well so i can put them under one region group and let's suppose i want to assign variables to region then i can use the same vars argument and i can just provide the name as region colon r vars and then i can apply the variables that i want so those will be applied to all these groups within which we have smaller groups asia and europe and within which we have individual hosts so let's suppose there is a variable name that is called company and it has to be applied across all the hosts so what you can do you can create the entry of that particular variable and you can group them accordingly so that they inherit the particular variable that you want let's suppose company name and that company name gets applied to all the hosts within these groups and the next one is host variables so the groups were to assign values to multiple machines at the same time but the host variables applies to one machine so this is how we have previously defined our inventory as well so this must be easy for you so we have the variables and the host name group or the one that we have already seen and each of the variables applies to the whole group so if you see this is this basically your host group myweb servers under which we have the host names and if i write my servers or my web server school onwards and i give the variable names here then these variable names kept up get applied to all the hostnames that we have but in host variables what happens if suppose specific requirement is there for a specific host and you have to provide specific configurations for a particular host name then what you can do you can just have my web server source group you can have the entry of the host name then the port manager that let's suppose this is one of the variables so port manager equal to 336x proxy and base url so let's suppose they are different for all the hosts then you can properly manage them individually as well so this is one of the benefits of using host variables and this is what most of the inventory files have you can as well define variables at one time so by passing variables at the command line using hyphen hyphen extra hyphen wires or hyphen e argument or you can request user input with the where's prompt so key equal to value format is that based on which you can you can pass the variables so ansible hyphen playbook we have already seen this how to execute a playbook so sensible hyphen playbook space name of the playbook hyphen hyphen extra hyphen wires and after which i have provided the arguments that i want to pass so version equal to 1.12.1 build equal to latest and we are using a key equal to value format here and the next thing that you can do is you can pass json as well for your arguments so here the same way we have defined hyphen knife and extra hyphen wires and here we have provided the json as well so this is the key value pair but this is not key equals to value but this particular json and one more thing that you can do is you can pass a yaml file or a json file as a variable format so the same thing applies here hyphen iphone extra hyphen vars and you can provide a json file or a ml file as an input now let's see a hands-on demo for the variables part and this should be fun and this would be very simple for you to understand so i hope you will do this along with me so this is the file that we had created previously in our previous demo so we'll take this file as a placeholder for our new section that we are going to experiment with variables so don't worry about this we'll do some small modifications and tweaks so that we get to know how we can use variables here so the first one that i want to do is i just want to replace this hard coded value for nginx and i want to replace this path because this is a custom path that i have hard coded i don't want this and i want a variable in that place isn't it so let's see how to define this variable so you know how to execute this playbook and you know how this particularly works so the next thing that i am going to do here is i am going to define a verse keyword so vars will allow me to provide variables to this particular playbook so in within which i will provide a variable name nginx underscore version and the version will be 1.12 and i'll provide one more variable that will be that will be my base path so that is basically my home slash ec2 hyphen user so as i already told you when you are creating variables you can replace them with the hard-coded values that you have just copy this and i'll just replace it here so what we need to do we need to provide double curly braces and we have to just paste the variable name here and the next thing that i want to do is i want to replace the base path so just copy the base path here and just paste it here provide the double curly braces and paste it here okay so you see that we are getting an error here what you can do you can just provide quotes i told you we will be facing errors okay so when you're using strings along with the expression please provide it with quotes so that it works you can do this for this one as well but it's not mandatory but for the usage of expressions along with strings you should provide the codes just copy this and let's go to our machine so i'll just remove the file that i already had let's create a new one just paste it so here we are having the two variables that we wanted to create engine xiphon version or nginx underscore version and this one base underscore path and this one will be the placeholder for our variable base underscore path and this will be for my nginx underscore version just let's save it and what i'm going to do is i'm going to delete this file that i have here in the host that we have previously copied so i'll just do rm hyphen rf sample txt and just run the file once again okay so one mistake that i have done here is basically if you can see it is not able to access this file path so i have not given the delimiter here i'll just give it and i'll try to execute this once again sorry for that but yeah you get to know if there is any error so it's fine if we got the file once again or not yeah we got it yes we have this so the next thing that i want to do is for you for you to actually have this understanding so let's go to this output once again and we'll see where we are able to print the version so ensure nginx is at the latest version so this is where we want to grab the value is ended so let's suppose i want to grab this particular value so this is inside the key std err underscore line isn't it so what we can do is we can grab that particular value using this particular entity or this particular key so how we will do that so what i am going to do here is i am going to register this variable register here register using register i'm going to create a new variable nginx underscore version so this version will be my output or the nginx hyphen v and i'm going to use it inside a particular print statement or a debug statement print the version of nginx and here to print the debug message there is a functionality for us that we can use that is debug that is one of the commands or the modules that you have and which takes an argument that is message and post switch we can write our message the current version of nginx is again use the curly braces use the variable name dot sgd err underscore lines of 0 so why am i doing this so if you see the entry here so this is stder lines and that is the first value of this one so this is the way i'm going to print it and this value is going to come from the output of this particular command i'm not going to and i'm not hardcoding the values here it is just trying to grab the value from here let's see if it works or not let's clear this we'll just delete the file once again and we'll create it just paste the values here or the code here and this is the debug message that we want to see and we are going to use the register value or the register module where actually we can assign a variable name to this which actually captures the output of nginx iphone v which is the command that is going to be executed on the particular host so let's see how it works now same way just run the playback this is the message that we got from there so current version of nginx is engineering version and that is the output that we are getting from the particular execution of this command isn't it so what we have learned today we have learnt how to create variable space and we have assigned variable names to this one and we have provided the values we have provided the base path variable and we have assigned it to a particular entry which is trying to copy the source location file to the destination location and we also have registered a variable in the same variable nginx underscore version is being used in the next task and we are not hardcoding any values here when we are just printing the output that we got from this one so this code is already available on the link in the description below so please clone the repository and you can make use of this code it is completely free for you and just make sure that you read more about this in the documentation and try to experiment on the various modules that we have and try to see if how it works and if you are able to execute them properly and i would give you a task so the task for today will be to put these under a variables file and use the function or the module that we actually can import the variable file onto this particular playbook and try to see whether it is able to execute it or not so that is one of the tasks that i want to give you in the past few sessions we have learnt a lot about modules and the ways we can use modules for our automation but what if we want to create our own ansible modules so can we do that let's see if that's a possibility so when we wrote a playbook previously we made use of the existing modules and parameters to execute the operation that is what we used to make our automation better so let's understand or let's try and understand this playbook and let's clarify these terms so the first one is hyphen name setup nginx server on my server list so this is our name string the second point host colon my servers so this is the host group that makes it a pattern as well so this is the host group and within all the host group all the hosts that are present on those machines that are available as a part of that host group the third one is become colon true as i said it is used for elevating privileges you can use become as true rtis and the fourth one task next we actually are defining the task using this task parameter and after that we have hyphen name and then we have the ansible module that is command here the command module helps you to execute commands on the shell prompt or as well as on the windows powershell but we use unesco command that's a separate thing but that is basically the operation that the command module does for us but along with that it takes the task keyword that is basically our arguments the next thing that we have done is we have given the parameter as well that is creates and we have given the path there is the path string so all these if you see we are having a collection of statements that are able to execute a certain operation so this ansible module that we have here will be used to install applications execute commands and operations on the host which are a part of your host group so that's what we are seeing here so this is the playbook that we are writing we are using the command module and when we try to execute this from our ansible master on to the host group that we have here so it gets executed on system a system b system c system d so that is how it gets propagated but our main focus for today is let's suppose if i have a custom requirement to create a specific module then how to do that for that you need to come to ansible module development getting started so this module will provide you all the basic requirements and the prerequisites that you need to create your own ansible module so first thing for us is to set up an environment and for that i need a ubuntu machine and we'll install the dependencies that we have here in the documentation so let's go ahead and create amazon instance for ubuntu so this is the amazon ec2 console you have launch instances here just click on that you have the ubuntu server yeah i don't have any problem that you have to do it with ubuntu i'm just following the documentation you can do it in any other operating system as well so don't think that it can only be done using ubuntu you can do it with red hat and amazon linux 2 as well so there is no harm in doing anything in any other operating systems just make sure that you follow the protocols and the documentation for you to achieve the same results that we are achieving here so we have the amazon server 20.04 lts version and 64 bit x86 architecture that's what i'm going to select select that and we'll quickly create an instance so let's wait for it to get started let's copy this public ipv address and we'll try and connect to that particular instance remember the username for ubuntu machines will be ubuntu not ec2 user so don't get confused if it does not work just give the username as ubuntu and just save it that's it so this is the one that we are using right now so just add and continue so you might be thinking that what we are trying to achieve here so the first thing that i want to tell you is that for everything that you want to achieve or you want to write let's suppose you're writing a code on python so for that you need python installed isn't it if you want to run it so just like that if you want to run ansible commands you need ansible to run it isn't it but for you to have your development environment set up you need the ansible core code itself and on that itself you are going to modify some of the files and you're going to add some of the files for your own customization and that's how it works so if you have your own ansible code or the root level code that you have then you can make changes to that and once you run that particular environment then you will be able to make use of the custom scripts that you have or the modules that you want to create we'll just see what is the present working directory here so home slash ubuntu and we'll come here and we'll clone the repository there's the first step and let's clone the repository for ansible so you will get the ansible code okay so now once your ansible code is cloned you can just do a ls here you will see the folder and sybil and within which you have all the ansible code that you need to run your own version of ansible the own version of ansible means if you have to customize something if you want your own code to be present in one of the command modules or anything that you want to customize you can do it here will not affect the overall environment that you have so now what it is telling is change the directory into the repository root create a virtual environment so what i'm going to do i'm going to do cd ansible and here it tells me that i can do it like this so my main aim is that i want to create so i'm giving it my b env so if you have to create virtual environment in python 3 you can give the command python 3 hyphen m that is the module name the module name is benv and you can give the name of the virtual environment that you want to create and then just press enter okay so if this doesn't work then you have to install this let's install the virtual environment now once it is done just try to create this so if it did not give you any error then just do a ls here inside the ansible folder you will see a folder name that is my v and b will do ls yeah see this is the one that got created so you can do a yveenv and you can if you want to activate it what you need to do is you can just go inside when and you will have this activate you can do dot slash activate it'll activate the virtual environment for you but let's come back to the documentation part so what it is telling next is you have to activate the virtual environment and you have to install the requirements.txt that we have here so for that what you need to do is it is telling slash dot what it is saying is you can do dot v and v slash bin slash activate so for us it is my v and v right that's it so once it is activated you will see a placeholder here for that particular environment which tells you that your virtual environment is active right now and within this environment you can install any packages and that will be local to that particular virtual environment it will not affect your overall python installation and now what we have to do is we have to install the development requirements so this is the requirements.txt and it has a lot of files and it has a lot of modules so you can check that out there's a file you can do a cat here requirements.txt and you will see these are the modules that have to be installed here there's also that so in order to install this what you can do you can just type install hyphen r requirements.txt and then just hit enter but before that i want to show you something like you can do pip freeze and see there nothing installed here so this virtual environment does not have any packages installed so just make sure that you check it ahead of installing it once again now that that has been installed what you can do you can just do upgrades again you see all the packages that were inside that requirements.txt have been installed now the next thing is to run the environment setup for each dev shell process you can just do this as well so hacking this folder is there hacking we have to activate this as well just do environment setup okay so the path python path and everything has been set so that ansible can make use of it now your development setup or prerequisites are done the next thing that you have to do is write your code and whenever you want to come back and you want to start working on this once again you can just copy this same thing let's suppose i want to move out i have i want to restart this machine then i have to come back and start working again on that particular environment isn't it so what i can do i can just let's suppose i disconnected from this machine okay remove it close it and this is the machine that i was currently working on right isn't it so i'll just double click on that and if you see currently i don't have that prompt for virtual environment what i can do i can just do a cd cd ansible and within which i can actually type myv and we activate and hacking environment activate it so now this is also done and my virtual visual machine sorry my virtual environment also has been activated so i'm good to go now so every time i come back and i restart the machine or anything that goes bad like if i want to restart the things once again i can just come back and execute the same command and i'll be good to go so here you have information like you can create your info modules and fact modules which gives you information about what the actual module is doing and you can have a documentation as well based on these info and fact modules but our main focus for today is if i have to create a module for myself so in order to create a new module navigate to the correct directory from your new module so within ansible there is a modules folder within which if i have to create a module i have to add my file there so lib ansible modules will go there so if i have to see here so we have lib inside which i have ansible inside which i'll have modules isn't it do a ls here and if you see most of the models that you are going to use in ansible will be present here itself see we have command module here if you open this and if you read the command module you will understand that it is just a python script it is telling you what to pass as an argument in the playbook and it will execute it based on the arguments that are provided there similar to that we have to create our own module as well so to create your own new module you can just type whim my underscore test module as it is given there and you have to create a new file here ansible already has a predefined format and documentation so based on that you have to write your own modules as well so for that what ansible tells us is that you can paste the content below on your new module file and it includes the required ansible format on documentation a simple argument spec for declaring the module options and a sample example code so this code that you see here can be straightaway used and we have to just modify this code to make a particular ansible module and that is as per the ansible standard so we'll not modify them we'll just tweak them for our requirement just copy this hit i just paste it here okay just see here now from the top so you have your shebang you have the copyright details the licensing module does you can write your own things here like short descriptions like this is my test model you can modify this there's no problem with that you can provide the information like what does it do what parameter it takes what argument it takes what type of argument it takes whether it should be a string or boolean you can provide everything here and you can provide an example code as well for users to use so this is the place where the actual execution happens so within this you have module arguments so here if you see it takes two arguments name and new you can tweak that based on your requirements so this just test module so when you pass a particular name here it is just going to return a string for you so that is just a basic purpose of this particular module this is just a test module so we'll just see how we can run this and then once we have executed this properly we will see how we can write our own custom module so you have this file now you can just save it by clicking on escape cool and double queue and if you do ls here you will find your test module also here as well where did it go in ghaji yeah so here my test dot p1 what is the next thing that it tells us to do so if i have to execute my module locally what i can do is i can create a temp args chart json so there is a temp folder here so temp i can what i can do is i can create a args.json here dash.json and this will be my input file so i can just copy this let me just paste it here so these are the two things that my test module actually takes name and new as i told you the name actually is a string that is hello and new is a boolean that is true so just save it for now you can run the module and test it and the next thing that it is selling is you have to activate it and run the development environment these two has already been done and the next thing that we have to do is we can just run this module so what is the command that we are showing here so this is just test command so python hyphen m and symbol.modules.mytest so my test is the module name and then it takes the argument from here so once you run this it will give you the basic formatting of what a particular result should look like okay so as i told you if you have made the new equals true then it will keep the status as or changed equal to true original message will be hello message is goodbye so that is the particular so that is the purpose of this particular module that once you pass a string here it will give you a message and the invocation will be module logs name hello and new equal to true so this is the basic input that have been provided this is just a simple execution so let's suppose i want to write a playbook for this so that option has already been given so i can go back to my ubuntu my home folder and i can create a playbook here test mod okay now i can just copy this and this playbook is very simple you have to provide a name and host is localhost it means that you are testing this particular module locally you're not going to install it on any other machine or any vm and we have two tasks here so first one is run the new module so here we are passing our own created module that is my underscore test that we created so that name of that particular module will be same as the python file name that we have provided there so my underscore test and then it takes name as an argument that is hello and new equals true and we are registering the output that we are collecting from this particular module in register that is our variable is test out and then we are dumping the test output to this particular debug message so that is as simple as it can get just save this so now we have the playbook so how to run a playbook we can just drive ansible hyphen playbook so what you can do is ansible playbook dot slash test not dot yaml so now if you don't specify any other module and the host does not have any other modules so it will take the localhost by default and if you see here we have the dump test output and there is the debug message that we are getting so changed equal to true failed equal to false message equal to goodbye and original message equal to hello so now what we have achieved is we have successfully run the development setup we have written a test code and we have successfully tested it on the local environment i have to write an ansible module which can take a string and encode it and give me the result and that same module should be able to decode if i provide a encoded string to it you understand the requirement so the same module it should take the string and it should encode it and also it should be able to process a encoded string or a byte string into a normal string that we have so it should perform both the base 64 encoding and base 64 decoding so such as the requirement for us so what i am going to create is i am going to create a base64 mod that will mansible module and the type of arguments it will take is the name that will be the string type will be either encode or decode and neo equal to true or false based on the states change that i want to do and the playbook will be exactly like this i'll register the output whether it is encoded output or the decoded output and i will dump it to my debug message so this is the particular requirement that i have right now and we are going to write some basic python commands so if you are not aware of python don't worry this is very simple anyone who has some basic understanding will be able to write this or you can just copy the same code that i have given here and i will be giving this on the git repository as well you can copy the same thing and you can try and experiment on that and try to learn things in a simple way so that when you try to achieve something big you will not have that initial hurdles so try and make use of the initial time that you have and try to learn things that you are able to understand and grasp don't worry if you are not aware of python it's not a mandatory thing here it is better to learn it but if you don't know it don't worry you can just learn this and you can see how simple it is and if you get the confidence now you can as well go ahead and learn python and execute your own operations as well so first of all we have to create our own module and we'll go to that particular path that we had the module path and we'll write our own python code i'm going to create a new file so my new file will be base 64 mod b64 mod dot python and just hit enter so this is the placeholder for me to write the code so my benchmark that i am going to set is basically so let's create a this 64 mod dot py and what i'm going to do is i'm going to copy the same content that i need and i'll just modify my code here so go ahead and copy this just paste it here okay so now this is done so i don't want to modify this terry jones 2018 copyright and everything and let's just add my name at the bottom and this is fine we don't have to mess up with the import statements the documentation is also fine and this is not going to be my test it will be base64 mod and this will be my test module okay so this is my base 64 module encode and decode so the first thing that you do go is you have to understand that it starts from main and it goes to main it is going to run module isn't it so run module is this particular function the module arguments that you see here is basically the arguments that you pass so the next thing that i wanted to give was type isn't it so type equal to dictionary and the string that is basically a string and required equilateral so that's a mandatory value for us now come down you don't need to change this not a problem this is the original reference dictionary so it is just creating the module object here there's the check mode analysis and after that this is the place where you're going to edit your code so it's written here as well manipulate or modify the state as needed where your module will do what it wants to do so this is the place where you're going to have your own code so first what i'm going to do is i'm going to check if module dot params i'll just copy this i don't want to do any spelling mistakes so the first thing that i want to check is it should have either encode or it should have either decode otherwise it should throw an error fund it should be either n code so i'll what i'll do is i'll give a flag here status flag so if it is encode then what it will do is it will set the lag to true or else if it is decode it will set the flag decode status equal to 2. and just copy these things and just keep it here as variable declaration decode and set the flags to false so this will be the default lag status okay let's save it so now our input validation is partially done i don't want to do much here you can as well do the input validation for your strings that you provide so now the next thing what we have to do is we have to raise an alert here so for that you see we have a message here like module exudation similar to that we have fail json here so with this we can actually raise error messages you can just copy this or you can just copy this as well let's delete this so what i have to check is if not encode status and not decode status if both of them are false then what i am going to do is i am going to row an error exception so that will be bad input and it should be encode or decode okay so this is the message that i have here so it will fail the json the next thing is once i have the input validation done i have to change string to base64 and return it so now what i have to do is so this is my encode part so if encode status so at a time only one operation can be there because there is only one type so just make sure that you don't get confused here so message will be the message is equal to module we are going to take the parameter params of what is the parameter name isn't it so name is the parameter and message underscore byte will be message dot and code of ascii so i'm going to just format it to the ascii characters and then i'll convert this to base64 encode so for that what i need to do if i need to use encoding i have to just import this import base64 here so it will be able to use that encoding functionality so post that what i can do is base64 underscore byte underscore string bytes underscore string is equal to strf i just want to convert it to str the base64 dot base b64 encode so that is a particular method within that particular base64 package message underscore byte so i'll take this and i'll pass it here and i'll use the base64 encode that is present in base64 that we encode that we just imported right now and i will store it in a string format here after that i have to show it to the user isn't it so there is a result json which i can make use of so this is the result json here see there's the resultation that is what i am going to use here so result of original message original message is equal to message which is basically your module parameters of name and then copy this once again just paste it three times so encoded message will be input and message will be 64 byte string and then message we have to give some message to the user so that will be basically your string your data your data is encoded to base 64. enjoy enjoy okay so your users can enjoy now so now what we can do is we can use the same logic here to decode the string just use the same thing here and we'll just change a few things change base64 to normal string and return it so here it will check for decode status so module status module parameters is name [Music] the next thing that gets changed here is message base 64. we'll change this to a new name so this will be my base64 underscore name message and this one will be base64 underscore bytes message so this will be base64 message dot encode of ascii and then base 64 byte string strf base 64 dot decode of base 64 bytes so we are taking this byte here and we are providing it here and we will just convert this so original message here will be base64 message decoded decoded message will be base64 byte string and message will be decoded to normal string okay just copy this and we'll just just paste it here so we have the decode status we have the encode status okay we have the validation [Music] we have the variable declaration of the status flag declaration we have given it the type is equal to dict of str yeah that's fine just save this now i'll create a playbook in our home folder i've created this so what exactly it tells is that it will take the host as localhost we'll take the name as hello that is a string that we want to encode so type equals to encode here status change equal to true and the test results will be dumped into that particular debug message so this is what i need so let's create another playbook here so this will be my playbook 64 64. yammer just hit i ctrl v paste it and this is the playbook that we have so the playbook is this so it execute localhost so it executes on the localhost and it uses base64 mod that we have recently created it takes hello as a string and it encodes it let's go ahead and run this okay so ansible hyphen playbook.playbook64 yaml just hit enter and it should basically encode your string so this is the original message there is the encoded message the message that i wanted to give to the user was now data is encoded to base 64. let's copy this let's copy this string that is basically your so that is basically your encoded string is ended so i'll just modify this playbook i'll just modify here and i'll show it to you let's just modify the playbook here i will provide now decode record and the string that i have is this one so this is the encoded string and i want to decode it so the type will be decoded now and will run the playbook now so we'll see whether it is able to decode or not okay yeah so the original message was this it is able to decode it to hello so that is one of the very simple requirements that i had let's suppose i give something else let's see what happens let's suppose i gave it like let's save it and let's run it what happened when there was a condition that we had put for input validation it has actually helped us to identify that that was a bad input it should be either encode or decode the benefit of using input validation is that whenever there is a wrong input it basically helps you identify what exactly the error was and based on our input validation we were able to supply a very good understandable message so that our users or someone who is going to use our module will be able to understand if there is any error so you have to do the same thing you have to follow protocols you have to follow the measures and you have to follow the standards and ensure that your code is going to be used by someone else so you make sure that your code is understandable enough and there are proper error handling mechanisms in place and all the messages that you write for your error handling should be meaningful and others should not face any problem while using your modules so that's it for today's session on ansible demo that is basically how to create your own ansible modules i hope it was easy to understand and i hope you will also experiment this by creating your own environment your test environment and installing the ansible sdk and you will let me know how did it go for you and if you faced any problems then please put them in the comments section below i'll try and answer them and if you wish to support me then the links to insta mojo paypal and patreon are given in the link in the descriptions and you can join the membership on the channel as well if you wish to support the channel so wishing you all the good health and success until then it's vitalik signing off [Music] you

Channel: Pythoholic

Views: 8,599

Keywords: Pythoholic, ansible, ansible 101, ansible on aws, ansible intro, learn ansible, ansible setup, ansible howto, ansible course, obtener ansible, ansible with aws, ansible tutorial, ansible inventory, ansible full course, what is ansible, install ansible, instalar ansible en ubuntu, cómo instalar ansible, what is ansible in devops, ansible playbook tutorial, learn ansible from scratch, ansible network automation, ansible tutorial for beginners

Id: MNGfPn0Yvs8

Length: 164min 21sec (9861 seconds)

Published: Sun Sep 12 2021