AWS Automation with CloudFormation, Boto, AWS CLI

so for today we are going to start with cloud formation this is one of the provisioning tools Amazon uses for automation and it is slowly becoming popular and popular that it is getting more easier to use in other words so lot of people have started using it and there are stories in the world that companies are completely using cloud formation alone let me repeat that only cloud formation for deploying their environments they are not using any safe word puppet or any other of mechanisms outside AWS to use is abuse environment standalone cloud formation templates stored in s3 but was in control using the get or some kind of a version control system and deploy your environment and if something is changed automatically your stack update will happen so proto machine is getting populous so let us go ahead and see what is no formation and how we can live with its permission to deploy out in developments so basically this is the agenda for the next one are what is a device for formation how to use it what are the different use cases what is the best practices and finally a demo which we will show I tell what is the code and what the heck the code exactly does so the question that you are trying to answer here is how do I approach in my infrastructure what are the tools that I have what are the difficulties will detect it into how you make the calling you so that is my there is no organizer it's taking up everything is good at the power bed okay so the question that we are trying to answer here is how do I provision being projected in my cloud what are the tools that I have what are the problems attempts are trying to solve by using any tools are existing there and what is the problem that they are there so why is it exist exactly it's a problem is this there are too many pieces of infrastructure that you need to configure it properly for infrastructure to work let us take the simple example that I want to post and website a very very simple website it is a single server website where the application core the web server code and everything config is in everything runs in a single server then I would still need to configure something like let me find out where is my mouse pointer so I can highlight it so I will still need a security group so that I am I can allow only restricted access so that port 80 port 443 is only used I will need a PPC so that my security group is tied in with my V PC and only certain IP server configured to my certain instances and then I will need a row two three four sit zone so that I can assign my domain names that is my friendly domain NW the wizard Kumar comm can be pointed to my server and I need a server also here and I need a drastic IEP if I am going to give the public IP to my route 53 or if you are using out of the tree itself I might not need this but depending upon the type of configuration I use I might need this also so something tables also will be required because I need an internet access for MIT PC so I will have to create a routing table so that all Internet traffic goes out and if I have static content in my website I will need an s3 bucket as well so although I said it is single server website depending upon if my website is having a lot of peak load and very low load let us say in the morning time I have 100 percentage of my average load and in my evening time I have only put 10 percent of my average load then I would need something like an auto sky to scale up my machine pump off my machine so think of all the bits and pieces that get it automatically added for a very single website that is a single serve web server instance I need to configure all these pieces to work together smoothly so that I can make my website run so I need to how some problems is any one of them than your instance or your web server or your company it is going to have a problem with it so how do you do this consistently let us say you set up the website once and some giant happens and you want to set it up again how do you do repeat the process you need to go through all the process again so this is where the error comes in and people are having troubles doing that because it is quite time-consuming and it is not easy to repeat it and quite often errors comes in because you are repeating 10 to 20 steps in each of those configurations and it is quite possible that one or two steps will be missed by the person who is doing it and if you are doing 10 20 steps you might ask somebody else to do it and that person might miss it and you are not able to clearly order it whether that person has completed his step let us say all the network is given to another network link right and then building the static website content and then putting it in s3 is given to another person so these people have to set up the permissions for each of them work very nicely if one of them has not set it up that is not going to work then how do you trace it where is the problem where is the audit logs of who is doing what in your account so if you need a pencil this is one of the means of setting up your environment you get all the problems that is listed in the part of all of these things so using the console is not an option because it is very very slow I would not recommend it and companies are no longer it is going to use this mechanism unless they are setting up a very very simple one even for a simple one I am saying think of just uploading and file robucket then the control workspace but when you are talking about a single web server with elastic IP number 2 P 3 know this is not going to be repeatable but all do not ever recommend this to a client of using the pencil only when something is really difficult and you are going to troubleshoot then you go to the pencil so what other means are there for me too provision my environment writer script is that even possible I can write a script yes you can but what if EPA call to the web cellular fails because when you are writing a script you are indirectly interacting with a division or on mind you are talking to the APA s that Amazon is configured and the APA has to give you the appropriate response to give you an example let us say your first command in a script is to style create NTPC so it should return back the vp cid based on that vp cid only you can create a subnet and then assign the subnet to NaCl so if the first call to create and VPC is failed how do you track that do you write a validation for each and every command output that is returning from the server so then your script will become have a lot of validations than the actual command itself and when you are writing a script how do you update your inner garment say for example you want to change the one small value in your script say the IP address of your public servers no longer needs to be one two 1.2.3.4 it needs to be 5.6 or 7.8 so if you change it how do you ensure that the entire environment that is not get rebuilt and only the IP address gets changed unless you run the whole script your IP addresses not going to be sent to the server so you cannot just run one line one command that means you are going to write one command that means you are using the pencil again not the script so this option is also having a problem so when you're running a script you cannot roll it back that is you cannot stop the script in mid of it and then say that whatever you've done so far if roll it back straight let us say there are ten steps and you're 70 step has failed or seven step is having a problem so what you do with the remaining six steps this is already is completed successfully how do you ensure that whatever is done in the first accepts stays there or how do you remove it so is there a roll that's option so if there is an old back option do you wait the additional comments inside your script itself and how do you pass the control saying that step number seven is failed and initiate the roll back so in other words you are writing a lot of management logic a script rather than provisioning logic in your script so you need to separate these two things the management of the environment and script and the provisioning logic that is the resources in your script so to manage and maintain a script properly I would not put too much management logic there I will keep it outside so that is a very good scripting practice that I usually follow and tends to be working out for me very well if you have a better practice share it with me I will also learn from you guys and finally there is one more interesting thing that we need to note and that is very very important how long should a phosphor sometimes what happens is because of internet Layton sees you might be sitting in India like me and then trying to create an environment is let us say Amazon Virginia region or an Oregon region that is internet latency it is be acceptable let us accept that fact how long should it wait a script should wait for an output from that command if when VPC is taking five to ten minutes to build do you think that script has failed or be completely wait for 5 to 10 minutes and then trigger a failure so how long we exactly wait for some activity to be completed VPC my thought is good example think of starting up a server especially if you're setting a server easy to server and attaching an EBS volume and let us say that you are going to attach a provision to a of EBS disk of let us say 100 GB alone just 100 GB we can put the terabytes but let us start GB of 100 G this 100 GB how many of you have done that under GB disk and try to kirusha provision it with position I ops so continuing here how long should I wait how long should I pause for that is the problem with when you are writing a script the API calls will not necessarily return to you like how much it is going to take for completing and certain action so writing a sip is better than using in control but it has the quick fall such as listed in the bottom so what you have one more option finally let's use float formation if you know it so let us go ahead and see what is cloud formation and we can do that so a single one line definition of what is throw formation is it is Amazon service for provisioning resources and you can create resources in a predictable and repeatable and automated way when I say predictable that means that you run the script or when you run the template it is start calling it as a template because it is not a script which is in template when you run the template any number of times you get the same predictable output if the resource is the certain IP address is already there the template will come back and say the author service it of this commercial service will come back and say you already have a resource which is having this IP address so I cannot redo this activity and it will throw an error and it will roll back on the previous s depending upon what is the options that you chose so it gives a very predictable output and when you if you run a script before or if you want to do the same steps in a script and it is trying to find the same result it doesn't give a very friendly error message it will rather give you an error message saying I cannot create an EPC so I'm exiting the script blah blah blah it will just control both of it the controllers come out to the prompt and it will give you an error message and you need to start finding out which step of your script failed and try to find out why it is and then both pick all the way through the script it is very very difficult to troubleshoot when you're running a script trust me that is very very problematic when your script is more than 100 lines unless you are a super coder you know exactly what is the problem so moving on what is repeatable predictable and repeatable is the more or less process because predictable is you know the outcome and you are going to expect it outcome repeatable is you are confident that if you run it ten times twenty times the same outcome will also come and in an automated wave because everything is done through control which has means of logging or the request which has a mean so for tracking all the requests and if there is any changes in any of the requests you can manage it and you can manage the verses of your script versions of your stack and roll back the stack or so if something goes wrong that is also possible and most of the activities is automated using APA calls and all the a peek or subscribe if you know the service called clothe trail anybody familiar with storage ok if you are not yes every SES here every APA call to get Lewis is tracked using Amazon Cloud trail and of course you need to go and enable it if it is a new account by default it will not be there but if you go ahead and enable it and point it to a particular bucket destination you can send all the floral notes to one pocket or depending upon how who you want to filter it and segregate it you can send it to multiple places the idea is to use flow trail logs and the Provo logs to understand what is going wrong with cloud formation when some template goes wrong so what can you do let us look at it little bit in more why what is the benefits of using cloud formation cloud formation allows you to create a templated provisioning system meaning let us say today you have set up a stack when I say is that I am talking about a network layer I am talking about data layer I am talking about application layer and then the web layer so we are talking about the three-tier architecture very simple example and you have set it up in India now so your manager comes up and says AHA sir you have done a great work I want you to do replicate it in Oregon vision of so how do you do that if you are doing it in pencil you do go through all the 200-300 clicks that you will do or if you're running a script you can rerun it again assuming that your script is flawless and having no issues but if you have a template all you have to do is just ask both of the corresponding this region and point to the same template and ask the template to be run again and at the same V PC stack and then the same network stack everything will be created as it is if your template is good your stack will be created any number of things item portable that is what they call it any number of things you run it you will get the same output as long as the outputs are not conflicting here so you can prove again we do the same thing batch scripts to batch scripts you're asking me yeah and I'm just asking can be is against and they just wanted a script yes you can do the same thing but the problem the bathroom as we discussed is the APA calls the roll back logs and then choosing the control between one pro to another flow and changing a particular line or configuration item in a script all those things are becoming a problem take that let me look at the same example of a three-tier architecture of data layer wavelet up layer and web layer and your manager says in software let us reverse the scenario the first scenario you have put it build it in America now and then everything is working fine and let us say you have 10 database servers for necessity and when you come to India or let us say Middle East where the traffic is going to be less and your manager say sir I shall go ahead and remove the 10 servers and just will be two servers so you are basically going to edit the script where it says 10 - number 2 - and you are going to ensure the core is editing it it's not making introducing any other error single patch script you can you can write it to configurable or parameterised but then means you are taking an input from your command prompt and validating it from your user if the user is introducing any extra characters that is again the responsibility of the person who is coding it so make sure that the person is a inputting the exact values it is required by your script so all this additional management logic is coming into your script - to answer your question it is possible to do it in bash script but that also introduces some additional management logic which makes a script management very difficult so keep those tropics in mind when you are trying to compose bash script land for computation management or provisioning management whereas when you come to close formation you can do that with something called the parameters we will look at how we can avoid parameter mistakes by using a drop down field or provisioning it using the predefined values so templated provisioning absolutely is possible it is safe forward just copy the template again again and front them as many times as you want what are the benefits are there infrastructure as code of course it explains for itself I don't have to even speak about it your entire infrastructure is written in adjacent or young will format toad formation uses both these formats so your templates can be slowed in that the infrastructure can be stored in a version control system it's interesting to think of it that way I have a configuration of a network layer database layer storage they are stored in a version control system and something changes I can go ahead and trigger the change and make sure that the new in process is getting built and you will again I'm using the going to you start using the word stack you track all the changes made use infrastructure step because it is in a version control system and if somebody is going and changing latest is a web service change from 10 to 2 for Indian region automatically is the corresponding India administrator so the Indian Service - managers will get a notification saying your web servers have been scaled down from 10 to do do you want to take some corrective action or is this the upload action go ahead and approve it you can trigger a workflow like that based on your track changes options by using in some success code and modifies an update resources in a controlled and predictable way so explanatory you can go ahead and update separate sections and make sure that your stack works every time when you run it in a different region and since it is version controlled and just an additional pointer when you are doing an enterprise level deployments all the stack templates are more or less is going to be inside your X 3 buckets and if even if you upload it to Amazon using the Amazon dashboard it is going to be a copy of it is going to be copy to the Amazon s3 bucket a packet will be created for you or the PI will be put it into a general packet in your account and then from there only a sack will be created and automatic you questioning will also be enabled for you so it is always checked for you for the poor that you are using in throat formation so in other words whatever you do is version control and practice so that it is very very predictable whenever there is a change happening in your code or in your infrastructure other words moving forward it is very declarative and flexible it is not very difficult to pick up you choose whatever the sources you want at the top of it and then you can customize your template through the use of parameters the parameters I chose about for example how many web servers you want and let us say you are an microsoft friendly company you want to run all your VM circa let us say B and Shirley for now or the PM's on windows-based machines you don't want to use any Linux based machines then you can ensure that all the a.m. is that your clients or users of his CloudFormation template using are going to see only Windows a.m. is because you are going to create a parameter which will list all the windows AMS and first use case was a windows friendly company next use cases a security friendly company then you can say that one is the latest operating system images only will be allowed for the users to be created so when your they try to choose an ami or the prop run option will list only the latest am ice next is then configuration management deployment use case that is let us say your company says that I need to have data center in the East Coast I need to have a data center in the West Coast and input the data centers I need to use a minimum of two availability zones then that also you can enforce on your users saying in Virginia out of six data centers use one and the C that is one A and one C and or again this is two data centers that you have that is AC one AC to use these two you can enforce this kind of behaviors and I repeated you are indirectly enforcing good practices or best practices on your users when I say use this I'm talking about code for machine uses the case you write the truth formation scripts might be different it might be the same person who is the end user also but when I am referring to excuses there are people who consume the curl formation templates and you want enforcing such good behavior or standard practices from your users by using parameters and validation fields so that is what it makes it very declarative and very flexible as well so resources and stacks how do I reuse my stacks we have some couple of nice image I just wanted to show the imagery skills so I just put up this slide let us go ahead and see that it might be true it might not be true but let us find out let us say you have a website one and the website one requires and auto-scaling that is elastic load balancer auto scaling and RADS so these are the three fields that we are talking about now and let us say that I have a one more website that is website - which also has a similar requirement that is elastic load balancer auto scaling and DynamoDB so if you see the website one perfect one required RDS and would say to require 1000 GB that is the only difference but both of them need and ELB and auto scaling so how do I avoid rewriting this code for TLB an auto scaling is there an option so that is where the nested templates are coming into picture that means that what I am going to do is I am going to separate my ELB and auto scaling code into a separate template let me call it as a network or a network template or network stack and what I am going to do is I am going to create one more template or one more storm machine stack what website one and what set one was referred to my year will be an auto scaling template and it will have code only for RDS and again it comes to website - I'll have one more template for nested stack and then would it be this way I am going to reuse yeah I was suspecting that let me just wind the back since you some of you might have not heard everything I'm just going to wind back a little bit so let us say you have a website one which is a requirement for creating a very simple website of RDS that is the database layer and then you are auto-scaling for your application layer and then you have ELB so the load balancer sends the traffic to auto scaling the application layer and application layer sets it to the database so this is website one website ooh is very similar but it is using dynamo DB as the backend it will be say auto scaling is editor motive is also there so if adding my ELB an auto scaling scripts two times and trying to maintain it in two different places whereas only the mega typically this different so I will remove the ELB an auto scaling code to a separate template so that this template will always deal with ELB and auto scaling and only the input parameters or values will change what is depending upon where it 100 set to and for website 1 I will create another template that is here and then you have only an RDS stack here it will refer to the network layers here so this is called nested stacking in other words I will run this when you do something like this what happens is this command will go ahead and check this stack it will run this first take the output from here what is the ESB value what is auto scaling value and pass it to my RDS node and my RDS will start running so when it comes to website - again it will go to this stack and my ELB and my auto scaling and pass the outputs pass the outputs to my website - and then it will start running my dynamo DB my apologies for the disturbance there so continuing so we have removed the repeatable code into a separate sack and pointing it if using nested stacks it is possible absolutely and this way you can reduce the management work or reuse the work that has been already done for the best practices here and there so this way your the network I let us say he is a network guy and he is doing all the network stack and ensures that your network Lee is very good and this guy is going to be very very happy because his work is not duplicated two times and these two guys are all so happy because website one template is becoming very very easy to manage he just he or she has to just pointed to the existing template and make sure that if something is goes wrong point the finger at network guy and say he can do that properly and I did my 1000 degree very well so this way everybody is happy and everybody is focusing on the core skills so use nested stacks for reusability and especially if you find yourself writing a script and rewriting the same script again our same logic again and again that means you are doing it wrong do not rewrite or to not repeat yourself this is a industry level best practices weather come to scripting whether it comes with templates do not repeat yourself again and again try to find out how to avoid that by moving certain segments of code outside of the logic moving forward how do I plan my stack this is one of the most important pieces of information that I am going to share whether it is for formation or butoh or Amazon CLA or any environment that you want to provision I would like you to focus on the next five to ten minutes here because this is common across however you want to do it when you want to plan something in the AWS cloud or any environment I would say this is so very good best practices and it is shared by Amazon itself which is not something that I am making up I picked up this best practice from one of Amazon's two conferences and I really liked it as well I do it I practice it myself it's kind of saying eat your own dog food what you preach you practice so when you are building a network stack or when you're building a throat environment the first thing that you need to focus is I am roles might be sound like counterintuitive why would somebody want to start with user ready because user ad was going to create on the server's no that is wrong absolutely because I am is a global role and it is applicable all over all the regions it is more easy to start I am role in the beginning and make sure the roles that are necessary to start a server and connected to the database all those things are built in before itself with the necessary permissions or in other words policies so you create a role I seen in s is very permissions and you start the server with the role whereas in a conventional environment what you do is you build your operating system do the security you put install your application then try to create the user ID and give the user ad permissions that is how happens in a traditional word but we are inverting the logic here we are going to start with your I am users groups and roles and policies and we are going to use that when our server is going to be provisioned so first is I am next the network layers the VP sees Internet gateways VPNs nerds anything else whatever you can think of networks that is what you are going to build next and this is going to be applying for particular now we are going to focus narrower and narrower to your region or rigor availability zone or your application step so we are going to build the network layer which boo is going to decide which region we are going to use it is going to decide what is the ACS we are going to use Internet gateways and all those things after we do that let us go up we are going to create our shared services something like which is going to be common across multiple availability zones the commonality elements of the shared elements across your network layer then you start building your back-end systems that is database layer or any pack ends for elastic as short as it is or a memcache that is the you take a layer of back-end services I have chosen an examples here rather than putting a service name but just what exactly it means you when you are talking about a back-end you are going to talk about data services and all those things then you build your printing services I am looking at it from an application perspective rather than from an Amazon service perspective so that is what the stack is building so we started from I am then the network layer then the shared services which sucks across all the services then your back-end layer then your front end where your customer is entering your business it might be customer facing website it might be a mobile print end or back-end or mobile app or your seller of your partner is using this services so finally you put together you get something like this now so once you have all the stacks available and 3d then you build your development they want able to integration or for product I mean your customer will give you the names for your enrollment this is just a simple naming convention that I have chosen absolutely you can welcome to chase excuse something like Dell UAT in to end testing equi I've seen all kinds of variation in my career that was something called just free flow I don't understand there will be tests in Atonement also when I was working for that account and there was a priest water in between test and prod I still don't understand that but anyway they have different naming conventions you go over and have different naming convention or it work comply with what your client requests you in other words more or less in production you will definitely have one or two enrollments behind your throught so that your testing happens and so that your code and everything is tested before you go online so I said one or two in adornments I am going to show you some deployment models which is completely inverted this thao test product logic that is a completely in the box work it is completely changing let us go ahead and see some of those at deployment logics now so before we see the deployment najuk there are some use cases here the slightly I'm touching the deployments if you notice the third point but we'll come to that now so the multiple use cases are there in cloud formation one is stack replication we saw that we are replicating from India to u.s. or China tonight is not there let us take Australia and just quick question - how many of you know that in China a the base is not there so getting bad stack replication you can replicate it all across Amazon's data centers and regions an infrastructure scale out yes it is possible you can configure auto scaling built into your template itself and you can configure all the necessary rules I'm going to get some water so you can scale out infrastructure scale out as well as gain scale in is possible because you are going to write in the auto scaling rules inside your templates itself or you can make it a configurable parameterised value say that so that your user can choose what is the CPU load that is expecting on his stack and then he can customize it or so the finally is the blue green deployments it's a very very interesting thing nowadays companies are going to have are already having only two environments in their closed deployments that is one is blue and one is green let us say the green one is a production serving like Africa and then you are trying to do some of the climates and keep some more developments so if you want to do testing what happens is the stat is updated on the the stack is getting updated on the blue environment and then every testing happens of the blue enrollment and once the stack is getting completed and once the stack is completely ready for Russian use the TLB or the route 53 is pointed to your blue enrollment knob so that your current green becomes blue and your group becomes green in other words you just slap the environments so you were in other words your test it becomes your prod because it has the latest version of the code which has been tested and running and yes rod is having older version of the code and becomes your testing environment for next level of deployments so you keep one swapping again and again until it's yours until you're ready for next requirements so the reason people do this is it makes it very easy for managing environments you don't have multiple environments to manage and it also ensures that you have a production copy of environment always ready and available if something goes wrong in other words you will glue which is used for testing can also be reverted back to the previous stack because remember all the stacks were all version we're in control and infrastructure is a code now you are going to deploy new code on top of the existing code so if something is not working you just go back to your world enrollment and say that this is the one I am learning and this is the version that hashtag that is a version six is the latest code working tested and find rollback to version six and everything comes back here so two environments Bluegreen deployments everything happens fresh as well and this is I talked about the deployment model where an existing environment is updated sometimes what happens is when you're doing container the base deployments the entire of blue Shack is rebuilt from ground up and then pasted and then it is going to be deployed to the cloud so this is how a Bluegreen deployment works and it is getting slowly popular and and DevOps based deployments almost always fall under Bluegreen deployments because it is very very easy to recreate environments when we are talking about the container the bridge to deployments so this is the one deployment model I spoke about that it will come again one more time in the slide I don't want to talk about it prosecutors core you already know that what it means let us move forward these are the different terminologies that we need to remember when we are talking about the clot formation these are the different components that clot formation has the first piece of the puzzle is template this is the place where all your parameters the first configurations and the configuration actions are written down this is the piece where you put in everything what you need and how do you put in you use either a JSON format or in Campbell format I am NOT going to say what you should choose personally I started using Yama now because I find it more easy to learn and more easy to understand when something goes wrong Jason looks more structured especially programming languages like adjacent because it has got of brackets lot of commas and it you gives a sense of order when you look at it but troubleshooting below why of you maintenance point of view is in my opinion at least is difficult because lot of commas lot of brackets means there is a possibility that I am going to miss one of them and the template is going to cry foul that I did not do it properly so I don't want to do that headache I started using Jason my one of my original templates when I started writing it or send Jason what the latest one that I wrote is in Campbell only and I really like it and I find lot of people with the same argument so I will leave that up to you and which format you want to choose you can choose Jason or a humble go ahead and try to learn whichever format you like if you already know one specialize in one continuity the same thing so that is the first part of the puzzle now so you have written the template but yeah sorry to interrupt you just a quick question so young witch who AM L AM L is nothing but at another markup language I will show you when answering the demo I'll show you how many ml looks like it's a format it's not a language if I've ever mentioned a language it is I would say it is a markup language meaning it's a syntax you can say how to write a file that's it it's not a separate language as its own definition which is silence okay so I can taste my question things okay so again if I made the error of saying either one of them is a language it is my mistake neither of them is a language post both of them is a format or a syntax think of it like an formula you have like you need to write it it's something in a particular way then we just follow the same thing that is what both of them are script object notation that is jason is another markup language which is a notepad based a tab or tabulating or space based or differences will be there we will see that when you look get a template so the first piece of the puzzle was rating the template itself assuming that we have done it what we do is the next step is be uploaded to Amazon pro formation service and Amazon cloud formation service has these features you can create a stack you can update a stack you can detect errors and do roll backs when we say roll backs it is step number seven has failed then you can have an option check box actually speaking you say that remove all the steps that you have done from 1 to 6 from my account do not keep any impurity sources that was created in step number 1 to 6 remove all of them and people remove it completely there it will be no traces of it but if you don't check this box which says no rollback you ensure that there is no rollback and you want to trouble with what went wrong then one two step step will be in your account so that you have to go and clean it up later so that is what a rollback means and error detection has I mean if you are having an errors keeping the stack is helpful to find out where is the error because everything goes away or everything disappears you don't know that was a problem but you have an error in your hand so troubleshooting that is the way you prefer shoot when you're creating a or formation template so finally assuming that you have created your template and put it into your service and a stack is created once your stack is created you can have service event our customer ID and so it it is not written properly but any the point is you can make your entire stack aware of events that is happening around it you can trigger lettuce is the throat formation stack is completed successfully then you can ask your third party services to start sending traffic or you can treasure some other lambda actions or some other actions in database itself or you can send an email using SES notification saying the sack has been completed service manager be happy everything went like line online successfully something like that you can trigger any downstream actions based on the stat events and it is fully customizable also and once again I repeat that I would want to create those management logic inside it sells which is quite possible I can create an SES notification or SNS participation inside my stack itself but that means that I am having additional logic in my stack which is not necessary for my stack itself so although it sounds very easy to put in SMS notification topic inside your stack itself for triggering a notification to your target users to not do that I wouldn't advise it those kind of notifications should be something separate what it should ideally happen is when a stack is completed you should take the success message or the status message and then subscribe to that channel or notification topic and then trigger one more action on top of that it should be outside the stack itself not inside the stack so moving forward let us look at what is a template in detail and what are the important things we are going to do is little bit of a biology class here it is going to break up the template piece by piece and learn that so looks like this I need some warm water to think can you ever order something so this is the biology class we are going to talk about on the left hand side isn't the Amelie template it is not the complete template it is just a description of what or all the important things but it is a centered Stags or all the keywords and the green color things are things that you will write most probably as we might have already read the version date is not mandatory s on today but I would very strongly recommend to put the version date in your templates the reason is with every version Amazon has certain kind of ApS supported and using this version date only Amazon will know what were ApS that you are expecting say for example today they have 10 services and there are a PA codes for toasts and services Amazon will support it and tomorrow they are going to launch a new service and there is a new API called and let me pay a call might be different from the previous 10 API calls then so to accommodate the difference Amazon will create a new version number and then everybody will have to start using the new version number to work properly with the new API calls when something like that changes you need to be aware that your Simplot is not going to work properly if you don't mention the version numbers because amazon wants to create those who kind of traffic separately and that is how they differentiate it using the version dates and suddenly you if you find all of the 100 templates you have written it's not having the version number or version date you will have to go and edit it all of them and that becomes exercise of its own so we don't want to do that later it is always a best practice and let us follow that put in the version number it is nothing but the date and amazon will automatically add it when you are using the console do not try to delete it because it is useless as of now so that is the most important bit of it and why it is important moving forward resources we are jumping little bit down but that is not should not be concerned the resources is the meat of the photo machine template and when I say meat it is the interesting bits of the Commission template so the resources this is the place where you are going to write our infamous on what are the resources you need I need a V PC I need elastic load balancer I need a PC - blah blah blah blah things this is where you write all those things a set of commands on how you operate it we will see that when you are try writing as its source so but remember that all those things that you need from Amazon you are going to write under the resources everything has to be intended for spaces or H pieces whatever you are using it should be consistent throughout the template this is the one important golden rule that we need to follow in Yama little place if you are going to use four spaces for the first line and the second line so if I am going to write something let us say AWS ec2 and below that I need to give two more spaces or four more spaces whichever I have chosen here so always use a consistent spacing whether it is two or four or one tab or two tabs in other words so that is the only requirement or only rule that camera template enforces on you and let me repeat if you are going to miss even one space general tab it will not work if the commas and brackets are a problem with Jason templates spaces are your nightmare in Yammer templates so and it is not very difficult to validate in a multiplet it is very very easy there are so many online tools you just have to copy paste your camera to some online tool and say this whereas validate my humble is fine and if you can come to you whether it is find out it will offer to correct it also so keep the spaces in mind that is moving back resources we have covered and descriptions thing if anybody wants an explanation I can go ahead and do that but I would feel this is very very self-explanatory here the practice if you you want to know what is inside your template and you want to give a very nice description to that for anybody who's using the template so you put in that information in your description field parameters as of now it is not required it is a place where you customize your template so that anybody can use it and it is the place I have you make your template general and available for reuse say for example your hard holy template of all your department saying the ABC department ABC department always likes d2 d3 large or c4 large and they always use Linux instances and the HR team also wants to use a similar setup but only difference is they don't want such a big servers they want to have a input system they don't want a c4 system so for that they have to they don't want to write the entire template if you parameter is only the type of systems then you can use the reuse template within both the department's so make it simpler so that multiple departments in your companies can reuse it that is why you need parameters and it is not mandatory but good practice mappings remember the use case about security I spoke about using the latest machine images or the use case about using multiple availability zones so if you are using an Oregon region and then you particularly favorable of particular cases in that Oregon creature you can enforce that behavior using methods that is for mapping sis and once again it is not a mandatory requirement in the template so as of now you see that only this sources was the only mandatory section in the template and so far nothing else is required that is what the first line means conditions again let us say you are asking an input for say your V PC and the input for an V PC is what anybody wants to volunteer what are the pieces of information you need for an V PC through the the reason we were talking about the conditions and V pcs is it is possible that when you are creating maybe you want to take the CIDR range from your user and an end user is in putting a CIDR range you need to validate whether that isn't valid CA th are they given someone pre-existing cadr or is that they are giving something like a slash 43 tip will not work of course so you need to validate and that validation you can enforce that PJ we're using conditions in other words let us say your company is using a particular series of IP addresses and you want the user to create the stack inside the particular series of IP addresses then you use that values you enforce that behavior by using conditions or say for example all the network elements should be using the same SSH keys then you can enforce that behavior by using a condition key which will force them to allow or choose only a certain set of keys that is in your account that key so I am talking about as it might be you generated or Amazon generated but you can enforce a behaviour using the squeeze so finally outputs you can write your scripts to give you outputs you can ignore this field completely if you are not expecting any output but if you want to trigger some notifications if you want to trigger some actions or if you have a nested stack especially as its face cross stack references you definitely need to write an output section because this output is an input for the next stack say this is a network stack you need to know the VP GID you need to know the subnet ID you need to know the internet acade way you need to know the let us say you're building a load balancer you need to know the domain name or the URL of the load balancer all those outputs will form your input for your application stack because application needs to receive the graphic from a particular security group where your ELB is residing so you see the flow here so vlv security group will have outbound rules to allow traffic from application group or in other words like that we'll be sending traffic to application group and application group will have an output - ELB group so this is how you take the input from one say your own stack and pass the input to another stack and make sure your stacks are all working nicely so output field is basically for that reason and nothing else if your cross track references 100% is necessary or if you have a single stack you don't have a lot of price tags it is still all so it's good to see let us say you have started web server and you want to know the public IP of the web server instead of walking into the server you can go to the template and get it and temperature will have a public review of the server ready and available for you so that is about the template and not any other where the class is completed let us go ahead and deploy it it is go to physics table so deployment styles blue green deployments if anybody is not understanding it I can happily explain it but I am going to give a person that you pasted edges it so on the left hand side is the in-place upgrade that means that you have a stack on the left-hand side running the latest version of the code all shiny new code has come now from the developers then what you do is from here you update it or play all the updates here and then this tag becomes this one and then you point all the traffic to here and then you get a new stack available it is very fast because you are having only one stack to update and then it is very cost efficient because you are running only one end or amount you are not running multiple environments and it is simple and data migration is not necessary because every data that is been attached to here is also available here it is an in situation upgrade it is pickup you are having a house and you are upgrading the same house again and again to make it move looking better and better with the newer furniture your kitchen all sorts of things it is very simple cost efficient and faster but the downside is if something goes wrong you cannot roll back because everything is updated everything is shiny new your whole back is going to be little bit difficult it is going to take little bit more time I would say to roll back especially we are using even a cloud formation template you need to find out what is wrong and then you roll back it takes um about 5-10 minutes was invited minutes but some businesses cannot rave even by 1/10 a minute sir let me put the question this way rather how many of you guys have seen an obeys with say google.com say of Microsoft calm we don't use Microsoft comma big calm but this is obviously 65 when you are using it to the enterprise almost never it queues and poor not word page or page cannot be displayed or I cannot show you the results and we see something like that because those guys have a deployment mechanism which completely hides those kind of error messages or it make sure that end user does not suffer any difficulty when they are deploying new port so blue/green deployments that is where it comes in because you have a deployment that is all running and going production traffic and taking care of a person traffic and you can see this arrow match here that is so true p3 is sending all the protein papers through your stack here so you create another stack and do all your testing clean do all your testing and create another environment that is this is your new template and from a new template you were created to your stack this is your stack and do all your testing when you're testing is successful your testing tema says that there is no blocking issue everything is fantastic let us go ahead and launch it then what you do is you make your wrote 53.2 your new step and your old stack is still there by the way so what happens is all the production traffic starts coming here and everything is fine you are not touching an existing stack that is your working stack is always there still there but you have a new stack which is ready to take care of Russian traffic so only don't say this you have a running food stacks which is full scale and you are going to incur a lot of cost for it but the upside is you are going to have a very very quick backup plan or a quick fail or plan because all you're going to do is point your throat with p3 traffic to here because it is running and your world working code is here and the new working code which is having the problem is not impacted or not rolled into this one so you are back is almost instantaneous I can say it's just pointing a rocket is being to the old one as simple as that very highly costly because you are running two different deployments but very very fault tolerant I can say because it doesn't show all the problems of your deployment to your customers and if customers don't you will know that you are upgrading on the background you change something on the background just let me experience in Google or Facebook so we have you have listened to me for some time now it is almost one and a half hours I'm talking so if anybody wants to volunteer and try to answer this question let us make it interactive so the few best practices to ensure that your staff is running error-free first is validate your templates how do you validate my template Amazon gives a nice little interface and it checks for syntactical errors it does not check you for cyclic errors for example once I click an error is let us say you you were trying to expect a load balancer and then you are trying to expect I'm not able to clearly formulate a cyclic error will come to that but anyway it doesn't check for your logic it just checks for your syntax whether it is Jason syntax or grammar syntax that is what I my zone validation are and test and then another best practice for making sure there are you are going to have an error free template this parameter is most of the fields where I was possible because if you parameterize and then do mappings and conditions you are going to restrict the type of input your users are going to give into a template and that way you are going to have a little bit of freedom I mean they have a little bit of free sale form to work with they don't introduce a special character which might break your template they don't use some x1 large machine or X large machine which will break the budget of the company so you param place wherever the input field we were taking into let us say you're asking for an Emmy ID from a customer then you do a validation who they're saying that am a hyphenate characters as whenever it has to be there and there is no special characters so you do a validation inside your template or enforce them by saying these are the Emmys these are the description for each of Ramy edits what is the one that you want to choose so things like that so this way you avoid errors and then use deletion policies when I say deletion policies I am talking about do not keep the if you if you know that your template is going to run fine if you'll know that your template this having no errors already rejected when you are running the template ask your users to enable the rollback policy so that when something is running and at the tensest if there is a problem you roll back the errors because in production environments you don't want to have resources which is half configured say they take example of the EBS volume that we spoke about you have an application server and you want a 100 GB provision to I of EPS discs available and you're waiting for that input and you while you're provisioning is happened you don't want to start writing data to it assuming that action is completed or if the provisioning is failed for some reason let us say Amazon has a development version is failed and you subsequent code should not run on that point because your disk is not available and your application server is another words not available although the request has been completed successfully so your subsequent code should stop processing and this whatever service that you have created up to that point all those things has to be rolled back you don't want those resources I am taking an example of 100 there are 100 GB of disk apart if it is a terabyte although Amazon so that is cheaper you don't want to have a lot of disk hanging here and there in your account which is not going to be you for certain reasons for troubleshooting purposes it might need it but otherwise you don't need it at all so do not leave your resources half-built or half-baked resources in your account always clean them up and finally in cumin myself actually happened to me I forgot to turn off yeah you are not alone it happened to me as well so yeah lunch a Dennis cook lucky for you and that is not the first time I am hearing that story as well and I have recommended a few people and 40 bucks is too big there means let us say 5000 rupees in India sorts of big money here and I would recommend this as a best practice but if you are a newcomer and if you go to Amazon with a honnest explanation most of the times the customers actual friend interrupts our understanding and they look at certain parameters also in the other context not like they just blindly queue out the discounts for your account or no cost delay they roll out most of the roll back most of the billing if you are a genuine customer if you like own Disney or those kind of cases and I have seen many people get some discounts if that is the first mistake that they have made and if it is a smaller amount not for a crazy amount like you start an x1 large and then say that I made a mistake and run it for 30 days I don't think so that is going to happen but you can try your luck but I wouldn't recommend it I would not rely on this all the time so because we are recording it and it might be uploaded also so I don't want to publicly recommend saying go to Amazon and say that I made a mistake after that it isn't so finally moving on use I AMS and tags almost always use I am wherever possible I cannot stress it anymore because quite often nowadays security teams and clients are coming up with this requirement in that bid itself use tagging policies so that you know what is inside a tag and water sources are filter by Laocoon when you are using a toad formation Amazon itself will add certain amount of tags to each of those resources but it is always good practice to you to have your own tags attached to it so use those these best practices when you are creating a flow function template and the first two or three helps you to ensure that your template is error-free and of course there is no replacement for a human logic if you can possibly get it validated by somebody that is also good so basically there are three or four ways to get your syntax validated you can use the Amazon API in other words you can connect template from your command line you can turn the template from the console or you can run it from something as a particle like a tariff or from Hachiko so there are so many throat management platforms are available they all use some kind of an APA to interact with Amazon some of them create a proper machine template itself some of them use Python or Java or come paint up a test became the factor is you can validate your template using a pH and and as I said earlier it checks only for your JSON syntax it doesn't take for your logical errors that is slightly the circular dependencies circular differences meaning no non-technical terms let us say that your step one you are expecting step number seven output and at step seven you go and say any step one this has happened in Railton I myself were filling to the scrap I was looking for a certain IP address of a certain machine when that machine is still not completed its process of coming up online and being ready so you might think that you have issued the command and you might want it to do some downstream processing the one activity we keep referring is the EBS volumes and steps number eight or nine you're looking for provision die-offs with a very high throughput but it is still not ready it is going to create some issues with your templates this will create a performance issue approaching a ops but there are certain dependencies like expecting IP address and trying to assign it to to a load balancer event occurs is still not ready so those kind of things also will happen take care of it logically there is no substitute for a human logic here as of now atleast Jonah's not paid anything to check for circular dependencies this is completely a manual work and your template structure don't try to create your server first try to create the layer stack that I showed you some time back the six or seven layers that we spoke about I am Network back-end print end and all those kind of things our logical organization don't start with the things that you are easier or most familiar with so that way you can have a very clear logical flow of what you are building now we are going to see parameters field remember the options where we are trying to customize the template to make it more interesting for people who are using it this is one of the examples here we are giving the option of making the parameters whether it is the stack is in staging or production or development you are giving these three different values so this is the variable name and this is the variable values the rest of all is like this is whether I want it with default value whether it is a type of value string or integer or characters blah-blah-blah things and this is a generic district description what is this field is doing and if you want to do constraint description saying I want it to be either one of these few new government's only or I want it to be you Oregon say for example you want to constrain discipline is your security architect says you need to use rated a question only no more a tech seven is supported something like more information for the uses so that he can make it inform the choice of what to choose or something like that are you sure you are going to deploy in production how you pick a necessary approvals something like that it's completely this Texas you write but in to inform the user so that is parameters with I think that's one more example for parameters let us see yep this example is about V pcs and subnets and security groups as you can see this is an Inc single parameter field well the previous one you gave the input here and not giving an input I am asking Amazon Excel to provide me the input for example I have given and VPC ID and I am giving something like la blah blah aw s : ec2 PPC ID blah blah blah - so what is this piece of syntax does is it goes to AWS it goes to easy - and it goes to my VP C's in that particular region where I have opened my console and it tries to find out what are all the B pieces I have and what are the IDS of those VP sees unlisted in this case if you remember in the previous parameter description we saw production development staging those were the 3 allowed options in this case what will happen is it will give me an option of V PC IDs in my account so that is what is going to happen when I give this command so what will happen here is it will likewise it will go and find out what are my subnet is listed for the VP XI that was chosen here and it will list those things here likewise it does the same thing for security group and it will do the same thing for key names also that is how you pick up values in your account say for example I talked about the key that particular key that needs to be used for all the services and your security architect has mandated that this key is 2048 with encrypted I want you to all of it to use it today this is the key and tomorrow he comes back and says mister Nezzer the key is hacked somebody's a crypto systems so we are changing the key go ahead and start using this in your template at this point of time you don't want to come and update your templates if you are hard-coded this field you will have to update in each and every time key changes you don't want to do that so you want to choose what is the latest a key that is available in the account and give the option for the user to choose that key you don't want to hard-code it in your template so if you write something like that automatically what is available in that account will be picked up and shown here so you remove the headache of managing the keys from your responsibility and pass it on to the security team so that they can manage the key but all over the account and likewise the security group world so you don't want to point out to particular security group where the circuit ports are open not open you completely outsource the responsibility to your security team assuming they are there of course assuming they up there and you use the security group IDs if they are not there those responsibilities also come to you and just because you know that what key names are they don't do that even in that case go ahead and parameterize it and then your scripts become very flexible and reusable so moving forward since to adjust to it so here you see the parameter field at the top of the rock formation console and then you have your V pcs that will have a drop down will have the V PC ID and it will list down your subnets and this will most probably change based on the V PC that you choose and you can see the security groups are also listed so that you can choose whatever security group you want for your account and remember these are always listed options when you say listed it means that it gives you option of choosing multiple choices here whereas V PC government you what the resource to be in a one V PC only you cannot put a resource in multiple PVCs so in other words it will become a drop-down option amazon has made something some of this logic very very simple so you cannot make an error in those kind of places so moving forward parameter constraints remember the IP address or cidr block we spoke about here I am particularly looking for a source IP address from my end user as you can see here this is the source CIDR for my ssh server or jump server and I am saying this is an allowed pattern and I want the characters of minimum of nine characters and a maximum of 18 characters in other words this will accommodate a 2.5 blah blah blah blah blah or the full octet and then this will accommodate the single-digit IP addresses also it will ensure that all the necessary IP address pieces are there and you cannot enforce a singular expression of million for that you are not introducing some alphanumeric or some crazy character in my field which is expecting an IP address good behaviors you input sometimes on be our users so that way it gets propagated all across these company mappings do you really want me to talk about a device and mappings again here mapping sister aren't type of VP C's here you see I am giving everything here I'm giving the user be PCC ad or sub next bah blah blah and everything everything is created here and you can see here a nice split if you want to use it for your any account you can use a nice slick but it is slash 16 try to play around with a smaller number let us say is like 17 or 18 and how you achieve 6 play split this is a six way split you have 1 - 0 - 6 so 6 into 256 something like a thousand 500 so how you find out how you split all those IDs into multiple subnets and what number should come here ideally so try to play around with that when you're experimenting and as I was saying your you can configure the locks groups and you can configure it once you compute log routes you can send you all the logs to your log group and close foundational is populated for you so that you can create it to a sub spunk or you can spiritual monitoring alerting system whichever you choose and then find out whether something was wrong or something is going good either way you can do it in other words cloud watch logs works very nicely with float formation and this is the interesting bit where you can integrate it with your see ACD pipeline let us say your app comets also happen to a version control system and remember now in the first exit gates are also coding interesting isn't it infrastructure is a code it is not anymore a database of the editing platform of the editing movies of crediting these gates are writing a template out so at all is nothing but a day where you poor goes to online our today where you do em up get all windows update those are not the days those are days are gone so you commit all your latest versions into version control system and whenever a version control system gets a latest commit it has the ability to send the web hook to any receiving dosing system you can trigger a pool model or a push model so this is a pool model so the arrow mark actually should be here hey come on I have a question go ahead so in a way somebody for commit every change that you make has to go into the version control system so everybody can see it pull it and update it so when the Wesen control system has an update it can trigger a notification to a configuration management system or an authorization system like tin cans or opsworks code works chef pocket whatever you want to call it and once it has the latest support then it can start deploying it using cloud formation commands or it can trigger it downstream actions in this case we are talking about AWS CloudFormation so we will go ahead and pick the template from s3 and it will send the template to to get of this code permission and the it abuse will send create the stack itself and as you see that the pointer is going to do and test so ever the testing is completed then automatically your blue-green deployment the test will become the live and this guy will become the test for the next requirements so this is a I wanted to make another slide but I didn't want to make it too big the test becomes I mean lie becomes the test after this blue green departments so that is for continuous integration let us move forward so this is a very very simple demo app that we are going to build today using code formation I did not want to make it very complicated it's very very simple it creates a we choose a region we create a V PC under the TPC we choose a particular region and we create the subnet inside that sometimes we are going to create an Internet gateway and security group and inside that we are going to create an web server and assign an HTTP server we are going to install and create a small sample website inside it website in the sense it's a single page will come to demo today bla bla bla so this is what we are going to move along hello hello there's a little bit of self-promotion here coming up next this is my github account and this is the template that we are going to use now I have given you a URL you can go ahead and check it out and you can see here there is a blah blah blah stuff about the saying WordPress actually just not WordPress we have I will eventually get to WordPress but anyways practice the small diagram that I showed you the architectural diagram that resource and impact we are going to use that a critical diagram has a reference and going to build that architecture and this is a nice description of what this template is supposed to do and the template itself is in this file this is the file if I will just click on that it will open it I open it already so the file looks like this so let this be as it is or you want me to explain it let me better do that let me just zoom out a little bit too so you can see as I see it a little bit bigger if you remember the undertow means that we saw a boarder template stack and all those things here is the ocean as I said I am using the ocean eating I want awkward and as a description of my stack although it says a VPC network start it also has a web server so there you see my dog food is not exactly complete and here is the important parts of it the parameters first thing is I am defining my CIDR block and it's saying that this is my allowed pattern and by default give this IP address although users can choose whatever you want by default give this IP address they don't want to change it they can just use the existing one itself and I'm giving them a description and also I'm creating my public availability zone my private availability zone my public subnet C idea and it just for easy reference I have given this name this names you can change it you can write whatever names you want I like to have a very descriptive parameter fields so that my users can understand what I'm trying to say instead of saying a c1 or PPR IVAC obviously it also works if your users are going to be smart enough to figure out what you are thinking if you not they're going to call you and ask you hey if I'm using your template I don't understand what this field means can you just tell me what should I put in here that means has bottom plate has failed in my opinion you are not having automated deployments people are calling you still so basically we created VPC and split it into subnets and CID X and if I scroll down I am creating an instance type I am going to enforce my users to use the t2 microbe only since this is a demo template even I myself don't like to have a pre-shock so I forced my template to use only this one when you are trying to do a demo no I will just add some more see what the value changes what is the proper option gives me and here see I am enforcing the key name by using a parametrized value here it automatically goes to my account and six out what is my keening and I am giving a default value also here so it is not necessary that you have to give atif or things because it is my account I know that this is going to be there I've just given it but the user will have an option to pick it directly and I am NOT hi pouring it by the way that's what point I'm trying to make so if they are going to choose a value it will automatically go to the phone and show all the values mappings I am restricting my users to to particular regions and here in so restricting them to availability zones I am restricting them to the type of ami Satyricon use in other words I'm asking the producer 6450 M a and the ami ID so you see the region map the mappings field is very very flexible you use it in multiple ways I showed you an example of how to restrict improve certain availability zones but I am actually doing an example of how to restricting them to a certain a.m. eyes so it is up to your imagination on how you want to use the mappings field and say for example if you're developing a dual stack you can take the value of dual stack here and say that you can only connect to a database don't try to pack you a brush and Atticus since it is in the same account they reduce account don't for you to production connect your question database so here automatically and puts the value of say DB equal to a database DB you are something like that so all those kind of behaviors you can enforce by using mappings field and finally the most important part of it resources so we have collected so far up to 50 lines if you notice we have collected input from our users on how to set up over to run what we have not set up in athing just collected the inputs now we are actually sitting up the V V PC and we are sitting up the Internet gateway and you are attaching the Internet gateway to you might be PC and why do we have to do that and the road table ensures that all the internet traffic that is see here and taking my internet traffic and sending it to make it where I am referencing to my Internet gateways remember I don't have the ID of those fields but I need to create them so what I do is I porn reference them by that name that I used here if you see I I gave them the name of Internet gateway and using the same reference here and it's asking what is the route table idea in you to use I am giving something like activity if you scroll up little bit I have created an RTP which is nothing but the customizer hoping table and this is another best practice that is coming from me do not edit the default row table when you create a V PC leave the defaults alone and create your own table always always always do not attach the defaults it is it might be very tempting because it is already there and just modify it for internet ropes but leave that alone because it is for Amazon to deal with things that is why they were created a default if they started a customer have to create it they would have left it to us so do not attach their create your own and then create a public route that is Internet traffic is going out then I am creating my subnet itself and you can see here very nicely I am creating a tag here dynamically my tag name key name is name whereas the value for the tag is very dynamically created I am creating using the name of the stack and using my public availability zone name also with a combination of these two my tag name will be created so my subnet will be nicely associated with something like episode - one a well X is the move or permission something like that we will see that when you are doing the demo itself and I am referencing it so skipping the private subnet which does the same thing and I am associating the subnet with my coding tables and other things so you here this is our TV our TV means a row table in my world and I'm creating the security group here you can see here I'm having inbound the protocol of 422 basically not necessary but if I want to troubleshoot or something is not working then I need 420 to access so I am doing up front port to port and opening it to the entire world this is not a security best practice again though do this in production or don't recommend this to anybody you need to give only specific IP address here so that certain people only can access your machine and since this is a demo leave it as it is and we are starting a web server so we definitely are expecting in traffic and we want to be the next Bill Gates so leave this to be 0 dot 0 dot 0 and we are setting up our webserver finally so far we have set up the network layer if you remember you building stack by one stack at a time that is one layer at a time so network layer is completed then you are going to come to a web server layer and see everything is referenced here my key name is automatically picked up from the previous values the user must have chosen and for image ID it is picking up from time map remember the region map by 3 from the treason map it is saying depending upon that region choose am iid and assign it here an instance state remember T to micro I showed earlier that will come here so everything is parametrized when I'm talking about my server I am NOT almost hard coding anything except for this one field where it says well I see two electric load function demo that is my home page that is expected to be started so this is the code that you want to write particularly if you are familiar with writing little bit of a bash this four lines must be very set forward you can fight if this is in Windows instance you can write it in PowerShell also or VB script orbiter is convenient to you and Amazon expects it to be URL encoded that is base64 encoded so I have to do it like this this is a syntax I cannot avoid these two lines I assume everybody is familiar with the user data so I am just going to leave it as it is because you should get a field is always expected to be you are base64 encoded or URL encoded and I am outputting to to three different values one of the value my outputting is my image ID what is the image that I am using and then I am outputting the public IP address of my ec2 instance so that's the brief explanation about this what I am going to to now is I'm just going to copy this into my notepad so click on raw disk encode a through C and then when they go to services I am going to type cloud formation and click on this one so which region am I am I am an episode now let me go ahead and quickly open my s3 bucket I just want to show you something there oh come on okay you can see here as of now let us take this as a reference there are five buckets and in two regions so let us go ahead and create our formation using the console itself completely I am NOT storing it in s3 I'm not storing anywhere I am just going to copy paste the code very simplest way not complicating it at all so when you go to the Commission - port this is a dashboard here and you have two options create a stack let us click on create stack and it is asking you how do you want to create your stack do you want to design it or you have an existing template which is in some location you want to choose it or you use a pre-existing stack this is all we defined where Amazon you can click on it if Amazon will build these templates for you or if you have it in your SP location you can click on this and choose your SP location also we don't want to do all these things right now go ahead and design your template click on this option so here is the interesting bits of the pieces pieces happens amazon says that you can use your left hand side resource types to build your template like for example you can drag and drop things and it will work I found it little bit difficult to work with but if you are comfortable to over head and try to build your template since I mean Amazon say stop using your Jason and Yama's you can come here and configure most of the pieces and get the code written for you automatically but I found it a little difficult to work with but if you are feeling good go ahead and try it and I am using an amble format so I am going to click on this option its HTML here so that let us click on that and what I need to do is here is you can see two sections components and templates I am NOT looking at the components I am going to look at templates so click on templates here and by default as you can see the version is automatically given so I am going to paste my code which might also have another version number so I'm just going to paste my code which I copied from github and go all the way on the top and you see there are two version numbers although it is not necessary or mandatory it's just remove one of them and you see here as soon as I change something in my template it has gone out of state and keep this a secret first you know that is not important the most important thing is this one you will see this tiny tick mark guys as it says valid validate template this is the option that I said you can sit for syntax errors if you have written a template and you want to ensure that it is compliant with the most of Amazon's guidelines of good template or syntax level you want to paste it and click on this button and you keep an eye out on this area because it quickly disappears after it cleans a notification so I am clicking on it it says validated template and if my template is good it will say something like template is valid and it will disappear so we are good here now we have copy pasted our code we have validated our template it is all error-free now we can go ahead to deploy it for decline you to call cloud what you as you do the same option next to it click on create stack so automatically you can see here automatically Amazon has uploaded to my s3 bucket here automatically so also I can click on next and go to the next screen let it go hide to this week and sold in quickly check out what has happened here if you guys can see it there is a new bucket created in my account in Mumbai region and my code has been already copied to this my code has been already cooperated to this from the template and these are all the code related information that is inside the bucket so anyway this is not for us it is for Amazon to manage it and you can see a new work template I did not change the template name so automatically Amazon's picked up the template name and stored it there so moving back on we have to do is in this screen click on next and remember the parameter fields all that I have chosen here is just giving me all that options that we chose some time back or provided in our template CIDR block I can write my own this is completely configurable I don't want to create a new subnet so I leave it as it is but what you can do is it is completely customizable so you can write whatever you want here you are but only thing is if you create a rock and you need to create a subsequent subnet inside the serial block as I said I am NOT checking for logical errors in my template animation also does not check for logical errors so if you create some other range here and then create to submit some others range it is not going to work it is going to throw an error so don't do those kind of take my template does not do all those things and it will fail and your template will not be deployed so don't try to take my template for those kind of things so let us start filling it up stack name I am going to say galaxy Co demo this is my stack name and do we have that no I don't have a type uterus I just go ahead and pick it up from here blah blah this is my default and what is the instance name we can give see this is all completely customized I can write whatever I want here and see here T 2 micro remembers in the field I showed you I want to enforce only one newer instance so it is there and nothing else that comes up here and I am going to choose the key that is already available here see you can see here there are two keys Mumbai trainer key whereas ETL ETS demo ki tamaki zero one I am going to choose this key because I have it already and my desktop if I need to copy a can use that so I am going to choose this here remember I did not hardcore this it automatically comes from my account and availability zones automatically from the Amazon source subnet you know whether you want to choose this or not so we have chosen our private to be in one year so we will make the public to be in one B so that is all that is to be done here click on next or if icon just go to previous quickly you can see your small descriptions this is a description field we gave in the template itself it just not something Amazon provides you can write this customized messages so that your users can make inform the choices and coming back what are the times you want to create here again I want to create galaxy demo or I could say see if the more run one I can add one more something like environment and let's do you can keep on adding whatever you want and mine is a simple demo an ordered me this word so I will leave it as it is and trim about this at once I will definitely want all of you to go here and get yourself in here is all these options very very interesting options remember I wanted speaker business popping keeping the management outside of the template itself so this is where you do that you create a topic subscribe to a topic and send an image when the stack is completed not completed all those things here don't do that inside your template and time order you want to wait how long you want to wait I am going to say something like I want to queue five minutes for my template to successfully complete if it is not completing we want to roll that this is the option yes roll back on failure if I click on no it will leave the resources in my account and stack policy is something like if you want to update our push notification or something in it unintentionally I mean you want to verify who can update your policy and then make sure the stack policy is updated when they are certain people are uploading data so you can have a configuration file and uploaded all those things bla bla bla but let us ignore it I don't want to do anything here this field is for new people explain this but this is just a summary we know this what is happening here so click on create now this is where the fun and magic starts as you can see here nothing is happened as usual Amazon likes to hit as the fresh one small time taxes oh this is a new feature taxes I heard about it in the Jenay or the bamboo conference I have not started using it but this is a new field I'm a new set of information that Amazon is segregating here stack sets so anywhere mean I want you to focus only on this point will come to this one later let's close on this one so if it says create in progress how do I know what is happening so click on that and then you probably have to click on one of these buttons let us do this one but one so you can see that as well this and you can see here already it has completed a creation of certain things for example might mov pc my internet acade way bla bla bla things okay so it constantly creates my network stack and everything I am just going to go to my ec2 is that one let me close this yes so as of now one running instance and you can see here this is the one that is created today just now when we started running it galaxies see if the more and one and let us go to our stack and see it is completed it is still work in progress and remember the output section as of now it is still always completed it is given and public IP address let me just put it in a chat window so I am let me put it in the browser also if my web server is running already we should be able to access it if it is a server is up and online available we should be able to access that website let me just go and check whether let us go to the VC section and see what all things have happened all the way photo VPC we click on the PCs so this is the one that my toad forms a template is created if I click on that if I go to click on tags we will see a whole bunch of them here as you can see here this is the tag that I created named and this is the environment that I created and these are the three tags that Amazon itself has created based on the tags or stack information that it has and remember the one combination of Wester I told you that is co 2 subnets you will find that there you can see the name tag for my subnet it is a combination of my stack name plus the availability zone that we are talking about okay somebody has already found out let me just go back to my float for machine template and you see here the status is shown as complete and just a hit refresh and I put two events come on ok it what it does not want me to do it doesn't want to okay select on this one and events you can see here the profile machine stack is complete and that means that all other resources that ash needs to be created has been completed on our website should be the default browsing so what I could have done gone ahead one step is I could have pointed a road 53 also I have a domain name but I didn't want to set it up last night because I didn't have time I put a pointed out to this piece or something like cover WMI failure to maintain calm will point to Metro formation that is all there for the cloud formation templates so we saw how VPC works or our proto machine works I'm sorry hello block for machine works how it sets up the entire network stack from VCC is Internet gateways sitting up the tags and routing rules I am going to open the default group that was created for us this is the group if I am NOT wrong and you can see here in my inbound rules I have both 80 and port 22 and as you can notice I did not log into the server I did not touch the control for anything else everything was set up so now I find that my stack is working fine I don't need the spork anymore I just need to remove this port in my configurations and also say that just assume that I am going to use HTTPS for my server and I want to ensure that the HTTP port 443 is also required down the line so I have two changes I have identified right now and I want to make those changes in my templates now so what do i do is i go to my template let me just copy the template you know what yeah so now we have put it into our notepad editor or any editor of your favorite choice this is sublime text that I use I for some reason I like this now one is the good so I have copy pasted our template to a editor and I am going to find the section where we are having security groups so that we can remove put 22 and add port 443 so let me just scroll down find out where is my security group here we go it all I have to do is I am just going to remove this section or in other words 22 is not required and 443 is required so I am just going to say it here for 4 3 here also for 4 3 and I am going to change the description of this field also saying enable HTTP and HTTPS access through port and I'm going to say is that the documentation is the responsibility of reporter developer so I will try to keep up with Amazon's expectations so I've just updated it and I am just going to copy it and I am going to update my stack I so this is how you update your stat did anybody remind me I gave the wrong number here thank you guys you guys are not telling me and I'm making a mistake so this is the change I am going to do but I'm not going to upload this because I'm going to up make this change in the existing stack in other words we are going to update an existing stack so let us see how to do that in the console itself so I the I came here so that you know that is where I am going to make this change this is the only piece or let us even better even better is just copy this section of it that is instant security group I am going to copy this I'm going to go back to my console which is having throat formation and this is my stack is here and I am going to click on this option and I am going to erase my option actions ohs update stack and it automatically brings me to this field and asking me whether I have the template anywhere is stored I am going to say no I don't have it anywhere I'm going to edit it in my designer only automatically Amazon goes to my s3 picks up the layer class template appealing and loads it here so let us do tonight unless they contradict getting hand you cannot be on like you know you're just the your voice is one step ahead of your to your screen actually okay then I need to speak slowly I cannot successfully I yeah because like we you are like moving everything up the fever's previous screen so we are not like trying to okay okay l then slowly is a better option I guess okay I get like the Indian politician who picked it let us scroll down and find out there is a security group below the thirty fellow yeah here we go this is the security group and I am going to make a small change in my security group I say keep on saying I don't want 22 M 1 4 4 4 3 so change that to 4 4 3 again change here to 4 4 3 and change this description also to say HTTP HTTP to port 80 of Draper's space down sorry so here it goes four four three so this is a one liter which you put it in our context no I am not using it them you're updating a stack we are not creating a new stack now I copied it to show you exactly where the edits are going to happen in real world remember we are going to use the source code version control system and that that version control system will trigger the change to your template and your positioning systems like Jenkins or any other configuration scripts might you might use your following mean I am just removing the automation parts of it and doing it manually how would updates to stack stack can happen yeah we understood this bit but know why you are asking why you copied that type yeah that's what I was asking so we could have just this copy pasted right so I wanted to know if that option works here is not so it shouldn't work because we are updating an existing stack and the code has to come from Amazon itself how much now and we done manually if once you come to this screen you can copy-paste it but I wanted to show you how to update stack works and when you click on it it designers automatically it will go to my s3 bucket and pick it up and come and show here okay but if you come to the designer directly and then try to put a code it will try to set up another environment like this for us in other words you were created another stack which you would also set up another web server and instead of updating you would of created another one so as always when you update it also make a validation check and you see there it says that I am validating blah blah blah and it is shaking and it says validate and then I'm going to create click on create stack let us see how that Amazon is smart enough to find out that it is an existing stack on me and it does that I have done this demo couple of times so it should automatically go and find out what is my changes and you say that some things have only changed it is uploaded to s3 and it will compile it and tell us in the next screen click on next and you see there automatically all the fields are chosen and it will not allow me to change the certain fields here blah blah click on next and click on next I am not changing any of those things here and I'm not see well sure whether you can see it it is showing computing changes and it is showing me the list of changes or list of resources that are going to undergo changes here and I am going to pause here to ask a question so Amazon is clearly documented what are the changes it is required so let us click on update and see how soon the orbits can happen and let me click on that did it happen already let me hit refresh say it says update in progress let us go to a V PC and see what is happening let us sit with us updates happened already or not so this is treated as one more V PC that is one more security which is getting created as you can see here there's one more that is getting created with port 443 with output 22 and let us go to overload formation template again and it is still working on it update is in progress it is updating the temple of template and the server everything let us wait for it to get completed so we have now two security groups and what's Amazon going to do about the old one but this is going to keep good or whether it's quite deleted let is wait and watch anybody wants to make a claim what will happen to the other security group the old one can see here the old security group with the port 22 is gone and the new security group with food for port 3 is only there so when you do a sac update and any redundant resources are there Amazon automatically cleans it up in your account you don't have that in your account anymore so that always your stack is clean and with resources only what you need so that is how we push an update to our server and if you want you can go ahead and not sure whether the URL has changed I'm just going to go ahead and switch it to another window to see whether the URL is to the same ok make worse and it is the same so basically what we have done today is or so far is we saw why provisioning is required why scripting is not good why control is not a good white cloud formation is really good how to write a cloud formation script what are the important things what is the parameters what is the mapping what is resources and what is outputs what is nested stacks and how to create a stack and how to update a step that is all the things that we have seen so far good

Channel: Valaxy Technologies

Views: 40,119

Rating: 4.8859315 out of 5

Keywords: aws, amazon web services, demo, introduction, register, enrol, sign-up, sign-in, console, hands-on, solution, contact, classroom, training, fast-track, online, trainers, certification, leader, assistance, instructor-led, vilt, virtual instructor led, cloud computing, cloud formation, automated, provisioning, scripts, best, practices, stack, nested, paramaeters, mappings, conditions, template, validate, reference, git, version control, blue green, deployments, stack update, stack creation, stack delete

Id: kyGR5WAqx6E

Length: 110min 37sec (6637 seconds)

Published: Sat Jul 29 2017